Understanding the Relative Scoring System


Understanding the Relative Scoring System:

An overview of the inherent flaws of a Rank-Based Scoring System and why a Relative Scoring System is a fairer way to score a multi-event sport.

By Mark Legg

In a multi-event sport like boomerangs, events are scored in several different ways: high times, low times, total points, total catches, etc. In order to determine the overall ranking from these disparate event-scores it’s necessary to have a scoring system that transforms the event-scores into an overall-score. Both the Relative Scoring System (RSS) and the Rank-Based Scoring System (RBS) serve that purpose but there is a fundamental difference in the way they carry out that transformation. The RBS transforms the rankings of each event into overall-points and the RSS transforms the scores from each event into overall-points. Fundamentally, that means that the RBS emphasizes the quantity of your rank while the RSS emphasizes the quality of your score. This is a seemingly subtle difference but, if we look more closely at how these two systems function, we will see they have very different impacts on the overall rankings.

Before we take a closer look at these two systems let’s be clear about the role of a scoring system versus the role of the rules in a sport. This will help prevent any misunderstandings and false arguments about the function and purpose of a scoring system.

  1. Although scoring systems transform event-scores into an overall-score, the event-score will always be your actual score for the event. If you have a 34s flight in MTA, your score in MTA is 34s no matter how the overall-points are determined. Event scores will always be used for awards and records. The point-values produced by scoring systems only count towards the overall.
  2. Scoring systems cannot and should not account for changes in wind conditions. This is the role of the rules of the sport such as: wind-speed limits, wind delays, make-up days, and alternate wind-events like TC 100 to make the sport as fair as possible. Scoring systems’ only function is to transform one set of numbers into another set.
  3. Just like variable wind conditions, other variables are controlled for by the rules of the sport and not the scoring system. These include: differences in equipment, skill level, height, age, etc.
...with those possible misunderstandings out of the way let’s take a closer look at these 2 types of scoring systems...
The RBS assigns overall-points to the rankings in each event: 1 point for 1st place, 2 points for 2nd place, etc. The overall-points of each event are then added together to determine the overall ranking. On the other hand the RSS transforms event scores into overall-points of 0-1000 using logarithms. These are then added together for your overall ranking. Although the math in the RBS is simpler than in the RSS, it is not necessary to understand the math to understand the difference between the results of the two systems. The easiest way to see this is to look at examples of how a thrower’s overall-score is effected by a change in his event performance. Let’s consider three examples.
  1. First, let’s look at a low score in Australian Round from a tournament of 12 throwers in Kiel 2015. The low scores were as follows: 14, 30, 37, 44, 46, 47. There were 3 NP so 14 was 9th place. In the RBS, If the thrower with 14 were to improve by double up to 28, he will still be in 9th place and receive the same 9 points towards his overall. However; in the RSS, since the overall-points are relative to the score, that thrower will gain overall-points for his improvement in score regardless of his rank. The benefit here is that the RSS always encourages you to make a better performance even if you are at a very low score.
  2. A second example looks at the effect of an exceptionally high score in the two systems. Thrower B is up in endurance and the top 5 scores currently are 42, 43, 46, 48, and 51. In the RBS Thrower B only needs to get the top rank to maximize his score. To get 1st place he only needs to get 52 catches. So even if he is capable of 68 he will gain no benefit from doing better than 52. However, in the RSS he will increase his overall-points as he continues to improve beyond 52. In both of these first two examples throwers benefit from improved performance at the high and low ends of the scores regardless of rank.
  3. A third example looks at the effect of the two scoring systems in the middle scores. Thrower C scored 54 in Accuracy and took 10th place out of 20. The rest of the scores are as follows: 31, 35, 38, 40, 44, 46, 47, 49, 52, 53, 54, 55, 56, 57, 58, 61, 63, 66, 70, and 74. Scores in the middle tend to be more densely packed with fewer scores at the high and low ends. Let’s consider that Thrower C improves his score by just 5 points. In the RBS we can see that just 5 points improvement in score can result in 4 improvements in rank all the way up to 6th place. A huge improvement in his overall for only 5 points! In the RSS the overall-score will also improve, as we have already seen in the other examples. However, since the difference in event-scores is relatively small from 54 to 60 the difference in overall-points will also be relatively small.
To summarize the 3 examples above: Within the RBS, very small changes in middle event-scores result in disproportionally large changes in the overall-score. Likewise, in the RBS, very large changes in high and low event-scores result in disproportionally small changes in the overall-score. However, in the RSS all changes in event-score result in proportional changes in the overall-score. The point here is that event-scores are of little importance in the RBS; only how many people you beat. Whereas in the RSS the overall-score is relative to the event-scores. This is a fairer way to score the overall and, additionally, encourages all throwers at all levels to always be their best regardless of their rank.

This unfair distribution of overall-points in the RBS causes another serious problem with respect to scoring the overall. In general, some events like Accuracy, Australian Round, and Fast Catch have more tightly packed scores than the other events which means the ranking in those events will have a greater impact on your overall-score. Likewise, MTA, Endurance, and Trick Catch generally have less tightly packed scores which will have a smaller impact on the overall-score. To make this easier to understand let’s consider 2 scenarios in which we compare 1st and 2nd place scores in ACC and TC. These scores are easy to imagine. In ACC 1st and 2nd are 72 and 70 and in TC they are 72 and 58. A difference of 2 points in a tightly packed event like ACC receives the same overall-points as a difference of 14 points in a less tightly packed event like TC. Not fair! The RSS fixes this problem. In the same way that scores are relative to each other in the event, they are relative to each other between the events. In this example both the 72 in ACC and in TC would receive higher overall values than the 70 and 58 but the difference in value would be greater in TC due to the larger difference in event-points which is fair and sensible. To get an even better understanding of how events are scored relative to each other, let’s look at another example in the RSS. Consider 2 throwers in 2 events. Thrower D scores 40 in ACC and Thrower E scores 80 doubling D’s score. However, in AR thrower D scores 60 in while E scores 30 (half of D’s score). In this case they would be in a tie after those 2 events despite how many other throwers were involved or how they ranked because they both doubled the other’s score.

There is yet another way that the RBS can unfairly and disproportionally reward overall-points in the middle scores. In MTA we use 3 timers and a backup to make sure that we can be as accurate as possible for the thrower’s time. However, mistakes still occur. So let’s imagine a scenario in which only 2 timers have valid times. In this case we use the worse of the 2 times. Now imagine that that error costs the thrower just 0.5 seconds and that he is close enough to other thrower’s times that he will lose 3 ranks in the RBS. That is a big loss for a very small error that is not his fault. However, even with that error, in the RSS those times are so close to each other that the difference in overall-score will be 3 or 4 points out of roughly 6000. A very small impact compared to the RBS! This is, obviously, more fair to the thrower.

As mentioned before, the RSS transforms event-scores into overall-points using logarithms. Those points basically range from 0-1000 and are distributed among all possible scores in each event. However, the overall-points are on a continuous scale which means it IS possible to score more than 1000. This simply depends on which event-score the 1000-point mark corresponds to. For example: the 1000p mark could be affixed to the world record, 90% of the world record, or even a score that is impossible to obtain like 150 in endurance. Since the scales of overall-points is relative and continuous it does not matter which event-score the 1000p is affixed to. The results of the overall will be unchanged.
Having said that, the current proposal affixes the 1000p to a score of 100 in; Accuracy, Australian Round, and Trick Catch; 80 in Endurance; 15s in Fast Catch; and 50s in MTA. Again, the nature of the math used to transform event-scores does not change the relative nature of the overall-scores regardless of where the 1000p mark is set. If a competitor scores better than the 1000p mark they will receive more than 1000 points towards the overall. In addition, with the 1000p affixed to a particular score in each event, all the other event-scores will also have a corresponding point-value. These points can then be put into tables and included in rulebooks, posted at tourneys, and kept in throwers kits. Since the tables will be standardized we will quickly become accustomed to using them at competitions and during practice. These standardized tables also have an additional benefit: they can be used for international, national, personal, and historic rankings.

 ­­­We have used the RBS for many years and it has served us well. It is familiar and comfortable. It is easy to use even without computer software. However, it is flawed! A summary of a few of these flaws includes: 1. The RBS gives greater impact to small differences in scores among the middle scores and lesser impact to larger differences in scores at the low and high ends, 2. In the RBS if a competitor doubles another competitor’s score he doesn’t receive double overall-points. In fact, he may only receive 1 placing point advantage if no other competitor scores between them, and 3. In the RBS small timing errors that are no fault of the thrower can cause disproportional changes in his overall rank. As our sport moves into the next stage of development we need to be willing to recognize and address those flaws and adopt a fairer scoring system. Fortunately, we have a new system that does just that. The RSS uses logarithms to score each thrower’s overall relative to the quality of his event-scores and not by the quantity of throwers he beats. It is fairer to all involved and is a more accurate way to score the overall. This paper is not meant to persuade you to simply throw out the old system that we are so accustomed too and dive straight into the unknown waters of this new system. It is however, meant to persuade you to consider the benefits of the Relative Scoring System and to be open to trying the RSS in tournaments during the next couple of years. It does not matter that the RSS is fairer in theory. We need to test it out to see if it is fairer in practice. I think the best way to do this is to use them side by side for a year or so at all tournaments in every nation. By doing this, I believe, that by the end of the 2017 season the boomerang world will have a consensus about whether we formally adopt the Relative Scoring System in 2018. If we do not have consensus, then we will still have the RBS to fall back on. There is nothing to lose by trying. But if we don’t try, we may miss the opportunity to make our sport fairer for all involved.