## A very simple ranking system

Posted by Doug on May 8, 2006

My friend Joe Bryant says that the BCS bowl matchups are like getting a shrimp cocktail at Morton's Steakhouse. Sure, it's better than what you normally eat, but at the same time it's frustrating and disappointing because you can see a bunch of far preferable alternatives right there in front of your eyes. I tend to agree. Nonetheless, it is not in any way an exaggeration to say that the BCS revived my interest in college football. Not because of the matchups the system has produced, but because it gave me an excuse to learn some very interesting mathematics.

As you probably know, the participants in the BCS championship game are determined in part by a collection of computer rankings. Those computer rankings are implementing algorithms that "work" because of various mathematical theorems. At some point, I'm going to use this blog to write down everything I know about the topic (which by the way is a drop in the bucket compared to what many other people know; I am not an expert, just a fan) in language that a sufficiently interested and patient non-mathematician can understand.

I'll start that off today by describing one of the most basic ranking algorithms.

The idea is to define a system of 32 equations in 32 unknowns. The solution to that system will be collection of 32 numbers and those numbers will serve as the ratings of the 32 NFL teams. Define *R_ind* as Indianapolis' rating, *R_pit* as Pittsburgh's rankings, and so on. Those are the unknowns. The equations are:

R_ind = 12.0 + (1/16) (R_bal + R_jax + R_cle + . . . . + R_ari)

R_pit = 8.2 + (1/16) (R_ten + R_hou + R_nwe + . . . . + R_det)

.

.

.

R_stl = -4.1 + (1/16) (R_sfo + R_ari + R_ten + . . . . + R_dal)

One equation for each team. The number just after the equal sign is that team's average point margin. In plain English, the first equation says:

The Colts' rating should equal their average point margin (which was +12), plus the average of their opponents' ratings

So every team's rating is their average point margin, adjusted up or down depending on the strength of their opponents. Thus an average team would have a rating of zero. Suppose a team plays a schedule that is, overall, exactly average. Then the sum of the terms in parentheses would be zero and the team's rating would be its average point margin. If a team played a tougher-than-average schedule, the sum of the terms in parentheses would be positive and so a team's rating would be bigger than its average point margin.

It would be easy to find the Colts' rating if we knew all their opponents' ratings. But we can't figure those out until we've figured out *their* opponents' ratings, and we can't figure those out until. . ., you get the idea. Everyone's rating essentially depends on everyone else's rating.

So how do you actually find the set of values that solves this system of equations? In high school you probably learned how to solve 2-by-2 and maybe 3-by-3 systems of equations by putting some numbers into a matrix, doing some complicated operations on that matrix, and then reading the solutions off the new matrix. Same thing here, except you've got a 32-by-32 matrix instead of a 2-by-2 matrix. If you wanted college football rankings, it'd be 120-by-120. I recommend using a computer.

It's more instructive, though, to solve it a different way. We'll start by giving everyone an initial rating, which is just their average point margin. I'll use the Colts as an example. Their initial rating is +12.0. Now look at the average of their opponents' intial ratings:

Opp Rating

===========

ari -4.75

bal -2.12

cin 4.44

cle -4.31

hou -10.69

hou -10.69

jax 5.75

jax 5.75

nwe 2.56

pit 8.19

ram -4.12

sdg 6.62

sea 11.31

sfo -11.81

ten -7.62

ten -7.62

Those average -1.2, so the Colts' new rating will be 12.0 - 1.2, which is 10.8. So after this calculation the Colts' rating changed from +12 to +10.8. But meanwhile, every other team's rating changed as well, so we have to do the whole thing over again with the new ratings. On the second pass, the Colts schedule looks a bit different:

Opp Rating

===========

ari -4.76

bal -1.49

cin 4.09

cle -3.85

hou -9.69

hou -9.69

jax 4.85

jax 4.85

nwe 3.09

pit 8.02

ram -5.16

sdg 8.62

sea 8.99

sfo -10.77

ten -7.30

ten -7.30

The average of these is -1.1, so the Colts' opponents aren't quite as bad as they looked at first. Indy's new rating is 12.0 - 1.1, which is 10.9. Uh oh! Everyone else's ratings just changed again, so we've got to run through the same procedure again. And again. And again. And eventually the numbers stop changing. When that happens, you know you've arrived at the solution. Take a look at the Colts schedule with the final rankings and you'll be able to convince yourself that this method works:

OPP Adj

WK OPP Margin Rating Margin

==============================

1 bal 17 -1.83 15.17

2 jax 7 4.76 11.76

3 cle 7 -4.22 2.78

4 ten 21 -7.57 13.43

5 sfo 25 -11.15 13.85

6 ram 17 -5.15 11.85

7 hou 18 -10.03 7.97

9 nwe 19 3.14 22.14

10 hou 14 -10.03 3.97

11 cin 8 3.82 11.82

12 pit 19 7.81 26.81

13 ten 32 -7.57 24.43

14 jax 8 4.76 12.76

15 sdg -9 9.94 0.94

16 sea -15 9.11 -5.89

17 ari 4 -4.98 -0.98

==============================

AVERAGE 12.0 -1.20 10.80

==============================

How to read this table: in week 1, the Colts beat the Ravens by 17. The Ravens were, all things considered, 1.83 points worse than average, so the Colts got a "score" of 17 - 1.83, or 15.17 for that game. In week 2, the Colts beat the Jaguars by 7. Jacksonville was 4.76 points better than average, so the Colts get an 11.76 for that game. Average their scores for each game and you've got their rating. The bottom line says:

The Colts' won their games by an average of 12 points each. Their opponents were, on average, 1.2 points worse than average. Thus the Colts were 10.8 points better than average.

Let's examine some of the features of this system:

**The numbers it spits out are easy to interpret**- if Team A's rating is 3 bigger than Team B's, this means that the system thinks Team A is 3 points better than Team B. With most ranking algorithms, the numbers that come out have no real meaning that can be translated into an English sentence. With this system, the units are easy to understand.**It is a**- this is a very important distinction. You can use these ratings to answer the question: which team is stronger? I.e. which team is more likely to win a game tomorrow? Or you can use them to answer the question: which of these teams accomplished more in the past? Some systems answer the first questions more accurately; they are called predictive systems. Others answer the latter question more accurately; they are called retrodictive systems. As it turns out, this is a pretty good predictive system. For the reasons described below, it is not a good retrodictive system.*predictive*system rather than a*retrodictive*system**It weights all games equally**- every football fan knows that the Colts' week 17 game against Arizona was a meaningless exhibition, but the algorithm gives it the same weight as all the rest of the games.**It weights all points equally, and therefore ignores wins and losses**- take a look at the Colts season chart above. If you take away 10 points in week 3 and give them back 10 points in week 4, you've just changed their record, but you haven't changed their rating at all. If you take away 10 points in week 3 and give back 20 points in week 4, you have made their record worse but their rating better. Most football fans put a high premium on the few points that move you from a 3-point loss to a 3-point win and almost no weight on the many points that move you from a 20-point win to a 50-point win.**It is easily imressed by blowout victories**- this system thinks a 50-point win and a 10-point loss is preferable to two 14-point wins. Most fans would disagree with that assessment.**It is slightly biased toward offensive-minded teams**- because it considers point margins instead of point ratios, it treats a 50-30 win as more impressive than a 17-0 win. Again, this is an assessment that most fans would disagree with.**This should go without saying, but**- I'll say it anyway. The system does not take into account injuries, weather conditions, yardage gained, the importance of the game, whether it was a Monday Night game or not, whether the quarterback's grandomother was sick, or anything else besides points scored and points allowed.

This system, like all systems, has some drawbacks, but it has the virtue of simplicity. It is easy to understand and it produces numbers that are easy to interpret. That is not to be sneezed at.

Furthermore, most of its drawbacks have easy fixes. For example, when computing a team's initial rating --- i.e. their average point margin --- you can tweak the individual game margins to make the initial rating "smarter." One way to do that is to cap the margin of victory at 21 points, or 14 points or whatever you want. You can explcitly incorporate wins and losses by giving the winning team a bonus of 3 points or 10 points or however many you want. To take it to the extreme, you could simply define all wins to be one-point wins and all losses to be one-point losses. This removes margin of victory from the scene completely. As usual, when you tweak the method to stengthen its weaknesses, you also weaken its strengths. In particular, if you use a modified margin of victory, the numbers don't have as nice an interpretation.

I'll close with some rankings. Here are the NFL's 2005 regular season rankings according to the original method:

Team Rating StrOfSched

=============================

1. ind 10.8 -1.2

2. den 10.8 2.2

3. sdg 9.9 3.3

4. sea 9.1 -2.2

5. pit 7.8 -0.4

6. nyg 7.5 0.7

7. kan 7.0 2.1

8. was 6.0 1.9

9. car 5.1 -3.2

10. jax 4.8 -1.0

11. cin 3.8 -0.6

12. dal 3.2 2.1

13. nwe 3.1 0.6

14. chi 1.4 -2.2

15. mia -0.8 -0.8

16. tam -1.0 -2.6

17. atl -1.2 -1.9

18. bal -1.8 0.3

19. phi -2.3 2.6

20. oak -2.8 3.0

21. min -3.5 -1.1

22. gnb -3.7 -0.8

23. cle -4.2 0.1

24. ari -5.0 -0.2

25. ram -5.1 -1.0

26. buf -5.8 0.2

27. nyj -6.4 0.8

28. det -6.7 -1.0

29. ten -7.6 0.1

30. hou -10.0 0.7

31. nor -11.1 -0.9

32. sfo -11.1 0.7

Here they are if every win of less than 7 points is counted as a 7-point win and if the margin of victory is capped at 21.

Team Rating StrOfSched

=============================

1. den 10.1 1.6

2. ind 9.9 -1.4

3. sea 7.1 -1.9

4. sdg 6.9 2.9

5. nyg 6.3 0.7

6. pit 6.1 -0.6

7. was 5.5 1.6

8. kan 5.4 1.7

9. car 4.8 -2.3

10. jax 4.8 -1.1

11. cin 3.8 -0.9

12. dal 3.6 1.6

13. nwe 2.8 0.7

14. chi 1.5 -1.8

15. tam 0.9 -1.9

16. mia 0.6 -0.7

17. atl -0.4 -1.3

18. min -1.8 -1.1

19. phi -1.9 2.1

20. cle -3.2 -0.2

21. bal -3.4 0.3

22. oak -3.6 2.6

23. gnb -4.9 -0.5

24. buf -5.1 0.3

25. ram -5.1 -0.7

26. ari -5.1 -0.1

27. nyj -5.8 0.8

28. det -6.0 -0.8

29. ten -6.7 -0.1

30. sfo -8.1 0.5

31. nor -9.2 -0.4

32. hou -9.8 0.5

Here they are with margin of victory removed altogether:

Team Rating StrOfSched

=============================

1. den 0.69 0.07

2. ind 0.66 -0.09

3. sea 0.50 -0.12

4. jax 0.42 -0.08

5. nyg 0.42 0.04

6. was 0.37 0.12

7. pit 0.34 -0.03

8. kan 0.33 0.08

9. cin 0.31 -0.07

10. sdg 0.29 0.17

11. nwe 0.26 0.01

12. chi 0.26 -0.11

13. tam 0.25 -0.13

14. car 0.24 -0.13

15. dal 0.22 0.09

16. mia 0.06 -0.07

17. min 0.05 -0.07

18. atl -0.06 -0.06

19. phi -0.14 0.11

20. bal -0.23 0.02

21. cle -0.26 -0.01

22. ram -0.28 -0.03

23. oak -0.36 0.14

24. ari -0.37 0.01

25. buf -0.37 0.01

26. det -0.41 -0.03

27. sfo -0.44 0.06

28. nyj -0.45 0.05

29. gnb -0.49 0.01

30. ten -0.50 0.00

31. nor -0.63 -0.01

32. hou -0.71 0.04

**ADDENDUM:** I need to clarify one thing about the simple rating system: it’s not my system. I didn’t invent it. In fact, it’s one of those systems that has been around for so long that no one in particular is credited with having developed it (as far as I know anyway). People were almost certainly using it before I was born. I like the system and use it a lot because it’s fairly easy to interpret and understand, and because the math behind it is nifty. But I just realized that I had never been clear enough about the fact that it’s not my system. I just use it.

Yes I have excel

Here's a video I made for Chase a long time ago, outlining how to do SRS in Excel 2007:

http://www.screencast.com/users/Sports-Reference/folders/PFR/media/cea8d1e3-ed91-431f-a0df-1336c03b1268

I have to warn you, it's probably pretty confusing. But it might help.

Also, you should check out Wayne Winston's book Mathletics, which talks specifically about how to calculate ratings (including SRS-like systems) in Excel.

http://www.amazon.com/Mathletics-Gamblers-Enthusiasts-Mathematics-Basketball/dp/069113913X

Hey thanks man that actually helps a lot I had to watch it a few times but I see what your cooking.

By the way whats up with the margin numbers on sheet one where its lists all the games. It looks like you subtracted 1.35 or .65 from the margin of victory of each game.

That's probably some kind of home-field advantage term; should be around 2-3 ppg (you subtract it from the home team's score margin and add it to the road team's margin).

Hey man

Whats the formula you use for SOS I can't see it clearly enough in the video. I can only get SOS by doing manually thats ok for football but ain't going to work very good for other sports.

By the way I got that book Mathletics interesting read.

SOS is the average SRS of a team's opponents. So if team's SOS is 1, it means on average, their opponents were about 1 point above average. It works fine for other sports.

Ya I know I want to use it for other sports I ment the actual formula he used in the excel video like

=average(if(sheet!$D$2:$C$207=A2,sheet$D$2:$C$207))

Thats what it looked like in the video but it was too small to see and when you zoom its too destorted to read and I've been at it for days but I just cant figure it out.

[...] Russell’s teammates who made an All-Star team (All-NBA team in parentheses). I’ve included the team’s SRS after the year as [...]

[...] keeps a nifty power ranking index they call “Simple Rating System“. Actually, several of the “sports-reference.com” sites keep such a statistic [...]

[...] let’s look at this claim more closely. The 2008 Patriots had an offfensive SRS of 2.3 (11th in the league) against a weak schedule (-2.4). They scored 25.6 ppg and averaged 5.3 [...]

[...] at an embarrassing 7-9, the Seahawks are worse than their record would suggest. According to the Simple Rating System, the Seahawks were the third worst team in all of football. Jeff Sagarin’s ratings had them [...]

[...] them to other champions from the past, and the 2010 Auburn Tigers are no exception. Using the Simple Rating System (SRS), let’s take a look at where the newest title-holders stand among BCS [...]

[...] which rates teams based upon Total Points Margin adjusted up or down for strength of opponent, as outlined at Pro-Football-Reference.com, to determine how the Seahawks fared in 2010 compared to [...]

[...] the Simple Ranking System in Perl. The Simple Ranking System is described on Pro Football Reference here. It’s important because it’s a simple – perhaps the simplest – model of the [...]

[...] spoken about the simple ranking system before, and given code to calculate it. I want to set up a “More” mark, and talk issues [...]

[...] been quiet a while, because I’ve been a little busy. A version of the simple ranking system, favored by Doug Drinen, is now coded as a CPAN module. CPAN, the Comprehensive Perl Archive [...]

[...] points above/below average, where zero is average. You can get a full explanation of the rating at Pro-Football-Reference. Without getting too technical, this rating basically shows how much a team kicked ass and took [...]

[...] Simple Rating System [...]

[...] Bad teams (91-120 in Sports-Reference/College Football’s Simple Rating System) [...]

[...] Bad teams (91-120 in Sports-Reference/College Football’s Simple Rating System) [...]

I just picked up "Mathletics" but I'm having an issue with the averageif in excel. I am poor at excel so if anyone could guide me on how to use averageif to come up with the MOV. I thank you in advance for taking time to help me!

Cheers!

thanks for your help by deleting my post. All I wanted was a little help.

[...] found that there is a correlation between SRS (Simple Rating System, a metric of success showing how far above the scoring margin mean a team plays) of elite teams [...]

[...] East Detailed Standings Explanation of Simple Rating System (SRS) · or hold your mouse over the header” [...]

[...] final record, and (2) the opponents Defensive SRS, or Simple Rating System, which is PFR’s quick-and-dirty measurement of team quality. SRS is complicated to explain, but easy to understand: an average team should have an SRS of 0, [...]

[...] absurd. An even easier, shorthand way to visualize the improvements for each team is using Pro-Football-Reference’s Defensive Simple Rating System. (You can click the link to read about the SRS generally.) For each team, here are the DSRS scores [...]

[...] absurd. An even easier, shorthand way to visualize the improvements for each team is using Pro-Football-Reference’s Defensive Simple Rating System. (You can click the link to read about the SRS generally.) For each team, here are the DSRS scores [...]

[...] by 16 to get predicted wins for 2011. SRS, MOV, and SOS are Pro Football Reference’s simple ranking system metrics. SOS is a factor in playoff wins, along with previous playoff experience. Home field [...]

[...] [...]