Ranking Presentation

• Warren Buffet offered a billion dollars for a perfect NCAA men’s basketball bracket.

• No one won.• Likely, he could offer the prize for 400 years

before someone won.• The difficulty is it had to be perfect, but lots of

matches are 50‐50.• However, a good ranking method will help get

many predictions correct (even if it can’t help call 50‐50 matches).

1(c) 2014 T. Russell Hanes

Ranking methods


Ranking methods


Ranking methods


Ranking methods

• This is a network graph of a simple, three team tournament.

• The Condorcet criterion is that the winner should be the winner of all head‐to‐head comparisons.

• Here, A dominated B (not about score, just the term we use instead of “wins over”).

• B dominated C. And A dominated C.• Therefore, A is the Condorcet winner.• This seems very simple, but real life ranking

methods violate the Condorcet criterion all the time.


Ranking methods

• First‐past‐the‐post elections can violate the Condorcet criterion.

• Here, we would normally say candidate A wins the election because it received the most first place votes.

• However, the Condorcet criterion says we should look at head‐to‐head comparisons only.

• If A had not run, his 43 votes would split between B and C, 30 to 13. Without A in the race, candidate B would get 67 votes, and C would get 33.

• Let’s look at all the pairwise comparisons.


Ranking methods

• For example, without candidate B, her 37 votes would go to A and C, 10 to 27. A would still win, but just barely.

• The most interesting point is that without C, all 20 of his votes would go to B, and candidate B would win.

• Therefore, B is actually the Condorcet winner, not A. C acted as a “spoiler.”

• Lots of voting methods have been invented to ensure the Condorcet winner actually wins.

• The simplest is the instant run‐off. People mark a first and second choice on their ballots. The lowest place candidate’s votes then get sent to their second choice.

• In this example, C would be eliminated, and his votes would go to B, and B would win.


Ranking methods

• An adjacency matrix shows dominances. For example, row 1, column 2 shows that B dominated A.

• This is the adjacency matrix for our election example.

• This matrix shows there is a clear Condorcet winner because it can be written in upper‐right triangular form (no dominances below the upper‐left‐to‐bottom‐right diagonal).


Ranking methods

• Adjacency matrices can be used to encode all sorts of network graphs. Here is a network graph of lifts (up arrows) and runs (colored lines) at Mt. Hood Meadows ski resort in Oregon.


Ranking methods

• Here are the lifts. Row 1, column 2 shows a skier can take a lift from point A to point B.


Ranking methods

• And here are the runs.• Adjacency matrices allow paths to be mapped

from lift to run to run back to a lift.• Adjacency matrices like these are behind Google

Maps directions.


Ranking methods

• How does the Condorcet criterion apply to figure skating?

• One difficulty is that judges can’t be allowed to judge competitors from their own country.

• Here, four nations are competing. Competitors A, B, C and D are from four different nations.

• Judges A, B, C, and D are from the same nations. Therefore, only judges B, C, and D can judge competitor A.

• The other difficulty is that scores are subjective.• Judge A seems to be loose with points, while judge D is

stingy. Since judge A does not judge competitor A, that seems to put competitor A at a disadvantage.

• Points totals, just like first‐place votes, is a flawed method.


Ranking methods

• The solution is to look at head‐to‐head comparisons. Judges C and D both judged competitors A and B, and both judges said A was the superior skater. So A dominates B, and we can ignore the judges’ actual points.

• Filling in all head‐to‐head comparisons, we see that A is the Condorcet winner.


Ranking methods

• If we look at total points, yes. Judge B has cheated to make competitor B the winner of total points.

• But no, one judge cannot effectively cheat if we look at head‐to‐head comparisons. All judge B has done is given competitors A and C his one tied vote.


Ranking methods

• Here is the revised network graph with judge B’s attempt to cheat. Competitor A still dominates competitor C, because judge D voted for A.

• A is still the Condorcet winner.


Ranking methods


Ranking methods

• A tie, or more specifically a Condorcet paradox, occurs when there is no Condorcet winner.

• In network graph terms, we refer to it as a cycle.


Ranking methods

• Consider this cycle in terms of the adjacency matrix.


Ranking methods

• There is no way to re‐rank the teams to avoid one dominance being outside upper‐right triangular form.

• We have to “override” one result to rank the teams – i.e., break the tie.


Ranking methods

• Tie‐breaking by total points is, just like elections and figure‐skating, a silly method.

• B would win under total points, but B was clobbered by A.

• Instead, look at the point differentials.


Ranking methods

• The point differentials show that the result we should “override” is C’s dominance of A. This result is the most likely to have been an upset or fluke.


Ranking methods


Ranking methods

Here it is. One can use point differentials to resolve the cycle.


Ranking methods


Ranking methods

• What happens if you break the tie B‐C‐D by flipping C‐D? You create a NEW cycle.

• We have to come up with more sophisticated methods. Rather than breaking cycles one by one, we have to find ways to rank the teams that MINIMIZE the total number of cycles.

• Note: We will use this running example through slide 48.


Ranking methods

• Is it better to have one extreme upset to override, like the result on the left? In that ranking, the worst team beat the best team.

• Is it better to have multiple, smaller upsets to override, like the result on the right? In that ranking, the second best team beat the first best, the third best beat the second best, and the worst beat the third best.

• There’s no a priori way to decide which one is better. It depends on how big the gap is between D and B. If B is much, much weaker, the outcome on the left is quite unlikely. If D and B are evenly matched, then the outcome on the left is more likely than the outcome on the right.


Ranking methods

• The probability of upsets follows a logistic curve. If the difference in teams’ strengths is close to 0, then it is a 50‐50 match. If the difference in strengths is large, the probability of an upset drops dramatically.

• In some sports, the curve is much sharper than in others. In football, for example, there are a lot of upsets. Even weak teams sometimes knock off much stronger teams –in part because the discrepancy in strength from best team to worst team is not that great.

• Ranking methods use various ways to “extract” from wins and points the strengths of each team. If a method finds great disparities in strengths, extreme upsets are unlikely. If a method finds minimal disparities in strengths, extreme upsets might occur more frequently.


Ranking methods

• This method is simple, easy‐to‐implement in Excel or code into other software, and transparent. It has a high likelihood to be accepted as an official tie‐breaker for some kinds of tournaments.

• It is somewhat sensitive to upsets.• This is a method of my own invention. The method factors in opponent strength.


Ranking methods

• The tournament results are in the upper‐left adjacency matrix.• For each win, a team can earn additional points for its defeated

opponents’ wins. Look at the upper‐right matrix. For example, A beat B and C. Teams B and C both won one match, so A gets one additional point for each win: 2 weighted wins.

• Team D beat A and C. Beating team A (2‐1) is worth 2 additional points; beating C is worth only 1. So D gets 3 weighted wins.

• The same for losses: a team can lose additional points for its defeating opponents’ losses. Look at the lower‐left matrix. For example, team B lost to A and C. Losing to A (2‐1) is worth 1 additional loss; losing to C (1‐2) is worth 2 additional losses. Therefore, team B has 3 weighted losses. (Read in column 3)

• Add the original wins and losses to the weighted wins and losses to see the final rankings. Translated into a three‐round tournament, A goes 2‐1, D goes 1.9‐1.1, etc.


Ranking methods

• Markov chains are simple to implement in Excel or other software.

• However, it is not very transparent and unlikely to be used as an official tie‐breaker.

• It is very sensitive to upsets. Even upsets at the bottom can change rankings at the top.


Ranking methods

• The idea behind Markov chains is simple: each win is a “vote” a team casts for itself, while each loss is a “vote” for the team that beats it. These votes are expressed as probabilities.

• For example, a fair‐weather fan who started out with team A would have a 2/3 chance of sticking with A and a 1/3 chance of defecting to team D.

• Note carefully: The direction of the arrows is reversed from a regular network graph.

• After wandering around all tournament, where is the fair‐weather fan likely to end up?


Ranking methods

• The top matrix [m] loads in the probabilities for our fair‐weather fan.

• The bottom matrix is [m]^n, where n is a very large number.

• The bottom matrix indicates that the fair‐weather fan has approximately a 30% chance of ending up with team A, a 40% chance of ending up with D, etc.

• These probabilities are used to generate the ranking. The probabilities also indicate some measure of relative strength.


Ranking methods

• Of course, we could use Markov chains with point differentials as well. Here are possible differentials for our tournament.


Ranking methods


Ranking methods

• Here is the graph for the Markov chain based on differentials.


Ranking methods

• The top matrix [m] is the probabilities for our fair‐weather fan, based on point differentials.

• The bottom matrix is [m]^n, where n is a large number.

• The bottom matrix shows the fair‐weather fan has a 55% chance of staying with A, a 27% chance of staying with D, etc.

• In this ranking, A comes out on top due to the large margins it ran up. A is rated far higher than D than D is rated than B.


Ranking methods

• These methods require some basic linear algebra, but they can be implemented in Excel.

• These methods are used by the BCS and other organizations to pick post‐season teams; however, given the lack of transparency, these methods are unlikely to be adopted as tie‐breakers for simpler tournaments.


Ranking methods

• The Massey method uses point differentials. The right hand matrix is point differentials for the first three teams, and finally, the last team’s differential is set to 0.

• t is total games played.• g is the individual matches any two teams played.• The final row of the left hand matrix is set to all 1’s.

The purpose is to make all the ratings sum to 0.• The r’s are the final ratings for the teams.


Ranking methods

• The teams are in order A, D, B, C.• Now we need to invert the left hand matrix and

left‐hand multiply on both sides to find our ratings.


Ranking methods


Ranking methods

• The Colley method does not use point differentials, only wins and losses.

• The meaning of t, g, and r are the same as in the Massey method.


Ranking methods

• Our running example in the order A, D, B, C.• Now we need to invert the left hand matrix and

left‐hand multiply on both sides to find our ratings.


Ranking methods

• The Colley method does not break the ties between A and D and between B and C. In more complex tournaments, the Colley method would break these ties.


Ranking methods

• This method makes a lot of sense when teamsclearly have bivariate strength. Football is a perfect example: the offensive team and defensive team are literally two different sets of players. This method works well for baseball: batting and pitching/fielding are two very different skill sets.

• The goal of this method is break a score into the contribution made by the offense (above or below average effort) and the opposing defense (above or below average effort).

• This is unlikely to be used as an official tie‐breaker, but it is easy to use to adjust matches mid‐tournament.


Ranking methods

• Here are six teams that have played two opponents each.

• The scores for each offense are recorded by row. For example, A scored 10 points against C, whereas in the same game, C scored 5 points against A. Therefore A won.

• The winner of each match is highlighted in orange.• The average offense (points scored) and average defense (points allowed) are recorded for each team.


Ranking methods

• Each team’s points scored in each game is divided by its average points scored.

• Thus, A’s score of 10 points against C is divided by A’s average of 8.5 to produce the 1.176. A’s score against C is above A’s average score. A’s score against E is below A’s average score.

• The adjusted defensive ratings are calculated for each team.

• For example, in column 1, A’s defense kept C and E both to below their average scores. Thus, A’s defense is strong.

• C’s defense is weak because both A and E ran up above average scores.


Ranking methods

• We divide the original scores by those adjusted defensive ratings.

• A’s score against C drops from 10 to 8.876, because C’s defense is weak.

• We use these adjusted scores to calculate adjusted offensive scores.

• A’s overall offensive score comes down because both of its opponents were weak.

• We can reiterate the process a couple times until the defensive ratings and offensive scores stabilize.


Ranking methods

• We stage all the hypothetical games.• In each game, we find the score by multiplying adjusted offense score against adjusted defensive rating. For example, A’s adjusted offensive score against B’s defensive rating indicates A might score 7.82 points.

• However, B’s offense times A’s defensive rating indicates B might score 9.9 points, thus winning.

• Actual wins are in orange; hypothetical wins are in yellow.


Ranking methods


Ranking methods

Date post:	19-May-2017
Category:	Documents
Upload:	t-russell-hanes
View:	216 times
Download:	0 times

Ranking Presentation

Documents