Debate Ratings - First Semester 2016-17

The final ratings for the first semester of the 2016-17 season are posted.

One unforseen consequence of posting the previous edition of the ratings when there were so many quality teams without the required number of rounds to be listed is that it artificially inflated the rankings of a lot of teams.  As a result, many teams that were previously ranked in the top 50 dropped a number of spots without actually performing any worse.  They were just bumped down as new teams were added to the list.  In the future, I'll have to consider whether it might be better to just wait until the end of the first semester for the first release.

I wanted to wait until the coaches poll was out to post the new ratings.  I will refrain from commenting in any detail about specific teams, but it is interesting to think about the differences in where some teams are ranked.  I doubt that there is a single factor that can explain all of the instances where there is divergence between the computer rating and the human poll.  However, if I were to make a couple of guesses about what might be at work, I think the following might be relevant:

1. It is possible that human voters are more likely to think in terms of team performance as a "resume" or "body of work."  Thus, teams that the computer ratings like because of strong head-to-head results might be disadvantaged if they have been to fewer tournaments (or less total prestigious tournaments).
2. It may be possible that human voters are more likely to value "elim depth" with less regard for the specific opponents that teams defeated (or lost to).  The computer ratings do give extra weight to elim wins, but what matters is *who* a team competes against in elims rather than which round they made it to.  Thus, the algorithm might be more impressed with a team that took down two highly rated opponents and dropped in quarters than a team that had an easy draw to semis.
3. For teams with a fewer than average number of rounds, it is possible that there could be a moderately outsized recency effect of their results.  Less data makes a team's rating more volatile, which means that they can move it up (or down) more quickly.
4. It might be the case that in some instances the computer algorithm could be less forgiving of teams with inconsistent results.  While this was only a quick dive into the data (and there are not very many data points to compare), it appeared on first glance that teams that possessed both a high rate of error (performed against expectation more often) and a large number of total rounds (which should tend to reduce error) performed slightly worse in the computer ratings versus the human poll.  Just or Unjust?  You decide.
5. Finally, UMKC.  Pretty much down the line, the human poll valued success at the UMKC tournament less than the computer did.

I hope to get my hands on the raw data from the coaches' ballots to see how much consensus/dissensus there was among the voters.  It could be useful to evaluate whether the divergence that we see with the computer rankings is within the range of human disagreement internal to the poll itself.

The usual disclaimers:

1. These are not my personal opinions.  The algorithm is set and runs autonomously from how I may personally feel about teams.  I do not put my finger on the scale.
2. The ratings are determined by nothing more than the head to head outcome of debate rounds.  No preconceptions about which schools or debaters are good, no weighting for perceived quality of tournaments, no eye test adjustments.  If you beat somebody, your rating goes up and theirs goes down.  If you beat somebody with a much higher rating, it goes up more.  If you beat them in elims, it will go up by more than if you do so in prelims.  That's it.
3. It is still early in the season, so the ratings are subject to a fair amount of volatility, especially for teams that have a fewer number of rounds.  They grow more stable over time.

For a sense of what the ratings number actually means:

• A 1 point ratings advantage translates roughly into 5:4 expected odds,
• 2 points is about 3:2
• 3 points is about 2:1
• 4 points is about 3:1
• 5 points is about 4:1
• 8 points is about 9:1