If you’re into the twitter scene like I am, you’ve noticed Gary Parrish tweeting after each round where the remaining teams were ranked in his pre-season top 25 (and one).  That got me to thinking about the old issue of the pre-season poll versus the final poll. It’s been another good year for pre-season polls. The AP version had the Final Four teams ranked 2, 3, 9, and 13. (Is this really that good? I don’t know.) As you might suspect, after a full season’s worth of data, the voters were arguably worse, ranking the eventual Final Four teams at 1, 6, 7, and 17 in the pre-tournament poll.

I’m not here to bash the AP voters for their late-season work. I’ve done that before and I hate repeating myself. I’m really here to support pre-season ratings. Because even in my system which should know better, the Final Four teams were more highly rated before the season than in the ratings posted on Selection Sunday.

The remaining four teams were ranked 1, 2, 8, and 11 before any games were played. It’s better but not appreciably different than the AP poll, which I actually have great respect for in the pre-season. (However, Crazy Uncle takes a bow for having Kentucky and Ohio State one and two, respectively, in the pre-season. According to pollspeak.com, no AP voter had these as their top two in either order. It’s unfathomable to me that not a single voter had Kentucky as their number one. Attention future voters: A little diversity of opinion isn’t a bad thing. Channel your inner Scott Mansch before next season. The poll will be better off for it.)

My pre-tournament ratings had the teams ranked 1, 2, 4, and 20. I’m not sure if that’s better than the pre-season or not, but obviously it was worse for Louisville who dropped from 8th to 20th over the course of the season. And looking at the Elite Eight, Florida was 12th in the pre-season and 19th entering the tournament. Getting beyond the tedious manual bookkeeping, I thought it would be interesting to see how the pre-season ratings stacked up with the pre-tournament ratings when filling out a bracket. Let’s compare!

Correct picks for each round

        Pre-season    Pre-tourney
R32       21/32          22/32
R16        9/16           9/16     
E8         7/8            5/8
F4         3/4            3/4
Title   UK over OSU   UK over OSU*

The pre-season ratings win! This is partly due to good fortune in the way the bracket was set-up. In the West region, the highest-rated teams in the pre-season were Louisville (8th) and Florida (12th). The top two seeds in that region were surprising Michigan State (24th) and Missouri (14th). This would have forced you to pick a Louisville and Florida regional final which seemed unlikely on March 12th but would have worked out for you. (It’s worth mentioning here that Michigan State continued the curse of the pre-season-unranked one-seeds.) In addition to those two, the pre-season ratings would have had Syracuse over Wisconsin, while the pre-tourney ratings correctly had Baylor instead of Duke in the Elite 8.

The only Final Four team missed by the pre-season ratings was Kansas. The Jayhawks may still have beat the Tar Heels with a healthy Kendall Marshall, but if that was your only Final Four miss, you’d have to feel good about your prognosticating given the Tar Heels’ unforeseeable lineup change.

I have to think that this is a rare occurrence. I hope it’s rare, anyway. It’s deflating to think a person could have tucked away an ordered list of teams in November, not watched a single game, filled out a bracket in March based on that list, and be on the verge of winning their pool on the eve of the Final Four. However, consider that if UConn had been the 9-seed in the West instead of the South, the pre-season ratings would have had them in the Final Four instead of Louisville. And if we looked at the average prediction error of actual tournament games using the two systems, I suspect we’d find that the final ratings did better. However, it’s probably not that unusual for the pre-season ratings to be competitive.

Let me make this clear: Do not use pre-season ratings to fill out your bracket in the future. That is not the point of this analysis. The point really is that 30-35 games are not enough to get a true measure of a team. The second-most common complaint about my system is that the pre-season ratings have too much influence for too long. All influence is removed on the third weekend in January, which seems to be far too late for most people’s tastes, but I’ve suspected that it would make the ratings better if I left the influence in even longer. (This suspicion has been supported by Nate Silver, whose bracket-forecast model includes the pre-season poll as a predictor.)

The only reason I haven’t done this is because I haven’t systematically investigated it. However, given what we’ve seen this March, this deserves a closer look. If the pre-season ratings make the system better in February and March, it would be irresponsible not to include them in the ratings mix to some degree for the entire season.

*The asterisk on the title game prediction from the pre-tourney ratings is because Ohio State actually leap-frogged Kentucky before Thursday’s round of 64 games based on NIT results. If you waited to do your bracket on Wednesday or Thursday, you would have had Ohio State as the champ based on the ratings.