Here at kenpom.com, we care deeply about how well our predictions do. It’s one thing to say you’ve picked 76% of winners, but that’s a meaningless statistic, especially in a sport with as many mismatches as college hoops.
That’s why this season we are tracking the performance of our forecasts. Eventually, we’ll use this information to calibrate the model a little better. I think people that follow this stuff closely understand that the percentages given to winning in my system are a little too aggressive. (For this reason, I water down the initial chance of winning in the win probability graphs.) But I won’t know how much until I aggregate the performance of predictions over the course of the season.
Here’s the breakdown on predictions through Tuesday’s games.
Prediction W-L Pct 50-55% 38-32 .543 56-60% 34-29 .540 61-65% 38-28 .576 66-70% 56-24 .700 71-75% 50-24 .676 76-80% 51-13 .797 81-85% 67-17 .798 86-90% 67-19 .779 91-95% 130-5 .963 96+% 194-5 .975
Ideally, predictions of a win chance of X% should hit X% of the time. Based on that, the predictions are holding up fairly well which to me means the pre-season ratings were well worth it. Actually, the pre-season ratings have helped keep the predictions from being too aggressive early in the season. Of course, I am pre-conditioned to want my choice to use pre-season ratings to be justified, so take my opinion in that context.
The top upset so far has been Texas Southern over Oregon State, where the Tigers cashed in on a 2.1% chance for victory. Undoubtedly, there are bigger shockers to come. (No, a team with a 43% chance to win doesn’t qualify.)