I have previously acknowledged my appreciation for the preseason AP poll. Once the season starts, the poll becomes a clerical exercise that isn’t useful in the analytical realm, but before the games are played, humans and computers alike are trying to do the same thing: figure out which teams are the best.
Get 60-65 experts together and you should have something more powerful than what a computer rating can provide. Wisdom of crowds and all.
For that reason, whenever I have the typical 6-10 outliers in the top 25 to start the season, I usually assume the AP poll has the better ranking. Wisdom of crowds will generally beat my simple algorithm.
But I never really tested how the preseason ratings do when it’s kenpom vs. the world, and instead of continuing to kiss the AP’s butt in the preseason, I wanted to know how good kenpom is when it disagrees with the computer. So I designed a study like so…
Step 1: Identify all the cases since the 2015 season where my preseason ranking for a team was outside the range of all AP voters. Essentially, find the cases where every single voter bet against my ratings. (The data on individual votes on collegepolltracker.com only goes back to the 2015 season.) To qualify, a team needed to be ranked in either the kenpom top 25 or the AP top 25.
This step gets subjective when the kenpom ranking is below the lowest AP voter and there are some “not ranked” votes. Reasonable judgment was used.
Step 2: Use the team’s final kenpom ranking to determine if the AP voters were correct. If the team’s final kenpom ranking is closer to the AP range than the kenpom ranking, the AP voters win. Otherwise they lose. (This also gets subjective for a few cases. I will discuss later.)
Now, the AP voters are not trying to predict the final kenpom ranking, but there’s no post-tournament AP ranking and in a general sense, the mission of the voters and the mission of the kenpom ratings are similar before the season.
As an example, last season Iowa was 23rd in the preseason kenpom while zero AP voters had them in the top 25. It was kenpom vs the world. Iowa finished 13 in my ratings and 16th in the AP (pre-tourney) and even with a first-round tournament loss, their body of work over the season was clearly that of a top 25 team. That’s a W for kenpom.
On the flip side, I had North Carolina 40th in my ratings while all but one AP voter had them in the top 25. It’s a safe assumption that that voter would have had the Tar Heels higher than 40th. And after a February 16th loss to Pitt that nearly dropped Carolina out of the top 50, kenpom had a chance. But we know what happened next and kenpom took the L on this one.
Over the past seven seasons, there have been 49 cases where it was kenpom vs. world. Here are those cases. The W’s I have given to kenpom are denoted with stars.
Preseason rank Final Team kp AP range kp 15 Duke 1 2-9 3 15 Syracuse 11 15-NR(20) 53 15 Michigan 14 16-NR(26) 74 15 UNC 21 2-13 10 15 G’town 22 23-NR(63) 22** 15 Baylor 23 25-NR(64) 14** 15 Iowa St. 25 5-21 16 15 SD St. 30 11-NR(3) 31** 15 Nebraska 35 11-NR(14) 121** 15 Utah 42 19-NR(44) 8 16 Villanova 5 6-22 1** 16 Baylor 10 15-NR(22) 24 16 Maryland 23 1-7 22** 16 UConn 35 13-NR(15) 26 16 Cal 47 9-NR(1) 28 16 LSU 49 14-NR(18) 81** 17 Clemson 23 NR(65) 45 17 Baylor 25 NR(65) 13** 17 UConn 41 12-NR(5) 96** 17 URI 43 14-NR(26) 34 17 Texas 45 9-NR(23) 70** 18 Villanova 1 2-9 1 18 Duke 6 1-5 3 18 Mich St. 10 1-8 6 18 TCU 14 17-NR(47) 24 18 Miami FL 27 7-25 37** 18 Minnesota 36 9-NR(5) 117** 19 Syracuse 8 9-NR(1) 39 19 Gonzaga 9 1-7 2 19 Texas 16 23-NR(63) 25 19 Iowa St. 24 NR(65) 15** 19 Va. Tech 31 8-NR(5) 13 19 TCU 34 13-NR(19) 40** 19 Wash. 45 15-NR(35) 48** 19 UCLA 46 12-NR(18) 102** 19 LSU 48 14-NR(28) 19 20 Maryland 16 4-14 11 20 Michigan 21 24-NR(64) 16** 20 Oklahoma 24 NR(65) 36 20 Oregon 29 7-NR(3) 17 20 Memphis 51 7-22 59** 20 Utah St. 68 11-NR(4) 41 T 21 Ohio St. 8 9-24 31 21 Tex. Tech 12 18-NR(57) 7** 21 Texas 14 2-11 15** 21 Iowa 23 NR(63) 13** 21 Florida 25 NR(63) 59 21 Oregon 29 7-NR(2) 82** 21 UNC 40 8-NR(1) 16
Count ‘em up and there are 23 wins for kenpom and 25 wins for the AP with one tie. (The tie is a case where the final kenpom ranking was not near the range of the AP voters and both the AP and kenpom were equally off.)
As I alluded to, there is some subjectivity in this. In 2015 I had Duke as preseason #1 while the AP voters had them between 2 and 9. Duke was a one-seed and won the national title but finished #3 in my ratings. It’s hard to really call that an L, but I’m going to bend over backwards against my own ratings here because, frankly, I feel sorry for the AP voters after going through this exercise.
The underlying method is a bit subjective, too. Because of Luke DeCock I can’t include 2019 Texas Tech, who I had rated 25th while every voter except Luke had unranked. Texas Tech was the national champ in a not-very-different alternate universe. That would have been a W for kenpom. (Obviously, one could find some additional L’s as well but clearly that’s not in my best interest.)
Winning nearly 50% against the world seems like a totally acceptable outcome for the computer. So, hooray for the computer. But I think this investigation is more useful as evidence that AP voters are cheating off each other.
Just consider it like this: If 65 subject matter experts are independently telling you that you are wrong about something, you should normally reconsider your opinion. How often should the voting consensus be right? I don’t think it’s 100% because unexpected personnel changes can skew the season for some teams.
And in some of those cases a few voters were close enough to the kenpom ranking that they wouldn’t have thought it crazy to be wrong by a few spots. But 50% has to be the absolute bare minimum, a veritable coin flip. Maybe 70-80% would be a reasonable expectation. And given that is about 50%, the range of AP voters is smaller than it should be. Instead of getting 60-65 independent opinions, we get 60-65 people heavily influenced by 4 or 5 opinions.
The groupthink behavior produces a situation where the AP voting bloc is a bunch of robots while the robots are the free-thinkers, and they don’t even have brains! Head on over to the ranking summary at mratings.com and you’ll get much more diverse predictions for most teams relative to AP voters. (You have to be careful because about half of those computer ratings just roll over last season’s final ratings.)
As an epilogue, here are the eight teams outside the AP range this season. It’s an unusual year, where a whopping 4 teams in the kenpom top 10 are outliers. There have only been 8 total over the past 7 seasons. And history would say kenpom is in trouble. The AP has won 6 of those battles in the past (although one of those was the generously gifted Duke case). And for good reason. It’s not like Texas, Tennessee and Virginia can easily improve on their preseason kenpom.
Preseason rank Team kp AP range 22 Texas 2 8-22 22 Tennessee 4 6-25 22 Virginia 5 10-NR(8) 22 UNC 9 1-4 22 Texas Tech 17 18-NR(33) 22 Va. Tech 21 25-NR(61) 22 Creighton 22 3-15