I have previously acknowledged my appreciation for the preseason AP poll. Once the season starts, the poll becomes a clerical exercise that isn’t useful in the analytical realm, but before the games are played, humans and computers alike are trying to do the same thing: figure out which teams are the best. 

Get 60-65 experts together and you should have something more powerful than what a computer rating can provide. Wisdom of crowds and all. 

For that reason, whenever I have the typical 6-10 outliers in the top 25 to start the season, I usually assume the AP poll has the better ranking. Wisdom of crowds will generally beat my simple algorithm. 

But I never really tested how the preseason ratings do when it’s kenpom vs. the world, and instead of continuing to kiss the AP’s butt in the preseason, I wanted to know how good kenpom is when it disagrees with the computer. So I designed a study like so…

Step 1: Identify all the cases since the 2015 season where my preseason ranking for a team was outside the range of all AP voters. Essentially, find the cases where every single voter bet against my ratings. (The data on individual votes on collegepolltracker.com only goes back to the 2015 season.) To qualify, a team needed to be ranked in either the kenpom top 25 or the AP top 25. 

This step gets subjective when the kenpom ranking is below the lowest AP voter and there are some “not ranked” votes. Reasonable judgment was used.

Step 2: Use the team’s final kenpom ranking to determine if the AP voters were correct. If the team’s final kenpom ranking is closer to the AP range than the kenpom ranking, the AP voters win. Otherwise they lose. (This also gets subjective for a few cases. I will discuss later.)

Now, the AP voters are not trying to predict the final kenpom ranking, but there’s no post-tournament AP ranking and in a general sense, the mission of the voters and the mission of the kenpom ratings are similar before the season. 

As an example, last season Iowa was 23rd in the preseason kenpom while zero AP voters had them in the top 25. It was kenpom vs the world. Iowa finished 13 in my ratings and 16th in the AP (pre-tourney) and even with a first-round tournament loss, their body of work over the season was clearly that of a top 25 team. That’s a W for kenpom.

On the flip side, I had North Carolina 40th in my ratings while all but one AP voter had them in the top 25. It’s a safe assumption that that voter would have had the Tar Heels higher than 40th. And after a February 16th loss to Pitt that nearly dropped Carolina out of the top 50, kenpom had a chance. But we know what happened next and kenpom took the L on this one.

Over the past seven seasons, there have been 49 cases where it was kenpom vs. world. Here are those cases. The W’s I have given to kenpom are denoted with stars. 

             Preseason rank    Final
Team          kp   AP range     kp
15 Duke        1     2-9         3
15 Syracuse   11    15-NR(20)   53
15 Michigan   14    16-NR(26)   74
15 UNC        21     2-13       10
15 G’town     22    23-NR(63)   22**
15 Baylor     23    25-NR(64)   14**
15 Iowa St.   25     5-21       16
15 SD St.     30    11-NR(3)    31**
15 Nebraska   35    11-NR(14)  121**
15 Utah       42    19-NR(44)    8
16 Villanova   5     6-22        1**
16 Baylor     10    15-NR(22)   24
16 Maryland   23     1-7        22**
16 UConn      35    13-NR(15)   26
16 Cal        47     9-NR(1)    28 
16 LSU        49    14-NR(18)   81** 
17 Clemson    23     NR(65)     45
17 Baylor     25     NR(65)     13**
17 UConn      41     12-NR(5)   96**
17 URI        43     14-NR(26)  34
17 Texas      45      9-NR(23)  70**
18 Villanova   1      2-9        1
18 Duke        6      1-5        3
18 Mich St.   10      1-8        6
18 TCU        14     17-NR(47)  24
18 Miami FL   27      7-25      37**
18 Minnesota  36      9-NR(5)  117** 
19 Syracuse    8      9-NR(1)   39
19 Gonzaga     9      1-7        2
19 Texas      16     23-NR(63)  25
19 Iowa St.   24     NR(65)     15**
19 Va. Tech   31      8-NR(5)   13
19 TCU        34     13-NR(19)  40**
19 Wash.      45     15-NR(35)  48**
19 UCLA       46     12-NR(18) 102**
19 LSU        48     14-NR(28)  19
20 Maryland   16      4-14      11
20 Michigan   21     24-NR(64)  16**
20 Oklahoma   24      NR(65)    36
20 Oregon     29      7-NR(3)   17
20 Memphis    51      7-22      59**
20 Utah St.   68     11-NR(4)   41 T
21 Ohio St.    8      9-24      31
21 Tex. Tech  12     18-NR(57)   7**
21 Texas      14      2-11      15**
21 Iowa       23      NR(63)    13**
21 Florida    25      NR(63)    59
21 Oregon     29      7-NR(2)   82**
21 UNC        40      8-NR(1)   16

Count ‘em up and there are 23 wins for kenpom and 25 wins for the AP with one tie. (The tie is a case where the final kenpom ranking was not near the range of the AP voters and both the AP and kenpom were equally off.)

As I alluded to, there is some subjectivity in this. In 2015 I had Duke as preseason #1 while the AP voters had them between 2 and 9. Duke was a one-seed and won the national title but finished #3 in my ratings. It’s hard to really call that an L, but I’m going to bend over backwards against my own ratings here because, frankly, I feel sorry for the AP voters after going through this exercise. 

The underlying method is a bit subjective, too. Because of Luke DeCock I can’t include 2019 Texas Tech, who I had rated 25th while every voter except Luke had unranked. Texas Tech was the national champ in a not-very-different alternate universe. That would have been a W for kenpom. (Obviously, one could find some additional L’s as well but clearly that’s not in my best interest.)

Winning nearly 50% against the world seems like a totally acceptable outcome for the computer. So, hooray for the computer. But I think this investigation is more useful as evidence that AP voters are cheating off each other. 

Just consider it like this: If 65 subject matter experts are independently telling you that you are wrong about something, you should normally reconsider your opinion. How often should the voting consensus be right? I don’t think it’s 100% because unexpected personnel changes can skew the season for some teams.

And in some of those cases a few voters were close enough to the kenpom ranking that they wouldn’t have thought it crazy to be wrong by a few spots. But 50% has to be the absolute bare minimum, a veritable coin flip. Maybe 70-80% would be a reasonable expectation. And given that is about 50%, the range of AP voters is smaller than it should be. Instead of getting 60-65 independent opinions, we get 60-65 people heavily influenced by 4 or 5 opinions.

The groupthink behavior produces a situation where the AP voting bloc is a bunch of robots while the robots are the free-thinkers, and they don’t even have brains! Head on over to the ranking summary at mratings.com and you’ll get much more diverse predictions for most teams relative to AP voters. (You have to be careful because about half of those computer ratings just roll over last season’s final ratings.)

As an epilogue, here are the eight teams outside the AP range this season. It’s an unusual year, where a whopping 4 teams in the kenpom top 10 are outliers. There have only been 8 total over the past 7 seasons. And history would say kenpom is in trouble. The AP has won 6 of those battles in the past (although one of those was the generously gifted Duke case). And for good reason. It’s not like Texas, Tennessee and Virginia can easily improve on their preseason kenpom.

             Preseason rank
Team          kp   AP range
22 Texas       2      8-22
22 Tennessee   4      6-25
22 Virginia    5     10-NR(8)
22 UNC         9      1-4
22 Texas Tech 17     18-NR(33)
22 Va. Tech   21     25-NR(61)
22 Creighton  22      3-15