Subscribe!
Follow me on twitter

The good stuff


At other venues...
  • ESPN.com ($)
  • Deadspin
  • Slate

  • Strategy
  • Whether to foul up 3 late
  • The value of 2-for-1’s

  • Philosophy
  • Brady Heslip’s non-slump
  • The magic of negative motivation
  • A treatise on plus-minus
  • The preseason AP poll is great
  • The magic of negative motivation
  • The lack of information in close-game performance
  • Why I don’t believe in clutchness*

  • Fun stuff
  • The missing 1-point games
  • Which two teams last lost longest ago?
  • How many first-round picks will Kentucky have?
  • Prepare for the Kobe invasion
  • Predicting John Henson's free throw percentage
  • Can Derrick Williams set the three-point accuracy record?
  • Play-by-play Theater: earliest disqualification
  • Monthly Archives

  • July 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • February 2012
  • January 2012
  • December 2011
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • April 2011
  • March 2011
  • February 2011
  • January 2011
  • December 2010
  • November 2010
  • October 2010
  • August 2010
  • July 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • February 2010
  • January 2010
  • December 2009
  • November 2009
  • October 2009
  • July 2009
  • February 2009
  • January 2009
  • December 2008
  • November 2008
  • October 2007
  • September 2007
  • July 2007
  • June 2007
  • May 2007
  • April 2007
  • March 2007
  • February 2007
  • January 2007
  • December 2006
  • November 2006
  • October 2006
  • September 2006
  • August 2006
  • July 2006
  • June 2006
  • May 2006
  • April 2006
  • March 2006
  • February 2006
  • January 2006
  • December 2005
  • November 2005
  • October 2005
  • September 2005
  • August 2005
  • July 2005
  • June 2005
  • May 2005
  • April 2005
  • March 2005
  • February 2005
  • January 2005
  • December 2004
  • November 2004
  • October 2004
  • September 2004
  • August 2004
  • July 2004
  • June 2004
  • May 2004
  • April 2004
  • March 2004
  • February 2004
  • January 2004
  • December 2003
  • November 2003

  • RSS feed

    More fun with the preseason AP poll

    by Ken Pomeroy on Sunday, October 21, 2012


    A while back, I expressed my support for the pre-season AP poll. A refresher: During the season I have no use for the polls because nobody really knows what they represent. But before games are played - and voters minimize the importance of scoring margin, game location, and schedule strength - everyone is trying to rank the best 25 teams in the country.

    As I prepare to release my own preseason ratings (in about a week), built strictly from a formula, I again wondered about the pre-season AP poll and how useful it is as a forecast tool. Not just for the team ranked at the top as was examined in the previous piece, but for teams ranked at each spot in the poll.

    So I went back through the 64-team era and recorded the eventual tournament seed for each team ranked in the preseason. That data is summarized below.

                             P r e a s e a s o n   R a n k
            1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
       1   19 13 11  9 12  3  7  2  3  2  6  3  1  0  0  1  3  1  2  0  1  2  0  2  0
       2    5  8  7  4  6 10  4  9  5  5  1  3  6  1  3  1  0  2  2  1  2  1  0  1  5
    S  3    3  3  4  5  5  3  3  1  2  4  3  3  2  4  2  2  1  1  1  4  0  4  3  3  1
    e >3    1  3  6  9  3 11 13 15 15 14 18 19 19 23 23 23 24 24 23 23 20 15 20 17 17
    e NT    0  1  0  1  2  1  1  1  3  3  2  5  5  7  6  3  8  5  6  7  7  2  6  8  4
    d %1   68 46 39 32 43 11 25  7 11  7 21 11  4  0  0  4 11  4  7  0  4  7  0  7  0
      %1-3 96 89 79 67 88 59 52 44 40 44 36 32 32 18 18 15 14 14 18 18 13 32 13 26 26
      %NT   0  4  0  4  8  4  4  4 12 12  7 18 18 25 21 11 29 18 21 25 30  9 26 35 17
    
    

    The top ranked team became a one-seed 19 times, or 68% of the time. It has received at least a three-seed 27 of 28 times (sorry, 2000 UConn), or 96% of the time, and has never missed the tournament (NT). Even after 28 seasons, the data is still rather noisy on a ranking-position basis. For instance, I wouldn’t take seriously the idea that the team ranked 17th in this season’s poll has a better chance at a one-seed than the team ranked 8th based on past performance.

    One can overcome some of the noise issues by grouping similarly-ranked teams. The following table takes all teams ranked within two spots of the ranking shown. So the column labeled “3” includes all teams ranked one through five. This offers a more coherent picture of what a pre-season ranking means.

                           5 - s p o t   a g g r e g a t e                                                
          1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
    %1         47 35 31 24 20 13 15 12 11  9  7  4  4  4  5  5  5  4  4  4  4    
    %1-3       84 76 69 62 57 48 43 39 37 32 27 23 19 16 16 16 16 19 19 20 22    
    %NT         3  4  4  4  6  7  8 11 13 16 18 19 21 21 20 21 24 21 23 25 24    
    
    

    The chances of being a one-seed get really slim once you get past the top 12 or 13, while the chances of missing the tournament altogether are very real for the teams in the bottom half of the poll.

    It’s striking that there isn’t much difference between the forecast of team ranked 15 or 25. So if you are feeling strongly that a team ranked 25 should be 15 or vice versa, you’re not actually talking about a significant difference. Even at the end of the season, with 30 games of data to rely on, it’s a challenge to distinguish between the 15th and 25th best teams. In the preseason, without that information, there are so many uncertainties involved that the difference is practically impossible to identify.

    This exercise also appears to give some quantification to the meaning of the regular season. In most seasons, there isn’t much suspense to the regular season for the top team or two. Exhibit A – the Kentucky/North Carolina game last season. There was already strong reason to believe that both would be one-seeds and that the result of the contest would have no impact on their future. (For those that fear the 96-team era - and I don’t look forward to it - that game was also an illustration that a game with no “meaning” in a championship sense can still generate a lot of interest anyway.)

    However, a team ranked as high as eighth has less than a 50% chance of being a top three-seed, necessary for a reasonable chance at a title. Teams ranked in the mid-teens have a one-in-five chance of completely missing the tournament. The uncertainty in the non-elite-team’s future provides the incentive to take advantage of each practice and game to get better. I’m not sure if an expanded tournament field would reduce that incentive, but I kind of like the idea that a team universally thought to be really good in November could find itself watching the tourney in March if things don’t go right.

    Finally, it’s worth discussing whether the pre-season poll is getting better. I’ve attacked this a few different ways and haven’t found a trend. (But if you’d like to take a crack, I’ve included the raw data at the end of this post.) Frankly, I would expect voters to be improving considering the increased information available in the internet age. There are two reasons I can think of that are preventing this.

    The simple explanation is that it’s more difficult to pick the top teams than it was nearly thirty years ago. Picking the top 20 of 250 teams is marginally easier than picking the top 20 of 350 teams.

    It’s also possible that the increased connectivity in the internet age is having a negative effect. The pre-season poll should represent the wisdom of the crowd, but that’s only true if everyone’s opinion is independent of the other voters. Independence was easier to achieve when people had to battle 1200-baud modems to connect to CompuServe and QuantumLink than it is now when everyone sees Andy Katz’s top 25 the day after the title game.

    Of course, not everyone is cheating off the answer key. That’s why I’ll always have Scott Mansch’s back, no matter how wacky his ballot is.

    Tournament seed for each top 25 team by year
                                  P r e a s e a s o n   R a n k
            1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
      2012  1  1  2  9  1  2  5  7  4 XX  8  3  2 10  4 XX XX  4  9 XX  6  3  7 12  2
      2011  1 10  5  1  1  9  1  2  2  3  4 11  9  3 11 XX  8  7 12  6 XX  7  9  3  2
      2010  1  5  8  1  2 XX  4  2  1  6  5 XX  8 11 XX  2 XX XX  9  3 XX 10 XX  7 11
      2009  1  1  1  6  1  2  7  2 XX  4  5  2  2  9  6  6 XX 10 XX XX  4 XX  3  3 12
      2008  1  1  1  1  2  3  2  5  8  4  6  9  2  7  2  9 10  6  9 11 XX  4  3 XX 11
      2007  1  1  1  3 XX  2  1  2  2  8 XX  6  3  2  7  8 XX XX 10 XX  4  8 10  7  5
    T 2006  1  2  1  6  1  6 XX  3  8  8  4  1 XX  6 10  5  4 XX  2  3  8  5  6 XX XX
    o 2005  3  2  5  1  1  4  2  2  2  3  1  9  5  4 XX  8  9  5 10 XX  6  1  4 XX  3
    u 2004  2  1  7  9 XX  4  5  5  6  2  1  3  5 XX  6 10  1  4  1  4 XX  3 XX  3  2
    r 2003  1  2  1  1  2  3  2 10  7  3  8  5  6 XX  5  *  1  3  6 13  7  9  8 XX 13
    n 2002  1  1  4  4  8  5  1 12 XX XX XX XX XX  8 XX 10 11  7 XX  4 XX  5  6  2  2
    a 2001  2  1  1  1  3  2  4  1  8 XX  3  2 XX XX  7  6  4  5  6  7 XX  4  6  5  2
    m 2000  5  2  1  7  3  8  2  5  1  1  8  6  1  5  8  4  4  2  4  9  5  3  6 10  6
    e 1999  1  1  2  3  1  2  6  6  4  2  3  5  9  7  3 10 XX  4  4  9  8  6 12 XX XX
    n 1998  1  1  1  1  6  6  3  2  2  6  4  2 XX  3 XX  3  7  8 XX 10  8 XX  4  7 XX
    t 1997  3  1  1  3  2  2  4  1 XX  2  6 XX XX XX 11 10  3  6  4  4  5  1  8 XX  7
      1996  1  2  3  4  2  1  1  6  5  4  2  6  5 XX  7 12  7  9 XX  6  2  9  6  1 12
    Y 1995  2  2  2  1  5  1  3 XX  9 10  1  7  7  4  6  9 XX  5  2  3  4  3 XX  1 11
    e 1994  1  3  1  2  3  5  3  4  4  6  4  5  5 XX  9  7  8  2  8  4  1  2 XX 10 XX
    a 1993  1  2  3  1  1  2  1 10  3  2  4 XX  4  4 XX XX 11 XX  8 XX  2  *  3  9 10
    r 1992  1  2  3  2  3  7  1  4  4  7  1  1  2 XX  9  6  5  5  4  6  9  9  7 XX  8
      1991  1  1  2  5  1  2  4  2  8  1  4  6  2  6 XX  8 11  7 10 XX 11  5 XX 11  4
      1990  1  5  2  3  3  2  8  5  4  3  3  4  7  8 11  1 10 XX XX  6 XX  4 XX XX  6
      1989  2  1  3  4  1  2  4  2  1  4  1 XX  6  3  7  4 XX  5 XX  3          
      1988  3  1  2  2  2  4  6  6  3  7  5  1  5  6  2  8  1  5  1  5          
      1987  1 XX  1  3  1  7  6  5  8  2  8  8  2  3  2  3 11  1 10 XX          
      1986  2  3  2  2  1  1  4  4  2  8  1  3  4 11  3  6  6  4  5  7          
      1985  1  3 10 XX  1  3  1  2  5  5  4  7  3  4  9  9 XX 12  3  2          
    
    

    *Teams ranked here were ineligible for the tournament