Subscribe!
Follow me on twitter

The good stuff


At other venues...
  • ESPN.com ($)
  • Deadspin
  • Slate

  • Strategy
  • Whether to foul up 3 late
  • The value of 2-for-1’s

  • Philosophy
  • Brady Heslip’s non-slump
  • The magic of negative motivation
  • A treatise on plus-minus
  • The preseason AP poll is great
  • The magic of negative motivation
  • The lack of information in close-game performance
  • Why I don’t believe in clutchness*

  • Fun stuff
  • The missing 1-point games
  • Which two teams last lost longest ago?
  • How many first-round picks will Kentucky have?
  • Prepare for the Kobe invasion
  • Predicting John Henson's free throw percentage
  • Can Derrick Williams set the three-point accuracy record?
  • Play-by-play Theater: earliest disqualification
  • Monthly Archives

  • July 2014
  • May 2014
  • April 2014
  • March 2014
  • February 2014
  • January 2014
  • December 2013
  • November 2013
  • October 2013
  • September 2013
  • August 2013
  • July 2013
  • June 2013
  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • November 2012
  • October 2012
  • September 2012
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • February 2012
  • January 2012
  • December 2011
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • April 2011
  • March 2011
  • February 2011
  • January 2011
  • December 2010
  • November 2010
  • October 2010
  • August 2010
  • July 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • February 2010
  • January 2010
  • December 2009
  • November 2009
  • October 2009
  • July 2009
  • February 2009
  • January 2009
  • December 2008
  • November 2008
  • October 2007
  • September 2007
  • July 2007
  • June 2007
  • May 2007
  • April 2007
  • March 2007
  • February 2007
  • January 2007
  • December 2006
  • November 2006
  • October 2006
  • September 2006
  • August 2006
  • July 2006
  • June 2006
  • May 2006
  • April 2006
  • March 2006
  • February 2006
  • January 2006
  • December 2005
  • November 2005
  • October 2005
  • September 2005
  • August 2005
  • July 2005
  • June 2005
  • May 2005
  • April 2005
  • March 2005
  • February 2005
  • January 2005
  • December 2004
  • November 2004
  • October 2004
  • September 2004
  • August 2004
  • July 2004
  • June 2004
  • May 2004
  • April 2004
  • March 2004
  • February 2004
  • January 2004
  • December 2003
  • November 2003

  • RSS feed

    On Wisconsin: The FAQ

    by Ken Pomeroy on Monday, January 9, 2012


    As you may have noticed, Wisconsin inhabits the #2 spot in my ratings this morning despite having lost five of its first 17 games, including a home game to Iowa and a lopsided loss to Michigan in the past week alone.

    This is an issue that is not going to go away this season. Even in a worst-case scenario for the Badgers, they are going to be highly ranked on this web site the rest of the season. This bothers many people and in order to manage the increasing number of inquiries into the matter, I have established what the tech geeks call a “FAQ document” for handy reference. What follows is that document. Please share with all parties you deem relevant to the matter.

    Q: Why do you think Wisconsin is #2?

    A: I actually don’t think this. Please stop assuming that I, Ken Pomeroy, personally believe every team is properly ranked. No system can do this. Even if you somehow had the time to rank all 345 teams, by the time you were done with this exercise, you would find some things that you didn’t agree with in the rankings you had just made. For fun, spend some time just try ranking the top 50. Through some miracle, if you’re happy with the position of each team when you’re done, just let the results of games come in for the next 2-3 days and then see if you’re still happy.

    The bottom line is that is impossible to rank 345 teams that will make any particular individual happy. There are always going to be some outliers in a system. Wisconsin is perhaps the biggest outlier ever in my system. And it’s only going to get worse, because they are going to lose more games and not drop very far because most of the teams they play the rest of the season are very good. But to get back to the question, I don’t believe they’re the second-best team in the country. Something like #20 sounds about right.

    Q: Who has Wisconsin beaten to deserve to be #2?

    A: As a reminder, the ratings are not meant to have any relation to the polls. (The first paragraph on the ratings explanation page is worth reading if you haven’t seen it before.) I don’t care who “deserves” to be #2. The #2 team in my system should be the second-best in the country. It is theoretically possible to be the second-best team in the nation without having beaten anyone of prominence.

    As an example, in 2010, Duke went 1-3 against teams ranked better than 20th in my system during the regular season. Then in the tournament they went 5-0 against such teams! It was crazy, but not completely surprising. The point is, for predictive purposes, simply looking at who a team has beaten or lost to is short-sighted. Obviously this methodology has not worked well in predicting the Badgers’ recent performance. Duly noted. In bulk, though, simply rating teams by the quality of their wins and losses is not going to make for a good predictive system.

    Q: The fact that Wisconsin is #2 invalidates all of your work. (More of a comment, really.)

    I disagree. Let’s say you are very good at performing some task. For the next 345 times you perform this task, I will judge you based on your worst effort. I don’t think you would feel like this is a fair way to measure your ability to perform the task. Because it isn’t.

    If you are going to trash the entire system based on the biggest outlier over the last six years, I suspect you had no desire to use the system in the first place. Either that, or your team is ranked lower than you think it should be.

    Q. When are you going to fix this?

    First off, I’m not sure it can be fixed. I mean, sure I can fix it so Wisconsin is ranked more reasonably. I could add some code to my algorithm like this…

    if Wisconsin:
    	do something to fix Wisconsin
    else:
    	do normal calculations
    

    But that really isn’t a good way to solve the problem. Needless to say, I’d like to fix it.  However, fixing Wisconsin can result in messing things up for others. Plus, there are a lot of other ratings systems already doing cool things, and I’m not in the game of copying others’ work, so that doesn’t leave a lot on the table for me to pursue. That doesn’t mean improvements aren’t possible, but regardless of what I come up with, I can promise you there will still be teams that you feel are mis-rated.

    For instance, one of the most respected ratings systems in the nerd world is the LRMC ratings. They had Wisconsin fifth before yesterday’s games. Last year, heading into the tournament, they had Belmont fourth, and their system outperforms mine! The Sagarin predictor, also deservedly respected, had Wisconsin second heading into yesterday’s games.

    If you would simply remove Wisconsin from the planet, the ratings on my page would look fine to me. I guess you could ignore the entire system based on the worst outlier, but I don’t think that makes a lot of sense. There’s still some insight gained from the teams that differ from the polls. (See: Saint Louis, New Mexico, UConn, Mississippi State, among others.)

    Q: Your work is flawed. (Not really a Q, either, I guess, but I get this all the time.)

    A: Ugh, I hate it when people say this. Of course it’s flawed. The thing is, your knowledge is flawed, too. If you are ignoring potentially useful tools because of a single issue, then your judgment is flawed as well. And I’m guessing you’ve never tracked the quality of your knowledge so you don’t even know how flawed it is. If you’re like most people, you think you’re knowledge is great because you remember the predictions you made that worked out and you forget about the ones that didn’t. It’s human nature.

    The difference between your flawed knowledge and my flawed system is that I am tracking the results of my system so you can identify all of the flaws for yourself. Ten years from now, you’ll be able to look at my site and see that Wisconsin was like 21-13 at the end of the 2012 season and ranked #5. Whereas the time you made some crazy prediction that didn’t pan out is quickly forgotten.

    I would say there’s still enough value in the work here to provide a useful reality check on your own knowledge. Used together, your flawed knowledge and my flawed tools can be more powerful than used separately.

    Q: Anything else we need to know?

    A: Well, since you asked, you might as well know that Ohio State will almost surely be the #1 team the rest of the season. Also, they will probably finish with more losses than Kentucky or UNC or Syracuse, because they will play a tougher collection of teams the rest of the season. Just to reiterate, if you are looking for a ranking of teams by wins and losses, a la the AP poll or the RPI, then you’ll be more fulfilled consulting the AP poll or RPI for your ranking needs as the season continues. (Sagarin and Massey actually have better systems for this purpose, though.) However, I believe you will be less enlightened by ignoring the work of predictive systems.

    Note also, that this does not mean I think Ohio State is the best team in the country, nor do I expect you to think that! (Although right now, I actually do believe the Buckeyes are the best, but I may change my mind as upcoming events warrant.) I didn’t think Duke was the best team two years ago, even after the tournament. But by the time the season ended, it was hard to argue they weren’t one of the best teams in the country, and were significantly underrated by the folks who take a resume approach to evaluating teams at the expense of all other information. And basically, I don’t think the Wisconsin situation invalidates the predictive methods used on this site, except in cases involving Wisconsin.