As you may have noticed, Wisconsin inhabits the #2 spot in my ratings this morning despite having lost five of its first 17 games, including a home game to Iowa and a lopsided loss to Michigan in the past week alone.
This is an issue that is not going to go away this season. Even in a worst-case scenario for the Badgers, they are going to be highly ranked on this web site the rest of the season. This bothers many people and in order to manage the increasing number of inquiries into the matter, I have established what the tech geeks call a “FAQ document” for handy reference. What follows is that document. Please share with all parties you deem relevant to the matter.
Q: Why do you think Wisconsin is #2?
A: I actually don’t think this. Please stop assuming that I, Ken Pomeroy, personally believe every team is properly ranked. No system can do this. Even if you somehow had the time to rank all 345 teams, by the time you were done with this exercise, you would find some things that you didn’t agree with in the rankings you had just made. For fun, spend some time just try ranking the top 50. Through some miracle, if you’re happy with the position of each team when you’re done, just let the results of games come in for the next 2-3 days and then see if you’re still happy.
The bottom line is that is impossible to rank 345 teams that will make any particular individual happy. There are always going to be some outliers in a system. Wisconsin is perhaps the biggest outlier ever in my system. And it’s only going to get worse, because they are going to lose more games and not drop very far because most of the teams they play the rest of the season are very good. But to get back to the question, I don’t believe they’re the second-best team in the country. Something like #20 sounds about right.
Q: Who has Wisconsin beaten to deserve to be #2?
A: As a reminder, the ratings are not meant to have any relation to the polls. (The first paragraph on the ratings explanation page is worth reading if you haven’t seen it before.) I don’t care who “deserves” to be #2. The #2 team in my system should be the second-best in the country. It is theoretically possible to be the second-best team in the nation without having beaten anyone of prominence.
As an example, in 2010, Duke went 1-3 against teams ranked better than 20th in my system during the regular season. Then in the tournament they went 5-0 against such teams! It was crazy, but not completely surprising. The point is, for predictive purposes, simply looking at who a team has beaten or lost to is short-sighted. Obviously this methodology has not worked well in predicting the Badgers’ recent performance. Duly noted. In bulk, though, simply rating teams by the quality of their wins and losses is not going to make for a good predictive system.
Q: The fact that Wisconsin is #2 invalidates all of your work. (More of a comment, really.)
I disagree. Let’s say you are very good at performing some task. For the next 345 times you perform this task, I will judge you based on your worst effort. I don’t think you would feel like this is a fair way to measure your ability to perform the task. Because it isn’t.
If you are going to trash the entire system based on the biggest outlier over the last six years, I suspect you had no desire to use the system in the first place. Either that, or your team is ranked lower than you think it should be.
Q. When are you going to fix this?
First off, I’m not sure it can be fixed. I mean, sure I can fix it so Wisconsin is ranked more reasonably. I could add some code to my algorithm like this…
if Wisconsin: do something to fix Wisconsin else: do normal calculations
But that really isn’t a good way to solve the problem. Needless to say, I’d like to fix it. However, fixing Wisconsin can result in messing things up for others. Plus, there are a lot of other ratings systems already doing cool things, and I’m not in the game of copying others’ work, so that doesn’t leave a lot on the table for me to pursue. That doesn’t mean improvements aren’t possible, but regardless of what I come up with, I can promise you there will still be teams that you feel are mis-rated.
For instance, one of the most respected ratings systems in the nerd world is the LRMC ratings. They had Wisconsin fifth before yesterday’s games. Last year, heading into the tournament, they had Belmont fourth, and their system outperforms mine! The Sagarin predictor, also deservedly respected, had Wisconsin second heading into yesterday’s games.
If you would simply remove Wisconsin from the planet, the ratings on my page would look fine to me. I guess you could ignore the entire system based on the worst outlier, but I don’t think that makes a lot of sense. There’s still some insight gained from the teams that differ from the polls. (See: Saint Louis, New Mexico, UConn, Mississippi State, among others.)
Q: Your work is flawed. (Not really a Q, either, I guess, but I get this all the time.)
A: Ugh, I hate it when people say this. Of course it’s flawed. The thing is, your knowledge is flawed, too. If you are ignoring potentially useful tools because of a single issue, then your judgment is flawed as well. And I’m guessing you’ve never tracked the quality of your knowledge so you don’t even know how flawed it is. If you’re like most people, you think you’re knowledge is great because you remember the predictions you made that worked out and you forget about the ones that didn’t. It’s human nature.
The difference between your flawed knowledge and my flawed system is that I am tracking the results of my system so you can identify all of the flaws for yourself. Ten years from now, you’ll be able to look at my site and see that Wisconsin was like 21-13 at the end of the 2012 season and ranked #5. Whereas the time you made some crazy prediction that didn’t pan out is quickly forgotten.
I would say there’s still enough value in the work here to provide a useful reality check on your own knowledge. Used together, your flawed knowledge and my flawed tools can be more powerful than used separately.
Q: Anything else we need to know?
A: Well, since you asked, you might as well know that Ohio State will almost surely be the #1 team the rest of the season. Also, they will probably finish with more losses than Kentucky or UNC or Syracuse, because they will play a tougher collection of teams the rest of the season. Just to reiterate, if you are looking for a ranking of teams by wins and losses, a la the AP poll or the RPI, then you’ll be more fulfilled consulting the AP poll or RPI for your ranking needs as the season continues. (Sagarin and Massey actually have better systems for this purpose, though.) However, I believe you will be less enlightened by ignoring the work of predictive systems.
Note also, that this does not mean I think Ohio State is the best team in the country, nor do I expect you to think that! (Although right now, I actually do believe the Buckeyes are the best, but I may change my mind as upcoming events warrant.) I didn’t think Duke was the best team two years ago, even after the tournament. But by the time the season ended, it was hard to argue they weren’t one of the best teams in the country, and were significantly underrated by the folks who take a resume approach to evaluating teams at the expense of all other information. And basically, I don’t think the Wisconsin situation invalidates the predictive methods used on this site, except in cases involving Wisconsin.