A treatise on plus/minus
01.10.11
As people who have followed my twitter feed have figured out, I’m more than a little uncomfortable with the use of plus/minus data in college basketball analysis.
I once produced something called an HD Box score which contained +/- data. It was a thrill to make the breakthrough to compute such data because after all, plus-minus captures everything. All of those hustle plays that lead to points are rewarded. The guy who’s able to force his man into repeatedly missing shots gets rewarded for this defense. It’s perfect. And I basically said some dumb things along those lines back then.
Only, my experience since then has led me to believe that it’s far from perfect. This well-intentioned analysis (which is accompanied with plenty of caution) indicates that Kyrie Irving was a huge drain on Duke at the beginning of the season. Would you go to Coach K and tell him with a straight face to bench Kyrie Irving (if he didn’t have that whole toe problem already)? With Irving’s offensive stats, he would literally have to be shooting for the other team to be a net drain on the Blue Devils. That’s just my conjecture, though. Legitimate conjecture that makes basketball sense, I would say, but conjecture nonetheless.
I decided to do a test to determine the value of plus-minus-related stats. I created a player who has absolutely no impact on the game. For both his team and his opponent, there’s a 3% chance of scoring one point on a possession, a 30% chance of scoring two, and a 15% chance of scoring three. These probabilities are unchanged whether the player is in the game or on the bench.
I am assuming that every game is 70 possessions long and that my player will be on the floor for 35 possessions and on the bench for 35 possessions. With minimal programming experience you can code up such a scenario, run it, and track Mr. Irrelevant’s plus-minus. I played 20 such games and tallied said player’s plus-minus for those possessions for which he was on and off the court. For a limited time you can go to this page and refresh it to do as many such tests as you want.
My first experiment resulted in the following output…
On Off MOV 1. 5 3 8 2. 6 -3 3 3. -1 7 6 4. -4 10 6 5. 11 -11 0 6. -1 4 3 7. 13 7 20 8. -8 7 -1 9. -11 7 -4 10. -9 23 14 11. -14 1 -13 12. 6 -4 2 13. -12 -7 -19 14. -13 -6 -19 15. 4 -7 -3 16. -13 -12 -25 17. 9 6 15 18. -6 -2 -8 19. -1 9 8 20. -3 -17 -20 ----------------- Tot -42 15 -27
If plus-minus were truly just, there would be nothing but zeroes on the list. Sure, there’s going to be some noise in any statistic, but this illustrates to me that the amount of noise in plus-minus is unacceptable. Heck, the amount of noise in final score is pretty amazing. My team is playing a schedule against teams it’s evenly-matched with. Yet in game 16 we were outscored by 25 points! You can imagine the game-stories written about that one. How my team was out-coached, or how the opponent is just a bad matchup for my team, or how my team quit, showed no heart, etc.
But I can assure you, none of that occurred! It was just a bunch of bits and bytes that produced the outcome. Think about that the next time somebody feeds you an explanation of why a lopsided result occurred in what was expected to be an even game. (For reasons too boring to explain here, this methodology overstates the variance in final margin by about 20%. Still, you get the point.)
In this case, my team was -42 with my player on the court and +15 with him off. Do the math and his value works out to -5.7 points per 40 minutes. The conclusion is that I need to bench this guy. But since this is an experiment, I know he actually has no impact on my team whether he’s in the game or not. It’s simply randomness that is creating these numbers. And keep in mind this is a more controlled environment than the real world. The quality of my player’s teammates remains constant. The quality of the players on the other team is constant. In the real world, there is variation in both of these things, which only adds more noise. In addition, the real world would contain games that are big-time mismatches, with plenty of garbage time. That would clutter things up as well.
Just for consistency, I decided to run my 20-game experiment 50 times. The results should give a more accurate assessment of the error associated with plus-minus. It wasn’t difficult to get a crazier result than the one described above. The third trial in this new run gave my player a -43 on the floor and a +52 off the floor. A few trials later, my guy was +48 on and -73 off. My dud had become a stud. I hope he doesn’t get hurt! For the 50 sets of 20 games, the average error in measuring my worthless player was 4.8 points per 40 minutes. There are few players in the country who, if they got hurt, would move the Vegas line for their team by more than a couple points.
One can imagine if we had an eight-man rotation of equal players that after 20 games we would see enough variation among players to convince some people that the best player was much better than the worst.
So where does this leave us? I will state my beliefs here:
1) In-game plus minus is useless. As you can see from the first trial above, there were games when my player looked very good and games when he looked bad. Sure there are things that a player does to help or hurt his team that are not reflected in the box score, but plus-minus for a single game is often not reflective of this. If a player has a significant effect on his team by doing things not accounted for in the box score, I suggest you actually see those things occur before giving the player credit or blame.
2) Multi-game plus-minus isn’t much better. I suppose there are cases where there’s enough of a difference over a few games to draw a meaningful conclusion about a player. But I think that pertains to the kinds of players for whom you already know the answer (like Greg Oden, although I don’t necessarily stand by everything I wrote in that piece).
3) Season-long adjusted plus-minus might have limited use. I haven’t talked about adjusted plus-minus, but in the simplest terms possible, it accounts for who a player is playing with and against. It’s useful in the NBA, but the NBA has many more games, more minutes per game, and its star players see a fair amount of time on the bench during non-garbage time. Even so, there are always some curious results with adjusted plus-minus and the most accurate version uses two years worth of data.
4) Respect randomness. This should really be the objective analyst’s credo because it’s where we have an advantage over people that fall in love with their own eyes. It’s true plus-minus captures everything that’s happening, but that includes a whole lot of random things that lead to a hoop or a stop. Things that have nothing to do with the ability of the player you want to analyze. In basketball analysis, we should be filtering out randomness, not embracing it.
In summary, plus-minus, while neat to look at, is a poor tool in college basketball analysis.
The kPOY: Up for grabs?
01.04.11
Let’s own up to it - this year’s player of the year race is still open to be claimed and to me, that feels somewhat unusual for January. Most of the players in the chase for the inaugural kPOY had rather sluggish efforts coming out of the holiday break. This opened the door for something like E’Twaun Moore’s 31-point effort against Northwestern, which vaulted him from obscurity to a legitimate kPOY candidate. However, based on what we saw against Iowa last night, I’d still be shocked if Jared Sullinger doesn’t stay at the top of the list from here on out.
(Standings through Sunday’s games.)
1. Jared Sullinger, Ohio St. (Rating of .496, Last week: 1st) Before posting 24 and 12 against the Hawkeyes, it was over two weeks since Sullinger scored 20. Though Melsahn Basabe and Jarryd Cole caused him a few problems, Sullinger’s numbers, big or not-as-big, typically come with a minimum of missed shots or turnovers.
2. Terrence Jones, Kentucky (.487, LW: 2nd) Jones was not the headliner in UK’s romp over Louisville. However, the entirety of Jones’ season-long quality in a number of different categories, combined with the Wildcats’ strong play, keeps him ranked highly for another week.
3. E’Twaun Moore, Purdue (.466, LW: 9th) Purdue’s tandem climbs again as the Boilermakers continue to string together dominant games. Moore’s aggregate 52-point/one-turnover performance against Michigan and Northwestern allows him to jump over a bunch of players.
4. JaJuan Johnson, Purdue (.459, LW: 7th) Johnson was less spectacular than his teammate, but 40 points and three turnovers is quite nice as well, and contributed to Purdue’s 1.25 points per possession of offense through two Big Ten games.
5. Jon Leuer, Wisconsin (.448, LW: 4th) Leuer’s production is coming back down to earth after going 7-18 on 2’s and 4-13 on 3’s against Minnesota and Illinois.
6. Jimmer Fredette, Brigham Young (.444, LW: 3rd) Fredette played sparingly in a rout of NAIA Fresno Pacific. He’s got a chance to make up ground with a good effort in tonight’s showdown at UNLV.
7. Nolan Smith, Duke (.439, LW: 10th) Smith had an outstanding game in a home win over Miami FL. He got the CG and posted 28 points on just 15 FGA’s and 6 FTA’s.
8. Kemba Walker, Connecticut (.435, LW: 6th) Like Leuer, Walker set an unsustainable pace with his early play, and gravity is starting to take its toll. Still, his team is not in the top 30 and therefore his presence on this list is a testament to how important he is to the Huskies. He gets another national showcase on Saturday at Texas.
9. Derrick Williams, Arizona (.411, LW: 5th) Losing to Oregon State in a game where he missed seven of ten free throws was enough to send his stock plummeting.
10. Jordan Hamilton, Texas (.374, LW: 8th) Hamilton scored 24 in a win against Coppin State, but needed a career-high 14 three-point attempts to get there.
Pre-conference preview blowout: part 2
(Ed. note: simulations run for the Pre-Conference Preview Blowout do not include Monday’s action. Nor does any of the text.)
This is part 2 of the Pre-Conference Preview Blowout, where I preview conference races by simulating them 10,000 times using my ratings. In this edition we find that Duquesne is sneaky good and you should pay attention to them. I’ll check back on this as the season winds down to see where I screwed up.
16. MEAC
Hampton 6015 Morgan St. 2811 Delaware St. 781 Coppin St. 155 Bethune Cookman 126 North Carolina A&T 95 South Carolina St. 13 Howard 2 Norfolk St. 4
There’s a new sheriff in the MEAC and its name is Hampton. Or at least it could be if this analysis is worth anything.
15. A-10
Temple 5495 Richmond 2714 Duquesne 1102 Xavier 403 Rhode Island 114 Massachusetts 80 Dayton 76 St. Louis 7 La Salle 5 St. Bonaventure 2 Charlotte 0
Let’s keep an eye on the Dukes over the next two weeks. Ron Everhart’s team doesn’t have the quality wins that would get his team some attention from the “Who have they beaten?” patrol, but his team has turned in nothing but quality performances this season. Maybe I am overrating them. But maybe I am not. I never would have given a second thought about them winning the A-10 without doing this analysis, but it doesn’t seem crazy. At the very least, expect them to screw up somebody’s tourney hopes.
14. SWAC
Texas Southern 5430 Jackson St. 4176 Mississippi Valley St. 272 Alabama St. 49 Prairie View A&M 30 Alabama A&M 28 Grambling 9 Alcorn St. 5 Arkansas Pine Bluff 1
Texas Southern is the 2011 version of 2010 Pine Bluff. The Tigers survived the pre-season barnstorming tour with just two wins, but losses included a number of respectable performances against decent teams.
13. SoCon
College of Charleston 5327 Davidson 1889 Furman 1467 Wofford 1251 Appalachian St. 47 Chattanooga 10 Samford 5 Western Carolina 2 Elon 1
Talk about unbalanced divisions. The top four teams here have put a ton of distance between them and the rest of the conference, and all four reside in the Southern South.
12. CAA
George Mason 5301 Drexel 3042 Old Dominion 1091 Virginia Commonwealth 539 James Madison 19 Delaware 8 Hofstra 1
Should be a good duel between GMU and Drexel, and from eyeballing it, the Dragons have the slightly easier schedule.
11. Sun Belt
North Texas 5260 Arkansas St. 2125 Florida Atlantic 1273 Western Kentucky 937 Denver 190 Middle Tennessee 124 Arkansas Little Rock 56 South Alabama 22 Florida International 13
Isiah, you still have a shot.
10. Conference USA
Central Florida 5190 Texas El Paso 1877 UAB 975 Southern Mississippi 880 Memphis 726 Tulsa 308 Marshall 21 Rice 19 Tulane 4
I get the feeling people think Memphis is a favorite to win the league despite their recent string of poor performances. In reality, UCF is the favorite based on its on-the-court play, and UTEP is a nice darkhorse thanks to a friendly schedule.
9. NEC
Quinnipiac 4927 Robert Morris 3115 St. Francis NY 1317 Long Island 417 Central Connecticut 122 Wagner 98 Fairleigh Dickinson 1 Monmouth 1 Sacred Heart 1 Mount St. Mary's 0
Quinnipiac has two losses by a total of six points, although almost every game has been an odyssey in late-game win probability swings.
8. MVC
Wichita St. 4826 Missouri St. 4578 Creighton 516 Northern Iowa 78 Illinois St. 1 Indiana St. 1 Southern Illinois 0
The Shockers and Bears are both 2-0 and if everything goes to plan, the race will be decided in the season finale between the two in Wichita.
7. America East
Vermont 4692 Maine 3174 Boston University 1721 Albany 212 Stony Brook 174 Hartford 17 New Hampshire 8 Binghamton 0
Vermont was picked fifth by the media (and fourth by the kenpom projection). BU was the media/kenpom favorite and still has hope despite starting the season 5-10 (0-1).
6. Ivy
Princeton 4677 Harvard 4567 Pennsylvania 236 Yale 223 Cornell 221 Columbia 68 Brown 7 Dartmouth 0
The nation’s most improved conference figures to have a good race for the most important one-seed of all. There’s a 21% chance of a tie at the end of the regular season which included one case of a five-way tie.
5. OVC
Austin Peay 4412 Murray St. 3501 Morehead St. 2054 Tennessee St. 31 Eastern Kentucky 1 Eastern Illinois 0 Tennessee Tech 0
This was widely thought to be a Murray/Morehead race but Peay’s 4-0 start, including a wild OT win over Morehead, has changed the math.
4. Big East
Pittsburgh 4405 Syracuse 2821 Louisville 990 Villanova 957 Georgetown 248 Cincinnati 223 Notre Dame 205 Connecticut 47 Marquette 46 St. John's 34 West Virginia 22 Seton Hall 2
OK, so I went a little overboard with this tweet. Pitt is the favorite, but it’s largely attributable to schedule differences. The Panthers get the Orange at home and also get to double dip with South Florida.
3. MAAC
Iona 4319 Fairfield 2704 Rider 2446 Siena 482 St. Peter's 39 Loyola MD 6 Canisius 4
Props to Rider, an after-thought in the preseason, but arguably the best team in the MAAC at this moment. However, the Broncs are a game down to Fairfield and Iona heading into Monday’s action.
2. MWC
Brigham Young 4228 Nevada Las Vegas 3166 San Diego St. 2479 New Mexico 124 Colorado St. 4
No games yet in the MWC, but you could pick any one of the top three teams and I wouldn’t quibble.
1. MAC
Ball St. 3776 Kent St. 1806 Buffalo 1553 Ohio 1243 Akron 1022 Western Michigan 338 Miami OH 213 Central Michigan 31 Northern Illinois 8 Bowling Green 7 Eastern Michigan 2
The conference that sent its nine-seed to the NCAA tournament last season also brings us the most wide-open race in 2011. I predict when log5 season comes around, seeding will not be very important this year, either.