Discussion of sports can often fall into an irrational abyss. But even the most levelheaded people lose their ability for rational conversation when it comes to officiating. In a rational world we’d hold the officials to the same standard as the players.

Playing basketball at the highest level requires among other things quick decision-making. We recognize that not every player can be Markelle Fultz or Malik Monk. Most of the players coming off the bench for a Big South team are working hard to be their best, too. For a variety of reasons beyond their control, they just aren’t as good as the players you see on ESPN promos.

Likewise, officiating basketball also requires quick judgments. Because of that, there will be a lot of mistakes made over the course of a game. And accordingly, there is a range of ability across the hundreds of people that officiate college basketball. Being a top official takes a lot of work and experience, and surely, talent. The great officials are working big games in power conferences. The not-so-great ones are subbing in the occasional Big South game.

This piece is not an attempt to defend officials, but I thought I’d take a stab at a method to identify the best college officials. You don’t often hear people saying things like “Official X is the best in the conference” or “Official X is better than Official Y” and I suspect that’s largely because there’s no data available to make such judgments.

The problem with this idea is that I have no way of actually grading officials. But officials do get graded by someone and I’m operating on the assumption that the best officials get assigned to the best games. That is surely not precisely true on a daily basis, but over time, if an official is regularly working the biggest games on the national stage, the folks that make such decisions probably think he’s doing a solid job.

In order to assess the quality of a game, I’m using the Thrill Score from FanMatch, imperfect though it may be. But it would seem to capture the things that are important here: opponent quality and game competitiveness. The only other thing we need to account for is the officials’ schedule. The busiest officials can top 100 games in a season. I’m going to value quality over quantity, so I’d prefer not to reward officials for bulk scheduling. In that interest, I’m taking average Thrill Score over the top 50 games that an official works in a season.¹

Based on this approach, here were the top 20 officials last season.

    Official        Rating
 1  Roger Ayers      63.2
 2  Mark Whitehead   61.4
 3  Mike Eades       60.6
 4  John Higgins     59.0
 5  Mike Roberts     58.6
 6  Doug Sirmons     57.7
 7  Bryan Kersey     56.6
 8  Keith Kimble     56.6
 9  Jamie Luckie     56.3
10  Pat Driscoll     56.1
11  Tom Eades        56.0
12  Joe DeRosa       56.0
13  Terry Wymer      56.0
14  Pat Adams        55.6
15  Brian O'Connell  54.7
16  Kipp Kissinger   54.2
17  Ted Valentine    53.7
18  Verne Harris     53.1
19  Randy McCall     52.6
20  Jeffrey Anderson 52.5

One fault with this method is that there’s a regional bias. Geography really puts a ceiling on game quality. Rogers Ayers is based out of Roanoke, Virginia, which is a good starting point to get some of the best games in the country. He gets a bunch of ACC assignments and occasionally snags an SEC, Big East or Big 12 game as well. He worked the game of the day yesterday between Villanova and Virginia. I don’t have any inside info on his schedule, but I think you can mark him down for one of the Duke/North Carolina games this season.²

Verne Harris is the best official in the West but is limited to the Pac-12 for high-visibility games. He also works the Mountain West heavily and the opportunity for big games in that league isn’t what it used to be. He even worked a Big Sky game Friday night. Some sort of regional ranking would be the fairest thing to do.

The problem with that is you have someone like John Higgins who transcends both regional attachment and the traditional bounds of hair-product use. He is willing to criss-cross the country during the season and has already worked three games at Cal and three games at Georgia this season. No matter where you are, it’s possible Higgins is nearby.

Here’s what the standings look like for this season.

    Official        Rating
 1  Roger Ayers      50.8
 2  Mike Eades       50.4
 3  John Higgins     48.1
 4  Terry Wymer      46.7
 5  Ted Valentine    45.7
 6  Jeffrey Anderson 45.6
 7  Jamie Luckie     44.5
 8  Keith Kimble     44.3
 9  Michael Stephens 43.1
10  Brian Dorsey     43.0
11  Bo Boroski       42.6
12  Doug Sirmons     42.2
13  Steve Olson      42.1
14  Verne Harris     41.7
15  Pat Adams        41.5
16  DJ Carstensen    40.8
17  Pat Driscoll     40.5
18  John Gaffney     40.4
19  Kipp Kissinger   40.4
20  Ray Natili       40.3

In addition to Ayers seeking a repeat of last season’s title, five officials appear in both last year’s and this year’s top ten. That’s not bad especially considering that two of last season’s top ten, Mark Whitehead and Bryan Kersey, moved into coordinator roles this season.

It should go without saying that year-to-year movement in the rankings is sensitive to some randomness in game assignments, too. It is unlikely that at this point in his career, Ted Valentine went from the 17th-best official last season to the fifth-best this season. True officiating talent can’t change all that much between seasons and if you were going to do this right, you would want a two-year rolling average and maybe more.

It’s conjecture on my part, but I’d guess that seniority alone plays a large role in assignments as well. Even when a younger official’s skill might be surpassing some of their older colleagues, they’re unlikely to get the prime assignments that their more experienced counterparts have been getting for years. It’s also plausible that the occasional official that makes the jump from college to the NBA is probably not rated all that well by this approach, either. Those officials are often young and not getting marquee games.³

Much like ranking college teams with an objective method, there is always going to be some sort of measurement error associated with the assumptions of the model. Short of having access to every official’s evaluations this method should put officials in the proper neighborhood in terms of their presumed ability. It’s better than nothing, which is what we have now. Hopefully, I can publish this on a more frequent basis sometime soon.