Bracketology season is on the horizon. Which metrics warrant major attention and how does the committee evaluate the different types?
The 2021-22 college basketball season is speeding along. With a month remaining until Selection Sunday, team sheets are flooding conversations. Bracketology is far from hard science, but the process of determining which teams might be worthy of NCAA Tournament inclusion has become more convoluted (in a good way) in recent years.
The introduction of the NCAA Evaluation Tool (NET) is the most well-documented change made by the selection committee, but it is not the only one. In fact, there are six core metrics, including the NET, that appear on official team sheets. The NET determines quadrant records and is thus the most important, but the committee also dives into resume and quality metrics.
The Kevin Pauga Index (KPI) and Strength of Record (SOR) are deemed the “resume” metrics. These measure the difficulty of a team reaching its record, based on the end results of games — rather than how those results came to be. “Quality” metrics generally focus on team efficiency (i.e., points per possession) to assess the caliber of performances. BPI, KenPom, and Sagarin are the quality metrics.
Juggling the value of resume vs. quality metrics is one of the more difficult bracketology discussions. Of course, boasting high rankings in both departments is the ultimate goal heading into March, but that isn’t feasible for every program. By the time the selection committee meets each year, there are teams that are held in much higher regard by one or the other.
With the 2021-22 college basketball season nearing its close, let’s dive deeper into how the committee appears to address this dichotomy and evaluate which teams are on each side of the fence at this point.
NOTE: This article was originally published on Jan. 5, 2022. It has been updated as of Feb. 16, 2022 to include up-to-date analysis of which teams are impacted most by the metrics discussed.
—Ten mid-majors eyeing at-large March Madness bids
—Ranking most likely first-time NCAA Tournament teams
—DPI: Game Predictions | Team Grades | Player Rankings
The selection committee’s approach to date
Conventional wisdom indicates that quality and resume metrics would eventually level out over the course of a season. With each game played, a larger sample size exists to draw the two closer. A team of a given quality should acquire a resume of similar stature. However, that isn’t always the case.
There are programs every year that enter Selection Sunday with sizably better rankings in one category versus the other. There are currently two years’ worth of data during the “NET Era” to evaluate how the committee addresses this debate.
The committee’s approach can be assessed — at least to some extent — using a model that projects NCAA Tournament seeds by averaging each team’s resume and quality metrics and weighting them evenly.
Below is a look at the teams that have been most overseeded and underseeded over the past two tournaments relative to the model’s placement. The rightmost column shows how the team ranked in quality metrics vs. resume metrics, which may provide a potential explanation for why they were seeded as such.
As it turns out, simply averaging resume and quality metrics and giving each equal weighting actually resulted in very similar seeding to what the selection committee did. That model projected 80 of 92 at-large worthy teams either perfectly or within one seed line – that is a pretty high rate. (Note: The term “at-large worthy” here refers to any teams seeded above the bubble cut line.)
However, of the teams that were underseeded relative to the model by two or more seed lines, five of the seven were rated better by resume metrics than in quality ones. Conversely, most of the teams overseeded benefitted from stronger quality ratings (aside from West Virginia). A preference for the KenPoms of the world appears to be in the committee’s brain.
Overall seed lines aren’t necessarily the most important part of bracketology, though. Selecting the correct field of team is obviously another crucial piece of the puzzle. So, do these same takeaways apply when looking at the bubble?
Evaluating how the bubble is addressed
The short answer is no. In fact, while the overarching theme of “overseeded” at-large teams is having better quality metrics, that is not the case as the committee progresses through the seed lines. The debate is treated essentially evenly through seed lines No. 6-8, with the average rankings matching up with one another.
However, from No. 9 seeds on – as well as the teams deemed to “First Four Out” – there is a significant shift towards favoring resume rankings. The change rewards the teams that compiled the strongest resumes, even if their efficiency numbers are not as high as some others under consideration.
This makes sense as the committee selects the teams that make the field before actually seeding them. As the seed lines draw closer to the cutline, it becomes more clear the teams that were selected based primarily on resumes.
When evaluating teams right on the bubble, No. 11 seeds and First Four teams are nearly identical when it comes to average quality ranks (plus-1.14 difference as compared to resume metrics). However, teams that actually made the tournament have been an average of 4.3 spots higher in resume rankings.
It is also important to look at the biggest extremes on the bubble.
The 2021 Wichita State team had the largest difference between quality and resume metrics of any at-large over the past two tournaments. The Shockers danced despite averaging just a 79.0 ranking in quality metrics. They were 41.5 spots higher in their resume metrics compared to quality measures. The 2019 versions of Temple (plus-28.0 in resume metrics) and St. John’s (plus-24.2) similarly made the field while being several spots higher in measurements based on results rather than efficiency.
In total, eight bubble teams (No. 10 seeds or lower) have made the last two NCAA Tournaments while being more than 10 spots higher in resume rankings compared to quality rankings. Only three such teams have done so while more than 10 places higher in quality measures.
While not a hard-and-fast rule by any stretch, it appears as though the committee has been favoring resume metrics over quality metrics over the past couple of seasons when it comes to the bubble.
How is the 2021-22 season shaking out?
Diving into this season, in particular, there are several teams emerging as leaning towards one side of the quality vs. resume spectrum. Based on the conclusions above, it is better to be on the quality side when already seen as a “lock” to reach the Big Dance. On the other hand, having strong resume metrics is more beneficial while flirting with the bubble.
Potential at-large teams right now (Feb. 16, 2022) roughly cut off at averaging a ranking of No. 80 better in an even weighting of resume and quality metrics. There are 75 teams currently meeting that criteria. Here is a deeper dive into which potential at-large teams have the widest gaps in resume and quality with a month left in the 2021-22 college basketball season.
Teams dependent on resume rankings
Generally speaking, these are the teams that have done the most with their respective opportunities this season. They have compiled solid overall resumes that would generally meet the standard of reaching the NCAA Tournament. Except for a few exceptions (Ohio and Stanford, namely), all of these teams would earn at-large bids if the entire selection process was based on the SOR and KPI rankings (there tend to be ~46 at-large quality teams per year).
Considering the committee appears to favor these resume metrics for selection (and the opposite for seeding), it is likely a good thing to appear in this chart if your team is hanging around the bubble conversation. The most notable bubblicious squads that appear in this chart are from the Atlantic 10: Davidson, VCU, and Saint Louis; all three of those are hovering right around the double-digit seed or first few out territories right now.
The Bracket Preview likely will not help much with regard to determining how the committee views the bubble this year. Even if the committee does favor quality metrics over resume metrics for its Top 16, that would not be any different than recent precedent. If, however, resume-oriented teams such as Wisconsin and Providence do exceed bracketologist expectations on Feb. 19, that would be great news for everyone on this list.
The Badgers and Friars are the two biggest teams to watch for at the Bracket Preview for resume-oriented teams.
Teams dependent on quality rankings
The flip side of the debate looks at teams that are more dependent on stronger quality rankings right now. Washington State is an extreme example but is also on the outside looking in right now with a wide margin between it and the cutline.
The best examples of bubble teams featured in the chart above are from the Big Ten with Indiana and Michigan; both are highly regarded by their quality metrics but lack the winning percentage or quality wins to be firmly in the field. Even Iowa to some extent is very reliant on its metrics that were boosted from beating up on subpar competition.
There are a lot of true “bubble” teams in this collection of squads.
Gonzaga and Houston are the teams to watch from a quality metrics standpoint during the Bracket Preview. The Bulldogs’ placement likely won’t mean much as they are expected to land a No. 1 seed regardless, but it will be worth noting if they land the No. 1 overall seed at this time. The Cougars are the other team to monitor and are perhaps more important due to a wider seed range. They could land anywhere from a No. 2 to out of the Top 16; both of those results are on the extreme end but would tell a lot if either happens. More than likely, Houston will be a No. 3 or 4 seed and will not draw too much attention.
Gonzaga and Houston are not as important to watch as Providence and Wisconsin because they are a bit closer to an even differential. Gonzaga and Houston both feature differentials of lower than 15 between their resume and quality metrics. Providence and Wisconsin, on the other hand, are more extreme examples – and thus more important to watch – as they exhibit differentials of over 20 each.
Overall analysis of the debate
The resume vs. quality rankings debate is not going to slow down anytime soon. There are only two seasons of data of the committee using these six metrics, and their application is sure to change over time. Still, adding KenPom and SOR – among others – to the team sheets has made the selection and seeding process more advanced than it was in the past.
Is the Big Dance about selecting “the best teams” or is it about selecting “the teams that earned it most” in the regular season? That question is at the crux of the debate. Based on the last two NCAA Tournaments, the selection process relies more heavily on resume while seed placement is more dependent on quality.
If that continues to be the trend, then it could lend more clarity on how to evaluate bracketology conversations over the final month of the regular season. That knowledge could also affect how teams put together their nonconference schedules. For now, though, these two very different types of metrics make for a tricky juggling act when trying to curate college basketball’s biggest stage.
You must be logged in to post a comment.