This season marks the first year that the D-III men’s NCAA tournament field is expanded to 10 teams. Six conferences have automatic qualifiers, leaving four at-large bids: one Pool B at-large bid for teams in leagues without automatic qualifiers and three Pool C at-large bids for teams not receiving bids in Pools A or B.
When the NCAA had only one Pool B and one Pool C selection just a few years ago, the process seemed pretty cut-and-dried and investigation after the fact showed the selections to be by the book. The addition of a third at-large bid added a little complexity to the situation, but the selection was still fairly transparent.
This season, the at-large bids came as a bit of a surprise to some, including USCHO’s own NCAA watchers.
So, for our fourth consecutive year, we’re doing a little forensic investigation — what we call CSI: NCAA. Since none of us were on the committee’s conference call, let’s see if we have any theories from forensic evidence to explain what could have happened to choose the 10 teams and decide the seeding.
As it is every year, the first step in the selection of teams was easy for the committee. Six teams in conferences with automatic qualifiers won their tournaments and received automatic bids: Middlebury in the NESCAC, Norwich in the ECAC East, Mass.-Dartmouth in the ECAC Northeast, SUNYAC champion Geneseo, MIAC champ St. Olaf, and Wisconsin-Superior in the NCHA.
Those six teams comprise Pool A.
With those six teams in place, four at-large bids were left for the committee to select: the single Pool B bid, for a team from the two leagues without autobids — the MCHA and ECAC West — and three Pool C bids, for teams that had not yet received a bid after Pools A and B were filled.
This is the second year that Pool B teams were also eligible for Pool C.
The selection committee and its two regional committees have five criteria on which to rank at-large teams: in-region winning percentage, in-region head-to-head results, in-region results against common opponents, the Quality of Wins Index (QOW), and results against ranked teams. The NCAA has been publishing a regional ranking of teams since February 14, using those criteria.
First, let’s look at the Pool B bid.
Top contenders for that would have been MCHA champion Milwaukee School of Engineering in the west region and Manhattanville of the ECAC West from the east region. A comparison between those two schools gives that hands-down to the Valiants, who won all three categories for which there were data (the two schools had no common opponents and no head-to-head contests.)
Manhattanville’s winning percentage was .840 to MSOE’s .786; the Valiants’ QOW Index, at 10.3846, was more than a point higher, and Manhattanville’s 7-3-1 record against ranked teams clearly outshone the Raiders’ 0-1 record, a loss in a close game against St. Norbert.
Now we move on to Pool C.
Each of the regional committees, prior to the final review by the selection committee, met via conference call to assemble their slates of teams for selection in Pool C. In the west, going by USCHO’s PairWise Rankings for that region, we can surmise that St. Norbert, Wisconsin-River Falls, and either St. John’s or Milwaukee School of Engineering were offered as possible Pool C teams.
(While the NCAA doesn’t specifically use the USCHO PWR, we believe that they do use some sort of pairwise comparison to help them visualize the comparison data.)
In the east, the absence of New England College as a Pool C selection suggests that they weren’t recommended by that region, or if they were, they were below the teams that were selected, Hobart and Elmira. The Pilgrims, whom many fans and USCHO’s D-III guru Chris Lerch thought would be a lock to be selected even if they did not win the ECAC East, did not make the cut.
Looking strictly at the USCHO National PairWise Rankings, which rank teams based on the selection criteria, New England College, Elmira and Hobart are the top three teams not selected. But NEC didn’t get the nod. Why?
First, let’s see how they stack up in the criteria.
Hobart beats New England College in three of four criteria:
Hobart vs New England College
WIN 0.7200 0 0.7692 1
QOW 10.2692 1 9.6154 0
H2H 0- 0- 0 0 0- 0- 0 0
COP 6- 0- 0 1 4- 1- 0 0
RNK 5- 7- 0 1 3- 5- 1 0
============================================
PTS 3 1
Advantage, Hobart.
Now, let’s compare Elmira and New England College:
Elmira vs New England College
WIN 0.6667 0 0.7692 1
QOW 9.5926 0 9.6154 1
H2H 0- 0- 0 0 0- 0- 0 0
COP 2- 1- 1 0 4- 0- 1 1
RNK 8- 7- 1 1 3- 5- 1 0
============================================
PTS 1 3
New England College beats Elmira in a straight reading of the criteria.
Now, Elmira vs. Hobart:
Elmira vs Hobart
WIN 0.6667 0 0.7200 1
QOW 9.5926 0 10.2692 1
H2H 3- 1- 0 1 1- 3- 0 0
COP 10- 6- 2 0 12- 4- 0 1
RNK 8- 7- 1 1 5- 7- 0 0
============================================
PTS 2 3
Hobart narrowly wins that, but Elmira did win three of four games this season against Hobart.
Well, that gets us nowhere. It suggests NEC should be ahead of Elmira.
Let’s dig further comparing Elmira and New England.
By watching the NCAA’s weekly rankings, we’ve come to believe that the NCAA treats some criteria as more important than others. Ranked teams are not in the same position as they fall in the USCHO PairWise Rankings. A review of that suggests that the Quality of Wins Index and the record against ranked teams weigh more heavily in rankings than other criteria, as teams with weaker schedules seem to be ranked lower by the NCAA than their PairWise Ranking would indicate.
First, let’s look at the Quality of Wins Index. (An explanation of the QOW is found in USCHO’s D-III PWR FAQ.) Elmira’s is 9.5926 (seventh nationally.) New England College has a QOW of 9.6154 (sixth), which is slightly better than Elmira’s. So that doesn’t explain it.
Looking at record against ranked teams, Elmira was 8-7-1 while New England College was 3-5-1.
That’s in Elmira’s favor.
The Pilgrims have the edge in winning percentage.
And NEC has the advantage in common opponents; both beat Hamilton and Skidmore (NEC beat Skidmore twice); Elmira lost to Trinity which NEC tied; Potsdam lost to NEC and tied Elmira. Only Trinity was a team above .500. So this may not have been regarded as terribly meaningful by the committee.
Did the committee look to secondary criteria? Here they are:
1) Out-of-region head-to-head competition
2) Overall Division III won-loss percentage
3) Results versus common non-Division III opponents
4) Results versus Division III teams ranked in other regions
5) Overall win-loss percentage
6) Results versus common out-of-region opponents
7) Overall Division III strength of schedule
8) Win-loss percentage during the last 25 percent of the season
Item 1 doesn’t apply. Item 2 is in NEC’s favor. Items 3 and 4 don’t apply. Item 5 is in NEC’s favor. Item 6 doesn’t apply.
Then there’s Item 7.
New England College had a higher winning percentage than Elmira but a nearly equal QOW Index. That means the overall strength of NEC’s schedule was lower. As it turns out, Elmira had only seven games against teams below .500, while NEC had 11 games against teams below .500, including two from the weaker ECAC Northeast. Advantage, Elmira.
Item 8, win-loss percentage in the last 25 percent of the season, is to Elmira’s advantage, but only slightly. NEC went 5-1-1 while Elmira was 6-1.
It appears that the committee decided to weigh record against ranked teams strongly, and to negate NEC’s winning percentage advantage by dismissing it as resulting from a weak schedule. While they reserve the right to weigh criteria as they see fit, they stretched those criteria to their limit to come up with this result.
But that leaves us another question: If Elmira was beaten in the comparisons with Wisconsin-River Falls, why did the Soaring Eagles get the bid and not the Falcons? Hobart does beat River Falls, but here’s the comparison between Elmira and River Falls:
Elmira vs Wis.-River Falls
WIN 0.6667 0 0.6852 1
QOW 9.5926 1 9.2593 0
H2H 0- 0- 0 0 0- 0- 0 0
COP 0- 0- 0 0 0- 0- 0 0
RNK 8- 7- 1 0 4- 2- 2 1
============================================
PTS 1 2
We have to conclude that the committee considered the winning percentages a wash and looked askance at the Falcons’ QOW Index.
Next, the committee was charged with seeding the teams. A page of comparisons won, like USCHO’s PairWise Rankings, generates this list:
1. Middlebury
2. St. Norbert
3. Manhattanville
4. Norwich
6. Elmira
7. Hobart
8. Geneseo
14. Wis.-Superior
22. Mass.-Dartmouth
28. St. Olaf
(The numbers are from the USCHO Men’s D-III National PairWise Rankings, not the NCAA’s.)
Using the rankings above, the committee first split the field. With a 7-3 split between the east and west regions, one first-round game is held in the west region and one in the east region.
While this is the most controversial part of the seeding process, the committee really doesn’t have any leeway on this issue. They are told they are to rank the teams within each geographical region and to limit flights. Here’s a section from the 2006 NCAA Men’s D-III Ice Hockey Championship Manual:
The Division III Men’s Ice Hockey Committee will rank the teams selected within each geographical region using the primary and secondary criteria. The highest-ranked teams in each region will be given consideration as first- and second-round sites. … The committee will pair the teams regionally, based on geographical location of all participants and final seeding. Flights will be kept to a minimum.
Fortunately, the rankings allow pairings that don’t require geographic rearrangement.
In the east, rankings would suggest this order of seeding:
1. Middlebury
3. Manhattanville
4. Norwich
6. Elmira
7. Hobart
8. Geneseo
22. Mass.-Dartmouth
(Again, the numbers are those of the USCHO National PWR, not the NCAA’s.)
The first-round pairing, Mass.-Dartmouth at Geneseo, with the winner at Middlebury, is what was announced.
But the other two suggested by this ranking, Hobart at Manhattanville and Elmira at Norwich, are not what were seeded. The seeding suggests that either St. Norbert is the top overall seed, despite being beaten in the criteria by Middlebury, or that Manhattanville loses to Norwich in a comparison. Neither makes sense. From this comparison, Manhattanville should have been ranked ahead of Norwich:
Manhattanville vs Norwich
WIN 0.8400 1 0.8000 0
QOW 10.3846 1 10.0769 0
H2H 0- 0- 0 0 0- 0- 0 0
COP 7- 0- 0 1 3- 2- 0 0
RNK 7- 3- 1 1 6- 3- 1 0
============================================
PTS 4 0
This one remains a mystery.
The western seeding was easy; St. Norbert is clearly above the other two squads, and Superior has a big comparison advantage over St. Olaf.
While our first three CSI investigations of the NCAA committee’s selections found that the committee followed its process to the letter, we don’t come to the same conclusion this year.
We’re left with some nagging questions: how did the committee manipulate its criteria to put Hobart and Elmira ahead of New England College? Also, why was Norwich seeded higher than Manhattanville? And if it wasn’t, why was St. Norbert seeded higher than Middlebury?
We’ll dig further, but I’m afraid this will end up in our cold case file.
