Comparing Face-Off Yearbook Rankings to Year-End Adjusted Efficiency Margin Rankings

Terry Foy published a sharp critique -- not in terms of throwing shade, but in terms of cleverness -- of Face-Off Yearbook's preseason rankings over the last three seasons and how they compare to year-end RPI rankings. The idea was to identify which teams have been historically overrated and underrated by the lacrosse annual. Some of the results were . . . interesting.

As I want to shove firecrackers up the RPI's rear-end, I decided to recreate Foy's analysis, using year-end adjusted efficiency margin rankings as a replacement for RPI rankings. I did this for a few reasons, but mostly because adjusted efficiency margin gives a better illustration of overall production from a team (it measures the difference between a team's ability to score goals and prevent goals on a per-possession basis) and is adjusted for strength of schedule. I could have used year-end Pythagorean win expectation rankings as the basis for this analysis, but I only thought of that immediately pressing the publish button on my Internet computing machine. So it goes.

Other important differences between the analysis that I did and the one that Foy put together:

• I'm eliminating High Point, Marquette, and Michigan from the analysis. There just isn't a large enough sample size from those three programs. (As it is, there's only three years worth of comparison going on, which isn't a long period to discover trends.)
• I included something else in my analysis: Standard deviation on ranking difference. That basically gives us an idea of how much variance there is in the ranking differences.
• Again -- I'm not considering the RPI because I'm an adult and I can do what I want.

Here's how the analysis looks:

Some notes:

• Face-Off Yearbook has been pretty consistent in ranking difference among teams that struggle to win Division I lacrosse games (you can see in this in the standard deviation rankings). This isn't surprising: Assuming that crab bucket theory applies to college lacrosse leads to such results. What's interesting about the top 10, though, is that Face-Off Yearbook has consistently whiffed on Delaware the last three seasons. That standard deviation value is kind of ridiculous: Face-Off Yearbook has continually been 10 ranking positions off on the Blue Hens since 2011. That's . . . well, let's be honest here: I sometimes forget that Delaware is even a state.
• The greatest standard deviation in ranking difference lies with traditionally-considered middle-of-the-road teams: Pennsylvania, Brown, Colgate, etc. This, also, is not surprising: Programs of this character are frequently the most difficult to project. You win some, you lose some, but Face-Off Yearbook has really struggled to figure out where the Quakers and Bears will end up on the Division I landscape.
• Overrated/Underrated: Air Force and Siena have been the two most undervalued teams by Face-Off Yearbook since 2011 (based on average ranking difference); UMBC and Delaware have been the most overrated teams (based on average ranking difference). What's scary is that Face-Off Yearbook has been consistently overvaluing the Blue Hens since 2011 and been consistently missing by wide ranges Delaware's placement on the national scene (small variance on the huge misses). So, there's one thing to take away from this: Bears will eat you given the opportunity.
• Other Important Notes:
• Overrated: The highest average ranking difference -- in terms of overrated teams -- in an adjusted efficiency margin environment (UMBC, -12.00) is lower than the worst value in the RPI environment (Lafayette, -12.67) . . . Army, in the RPI analysis, is considered the fourth-highest overrated team; in the adjusted efficiency margin analysis, Army is a notably underrated team (the Black Knights' difference value ranks 21st) . . . Stony Brook is in a similar position to Army: The Seawolves are the sixth-highest overrated team in the RPI analysis; in the adjusted efficiency margin analysis, the Seawolves are an underrated team, sliding in around the same position as Army (those are important flip-flops; please note that as there will be a test later) . . . Lafayette has been consistently overrated, as has Navy, Rutgers, and Mount St. Mary's; I'm not sure why Face-Off Yearbook has done that over the last few seasons (and the RPI is sort of telling the truth there).
• Underrated: Hobart goes from the most underrated team in the RPI environment to an overrated team (in fact, only 14 teams are more overrated than Hobart according to the adjusted efficiency margin analysis) . . . Air Force enters the picture as the most underrated team, moving forward from the 20th most-underrated team in the RPI model (this is primarily due to some mega misses from Face-Off Yearbook in 2012 and 2011); that feels right . . . The most notable miss (Air Force, 13.00) in the adjusted efficiency margin environment is bigger than the most notable miss in the RPI environment (Hobart, 9.67) . . . Detroit, in the RPI environment, looks like a program that Face-Off Yearbook has figured out (no average ranking difference); in the adjusted efficiency margin environment, though, the Titans are the third-most underrated team in the 60-team field examined (this is heavily influenced by a big 2011 miss).

Any other thoughts? The comments are yours.