FanPost

Postmortem on Draft Predictions - Assessing Accuracy

With very little to do between the Stanley Cup Finals and the NHL Entry Draft, it seems everyone in the hockey media does some kind of mock draft. In Canada, Bob McKenzie's ranking seems to carry a good deal of credibility, not just because he's a credible hockey insider, but also because of his methodology. I heard him speak once about how he determines his rankings and he described his process as being an "aggregator" of draft predictions from multiple sources. If I recall correctly, he said that his rankings are not his own opinions, but basically a consensus of the information he collects from others.

I found his methodology interesting, especially since he has a pretty good track record of accurately predicting the top 10 positions. In his rankings on the TSN website, he also includes the rankings from other reputable and well-known sources from around the hockey world: Craig Button (TSN Director of Scouting), The Hockey News, International Scouting Services, hockeyprospect.com, McKeen's Hockey and NHL Central Scouting.

To my knowledge, no one really does a post-draft analysis to see how well these various sources did with their predictions, so I did a little spreadsheet. Predicting draft order beyond the first round is difficult (and not all sources make predictions beyond the first two rounds), so I only focused on the first round. To assess accuracy, I compared each source against the actual draft order for the first 30 selections and then calculated the difference between the prediction and the actual draft position. For my purposes, it didn't matter whether the prediction was high or low, just how far it was from the actual selection. The result is an average of the difference (prediction minus actual selection) for each source. (NHL CSS was left out due to their unique system of separating players into categories)

Here's what I found (from most accurate to least accurate):

Source Avg Diff
Bob McKenzie (TSN) 3.77
The Hockey News 4.53
McKeens 4.63
International Scouting Services 5.60
Craig Button (TSN) 6.87
Hockey Prospect.com 7.87

I also looked at the variance of the predictions, noting the range, the average of the predicted draft number and the actual draft number. It's interesting to see situations where there was general consensus and others where there was a huge range of predictions. For this chart, I looked at a the top 50 players in the 2014 draft.

2014_draft

I guess Bob McKenzie's method works best. As a self-described "aggregator", I kind of expected his rankings to come in at or near the average for each player, but that wasn't the case. Interestingly, there were a number of players where he was at an extreme end of a large range -- Jake Virtanen, Kasperi Kapanen, Adrian Kempe, John Quenneville, Thatcher Demko and Roland McKeown are examples (see chart link below). I can only guess why, but it would seem logical that, as an insider, Bob McKenzie informs himself with sources that he does not reference.

2014_draft__tsn___1_

*Note: There were four players who were selected in the top 50 who were not ranked there (Jayce Hawryluyk, Mason McDonald, Vitek Vanecek & Joshua Jacobs) and another four players who were (on average) ranked inside the top 50, but not selected there (Roland McKeown, Jack Dougherty, Brycen Martin & Anton Karlsson).

The Copper & Blue is a fan community that allows members to post their own thoughts and opinions on the Edmonton Oilers and hockey in general. These posts do not necessarily represent the views of the staff of The Copper & Blue.

X
Log In Sign Up

forgot?
Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join The Copper & Blue

You must be a member of The Copper & Blue to participate.

We have our own Community Guidelines at The Copper & Blue. You should read them.

Join The Copper & Blue

You must be a member of The Copper & Blue to participate.

We have our own Community Guidelines at The Copper & Blue. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9355_tracker