Postmortem on Draft Predictions - Assessing Accuracy

With very little to do between the Stanley Cup Finals and the NHL Entry Draft, it seems everyone in the hockey media does some kind of mock draft. In Canada, Bob McKenzie's ranking seems to carry a good deal of credibility, not just because he's a credible hockey insider, but also because of his methodology. I heard him speak once about how he determines his rankings and he described his process as being an "aggregator" of draft predictions from multiple sources. If I recall correctly, he said that his rankings are not his own opinions, but basically a consensus of the information he collects from others.

I found his methodology interesting, especially since he has a pretty good track record of accurately predicting the top 10 positions. In his rankings on the TSN website, he also includes the rankings from other reputable and well-known sources from around the hockey world: Craig Button (TSN Director of Scouting), The Hockey News, International Scouting Services,, McKeen's Hockey and NHL Central Scouting.

To my knowledge, no one really does a post-draft analysis to see how well these various sources did with their predictions, so I did a little spreadsheet. Predicting draft order beyond the first round is difficult (and not all sources make predictions beyond the first two rounds), so I only focused on the first round. To assess accuracy, I compared each source against the actual draft order for the first 30 selections and then calculated the difference between the prediction and the actual draft position. For my purposes, it didn't matter whether the prediction was high or low, just how far it was from the actual selection. The result is an average of the difference (prediction minus actual selection) for each source. (NHL CSS was left out due to their unique system of separating players into categories)

Here's what I found (from most accurate to least accurate):

Source Avg Diff
Bob McKenzie (TSN) 3.77
The Hockey News 4.53
McKeens 4.63
International Scouting Services 5.60
Craig Button (TSN) 6.87
Hockey 7.87

I also looked at the variance of the predictions, noting the range, the average of the predicted draft number and the actual draft number. It's interesting to see situations where there was general consensus and others where there was a huge range of predictions. For this chart, I looked at a the top 50 players in the 2014 draft.


I guess Bob McKenzie's method works best. As a self-described "aggregator", I kind of expected his rankings to come in at or near the average for each player, but that wasn't the case. Interestingly, there were a number of players where he was at an extreme end of a large range -- Jake Virtanen, Kasperi Kapanen, Adrian Kempe, John Quenneville, Thatcher Demko and Roland McKeown are examples (see chart link below). I can only guess why, but it would seem logical that, as an insider, Bob McKenzie informs himself with sources that he does not reference.


*Note: There were four players who were selected in the top 50 who were not ranked there (Jayce Hawryluyk, Mason McDonald, Vitek Vanecek & Joshua Jacobs) and another four players who were (on average) ranked inside the top 50, but not selected there (Roland McKeown, Jack Dougherty, Brycen Martin & Anton Karlsson).

The Copper & Blue is a fan community that allows members to post their own thoughts and opinions on the Edmonton Oilers and hockey in general. These posts do not necessarily represent the views of the staff of The Copper & Blue.

Trending Discussions