FanPost

Analytics is a Method, Part 2: What's Wrong with the Devils?


In a previous post, I argued analytics ought to be viewed as a method of solving problems and that the manner in which it has been mainstreamed by recent case studies (notably the Wild, Kings and Leafs), is problematic. That is, analytics has been mainstreamed under the framework of "analytics as possession," which has, in part, encouraged a variety of unproductive arguments, esp. those assuming analytics fails to account for a variety of outcomes and/or variables.

In this post, I'd like to examine a counter case study, which I hope will highlight the way analytics at its core is about solving problems and not about any specific tool, number or combination of tools and numbers.

The New Jersey Devils are an outstanding puck possession team. Here are charts of the last two years showing where the Devils stack up 5x5 by FenwickClose (3rd, 5th):

2z5olxx_jpg_medium

2lnngi_jpg_medium

(both charts via extraskater.com)

The Devils are in elite company among possession teams. And, the vast majority of these teams are winners. Or, possession correlates very high with winning hockey games.

When we look at where the Devils sit in the standings, however, it is apparent that their elite-level possession is not translating into an elite-level win record. Here's their current spot in the standings:

Dwqw7q_jpg_medium

In an environment where the Leafs possession failures are dominating the mainstream analytic conversation, the Devils present an interesting case study. That is, the way in which the Leafs stats/anti-stats debate has been framed is misleading. It has constructed a hyperbolic situation in which "analytics" is synonymous with possession and whereby the validity of analytics is dependent upon the success/failure of possession to accurately predict outcomes.

To the analytics community this hyperbolic situation is surely absurd. To the nay-sayer and the casual observer, however, no doubt this is how they have encountered analytics: as a question of possession and prediction.

To this group, I think, perhaps the best way to demonstrate analytics as a method, rather than a set of tools or numbers, is to show how it might approach a different kind of problem. The Devils represent a different kind of problem insofar as they invert the situation of the Leafs (they are an elite possession team faltering in the standings).

To a naif, surely the mere fact of the Devils ought to debunk analytics. If possession can't account for the Devils' losing season, how can it account for anything?

And yet, analytics as method merely sees this as another problem to be solved.

So, what is wrong with the Devils?

To start with, they have been extraordinarily unlucky. They enjoy a league high 16 overtime loses and are an incredible 0-11 in shootouts.

They are 27th in 5x5 save percentage. (last year they were 26th)

They are 24th in 5x5 shooting percentage. (last year they were 28th)

They are 29th in PDO.

But, it's not all luck.

Part of the problem is also roster deployment. The Devils have been loath to unseat Martin Brodeur from his putative number one goaltender position. This has surely cost them goals and shootout wins.

Another part of the problem is roster construction. While built to dominate possession, the Devils are a very poor shooting team. Now, Edmonton Oilers fans will recall fondly the debate about Jordan Eberle's unsustainable shooting percentage. However, over time the shooting talent of a given player settles into a sustainable historic average. In the case of the Devils, especially without Ilya Kovalchuk, we appear to be witnessing a team constructed without a lot of true talent shooting ability.

Beyond luck-like categories like shooting and save percentage, however, there is another problem with the Devils.

While they are 1st in the league in fenwick against per 60 minutes, they are 27th in fenwick for per 60 minutes. That is, while they are exceptional at limiting unblocked shot attempts, they are very poor at generating their own.

Sure, they generate enough to cover the tiny amount they allow such that their fenwick percentage is phenomenal. However, this percentage fails to acknowledge that when the Devils are playing neither team is generating many shots. This is simply a product of the anti-entertainment brand of hockey the Devils employ.

Another way to express this is simply with shots on goal per 60. The Devils are 1st in the league in limiting shots against and 29th in shots taken.

If we compare these shot splits with other elite possession teams, say the Chicago Blackhawks (2nd in 5x5 FenwickClose%) and the San Jose Sharks (3rd in 5x5 FenwickClose%), we see a very different picture:

Chicago is 4th in shots against per 60 minutes (2nd in FA/60) and 2nd in shots for (7th in FF/60).

San Jose is 6th in shots against per 60 minutes (7th in FA/60) and 3rd in shots for (1st in FF/60).

That is, not only are these teams winning the percentage of unblocked shot attempts battle (fenwick), they are also playing a style that generates a lot of shots on net. The Devils, however, simply aren't constructed and/or deployed to generate enough attempts to score goals.

This is hardly an exhaustive look at the Devils (hint: go here for more in-depth coverage). We haven't even covered what's weird about how lucky this team has been on special teams

What I'd like to suggest, however, is twofold.

First, analytics is best understood and deployed as a kind of method of problem solving and not as a set of tools or numbers in the abstract.

Second, the way we talk about analytics and the case studies that get mainstreamed affect how analytics will be perceived and understood. In order to complicate the framework in which the current mainstreaming of analytics has taken on, alternative case studies should be sought that can showcase some of the other problems analytics can solve (at least provisionally).

X
Log In Sign Up

forgot?
Log In Sign Up

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to SB Nation going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to SB Nation going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join The Copper & Blue

You must be a member of The Copper & Blue to participate.

We have our own Community Guidelines at The Copper & Blue. You should read them.

Join The Copper & Blue

You must be a member of The Copper & Blue to participate.

We have our own Community Guidelines at The Copper & Blue. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9355_tracker