The dominant statistic in the evaluation of goaltenders is save percentage. A goalie with a high save percentage is considered good and a goalie with a low save percentage is bad. Simple enough really. Now, it's quite obvious that the reality is somewhat more complicated than that. One of the problems that we don't often consider is the rather small spread between a good save percentage and a bad one. Although there are players with more extreme save percentages I usually consider anything above .920 excellent and anything under .900 poor. That's only two goals every one hundred shots. There's an awful lot of things other than "goaltender skill" that can eat that difference. After the jump we'll take a look at one of those things, namely the distribution of shots in various game states.
The following chart should be rather self-explanatory. I've included every goaltender who has faced a minimum of 400 shots so far this season and broken down each one's shots by the percentage faced in each game state. Remembering that the average goalie has a save percentage around .915 when on the power play and at even strength but around .870 while penalty killing, it's easy to see how shot distribution by game state might impact a goaltender's overall numbers. I've listed the goalies who benefited from a favourable distribution first.
So there is actually a pretty wide spread with Martin Brodeur taking home the most favourable distribution without a problem. The differences between teammates are also pretty interesting. In Tampa you have Mike Smith near the very top and Antero Nittymaki in dead last. I would assume that goaltenders have very little influence on these distributions and that they're mostly a function of team skill. If that's the case, we haven't reached a sample size where that's showing itself clearly.
So what difference does this make on overall save percentage? Well, if we take the numbers for an average goaltender and give him Martin Brodeur's shot distribution his overall save percentage would be at .909. If we give the same goaltender Antero Nittymaki's shot distribution his overall save percentage would be .904. A .005 shift in save percentage at the extremes is both not much and quite a bit. It means that for most goalies the effect will be small but that when we're comparing two guys near the margins (say Bryzgalov and Miller for the Vezina trophy) it could make a substantial difference. We've looked at save percentage here but a higher distribution of shots on the PK will also have an impact on the other prominent goaltending statistic, goals against average. When we're making comparisons between goalies I think it's worthwhile to take into consideration.
Nonetheless, the effect is still small. To illustrate this I'll use an example from the Edmonton Oilers. Jeff Deslauriers currently has the fifth most favourable distribution out of the 45 goalies who meet the criteria. Right now his overall save percentage is .906 but if we take his .913 EV save percentage and his .873 PK save percentage and plug them into J-S Giguere's shot distribution (he's had the fifth least favourable distribution) then his overall save percentage drops modestly to .904. On the one hand, this is a small difference. On the other hand this is 10% of the difference between an excellent goaltender and a bad one. Such is the nature of goaltender evaluation where the difference between poor and excellent is so small.