At the 2014 Sloan Sports Analytics Conference in Boston, there were some very bright minds discussing evolving uses of statistics and different strategies that can result; really, just from sharp minds discussing some interesting ideas. Then there was the Hockey Analytics panel.
Okay, it wasn't necessarily that bad but this was my fifth year attending and every year my sense has been that the NHL lags so far behind baseball and basketball in its adoption of analytic concepts that it can become a frustrating exercise.
This year's Hockey Analytics panel included Calgary Flames president Brian Burke, Boston Bruins Assistant GM Don Sweeney, Washington Capitals Assistant GM Don Fishman, Merrimack College Head Coach Mark Dennehy and stats analyst Eric Tulsky. The panel was moderated by Dale Arnold, from NESN.
Houston Rockets GM, and Conference Co-Chair, Daryl Morey noted that every NBA team has stats analysts. In the NHL, there are a few teams that have staff positions for someone in analytics. The Tampa Bay Lightning, for example, have Michael Peterson, who was at the conference, doing work as a statistical analyst and the New Jersey Devils recently had a job for a Director of Analytics posted on NHL.com. It's no coincidence that the Devils' new ownership group, headed by Josh Harris, is also involved in the NBA, owning the Philadelphia 76ers, so they come from a world where analytics is simply a part of doing business.
(As a complete aside, it blew me away that the Devils, who have traditionally guarded information like the CIA, would publicly announce intentions to hire a Director of Analytics, the kind of thing that most teams generally tend to keep secretive and under the radar.)
Part of what made the Basketball and Baseball Analytics panels at Sloan successful is that they didn't waste time discussing whether there is value in analytics. For whatever reason, hockey panels at Sloan insist on having a counter voice, one loud voice belonging to Calgary Flames president Brian Burke, that this sport can't be helped by objective statistical analysis. On one hand, it's good to have dissenting opinions so that every thought isn't lost in an echo chamber but, given the state of acceptance of hockey analytics, it could probably use a more positive approach at an analytics conference.
Burke started to acknowledge, "Statistics have value; ignore at your peril," he said, before getting back on familiar ground, "But it's an eyeballs business." It's understandable when anyone has been in the business for so long that they aren't inclined to jump in at new methods of analysis, but Steven Burtch had a good take on the tendency to rely on eyeballs.
It's been said more than a few times, that use of analytics doesn't guarantee anything, which is an odd standard to apply since not using analytics and sticking with eyeballs, hunches and gut instinct most certainly doesn't come with any guarantees of success either.
The funny thing about using analytics is that, somehow -- likely through the famous scene with the scouts around the table in the movie Moneyball -- there persists the idea that anyone wants to use analytics without actually watching players play.
At last year's Sloan Conference, Tulsky's paper on zone entries was a valuable insight into the game and he came up with the data by watching the games in minute detail, providing real analysis on numbers that weren't provided elsewhere. All the game-watchers in the world couldn't tell you how much more important it is to carry the puck into the offensive zone rather than dumping it in and, by quantifying the results, Tulsky showed that, all things being equal, dumping the puck in should be a last resort. For what it's worth, I watch games differently having seen that data.
There were several instances in which it was clear that the panel wasn't really up to speed on hockey analytics. While moderator Dale Arnold had researched Tulsky's work, and knew to ask about Score-Adjusted Fenwick rating, they also veered into a discussion based on the premise that advanced stats don't capture the essence of a player like Patrice Bergeron, which is crazy because advanced stats do a superior job to traditional stats in that respect. Hockey analytics LOVE Bergeron.
Where the panel lost its way, in some respects, is that when Tulsky discussed how modern NHL analytics are based on shot attempts, because that offers more valuable statistical evidence, there was no follow-up. No one argued the point -- because what kind of footing would they be on if they dared? -- but it was left to hang out there until the panel moved to a new topic of conversation.
Some of those topics:
- Fishman marveled at the clutch performance of some players, referencing Alex Ovechkin's ability to score in key moments and noted that's what analytics miss about Ovechkin is his desire to have the puck on his stick with the game on the line. I think it's been established enough, throughout the sports world, that clutch might be a thing that exists, but is entirely unpredictable. Like character, this is the kind of thing that gets assigned to players after the fact.
The funny thing is that I used analytics to defend Ovechkin's performance this season and I might suggest that the reason analytics may not capture the essence of Alex Ovechkin is that the most basic of stats, goals, does a pretty bang-up job. How far down the list of the league's leading goal-scorers would you have to get before you find one that didn't want the puck on his stick with the game on the line? 50? 100?
- So while I take issue with Fishman's stance on measuring clutch performance, Fishman also cited the importance of using objective data, as opposed to teams using their own measures because those measures come with more inherent biases.
Many NHL teams talk about tracking scoring chances, which is fine but, from my perspective, the concern in tracking scoring chances is 1) its correlation to Corsi and 2) at no point have I been under the impression that teams track scoring chances league-wide, which means covering only games involving their team. That kind of misses the point of analytics in many respects.
While it's useful to have more data on your own players, you also get to see them for 82 games and every practice, so there shouldn't be any grand surprises about the value of players that you see that often. Applying real value to data, in my opinion, has to involve measuring players league-wide so that there are some comparable baselines and insights that might help in player acquisition or determining contract value.
It's one thing for the Bruins to say how their scoring chance data tells them just how valuable Patrice Bergeron is; it's another to have any idea whether or not there are comparables elsewhere in the league.
- There was some brief discussion of coaching analytics -- Michael Parkatti had done some work in this regard -- and it's probably an area ripe for further exploration. Burke, speaking from the executive office as a GM or President tends to cut his coaches some slack, saying "The coach starts the game with the players we give them," which is true, but that doesn't mean that there isn't something to be gleaned about how a coach can affect a team's puck possession.
Sweeney, who played 1223 regular season and playoff games for the Bruins and Stars, noted that players are visual learners and need to see examples that merge with analytical insight. This is the right approach, merging analytic findings with real-life, actionable solutions. It's one thing to know, due to analytics, that there is an area of concern; it's another thing to have an idea how to fix the issue and express it clearly to those that can make changes.
- Burke said if there was an analytics package that he believed would work, he'd buy it in a second: "We're all looking for that edge."
Burke and Toronto Maple Leafs GM Dave Nonis, his protege, are in lockstep - as both have publicly stated they will pay for analytics if they see value. They just don't see it yet.
This goes back to the standard to which some hold analytics. For someone who freely admits he's made mistakes -- a function of a long career of drafting, signing and trading players -- it's incongruous that Burke wouldn't see the possibility of potentially limiting those mistakes. It's not about being 100% right on all transactions -- analytics isn't about guarantees -- but if analytics made you "right" 60% instead of 55% of the time, then wouldn't that be pretty useful?
Strangely enough, the Maple Leafs (represented by VP and Assistant GM Claude Loiselle) were one of a half dozen or so teams with hockey operations people in attendance. While the Leafs have shown a general disdain for analytics, Loiselle's presence at the conference raises the question of whether they could be willing to learn a little -- at least to have a better idea what it is that they are so readily dismissing.
- Not surprisingly, Burke is really big on the character of players they draft. I don't dispute need for character, but do think it's subjectively over-valued and often assigned retroactively. The guy whose team wins or that has a long career is determined to have high character.
It's one thing to tell a colourful story about Trevor Linden that shows his character and another to base hockey decisions on that information. Basically, there are plenty of guys with high character that aren't necessarily great hockey players.
As Houston Astros GM Jeff Luhnow noted on the Baseball Analytics Panel, "There is no correlation between being a nice guy or a good person and being a good baseball player." I've yet to see the evidence to suggest that hockey functions differently.
- Perhaps more interesting than the official Hockey Analytics Panel was discussing some of these concepts with others in attendance. In addition to Tulsky and Peterson, Sault Ste. Marie Greyhounds GM Kyle Dubas, Hockey Prospectus Editor-in-Chief Timo Seppa, as well as Michael Schuckers, Brian Macdonald -- both of whom have presented at the conference previously -- were all there to discuss and debate issues. You could probably have a more enlightened panel with some of the people in that group.
Probably the most interesting discussion to me involved the merits of swing players, those who could play both forward and defence. After attending the Baseball Analytics Panel, in which they discussed the possibility of teams shifting their best defenders around the field according to a hitter's tendencies, Dubas, Tulsky and I talked about how it might apply in hockey terms.
Most recently, players like Dustin Byfuglien and Brent Burns are prominent players that have played both forward and defence; Sergei Fedorov took some turns on the Red Wings' blueline and Christoph Schubert had that flexibility with the Ottawa Senators.
Going further back, Rick Chartraw did it for the Montreal Canadiens in the late 1970s and while Paul Reinhart and Phil Houseley were primarily defencemen, they both logged some time at forward, as did Red Kelly as far back as 1960, so it's not like it's new to have players who can fit into the lineup in different spots.
Our discussion, however, focused more on the idea that a team might, under some circumstances, go with four forwards and one defenceman in standard 5-on-5 play. Certainly, when trailing by a goal, it could make sense for the Jets to slide Byfuglien back on defence to upgrade their overall attack.
While we debated the possibilities of how it might work, and what it might take to convince, coaches, scouts, players and agents to look at players in a slightly different way, the ultimate conclusion was that there could be real value in such a player, perhaps more than would be expected. Somtimes, the player who can't fit full-time on defence gets lumped into a fourth-line forward role (like Chartraw and Schubert) but having a player that is strong enough to handle both responsibilities could be a real asset.
Anyway, the weighing of pros and cons with some smart guys made for an interesting discussion.
- While the NBA's use of SportVu Player Tracking appears to be the wave of the future, there appears to be little appetite for it in the NHL to this point. Tulsky suggested that the teams that get in on the technology first may have the opportunity to gain a competitive edge but the margins will get decidedly slimmer if the technology was to ever be carried league-wide, like it is in the NBA. Once everyone has the data, then the advantage goes to the best analyst.
- Tulsky says it's hard to come up with a hockey version of WAR right now, but it's possible some day. As someone who uses statistics to generate my NHL Player Rankings, I accept Eric's positon, because it surely can get better, but I still find it useful to have a catch-all number to at least provide ballpark value on players. Michael Schuckers, who has presented at the conference in previous years, developed his Total Hockey Rating (THoR) in an attempt to come up with this kind of value.
ESPN Insider NHL Draft/Prospect Writer Corey Pronman weighed in, via Twitter, making a point with which I agree: In order to do a proper stats analysis of a "player", need to examine 9-13 micro stats. Knowing how to combine & weigh is a bit of a skill.
So, even if there is room for improvement on the Sloan Sports Conference Hockey Analytics Panel, that's almost fitting for a sport that has been slow to come around on the use of advanced stats. As Tulsky has found, there is progress being made and it's easy enough to see how the trend towards advanced stats tilted in other sports so that, eventually, it will be part of standard business in the NHL too.
Some day it will be accepted that having more knowledge at your disposal is better than less.