SI’s power ratings and measuring team defense

Sports Illustrated publishes weekly power rankings, and last week’s ratings featured a surprise—the Nats were ranked first in the National League and second in MLB. This was surprising because the previous week they’d been ranked sixth, and during the intervening week (May 14–20) they had gone 3–4 against the Padres, Pirates, and Orioles, giving no indication that they deserved to move up.

The reason, of course, was a change in methodology, specifically for the measurement of team defense. SI‘s explanation was that their power rankings were based on Fangraph’s WAR, but the WAR statistics for fielding, which are based on ultimate zone ratings (UZR) weren’t able to adequately adjust for team fielding. In particular, the UZR is based on standard assumptions about where fielders are positioned. With the growing use of defensive shifts, they found that some teams were successful getting defensive outs even though the team didn’t show up well according to UZR.

SI announced that they had replaced the defensive component of WAR with a measure based on “the rate at which teams turn balls in play into outs.” Those of us who are familiar with the old Bill James Baseball Abstracts recognize this statistic as the defensive efficiency rating” or DER.

I have to say that I too had been puzzled by the difference between defensive WAR (and the UZR on which it’s based) and DER. Their fielding WAR through May 26 is a below average –2.4, while their DER of .730 is much better than the major league average (.711) and ranks sixth among major league teams.

DER, of course, doesn’t measure everything on defense, and there is certainly a case that it leads to an overrating of the Nationals. For example, it omits errors (where the Nats have been about average, with a .985 fielding average) and double plays (where the Nats have been far below average, with only 73 double plays turned, though it’s unclear how much that late rate is due to their power pitching staff).

I guess what I take from this is that the measurement of fielding is still a bit fuzzy compared with what’s been accomplished with batting and pitching. In the not-too-distant future, I expect we’ll have measures that will decompose fielding into positioning and reaction to the batted ball. Then, maybe we’ll get fielding statistics that are reliable enough to be informative as the season goes along. For now, I think the operative warning for all fielding statistics continues to be caveat emptor.