Lies or damned lies?
Moneyball flaws can be found in finance. “The Sabermetric Revolution,” a new book just in time for the start of the Major League Baseball season on March 30, debunks some of the numerical craze sweeping the sport. Two Smith College professors expose how many of the statistics rely on poor measurement, dodgy theory and over-extrapolating small data sets. Even good metrics tend to get arbitraged quickly. Just like on Wall Street.
While baseball’s statistical obsession can be traced back decades, Michael Lewis’s 2003 bestseller “Moneyball” and a subsequent film adaptation elevated the number-crunching to a new level. Lewis spent a season with the Oakland A’s charting how general manager Billy Beane purportedly upended convention by using data analysis techniques, or sabermetrics, to find inefficiencies.
The first chapter of “The Sabermetric Revolution” deftly deconstructs “Moneyball” to reveal a great yarn that isn’t exactly supported by the math. As the authors write: “Lewis’s storytelling works better with simplicity and with heroes.”
“The Sabermetric Revolution” scrutinizes the development of baseball statistics in the decade since “Moneyball” was published and assesses their contributions to the quality of the game and team fortunes. The discipline has expanded enormously, with armies of Ph.D.s replacing a handful of analysts with lesser academic credentials working in the sport’s management suites.
Professors Benjamin Baumer and Andrew Zimbalist aren’t out to invalidate sabermetrics. At its simplest, focusing on new and improved performance measures like on-base percentage instead of historically popular ones like batting average have indeed helped sharpen the process of evaluating players. The likes of Kevin Youkilis, who Lewis refers to as the “Greek God of Walks” for his ability to reach base without necessarily getting a hit, have become more coveted than they would have been two decades ago. Swift market adjustments, however, also make fast work of such bargains.
That has pushed sabermetrics deeper into the spreadsheets, where the numbers can get fuzzier. For example, gauging a player’s fielding ability more accurately is a hot area for scrutiny these days, but “the accurate measurement of player contributions on the defensive side has proved far more elusive to sabermetricians than the corresponding offensive components.”
Such play-by-play analysis is subject to scorer biases and the random effects are so large that there is little stability from season to season. What’s more, because a player’s abilities vary naturally over a period of years, it becomes almost impossible to separate analytical signal from noise.
Indeed, there is even little agreement on the relative influence of fielding compared to batting and pitching, although it seems likely that traditional scouts (and therefore, baseball’s salary structure) have tended to undervalue it. And the best metrics for fielding, such as Ultimate Zone Rating, are proprietary, which means any biases can’t be assessed.
It’s all eerily reminiscent of the way bankers and traders operate. Risk management metrics, such as Value-at-Risk, include statistical assumptions – most notably Gaussian ones – that make them prone to break down in periods of high market turbulence. In many cases, they also suffer from over-extrapolation of small sample sizes, for example in estimating volatility from limited trading period data.
Good anecdotes are rife, especially in fashionable areas such as technology firms going public. Further, the use of black-box formulas to value securities and perform other tasks makes it difficult for top management, regulators and investors to ascertain their true value until it is too late.
“The Sabermetric Revolution” delves into the broader business of baseball, but the authors leave ample territory to explore. Contract patterns for players with more than six years of experience and in particular the interaction between the duration of such agreements and a player’s annual value, for example, seem like fertile ground. The best players in each winter’s contract season – Robinson Cano and Jacoby Ellsbury this year – are likely to be overpaid, generally by way of longer-than-rational deals.
Conversely, “bounce-back” players, or ones whose careers have been derailed by injury and are expected to return to form, are quite common and may well be undervalued. The contract of only $750,000, or $6 million including maximum incentives, granted by the Red Sox to former star center fielder Grady Sizemore may be one such case. It’s also fairly clear that, if the fielding metrics are anywhere close to correct, incremental ability with the glove is cheaper than incremental hitting or pitching ability.
Baumer and Zimbalist have written a thought-provoking survey of how sabermetrics has developed, and even progressed, since “Moneyball.” Most teams now use it in some form and the advanced math has helped a handful of them win a little more. For financial whizzes, “The Sabermetric Revolution” is a cautionary tale on the limits of statistics. Winning at baseball is just as tough as beating the market.