The predictive power of K:BB ratio: you'd be better off using a psychic

Ah, K:BB ratio, beloved metric of the ESPN fantasy baseball analyst.

By the end of last season, ESPN must've comment upon every pitcher's K:BB ratio in at least one fantasy baseball 'news' bulletin. The assumption appeared to be that a great K:BB ratio indicated that despite a bad ERA, better days were on the horizon (hello James Shields, Jason Hammel, Scott Baker et al), whilst a bad K:BB ratio but good ERA suggested luck and a forthcoming regression to the mean (hello Trevor Cahill).

Trouble is, K:BB ratio is not very predictive - i.e. it is easy to have a bad K:BB ratio and good ERA or a good K:BB ratio and bad ERA. This is quite evident from the list of last year's qualified starters ranked by K:BB ratio - in amongst the good pitchers like Roy Halladay and Cliff Lee, you'll see the undraftable likes of Doug Fister and Rick Porcello.









 Qualified Starters sorted by K:BB ratio

And here is the predictive value of K:BB ratio in graphical form (for the method behind the madness, click here):

An R^2 of 0.05 is rubbish, and means that K:BB ratio is less predictive of future performance than ERA - i.e. you'd be better off assuming that a pitcher with a high ERA will continue to have a high ERA and vice versa, ignoring the K:BB.

There are things that are better than ERA at predicting future performance, but K:BB ratio ain't one of them. Stay tuned to find out what they are.

Sabermetrics and Moneyball are wrong

A new book that revels in small sample sizes and random pigeons (probably, I haven't read it) conclusively proves that Moneyball and Sabermetrics are wrong. After all, what have the Red Sox won since Theo Epstein took charge?

http://www.prweb.com/releases/2011/03/prweb5139404.htm

How to Pick a Starter Part I

There will, in the upcoming baseball season, be many, many blogs trying to identify those pitchers who've been unlucky (e.g. James Shields), and should be acquired at all costs, and those pitchers who've got great numbers which are little more than luck (e.g. Trevor Cahill), and therefore must be traded ASAP to some poor sap who doesn't follow advanced statistics.

So which stats should we use to judge future pitcher performance? Can stats divide the good from the lucky and the unlucky from the rubbish? To keep things simple, we'll begin with looking at what past ERA tell us about future ERA.

The Method

From Fangraphs, I downloaded the pitching stats of all qualified starters going back to 2005. In this initial analysis, I asked how consistent a pitcher's ERA is from year to year. To do this, I plotted for each pitcher his ERA  in year 1 vs his ERA in year 2 - and to increase sample size (n=267), I repeated this for each pair of years - for instance, in today's graph, the data is the correlation between 2005 ERA and 2006 ERA, 2006 ERA and 2007 ERA etc, all the way through to 2010. I've treated each yearly pair as an individual data point so that many pitchers stats appear more than once. By looking at the correlation between all these individual pitchers' ERAs from year to year, we can judge how good ERA is at predicting future success (at least as measured by ERA).

How good is this year's ERA at predicting next year's ERA?

We can measure the correlation between data points by calculating the R^2 for a line of best fit through the data points. If every pitcher's ERA was identical from year to year, R^2 would equal 1. Obviously this is impossible. Instead, the (low) R^2 of 0.13  for the correlation between this* year's ERA and next* year's ERA suggests that an individual pitcher's ERA is very variable from year to year.

Helpfully, this data also gives a baseline, so we can ask whether a given pitching stat (e.g. WHIP, K:BB ratio, FIP, xFIP) is better (R^2 >0.13) or worse (R^2<0.13) than ERA at predicting future ERA and in this way identify the stat that is the best predictor of future pitching success). Those exciting analyses are coming very soon.

*where this and next can be any pair of consecutive years.

Do you have a soul?

From interpretationbydesign comes this handy guide:

More WAR: Fangraphs vs. Baseball Reference. Roll Up, Place Your Bets, Fight! Fight!

Trevor Cahill 2010 = 4.1 Baseball Reference WAR
James Shields 2010 = -1.3 Baseball Reference WAR (that's minus 1.3 WAR)

That looks more like it (remember Fangraphs had them both at 2.2 WAR).

So why the big difference?

It's all to do with the difference in how the two sites calculate WAR.

Fangraphs uses FIP to calculate WAR - which is largely a predictive, rather than descriptive stat - so Fangraphs pitcher WAR reflects what a pitcher should've done - instead of being credited with what they actually did - this is the FIP formula - ((13*HR)+(3*(BB+HBP-IBB))-(2*K))/IP + constant* - theoretically removing luck - you'll see no mention to hits given up. In 2010 Trevor Cahill had a very low BABIP and gave up relatively few hits, (hence the low ERA) but struck no-one out - thus his FIP is rubbish, resulting in a low fangraphs WAR (and the K:BB ratio fans out there will be excited to see that K's and BB's are factored into FIP)

*the constant adjusts FIP to put it on a scale similar to ERA

In contrast, Baseball Reference WAR is calculated from the number of runs a pitcher allows As adjustments are made for whether he gives up more or less runs than the average pitcher playing in front of his defence, Baseball Reference WAR gives a pitcher credit for 'luck'/preventing hits/runs, and penalizes a pitcher for giving up hits/runs, and therefore is more descriptive than Fangraphs WAR. It also takes into account quality of opposition faced - whereas on Fangraphs, they tend to mock the idea that putting up good stats in the AL East vs AL West should weigh into Cy Young discussions.

There is a discussion on Fangraphs here, justifying their approach - both are valid, I guess, but it seems perverse to credit batters for luck, but penalize pitchers for luck when calculating what is supposed to be the same stat.

To be honest, if I were going to introduce a stat called 'Wins Above Replacement' and calculate it on a year-by-year basis, I'd credit a player for his actual performance that year, not what he should've done if he hadn't been lucky/unlucky. But as it is, both Fangraphs and Baseball Reference WAR can be useful. If you want to know what a pitcher actually contributed in a given year, use Baseball Reference WAR. If you want to know what they are likely to do this year (perhaps more fantasy relevant) use Fangraphs WAR - if FIP is useful for calculating future performance. We'll cover how to predict future performance in some exciting upcoming posts - with fancy graphs and original analysis.

And finally, the WAR-related video of the day is an '80s classic:

Quality Starts > Wins

Rob Neyer on Quality Starts:

http://www.sbnation.com/mlb/2011/2/13/1991203/running-down-quality-starts

in this 2007 piece at Baseball Prospectus -- that I reported that in 2005 all the QS added up to a 2.04 ERA ... and the non-QS, 7.70. Not that Quality Starts mean anything. Not at all.

In other science/baseball news, Josh Beckett marries a Rocket Scientist. Maybe she can help him better locate his pitches, or alternatively track the trajectories of all the home runs he gives up?

And truly, this is a Quality Start:

WAR - What is it good for?

Last season in baseball......2 pitchers of equal 'worth' - I give you fangraphs 69th and 70th (of 92) most valuable (by WAR)  qualified starting pitchers of 2010:

James Shields - 2.2 Fangraphs WAR but a disastrous fantasy starter - 5.18 ERA, 1.46 WHIP (#99 in our league)

Trevor Cahill - 2.2 Fangraphs WAR and a stud fantasy starter - 2.97 ERA, 1.11 WHIP, i.e quite excellent (#18 in our league).

Surely some mistake? How can this be?

Cahill's K:BB ratio was 1.87, Shields 3.67 - could this help us answer our conundrum?

To be continued..................

And open for comments