** The Marc Pelletier Challenge
Part II - The Pelleiter System
dealing with Pitchers **

Introduction

Last month, I sketched the basis of a WAR-based evaluation system for Strat (WAR=Wins Above Replacement level). In summary, WAR-strat is determined by taking into account all contributions that you can find on a Strat card:

 (Offensive Runs + BaseRunning Runs + Fielding Runs + Positional Adjustment),  sum of which must then be:

·         adjusted for the expected playing time of the player,

·         adjusted for the replacement level of your league,

·         and converted into wins.  

Finally, I went to great length to explain how weights should not be perceived as static. A run is a run is a run, but singles are not created equal. A two-out single with bases loaded has more weight than any other type of singles, so you would be wrong to attribute the value of 0.47 (the average weight of a single) to a clutch single — this is why many rating systems that I’ve seen around in Strat typically underestimate the value of clutch.

The higher the scoring environment, the more valuable singles and doubles are (the opposite is true in low-scoring environment) and the reason for this is because the run expectancy related to singles and doubles increases in a high-scoring environment --- they have greater chances to produce runs. This explains why the weights for clean-up hitters are usually higher than hitters at the low end of a lineup --- they hit in a high-scoring environment. This explains why on-base pays so much more in Coors Field than in the Marlin's stadium whereas; I should add that the value of homeruns is more constant throughout environments: its value is always 1 run plus something (the contribution of advancing runners), and while the value of that “something” might fluctuate, since it can’t go lower than 1 run, then it has less room to fluctuate. I’ll illustrate this effect with an example shortly. 

Scoring environment and pitching

I now turn to how should we best evaluate pitchers. The easier way to measure the value of pitchers would certainly be to use the very same linear weight formulas that we used for the offense, multiply that by the chances that you have on the SOM rating file that is based on the pitcher’s card, and finally consider usage.

This said, you might realize that what I wrote earlier about the importance of considering the scoring environment will have an important implication for evaluating the pitchers. Pitchers are among the greatest contributors of settling a high-scoring or low-scoring environment. A bad pitcher who allows a lot of on-base creates a high-scoring environment by himself, whereas Arrieta's card creates a low-scoring environment with such low on-base chances that are on his card. So if an average single has the weight of 0.47, what weight should we use for Arrieta? Surely lower, but how much lower? 0.40?

But on the surface, it sure looks like a better rating system for pitchers should assign to each pitcher its proper linear weights generated by the scoring environment the pitcher creates. While this looks like an impossible mission, in fact, it can done relatively easy thanks to the availability of Excel files that integrate a mathematical function called Markov chain. The most accessible file I found is one created by Steve Staude, on fangraphs, which you can download here: http://www.fangraphs.com/blogs/team-specific-hitter-values-by-markov/. In fact, this spreadsheet is capable of not only generating specific linear weights for each pitcher, but it can also generate for each pitcher the (Markov) Expected Runs per game which can more or less be interpreted as the expected ERA. However, the downside of using this approach is that it works properly only if you project the ratings of a pitcher on 216 plate appearances --- so you need to estimate what a typical offense (in your league) will do in 108 chances, what a typical defense will allow on the 30 chances found on the X-chart, and add those value to the ratings that reflect the pitcher’s card.

I did this exercise with the new 2015 rating file. Here is the output for two pitchers, the 2015 ace Arrieta and journeyman Feldman:

--------

Here is the top value for SP and RP (based on a neutral stadium, this is based on RA-adjusted, and not WAR, which must integrate usage, the WAR rankings will come later):

 The numbers appear high, but they include both earned and unearned runs (defensive miscues coming from an average defense were included as hits allowed).

When I compare this ranking with the one obtained more traditionally, using the same weights for all pitchers (e.g. 0.47 for singles), the changes are more subtle than I expected.  Most pitchers change rank by one or two spots. The impact is more important for only a few pitchers. The biggest “loser” among the potential top 50 starters is Texas’ C. Gonzalez, who goes form 32nd on the traditional ranking to 39th (that’s roughly equivalent to losing 0.15---going from 5.00 to 5.15). Among relievers, the impact is slightly more important: Brewers’ Rodriguez falls back to the 10th best card and Houston’s Harris to 13th best card---losing both 5 positions. Garrett Cole is a beneficiary of the new ranking, so in his case, it might be explained by the combination of a great gbA numbers and the relatively higher on-base as compared to pitchers of a similar value.

Pitchers usage

Those rankings I shared above are solely based on the value of the card (supplied by league average), but they don’t consider usage. A 2-inning reliever who can close might provide more value (and get a better WAR estimate) compared to a 1-inning reliever with a better card.

Usage is in part influenced by the endurance rating of the pitcher. A 7-inning starter (S7) is likely to pitch more than a 6-inning starter (S6). This said, usage is also determined by the quality of the pitcher. Pitchers who allow a lot of on-base are more likely to reach their fatigue point (or induce the countdown of the SADV {Super Advanced} fatigue system in the online game), which will limit the number of innings pitched. Slugging can also have an impact, especially in inducing fatigue based on the 5-run inning ruling. And of course, if you play with the same rules as the online game, so-called “star” pitchers will have more starts than “regular” starters. Taking all together, Arrieta is the pitcher expected to pitch on average the highest number of innings with 330 innings:

Relievers also have an endurance rating (form R4 to R1) which will influence the number of non-fatigued expected innings, and they have a closer rating, which allows more opportunities (in the 9^th inning to pitch in non-fatigued mode).  I should add that these inning estimates are useful only if you can estimate how many innings are squared in 216 plate appearances. Since the usage rules in the on-line game are rather relax, there are 10 relievers who are expected to pitch 160 innings or more:

Leverage index

A final issue that can have an impact on WAR is the leverage index (LI), which is an attempt to quantify the probability that a reliever is used in games that count rather uselessness situations. Closers and set-up men are usually more involved in tight situations compared to a 7th or 8th reliever who job is to mop-up situations. In real-life baseball, by definition, starting pitchers have a 1.00 leverage index, closers have a LI between 1.75 to 2.2 and set-up men have a LI an index around 1.5.  The net impact is important: if a reliever saves 20 runs per year compared to replacement player, and his leverage index is 1.5, the net impact will be closer to 30 runs (20 X 1.5), although I must add that, as I write this, I am still trying different solutions to make sure I don’t overestimate by this index the impact of relievers. I kinda believe that relievers that can throw up to 150 innings are probably throwing in more useless innings compared to real-life set-up men and closers who are limited to 60-70 innings.

Conclusion

You’ll find below a provisional account of the top WAR for pitchers. As I am still struggling to incorporate accurately the value of relievers, I prefer for now to keep the WAR estimation separately for starting pitchers and relievers with the WAR for relievers are probably overestimated. This being said, it is obvious that the online game allows the exploitation of the super-reliever strategy and there is no doubt that this strategy is successful. So I’m leaning towards thinking that the WAR estimation for super-relievers might not be far from reality.

Perhaps this is a platitude, but you should bear in mind that these WAR values don’t add up within a team. If you pick up 4 super-relievers who's worth is above a value of 3 WAR, your team won’t get 12 extra wins, since a few of those super-relievers will mostly be involved in games with low leverage, doing a mop-up role. The WAR estimation with regards to relievers is closely linked to the specific roles of a closer and the set-up man. It represents more a potential rather than an actual prediction. Of course, if you play the online game, you must also control with sufficient ease the bullpen settings to have the proper pitcher play the role you have planned. The interaction between the right bullpen and the rotation must also be considered.

Furthermore, I should repeat something I wrote in the last article, which is that, although my ratings could be adapted for any kind of league, they are definitively set for the 80 Million on-line game. This is clearly apparent in the high ranking of lefty pitchers. In the 80 Million on-line game, expensive lefty crushers are most often left on the bench as Strat players feel that they don’t play enough to justify their salary. As a consequence, the league average vs lhp is almost equivalent to the league average vs rhp. In face-to-face leagues with no salary restrictions, league average vs lhp is usually better than league average vs rhp, and this lowers the value of lefty starters.

There are several issues I didn’t talk about which are more technical: how to estimate the impact of holding, balks and wild pitches, which could take several paragraphs; plus, in the process of estimating WAR, how to go from a system based on 216 PA to a system based on innings pitched (the transition is more complicated than it seems). As I intend in the future to make available my spreadsheet with a full example of a player and a pitcher being evaluated, these details may be provided to whom it may be of interest.

Provisional ranking of pitchers by WAR (usage included)

 
(Note, while I don’t expect the rankings above to be changed, the scale on which WAR is based might evolve more ...)

Marc Pelletier


If any members of our newsletter would like to speak to Marc directly, you can reach him at his email at: marcalain.pelletier@gmail.com )

-----------------------