** The Marc Pelletier Challenge
Part I - The Pelleiter System
dealing with Hitters **

Introduction

Wolfman asked to write a few words about my rating system. I’ll focus in this article on a few issues, but I won’t go through all the formulas, as it will be too laborious. For those among you that would like to implement my rating system for your use, Wolfman and I will find a way to convey the information---more details will be provided in the February edition.

My system is based on WAR, which stands for “Wins Above Replacement level”. Most of you are probably familiar with WAR or at least heard about it. Fangraphs defines WAR as “an attempt by the sabermetric baseball community to summarize a player’s total contributions (…)  in one statistic”. I like the idea of building my system on a meaningful and straightforward statistic. So contrary to other ranking systems used in Strat that I am aware of, my system is not based on arbitrary units.

In theory, my rating system could be used for any league. In face-to-face leagues with no salary cap, you could use the system to say for example “I should get this rf over that ss because the rf is 3.2 wins above replacement level whereas the ss is only 2.5 wins above replacement level”. In the on-line game, in leagues with salary cap, I put a pricetag for each WAR, so this allows me to see which players are the bargains. This said, leagues have different rules, and so some parameters that I use in my rating system are specific for the on-line game. For example, in the online game, about 75% of the single “one-base” or the single “two-base” that you see on the card are transformed into single (of), involving a decision coach to send the runner or not for an extra base. So speed has more impact in the on-line game than in face-to-face leagues. But of course, the parameters that are set for the on-line game could be adapted to face-to-face league rather easily.

WAR-s (WAR for Strat)=

(Offensive Runs + Base Running Runs + Fielding Runs + Positional Adjustment),

the sum of which will be then adjusted for replacement level and for expected playing time, and then convert this value into wins. To illustrate, here are the numbers for the 2014 Anthony Rendon card:

The limits of traditional linear weights formulas

So the challenge in calculating WAR is first and foremost to calculate the sum of runs a player will contribute to his team. The core way to perform this is by using a linear weight formula. There are many linear weights available in the sabermetrics market, with different pros and cons. The one that was chosen by fellow on-line Strat player, Dean Carrano, is the New Estimated Runs Produced (NERP) (Dean’s article is available on this website: http://www.mfooz.com/bblog/wp-content/uploads/offense-vs-defense.pdf). While Dean expresses his formula differently, the NERP formula is mathematically identical to this:

= BB/HBP* 0.33 + SI*0.48 + DO*0.80 +TR*1.12 + HR*1.44 - OUTS*0.085 - GIDP*0.418 + SB * 0.2 - CS*.333.

First, they fail to specify values for events specific for Strat. Strat is a very close approximation of baseball, but it has also its own logic. For example, the NERP formula provides a weight for GIDP, but gb(A) is not exactly GIDP. In some occasions, gbA has some positive contribution, for example by advancing a running from second to third with a gb(A) hit to the first baseman. In the NERP formula, outs have the same weight, but gb(C) and fly(B) are more valuable than gb(A). Single** are more valuable than single*, but it doesn’t show in the NERP formula.

Second, many linear weights formulas such as NERP are “context free”, meaning that they give the same weight to all events regardless of when or where they occur and regardless of the “scoring environment”. All singles for example are given the value of 0.48. But in fact, the value of a single fluctuates depending on whether they are no outs or two outs, with the men on base or not, or even depending on the ballpark or the quality of the pitcher.

The best example of this is of course clutch singles (when a horseshoe (Ω) (or a $ on the on-line card) transforms an out into a single**). Clutch does not appear in the NERP original formula, but even if we were to estimate its value based on the linear weights provided by the same NERP formula, we would inaccurately estimate its value by a good margin. Specifically, a clutch single is not worth 0.48 runs (the value of a single) or even 0.58 runs (if we add the value of an avoided out). Common sense tells us that these values are manifestly wrong since a clutch single ALWAYS generate a run, in absence of which you wouldn’t score (I leave aside the case that, in some versions of the game, a runner can be thrown out on a clutch single).

Actually, clutch singles sometimes generate two runs, and someone they open the door to a big inning that would have not been possible otherwise. Scoring environments also influence the weights. Sabermetric analysists have demonstrated that the weights fluctuate whether you are playing in a low-scoring environment or a high-scoring environment. Walks, hits (mostly singles and doubles) and outs are particularly affected by this phenomenon. In a low-scoring environment, their respective values tend to get closer to zero whereas in a high-scoring environment, their values tend to get closer to one. (see www.tangotiger.net/customlwts.html to see how weights fluctuate depending on how many runs are expected. For the value of outs, please refer to my word of caution later in this article.) So weights are affected by the ballpark you use, but more importantly, when it comes to pitching evaluation, weights change depending on the quality of the pitcher. A bad pitcher generates by his own fault a high-scoring environment, and an ace pitcher generates a low-scoring environment. So weights have to be flexible enough to reflect this. In using a traditional linear weight like NERP, you can’t.

The Run Expectancy method

The solution is to find a way to create weights on our own, based on baseball analysis, but specified to each Strat event and specific to the Strat environment. One method that allows this solution is “the run expectancy method”.

I’ll explain the method in a few words for the most curious among you, but if you are simply interested in knowing the weights of each event, you can skip this paragraph and jump to the next. Basically, you start with a run expectancy matrix. You can freely download the matrix that has been specified for the 2015 season on the Baseball Prospectus website (look for Run Expectations).

(Key above for Men on Base - each of the three numbers represents either:
0 - no man on 1st, 2nd or 3rd; 1- man on 1st, 2 - man on 2nd, 3 - man on 3rd)
 

Calculating the weights of gbB and gbC are kinda tricky, since the rules governing when the player scores from third base vary on a number of parameters, including whether defense plays in or not. But for the sake of this exercise, I assume that a runner on third never scores on a gbB and always score on a gbC. 

As for clutch, a clutch single is worth 1.42 runs (on the positive or negative side, depending on whether an out or a hit is inverted), quite higher than the value of a “normal” single. But be careful to not multiply this value with the clutch chances: the horseshoe (Ω) (or $ for the on-line version) comes into play by inverting the result only in clutch situation, which happen roughly 12% of all at-bats. So the net of value of having a positive chance of clutch is 0.17. The same is true of pb/wp/bk---a pitcher having a pb-7 does not mean that he will allow 7*0.29 runs= 2 runs per 216 chances—it depends on the frequency of the event. But after you completed a season, if you know that Mr. X has allowed 7 passed balls, and then you could assume that this has roughly cost 2 runs to your team.

Of course, these weights are useful if you know, for any individual card, the chances for each event. If you are playing historic seasons, or the online version called ATG8, a rating file with over 4000 players can be downloaded on the diamonddope website which contains most of these ratings. I also believe that a script called “Grease monkey” provide some of these ratings online.   Unfortunately, if you play 2015, these numbers are not easily accessible, since the SOM rating file doesn’t give them straight-forwardly. Of course, you can read the cards individually. There are also ways to tweak formulas to roughly estimate and sometimes deduce the chances of singles, doubles, even sometimes the combined chances of gbC and flyB on individual cards (to give an example, Soler vs rhp does have more than 3.7 chances of gbC/flyA/flyB combined). But the following formula is more general and can be used as a general sketch:

0.17*H + 0.30*TB + 0.05*HR + BB/HBP*0.33 – 0.18*gbA – 0.11*K  –0.04*gbC/flyA [if available] – 0.10* (all other outs)  + 0.17*clutch + stadium adjustment for ballpark single + stadium adjustment for homeruns + “weak” adjustment + lefty/switch/righty adjustment

This formula is fairly close to the NERP formula, but it’s more precise and it handles clutch and gbA better. But whenever I have the chance, I use the weights for each event, particularly when it comes to evaluate defence, where you know by looking at the charts, the probability of getting gbA/gbB and gbC (or what SOM calls G1/G2/G3 in the super-advanced charts). Well the weights provided here stand for a typical 20XX season, it’s obvious that the same run expectancy matrix can be used in different environment (Coors Field, or at Forbes against all-time great pitchers).

The impact of a player usage for the Offensive Run contribution

As you probably all know, there are 108 chances on a Strat card, and these chances are usually responsible for 50% of the results from 216 plate appearances (PA)- or 216 rolls (pitchers’ cards and defensive-X chart make up for the rest). At this stage, it’s very tempting to just multiply by 3 (or some factor close to it), and assume that what you get is the offensive contribution a player for a full season or so.

Intuitively, this strategy makes perfect sense, but it’s wrong.  Doing this is possible only if you take for granted that the number of rolls a full-time player will have over the course of a season is constant for everyone, other things like injuries being equal. In truth, the number of rolls will vary by two factors that are not equal among players: what on-base a player has, and what position slot this hitter holds in a line-up.

Let’s first discuss about the variation caused by on-base. Every team disposes of 162 games X 27 outs to win ball games. So what is constant is that every team will have 4374 outs or so at its disposal in a full season (give or take some for extra outs for overtime games and unnecessary 9^th innings when you win at home). If team A has a .400 team on-base, and team B has a .300 on-base, both team will nevertheless finish the year with 4374 outs.  However,  team A will have many more PA than team B, which means owner A will roll the dices many more times than owner B until both teams reach 4374 outs. And since teams are made up of players, players with higher on-base are the one responsible for the extra value generated by rolling more dices.

Consider two players with the same run production as estimated by computing the 108 chances that are present on their cards—let’s say a slugger, Mr. Cespedes and an on-base machine, Mr. Vitto. So we are assuming that both Cespedes and Vitto are worth 30 runs (based on 108 chances, or 50% of 216 rolls). Even though they have the same offensive value after 216 rolls, we know that Vitto will generate fewer outs than Cespedes, since Vitto on-base is better by, say, 20 chances. If Vitto has 20 lesser outs per 216 chances, the team that employs Vitto will not reach 4374 outs at the same pace than the team that has Cespedes. In fact, if the Vitto team has an on-base of 0.350, it will take slightly more than 90 PA to reach 4374 outs compared to the very same team that has Cespedes instead. By plugging in league average value (assuming here a 80M league on the online game), the extra on-base that Vitto generates will give to his team almost 10 extra runs. If you don’t adjust for Vitto’s on-base, your seasonal run prediction will be underestimated by 1 full win. Of course, the full impact of this adjustment will depend on the type of league you are playing in, but you get the idea.

The other issue is lineup positioning. Players at the top of a lineup will always have more PAs than players at the bottom, simply because their chances to getting an extra at-bat is higher. Assuming that a full season represents 688 PA per player, we can expect the following PAs per batting order:

1 ... 752 (almost 10% more than the 5^th slot)

2 ... 736

3 ... 720

4 ... 704

5 ... 688

6 ... 672

7 ... 656

8 ... 640

9 ... 624 (10% less than the 5^th slot)

A lead-off hitter is likely to have roughly 140 additional PAs compared to a 9^th hitter, and 70 PAs compared to a likely 5^th hitter. If that lead-off hitter is worth 6.0 offensive WAR based on an estimated 688 PA, a simple multiplication based on the rule of three suggests that the true contribution might be 6.5 WAR.

That being said, the number of PAs (or of rolls) is not the whole story with regards to lineup positioning. Common sense tells us that your most important hitters should hit in the 3^rd or 4^th holes because these positions have more important at-bats than any other positions, and this observation has been backed-up by sabermetrics. In other words, even though the 4^th hitter has 50-60 fewer at-bats compared to a lead-off hitter, the importance of these at-bats make up for the fewer at-bats. In turn, a lead-off hitter has more at-bats than any other hitter with bases empty, and hitting with bases empty has less impact than hitting with men on base, which cause the weights to reduce a little bit. So even though lead-off hitters do have 10% more PA than someone hitting 5^th, their true impact in turn of offensive run production is not as high, it’s more in the 5% range.

I should add that sabermetrics studies also showed that the 2^nd role is as important as the 4^th slot and might be even more important than 3^rd. The lesson here is that, when evaluating offense, a premium of roughly 5-7% should be applied to any hitter that is likely to hit between the first and fourth holes of a lineup: without that adjustment, the run value based on a Strat card will not reflect the overall run production over the course of a season. A player whose card is estimated at 6.0 offensive WAR and likely to hit in the first four slots will really contribute to 6.4 WAR over the course of a season. Conversely, players expected to hit down in the lineup should have a lowering of their offensive contribution by up to 10% (up to 8% in non-dh leagues). Of course, the logic has to be applied separately vs. lefty and righty pitching.

I’ll admit, it’s a small adjustment, and some readers might not like the idea that a coaching decision impacts how we assess a player’s overall contribution. But even though the difference between 6.3 WAR and 6.0 WAR may look small, I believe it does have some impact when you evaluate which player to choose. Imagine for example that you are hesitating between a gold glove shortstop who is likely to hit down in the line-up and a silver bat ss who has an average glove. You do your homework, calculate the offensive, running, and defense run contribution of both strat cards,  and you find out that it’s a wash, both are valued 6.0 WAR. In fact, the offensive player is the better choice---his usage in the top four slots of a lineup will drive his contribution over the gold glove.

The same logic applies the other way round when it comes to evaluate between two weaker players. For example, if you are hesitating between a gold glove ss with such a weak bat that his offensive contribution is negative (below replacement level) and a 3-rated ss who is likely to hit 6^th or 7^th, and if both have the same WAR based on the card evaluation, than the gold glove is likely to be the better choice because his usage at the bottom of a line-up will limit Between you and me, both decisions described here are commonsensical, and I think it’s a positive aspect of my rating system to reflect that common sense.

There are some limitations to my rating system with regards to calculating the offensive contribution: the weight I calculated assume that no outs are made on the basepaths (no out on base), which of course is not the case. With the "add baserunning decision" set on, this limitation affects all events except homeruns, so there is a case to provide more weight to homeruns, but I resisted to do it. Furthermore, I didn’t assign any value for bunting and hit-and-run. It’s obvious that players with weaker bats can upgrade their offensive contribution, especially with a hit-and-run rated B.

So the final yearly estimation for offensive run production will look like this:

Season offensive run estimation=

Offensive runs (based on 108 chances) * 3.1 + on-base adjustment + lineup positioning adjustment – league adjustment (to adjust for pitcher’s quality)

Summarizing the rest

I won’t get into the details of how I calculate the value of running and defense, but I do integrate the most important statistics: I consider not only stolen bases and caught stealings, but also catcher’s arm, pitcher’s hold, catcher’s overall defense, the probability for a runner to be held at first and the ensuing consequences (turning double-plays into singles), runner’s speed combined with outfielder arms and of course both range and errors from defensive players. To calculate these values, I use the weights above when I can, but many of the formulas are derived from simulating seasons on the Window games.

I shall perhaps conclude with a few words on the concept of “replacement level”. In sabermetrics, the replacement level player is defined by the contribution of a typical player in a .350 team or so. In no-cap leagues (and most face-to-face leagues have no cap, you just go with your best players), I find that this definition roughly reflect the value of a team that has to pick after every other team has made their draft picks. For this reason, I consider that a player is a replacement player is typically the X^th+20% best ranked player of any position, X being the number of teams in your league. So in a 12-team league, I usually take the 15^th best player at any position against each lhp and rhp (for more stability, I might average the value of 5-6 players, for example, the average value of the 12^th to 17^th best player combined). In 80M cap league, for the online game, I consider a player at 2.5M to be at the level of replacement.

The replacement level has two more functions. First, it serves as the basis to determine whether a player will be considered by my ratings as a platoon or an everyday player. If the value of a player goes below the replacement level vs either lhp or rhp, that replacement level is given to that player on his weaker side, which is then added up to the value on his strong side. Doing so neatly allows platoons to fit appropriately in a system ranking and to have an appropriate estimated price tag in the on-line version. Second, similarly to platoon players, the value of missed games due to injuries is also replaced by the value of the replacement level rather than by zero. This allows the rating system to correctly attribute the value of injury-prone players in no-cap league, where the quality level is much higher than in leagues with money cap, and of players used in platoons.