Offense vs. Defense article

[MARCPELLETIER]

Hi Maligned,

I understand and agree and any linear-weight based system (LWBS) needs to make an adjustment for the problem you and bbrool raised, but I believe that a simple adjustment could fit the problem, or at least provide a sound approximation of the value generated by the extra at-bats.

In my opinion, though, in all due respect, I think you make the problem more complicated than it seems. For example, I don't believe the issue has got to do with linear vs multiplicative evaluation. I don't believe so because, on a team basis, any LWBS can perfectly predict the run production (when predicting an offensive team performance, we know what the players did with the extra PAs).

The issue is about how to attribute the value of the extra PAs to the guy with more on-base.

At first, I thought that playing with the outs value could do the job, but I realized afterwards that, if such solution should work, the value for outs would have to change from players to players depending on their on-base, and obviously that makes things complicated.

Here is a little thought experiment. Imagine that we have nine players whose NERP are exactly 108 runs for a season of 648 PA (0.166 RUNS per PA). Imagine that we take away 16 outs from the ninth hitter to replace them with 16 walks.

The value of this ninth hitter has increased of 6.83 NERP-runs.

Let us say that the lesser 16 outs generated 24 PA (16 outs, and 8 hits/walks, roughly a 0.333 on-base percentage). To simplify things, let's imagine that all the extra at-bats generating by this ninth player are equally shared by all eight other players, and that the production per PA is not changed. Overall, the value of the eight other players is still 108 RUNS per 648 PA, but we must now calculate their production over 651 PA.

When running all numbers, we get that the team, as a whole, generated 10.83 more RUNS. We have already attributed 6.83 more RUNS to the ninth hitter, so there is an extra 4 runs to be attributed. These extra runs cannot be attributed to the eight other players, as their value hasn't changed. From a strat point of view, the value of their cards has remained at 108 RUNS per 648 PA, so it has to be attributed to the 9th hitter.

So, in need for terminology, let's call the 6.83 extra runs generated by the ninth hitter as the explicite value of the player, and the 4 extra runs as the implicit value of the player.

So the issue is real. The question is then: is there a way to change formulas in order to include this implicit value generated by a player.

I believe there is. In my example, I worked in an environment that produced 6 runs per game (108*9=972/162=6), or 0.666 runs per inning, or 0.222 runs per out. So simply calculating 0.222 runs per out X 16 out = 3.55 runs (the reason it undershoots the target of 4 runs is because the way I set the example: by posing that players generated 4 runs with the extra 24 PA, I implicitely assume that their slugging would be greater than expected...had they performed fully according to their average, the eight players would have generated 3.55 runs, and not 4 runs).

Rather than working with examples, we could work with averages. In today's baseball, there is approximately 0.555 runs per inning, or roughly 0.185 runs per out. The average on-base is .340. If a player has a .400 on-base per 216 PA, then he is likely to generate 13 fewer outs per 216 PA than the average player. We can then attribute the implicit value of 0.185 X 13 = 2.4 runs per 216 PA to the 0.400 on-base player.

So all Dean or any other person working with linear weight based system needs to do is add to the formula:

(number of outs generated by a player per 216 PA - number of average outs by a player per 216 PA) X 0.185 X K

where K is a multiplicative term that takes into account the stadium. In normal settings, 1-8/1-8/1-8/1-8, K=1.

[maligned]

I think if I showed you the spreadsheet I created about 18 months ago for the 2007 cards, you would agree it's not that complicated to figure team context the way I suggested.
With that said, I agree that your method is much simpler and easier for anyone to apply. I followed what you laid out and your simple process for giving essentially a bonus or penalty for greater or lesser OBP in predicting future Strat results is excellent.
I'm still not comfortable with the phraseology that NERP "predicts" results. This is misleading. NERP takes raw data and tells us how many runs a team scored in the past. It is brilliant for this. But we still have to consider a standard number of outs as our basis for prediction of future results.

Again, your recommendation does this wonderfully.

[MARCPELLETIER]

Thx maligned. I guess I should say that NERP estimates the value of a card, rather than "predicts results".

I must also add that I have some other issues specific to the NERP. I certainly disagree with a formula that gives a greater value to a SB (0.16) than to a CS (0.09). Not sure how Paul Johnson got away with that, but the general consensus is to attribute to the CS at least twice as much as the value attributed to the SB.

Perhaps one reason why he ended up with these results is the consequence of not taking in consideration the running ability of teams. Generally, teams with high CS numbers are also faster, and running ability generates runs by itself. If one gives the proper value to CS without taking in consideration stats related to running ability, then you might end up underestimate the offensive performance of these teams.

Just guessing, though.

[maligned]

Yeah, I follow you. I prefer to use NERP independent of the SB/CS components in the Strat context. Instead, I simply use the rest of the formula, then give small bonuses for stealing and running ratings based on other data. This obviously throws off the NERP result a bit; but in the end, I'm just looking for a final rating value that can compare players the most effectively (Player A is 5.5 runs better than Player B) rather than tell me exactly how many runs a particular player will contribute.

[Mean Dean]

We might well be arguing over semantics at this point. I just don't think that the distinction you're drawing -- claiming that ERP "predicts how many runs a team scored in the past" whereas your solution does something totally different than that -- is a meaningful one. If your team's OBP is within the ranges that actual team OBPs have been in throughout history, then the value of the out that ERP is using is about right. The fact that the value of the out does vary depending on the run context you're playing in (which, BTW, includes more than the other guys in your batting order...) doesn't make ERP meaningless. It does make it less precise than the improvement that you're suggesting... with the corresponding advantage that you don't have to re-calculate the formula for every team you play as. For purposes of my article, which is designed to be as user-friendly as possible, I value the latter over the former. But, especially if you are comfortable enough with spreadsheets, you might well value the former over the latter, which is ok too.

[MARCPELLETIER]

[quote:7235bd3a09]with the corresponding advantage that you don't have to re-calculate the formula for every team you play as. [/quote:7235bd3a09]

Dean, with my solution, which seems to satisfy maligned, at least as a good approximation, you do NOT need to re-calculate the formula for every team or every player.

You just take your NERP formula of your choice and you add:

(number of outs per 216 PA on a players'card (ajusted for pitching/defense, just like you already do) - number of outs by an average player per 216 PA) X 0.185 X K

where K is a multiplicative term that takes into account the stadium. In normal settings, 1-8/1-8/1-8/1-8, K=1.

Also, the problem raised by maligned does not simply apply to teams with extraordinary on-base: it also (and most importantly) affects ratings of individual players with outstanding on-base. And such players are usually the ones around whom we build our teams, Babe Ruth in ATG4, Bonds in the 200X era. I find it extremely valuable to have a rating system which could tell me that Ruth + cheapy 3b + cheapy ss is better or worse than, say, 21M equally spent on 3 very good players.

BTW, since I posted about this discussion, I looked up at sabermetrics sites, and I found out that the problem raised here is one known to affect formulas such as NERP in sabermetrics circle. One solution I read was to add roughly 0.12 X PA for every additional PA generated by a player. If you break down the probabilities, it comes pretty close to the suggestion posted above.

[maligned]

Right, again, I'm not questioning at all the value of the out in the NERP calculation. I'm questioning how we value in a rating system the "implicit" run production (using Lucky's word choice) variation between different players. NERP still works perfectly to take raw data and tell us how many runs a team scored. It is not faulty at all in this regard.
By simply considering performance per 216 PAs, we're still doing nothing different from my oversimplified example of the 4 PAs of the .500/.750 hitter vs. the .250/1.000 hitters. It has nothing to do with any problem with NERP's numeric value of outs; it has everything to do with wanting to predict how those two players' teams will perform in their 4,374 available season outs...knowing that PAs are completely variable based on outs recorded.

Dean, I need to mention again, also, that I really appreciate all your work on the paper, the theory, all of it. I feel bad with some of this discussion because it could seem like I'm somehow trying to undermine your very hard work. This is not the case at all. As much as anything, I've been trying to wrap my own mind around how effective ratings should work.
I used your ratings ideas 18 months ago, felt like they were valuing SLG too greatly (in both the offensive and defensive contexts), and started researching if there could be a reason. I'm telling you that your final ratings will provide some significant shifts for high and low OBP guys when you incorporate an element that rates based on how a team functions--not based on per/PA values.

[MARCPELLETIER]

Dean,

I just saw your posts on baseball fever, with your observation that ERP works better than NERP. You wonder why ERP works better than NERP.

I would suggest, just a wild guess, what I wrote in a preceding post: the NERP has the "correct" (standard) value for SB and CS (respectively 0.2 and -0.33), and does not take any other running stats in consideration, so the formulas are underestimating the value of faster teams.

It's only a guess, but if my guess is correct, you should see that RMSE of NERP gets closer to ERP once you don't consider sb and cs.

Also, there is one thing that came across our discussion, but could perhaps be misinterpreted.

In the Excel file, when you check if the formula works fine to estimate the run production of any given team, you should NOT make the adjustment I proposed above (it will increase the RMSE).

The adjustment I proposed is only intented when evaluating an INDIVIDUAL card.

And the reason you need to make that adjustment is because your ERP formula is intended to measure run production with OUTS intended be equal (across team), whereas your evaluation of individual cards is assessed with PAs (not OUTS) intended to be equal (across players, all have 216 PAs).

So this is why you are missing the "implicit" value of players with strong onbase. My proposed adjustment enables you to make the translation from one system to the other.

See also this discussion on the same topic.

http://www.insidethebook.com/ee/index.php/site/comments/reconciling_linear_weights_and_runs_created/

[MARCPELLETIER]

[quote:cfa6b5f388]
I just saw your posts on baseball fever, with your observation that ERP works better than NERP. You wonder why ERP works better than NERP.
[/quote:cfa6b5f388]

Hey Dean,

Sorry, me again, but forget about my suggestion. After looking in depth to your Excel file that I could downloaded from the baseball fever forum, I realized what caused problems.

When people use the sabermetrics formulas to evaluate past performance, they always used an adjusted value for outs. The value of outs, over a period, is always adjusted for that period.

In your Excel file, you took the values for outs as found in the prints. But those values worked for periods covered in those prints. They are not adapted for the more "offensive" era of 2000-2007.

After adjusting the formulas to have the rightful value of outs, I end up with BsRW being the best indicator (the least MSRE), followed by NERP, which was better than ERP. All indicators, except for RC, were in the low 20s, as it was predicted by tangotiger.

So NERP does a better job than ERP, after all, which is not surprising given the wrongful values of CS/SB provided in ERP. In fact, I would have expected the difference to be greater, which again probably says something about the value of running not being assessed by linear weights formula, which tampered the real difference between the value of SB and CS.

[childsmwc]

I saw you guys dusted off this thread. I can tell you with certainty that the pricing for all strat cards includes an incremental average run per out saved as you have described Lucky, that is exactly how you address the issue but keep it simple. The average value that was used was .38 additional runs per out either positive or negative off of the "average" player. I would have to dust off where this number came from, but it is probably based on the strat average environment and not the average MLB environment.

I also initially had a problem with Paul Johnson's formula for SB and CS factors, until I additionally layered on the value of outs for reduced team PA's because of that CS. Once you consider that the CS has a negative value and the out has a team consequence as well, I found that a base stealer in his formula had to be about 66% effective to have a positive contribution in this area, which is in line with most sabermetrics.

In pricing the cards, outs include outs made, plus CS, plus an additional out for expected DP's. This is compared to an average players out consumption and the +/- *.38 is layered onto his NERP.