In an interesting week on fangraphs.com, Jack Moore published several interesting and probably long overdue articles on reliever leverage and how it pertains to their value as indicated by their free agent salary (Overdue generally, not from Jack Moore specifically). Through these articles, Jack Moore analyzes WPA as a possible measuring tool for determining reliever salary as opposed to the conventional $/WAR measures because relievers, unlike other players, largely have their leverage determined for them by coaching decisions. As a result, relievers signed to be put in high leverage situations by their coaching staffs probably have their value unfairly determined to be albatross by traditional measures.
With this new and probably superior way of thinking of high leverage relievers available to us, it makes sense to analyze reliever performance incorporating both expected performance in terms of WAR and in terms of leverage. Jose Valverde, in particular, is an interesting case from a number of different perspectives. Valverde has outperformed his expected FIP and xFIP over the course of his career by a considerable margin, one that probably cannot be ignored. In addition, Valverde’s actual value to the Tigers must be analyzed in terms of his leverage as well as traditional value measurements.
When you combine all these facets together, what you have is a big complicated mess as to Valverde’s actual value that I will attempt to sort through in an efficient matter if at all possible. Valverde has, at least in smaller samples, demonstrated an ability to somehow ‘survive’ 9th inning save opportunities while giving up the majority of his earned runs in non-save situations. Is it even remotely possible that this could be a skill? Does Valverde have a second gear when he is saving games? As I said before, Valverde is an interesting case.
For his career, Valverde has 242 saves to 30 blown saves. Over the last two years as a Tiger, Valverde has 75 saves to 3 blown saves, a ratio far superior to his career numbers. Valverde has increased his GB rate and his HR/FB rate is down however, Valverde’s K% has decreased while his BB% has increased. As a result, it is difficult to say that Valverde has shown any significant improvement as a pitcher even if his peripheral profile is a bit different.
Since Valverde is clearly a pitcher who is ‘changing’ since his arrival with the Tigers for good or bad, I will try to look only at his stats as a Tiger. Furthermore, I will assume at the beginning of this analysis that Valverde cannot actively change his walks, strikeouts and home runs based on whether he is trying to save a game or not. Valverde has pitched 135.1 innings as a Tiger with 132 K’s, 66 walks, and 10 home runs. While Valverde has largely outperformed his FIP and xFIP, he matches up very well with the more robust SIERA regression with a 3.07 SIERA as opposed to a 3.02 era career. Despite the discrepancy over the last two years, I think SIERA correlates well enough with Valverde that we can use it as an approximation going forward for Valverde’s expected performance.
Next we have to consider Valverde’s leverage. Unfortunately WPA (Win Probability Added) is entirely results based rather than regressed based on expected performance. In addition WPA does not consider marginal benefits (the over replacement part of WAR). Valverde’s cumulative WPA for the 2011 season was 4.168 and 1.31 in 2010. Valverde’s WAR over the same two year period was 1.6. As you can see, WPA gives credit to Valverde for over 5 wins while WAR gives him credit for under 2.
Valverde’s actual value to the Tigers over this time is probably somewhere between those two numbers. Valverde did add about 5.47 actual wins in probability form to the Tigers, but his replacement could have done feasibly almost as well. Further, as we can see from the numbers, Valverde had a 3 era in 2010 and a 2.24 era in 2011 as opposed to a 3.47 and 3.58 SIERA in both seasons. This makes it highly likely that Valverde benefited from some good fortune, good defense, or both.
If we first regress Valverde’s two year performance to his SIERA we get:
63inn/9inn = 7 x 3.47 = 24.29 runs; 72.33 inn/9inn = 8.036 x 3.58 = 28.77 runs + 24.29 runs =~ 53 runs in 135.33 innings
This result gives approximately .392 runs/inning regressed. This essentially provides a probability for Valverde giving up a single run in an inning. We can roughly estimate the probability of Valverde giving up additional runs by treating each run as an independent event per inning making the probability p = .392^N where N = runs.
When we consider leverage we really are only concerned with situations in which the score is close, possibly within 3 runs or so each way. The greatest gains in WPA occur when the score is tied or the lead is one run and Valverde pitches a scoreless inning. For the purposes of this study, I will analyze Valverde as if WPA is park neutral for simplicity. In this case, the home team at the top of the 9th inning has odds of winning of 83.8%, 92.7% and 96.7% for one, two and three runs respectively while they have a 50% chance of winning when entering tied but a 63.8% chance entering the bottom of the inning tied. For the away team it is 80%, 91% and 96% in the bottom of the 9th.
Now Valverde, the opposition, and the park environment will all have some impact on these win odds; an inherent flaw in this analysis. However, these numbers give a good approximation without getting too detailed.
Additionally a problem we have in this attempt is that this analysis would require normalizing leverage because the leverage Valverde actually experienced is not predictive. However since I am more concerned about Valverde’s value as depicted by WPA, WAR and my middle ground analysis, I will use his actual opportunities.
Between the 2010 and 2011 season, Valverde appeared in 14 one run away games, 9 one run home games, 15 tie games, 18 two run away games, 11 two run home games, 6 three run away games, 16 three run home games, 7 four run away games, and 10 four run home games. In addition he appeared in a couple eight inning two run 1 outs situations which are high enough leverage to matter.
I am now going to use a simple cheat to determine how much of his WPA he would be expected to capture because there is no precise mathematical formula to find the win probability at different run levels. I will discount Valverde’s WPA based on the probability of him capturing the positive WPA with an attempt to estimate the negative value of the blown chance added in (note there are additional levels of "blown" chances if he gives up more than the tying runs but probability is low enough that the subtracted value is not enough to play out, I suppose I could more accurately use some series to get the value but I decided not to go that far):
Away:
1 Run = .2 wp x 14 games x (1 - .392 SIERA) - .3 x 14 x .392 = .357 adjuted WPA
2 Run = .09 wp x 18 games x (1 - .392^2 SIERA) - .41 x 18 x .392^2 = .55 adjusted WPA
3 Run = .04 wp x 6 games x (1 - .392^3 SIERA) - .46 x 6 x .392^3 = .112 adjusted WPA
4 Run = .018 wp x 7 games x (1 - .392^4 SIERA) - .482 x 7 x .392^4 = .11 adjusted WPA
Home:
1 Run = .162 wp x 9 games x (1 - .392 SIERA) - .2 x 9 x .392 = .321 adjusted WPA
2 Run = .073 wp x 11 game x (1 - .392^2 SIERA) - .297 x 11 x .392^2 = .18 adjusted WPA
3 Run = .033 wp x 16 games x (1 - .392^3 SIERA) - .329 x 16 x .392^3 = .17 adjusted WPA
4 Run = .015 wp x 10 games x (1 - .392^4 SIERA) - .347 x 10 x .392^4 = .06 adjuted WPA
Ties = .138 wp x 15 games (1 - .392) - .138 x 15 x .392 = .44 adjuted WPA
Total Adjusted WPA = 2.3
As a result, when we adjust Valverde’s leveraged situation with some linear weight calibrations (SIERA) we can see his win probability added is much closer to his WAR although still slightly higher because of the fact that he is intentionally put in high leverage spots. According to the market that means Valverde was actually worth something around 10.35 million to 11.5 million over the last two years a/k/a between 5-6 million a season.
Now there are still two things to take away from this analysis. First, these are largely rough estimates based on all the factors I mentioned above. Second, the adjusted WPA is still NOT in Above Replacement or WPAAR (Win Probability Added Above Replacement) form. The key for the 'above replacement' adjustment is to determine the proper replacement in this case and then to discount that from the adjusted WPA value. In this case, we can probably assume Valverde would be replaced by the AVERAGE BULLPEN ARM. Keep in mind here I said, average bullpen arm and NOT minor league replacement player. This is because we are accounting for leverage in this calculation and Valverde’s leverage situations are likely to be overtaken by somebody with major league caliber talent. In other words, his replacement is already a major leaguer in the same bullpen, he’s just pitching low to no leverage situations. (This assumes the high leverage set up guy moves up to closer, and non leverage pitcher comes into high leverage spots. There is a leverage difference between setup and closer on down the line so this again is a rough estimate.) Third, I have not attempted to study whether Valverde could possibly adjust his performance based on the leverage at hand. Although I believe that to be unlikely, it is probably a scenario that should be considered in light of the fact that Valverde has not “failed” at anywhere near his expected rate.
If you then want to consider WPAAR, from 2010-2011 the average relief pitcher in the MLB has a SIERA ER/IP of .392! That is exactly the same number as Jose Valverde! As a result, when you consider leverage value, Jose Valverde has a 0 WPAAR (based on his own "chances" the replacement pitcher would have an identical WPA value)! Now based on how we have defined everything it is important to consider that a relief pitcher with a WPAAR of 0 still has a positive actual WAR. What is also says is that the average MLB relief pitcher should be able to get you a WPA of around 2 over two years, or 1 WPA/year. What this might indicate in Jose Valverde’s situation is that he might not offer you any value above the average reliever you could put in a high leverage situation such as the role of “Closer”.
The only thing I have yet to consider is whether Valverde might have the ability to better his production based on high leverage situations so as to be a positive WPAAR player.
No comments:
Post a Comment