The Playoff Odds - An Explanation and AMA
Aug 12, 2018 16:58:50 GMT -8
Nationals GM (Preston - Old), White Sox GM (Aidan), and 2 more like this
Post by Rockies GM (Dan) on Aug 12, 2018 16:58:50 GMT -8
Explaining the Playoff Odds Model
The podcast isn’t the first time I’ve heard/read someone from the league dismiss the playoff odds. Now, while I’m open to peer review, it does dishearten me to hear someone just say “I don’t trust them” out of hand without learning how they work and why they say the things they do. In the end, what that says to me, is I’ve been maybe a bit too proprietary of my work and methodology. So, in the spirit of transparency. Not only will I explain everything I can here, I will also open this up to any and all questions. If you wonder why or how I do something in this, please feel free to ask and I’ll respond.
Accumulation of Statistics
Before I get into the nitty gritty of simulation, let’s talk about where I get my numbers. At the beginning of the season, I pull all projections from Fangraphs Depth Chart projections, which attempts to combine ZIPS and Steamer. In the middle of the season, I add the Depth Chart RoS projections to the statistics already accumulated by the teams in the league (pulling them straight from the Standings>>Season Stats tab in the league.
I automated a process by which the program creates a Firefox browser, signs into Fantrax and grabs the .csv of each team available from the team’s home page. I then parse this list and check each and every player against the large Pitching and Batting projection .csv files gathered earlier. I double check this by listing every player I can’t find in the Java console. I’ve got methods in there for first names that don’t match between databases, and names with Jr., as that can throw off the simple search.
Once it pulls projections for all players, I modify those numbers. There is a loose correlation between number of AB/IP expected per week and the percentage of a stat that is accumulated (since you cannot play every player on your roster at all times). Those formulae are:
xRuns = totalRuns*(1.3939-.00447015*(AB/week))
xHR = totalHR*(1.44006-.00469675*(AB/week))
xRBI = totalRBI*(1.77222-.00720542*(AB/week))
xSB = totalSB*(1.38918-0.00462705*(AB/week))
xW = totalW*(1.03494-.002418843*(IP/week))
xK = totalK*(1.033395--.002332322*(IP/week))
xHLD = totalHLD*(1.097794-.00406*(IP/week))
There are 5 categories that do not adhere to this correlation:
- Saves – I count all saves. Generally, there is no historical drop off between the total saves accumulated vs saves counted. The reason for this is the fact that there are only 30 closers in the league, therefore players expected to earn saves are generally played every day.
And then there’s the rate stats, which don’t fit the AB/IP per week correlation. There’s no way for me to really guess who any team will use, so what I do is the following:
- Add up every players rate stats (AVG, OPS, ERA, WHIP) multiplied by their projected IP or AB, whichever the relevant stat
- Divide the sum of all players’ rate stats by the total IP or AB in order to get a good, weighted number
o The benefits here being that your hitting stars, more likely to get ABs count more towards the hitting stats
o On the pitching side, it’s a good way to weight the starters and relievers contributions towards the rate stats. If you’re the Astros, where your relievers accumulate a lot of innings, it’ll be affected thus. If you are starter heavy, like the Giants, that will be weighted
Finally, I add the expected counting stats to the already accumulated stats in the middle of the season. For the rate stats, the formula looks a little something like this (for AVG, for example):
- ((AccumulatedAVG*AlreadyPlayedWeeks) + (ProjectedAVG*WeeksToPlay))/Total Weeks
Finally, a note that if a team is not expected to accumulate the minimum IP/week or AB/week, I assign a BA/OPS of .000 and an ERA/WHIP of 10, to account for the automatic loss of those categories in Fantrax.
And that’s how all the teams get their stats.
Simulation
Now that we’ve got all of our statistics, it’s time to simulate. But to do that, we need to first sort and rank the teams by stats. Then, I assign a winning% to each stat, based on the winning percentage in league history of that position in the rankings (which had a strong-ish correlation). The number rank I use in the formulae corresponds to what you’d see on the Fantrax Standings>>Season Stats page (30 points to the league leader in that category; 1 to the worst statistical performer). These end up being the main numbers the program uses.
I have hard-coded the schedule into the program. When I start the program, it simulates the season week-by-week. Each game works the same way:
Team 1 has a winning percentage in the runs category of A
Team 2 has a winning percentage in the runs category of B
The accepted formula in simulations (Bill James’ Log5 Formula) is the following:
Winning Percentage for Team 1 = (A*(1-B))/((A*(1-B)) + ((1-A)*B))
Or, in other words – the chances that team 1 wins multiplied by the chance team 2 loses, divided by that same number plus the chances that team 1 loses multiplied by the chance team 2 wins.
Thus, if Team 1 wins the runs category 60% of the time and Team 2 wins the category 40% of the time:
W% = (.6*(1-.4))/((.6*(1-.4)) + ((1-.6)*.4))
W% = .36/(.36+.16)
W% = .6923
So, when these teams play, Team 1 wins runs 69.23% of the time.
In each and every matchup, I do winning % for each stat. Then, I do a simple random number simulation and come up with the winner for each stat in this theoretical game. I then record a win, loss or tie (using actual league tiebreakers)
I simulate the entire season game-by-game using this algorithm. I then determine playoff teams, breaking ties by league rules, put them into ArrayLists by seed, play the matchups and eventually come up with the world series champ.
Then, once that whole season is completed once, I do it 99,999 times again. That gives me a good enough sample size to eliminate random noise.
The Numbers You See
I get that seeing that the Colorado Rockies having a 95% shot at the division with 4 games left and a one game lead can look like I’m putting bias into the program, but that’s simply not true. Those numbers can seem callous and cold, I understand, but it’s based on specific games and likely outcomes.
With 100,000 season simulations, you get almost every conceivable outcome. True, the Washington Nationals aren’t going to show you a 0-21 season, but that isn’t even realistic in this fantasy simulation. Many different outcomes go into that average wins number, and even that doesn’t give you a full appreciation for the variety of the outcomes
FOR EXAMPLE, below you can see the win spread for each team by division if the season started today with a blank slate but with the same team rosters and accumulated statistics as it stands today.
How to Read These Graphs
The x-axis is the number of games won. The y-axis is the percentage that outcome occurs. In general, you want to be the team with the top-right line in your division (the right-most and most top-heavy towards the right).
The podcast isn’t the first time I’ve heard/read someone from the league dismiss the playoff odds. Now, while I’m open to peer review, it does dishearten me to hear someone just say “I don’t trust them” out of hand without learning how they work and why they say the things they do. In the end, what that says to me, is I’ve been maybe a bit too proprietary of my work and methodology. So, in the spirit of transparency. Not only will I explain everything I can here, I will also open this up to any and all questions. If you wonder why or how I do something in this, please feel free to ask and I’ll respond.
Accumulation of Statistics
Before I get into the nitty gritty of simulation, let’s talk about where I get my numbers. At the beginning of the season, I pull all projections from Fangraphs Depth Chart projections, which attempts to combine ZIPS and Steamer. In the middle of the season, I add the Depth Chart RoS projections to the statistics already accumulated by the teams in the league (pulling them straight from the Standings>>Season Stats tab in the league.
I automated a process by which the program creates a Firefox browser, signs into Fantrax and grabs the .csv of each team available from the team’s home page. I then parse this list and check each and every player against the large Pitching and Batting projection .csv files gathered earlier. I double check this by listing every player I can’t find in the Java console. I’ve got methods in there for first names that don’t match between databases, and names with Jr., as that can throw off the simple search.
Once it pulls projections for all players, I modify those numbers. There is a loose correlation between number of AB/IP expected per week and the percentage of a stat that is accumulated (since you cannot play every player on your roster at all times). Those formulae are:
xRuns = totalRuns*(1.3939-.00447015*(AB/week))
xHR = totalHR*(1.44006-.00469675*(AB/week))
xRBI = totalRBI*(1.77222-.00720542*(AB/week))
xSB = totalSB*(1.38918-0.00462705*(AB/week))
xW = totalW*(1.03494-.002418843*(IP/week))
xK = totalK*(1.033395--.002332322*(IP/week))
xHLD = totalHLD*(1.097794-.00406*(IP/week))
There are 5 categories that do not adhere to this correlation:
- Saves – I count all saves. Generally, there is no historical drop off between the total saves accumulated vs saves counted. The reason for this is the fact that there are only 30 closers in the league, therefore players expected to earn saves are generally played every day.
And then there’s the rate stats, which don’t fit the AB/IP per week correlation. There’s no way for me to really guess who any team will use, so what I do is the following:
- Add up every players rate stats (AVG, OPS, ERA, WHIP) multiplied by their projected IP or AB, whichever the relevant stat
- Divide the sum of all players’ rate stats by the total IP or AB in order to get a good, weighted number
o The benefits here being that your hitting stars, more likely to get ABs count more towards the hitting stats
o On the pitching side, it’s a good way to weight the starters and relievers contributions towards the rate stats. If you’re the Astros, where your relievers accumulate a lot of innings, it’ll be affected thus. If you are starter heavy, like the Giants, that will be weighted
Finally, I add the expected counting stats to the already accumulated stats in the middle of the season. For the rate stats, the formula looks a little something like this (for AVG, for example):
- ((AccumulatedAVG*AlreadyPlayedWeeks) + (ProjectedAVG*WeeksToPlay))/Total Weeks
Finally, a note that if a team is not expected to accumulate the minimum IP/week or AB/week, I assign a BA/OPS of .000 and an ERA/WHIP of 10, to account for the automatic loss of those categories in Fantrax.
And that’s how all the teams get their stats.
Simulation
Now that we’ve got all of our statistics, it’s time to simulate. But to do that, we need to first sort and rank the teams by stats. Then, I assign a winning% to each stat, based on the winning percentage in league history of that position in the rankings (which had a strong-ish correlation). The number rank I use in the formulae corresponds to what you’d see on the Fantrax Standings>>Season Stats page (30 points to the league leader in that category; 1 to the worst statistical performer). These end up being the main numbers the program uses.
I have hard-coded the schedule into the program. When I start the program, it simulates the season week-by-week. Each game works the same way:
Team 1 has a winning percentage in the runs category of A
Team 2 has a winning percentage in the runs category of B
The accepted formula in simulations (Bill James’ Log5 Formula) is the following:
Winning Percentage for Team 1 = (A*(1-B))/((A*(1-B)) + ((1-A)*B))
Or, in other words – the chances that team 1 wins multiplied by the chance team 2 loses, divided by that same number plus the chances that team 1 loses multiplied by the chance team 2 wins.
Thus, if Team 1 wins the runs category 60% of the time and Team 2 wins the category 40% of the time:
W% = (.6*(1-.4))/((.6*(1-.4)) + ((1-.6)*.4))
W% = .36/(.36+.16)
W% = .6923
So, when these teams play, Team 1 wins runs 69.23% of the time.
In each and every matchup, I do winning % for each stat. Then, I do a simple random number simulation and come up with the winner for each stat in this theoretical game. I then record a win, loss or tie (using actual league tiebreakers)
I simulate the entire season game-by-game using this algorithm. I then determine playoff teams, breaking ties by league rules, put them into ArrayLists by seed, play the matchups and eventually come up with the world series champ.
Then, once that whole season is completed once, I do it 99,999 times again. That gives me a good enough sample size to eliminate random noise.
The Numbers You See
I get that seeing that the Colorado Rockies having a 95% shot at the division with 4 games left and a one game lead can look like I’m putting bias into the program, but that’s simply not true. Those numbers can seem callous and cold, I understand, but it’s based on specific games and likely outcomes.
With 100,000 season simulations, you get almost every conceivable outcome. True, the Washington Nationals aren’t going to show you a 0-21 season, but that isn’t even realistic in this fantasy simulation. Many different outcomes go into that average wins number, and even that doesn’t give you a full appreciation for the variety of the outcomes
FOR EXAMPLE, below you can see the win spread for each team by division if the season started today with a blank slate but with the same team rosters and accumulated statistics as it stands today.
How to Read These Graphs
The x-axis is the number of games won. The y-axis is the percentage that outcome occurs. In general, you want to be the team with the top-right line in your division (the right-most and most top-heavy towards the right).
