[OC] Is playoff expansion really worth it? I simulated 50 million NFL seasons to find out! Part 1: Introduction

Posted on: Apr 23, 2018   |   Posted by: Refs4Pats NFL Blog2

This post is heavy on tables, so apologies in advance to anyone with a small monitor. This is going to be a multi-part series due to the large amount of data. I don't want to just bury you in tables. Introduction, a.k.a. the skippable part About a year and a half ago, /u/jaguargator9 made a post called "Analyzing what would happen if the NFL added a 7th team to the playoffs." He used the real NFL standings and results to predict how a 14-team bracket would have played out every year. While his analysis is interesting, it contains many flaws, not the least of which is the minuscule sample size. So, you may ask, how do you solve a problem like this? Even today, we only have two more seasons of data! The answer is simple: I wrote a program to simulate millions of NFL seasons. Methodology, a.k.a. explaining Elo and avoiding super-teams Due to the need to produce a large number of results, my model could not be very complex. I settled on the Elo rating system. For those of you who don't know what that is, it's a single number that predicts a team's chance of victory. An average team has an Elo rating of 1500, with about 44% of teams within a hundred points. A team with a 100-point advantage in Elo has about a 64% chance to win the game; at 200 points this becomes a 76% chance. The data I used was since the NFL moved to the 8-division format, from 2002-2017. Elo ratings were converted from Pro-Football-Reference's Simple Rating System by /u/bwburke94, who is the one who inspired me to actually post these results instead of sitting on them for another year. The best team in that timespan was the 2007 Patriots, with an Elo rating of 2002. The Patriots would win 95% of the time against an average team. The worst team was the '09 Rams, rated 1064. They would win about 7.5% of games against an average team, and would beat the '07 Patriots about once every 222 tries. Of course, most of this is irrelevant, since I'm not using real teams in this simulation. I instead generated them using a normal distribution based off the real teams. This was done so that I wouldn't have the '07 Patriots show up every 16 seasons. Using the same normal distribution for all teams, a team of their caliber would appear about once every 46 seasons - still far too common. Once I adjusted for the fact that teams tend to retain some of their skill from the previous season, it became once every few hundred seasons. (Keep in mind that the Patriots' ridiculously high rating comes from only on-field regular season results. Their actual ability was certainly lower than that, especially considering that they would end up losing to a team with an Elo rating of only 1578 - an 8% chance of happening if the ratings were perfectly accurate.) Anyway, that's enough talk. Let's get to the results! The Results I simulated 50 million seasons, meaning this data set consists of an astounding 1.6 billion teams, which should be more than enough to alleviate any sample size issues. Let's start with the basics. What is the distribution of wins among all teams, and how does it compare to the real NFL? WINS 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Simulation 0.19% 0.85% 2.14% 4.02% 6.27% 8.52% 10.43% 11.67% 12.08% 11.59% 10.30% 8.41% 6.21% 4.04% 2.19% 0.89% 0.21% Real NFL 0.39% 0.59% 3.13% 2.73% 8.59% 8.01% 8.79% 11.13% 11.91% 11.13% 11.13% 8.20% 7.03% 5.08% 1.37% 0.59% 0.20% I'll also show this one in graph form, since the table makes it look worse than it is. It's fairly accurate, as close as I could get it on a simple model. One thing that sticks out is the relative lack of 3-13 teams in the real NFL. Since this entire experiment is based off playoff seeding, I'll also provide the records that earned each playoff (or non-playoff) seed. A blank cell means it never happened, either because it's impossible (such as a 2-14 team winning their division) or ridiculously improbable (a 9-7 15 seed.) WINS 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1 Seed 0.00% 0.00% 0.30% 4.48% 18.78% 31.93% 27.64% 13.56% 3.32% 2 Seed 0.00% 0.00% 0.34% 5.77% 25.28% 37.93% 23.46% 6.48% 0.72% 0.02% 3 Seed 0.00% 0.00% 0.02% 0.71% 7.27% 27.45% 38.53% 21.05% 4.60% 0.37% 0.01% 4 Seed 0.00% 0.00% 0.02% 0.44% 3.42% 13.78% 30.02% 32.77% 16.05% 3.24% 0.24% 0.01% 0.00% 5 Seed 0.00% 0.01% 0.66% 10.04% 32.65% 34.93% 16.87% 4.30% 0.52% 0.01% 6 Seed 0.00% 0.18% 6.82% 36.51% 41.50% 13.44% 1.50% 0.05% 0.00% 7 Seed 0.00% 0.02% 2.10% 26.02% 50.50% 19.55% 1.77% 0.04% 0.00% 8 Seed 0.00% 0.34% 11.28% 49.16% 34.79% 4.33% 0.10% 0.00% 9 Seed 0.00% 0.04% 3.03% 32.01% 51.44% 13.00% 0.47% 0.00% 0.00% 10 Seed 0.00% 0.00% 0.60% 13.88% 51.62% 31.20% 2.67% 0.02% 0.00% 11 Seed 0.00% 0.11% 4.64% 35.07% 48.88% 11.01% 0.29% 0.00% 12 Seed 0.00% 0.03% 1.52% 18.63% 50.49% 27.13% 2.19% 0.02% 0.00% 13 Seed 0.00% 0.01% 0.64% 9.68% 39.74% 41.15% 8.55% 0.23% 0.00% 14 Seed 0.01% 0.44% 6.31% 29.33% 44.31% 18.17% 1.43% 0.01% 0.00% 15 Seed 0.02% 0.68% 6.52% 24.95% 40.11% 23.71% 3.90% 0.11% 0.00% 16 Seed 3.02% 12.85% 27.19% 32.42% 19.52% 4.69% 0.31% 0.00% Screenshot for those who can't see the full table. Interpreting the Results Most of what we see here makes sense, but there are quite a few things that I have to note. There were noticeably more 16-0 teams than 0-16. I'm not too surprised, as the model gives more exceptionally good teams than exceptionally bad ones. A good number of the 0.00s above only happened once or twice in fifty million seasons, so don't read too much into them. With regards to the 7-seeds, the reason I made this project to begin with, over half of them went 9-7, with 28% going 8-8 or worse. The "doom scenario" of an 11-5 team missing out happened only a handful of times per century. Although it's not in the data above, I should mention that about 1% of divisions had every team go 9-7 or better. In the current playoff structure, at least one of that division's teams has to miss the playoffs, and that team is likely to have gone 7-3 or 8-2 outside the division. In fifty million simulations, there were multiple winless teams about 44,000 times. That's less than once per millennium. So don't worry, Browns/Jets/Bears fans, you probably won't be going 0-16 and picking second. Next time, we'll get to the actual point of this project: How does adding a 7th team per conference impact the playoffs? In fact, let's take it much further - why stop at 7? submitted by /u/JamesBCrazy [visit reddit] [comments]