There are stats all over the internet, from numbers like points per game, which have a direct cause (and correlation) in which team wins the game, to stats like a quarterback's record versus another team which is more correlation that causation.
With that in mind, I am going to build the foundation for what a "stat" is, its "type" and its "scope".
The word statistics (or "stats") is defined as a numerical fact or datum, especially one computed from a sample (dictionary.com). The word fact is defined as something that actually exists, is known to exist or happened.
For example, Baltimore won SB 47. This is a fact. We know it happened. However, Baltimore won SB 47 and Baltimore was the best team in 2012 are not the same statements. Baltimore may have been the best team, but BAL also lost 6 games that season and was still outscored by Denver 66-55 in the two games they played. Its imperative that we understand the difference between fact and truth.
I have come up with a definition tailored to fit stats in sports, specifically the NFL, so that we can build a strong foundation. I define stats as a number, or collection of numbers, that can represent plays on the field (completion, rush, sack, interception...etc), is unaltered in any way (the stat isn't weighted in any way which can allow bias to effect its validity), and finally it can't be argued.
Stats can be in a percentage form, which is simply moving the decimal, but the value is still unchanged.
Examples of stats:
- Yards in various forms: rushing, passing, total, yards per carry, yards per attempt
- Points in various forms: points per game, points per possession, points per play
- Turnovers in various forms: TOs per game, TOs per play
- Penalties in various forms: penalties per game, penalty yards per game
- Percentages: third down percentage, completion percentage, red zone percentage
As long as no modifier outside of 100 (when used for percentages) is used, then these numbers are the basic stats you will see on almost every football website. You can argue their value, but you can't argue the specific totals.
This is the most simple category. Stats are used to describe what has happened and predict what will happen. Some stats do a great job describing what has happened but are a terrible indicator of future success.
Simply put, how much value do we assign each stat? Does a stat just measure an aspect of a unit or the entire unit and is it even worth using?
Most people in the NFL community (fans, broadcasters, TV analysts) use the general "box score" stats to discuss teams instead of "advanced" metrics.
So what does all this mean? The basic stats do one thing - they measure events. A play is either a rush, pass attempt, turnover, penalty or a score. A pass attempt can be completed, incomplete or a sack.
Our job is to determine which stats are important and which ones have limited value. This will allow us to determine its "scope". Then we can study teams based on the stats that do matter while ignoring the ones that have limited value.
STATS vs. RATINGS
While most people may combine these two, I am going to identify a difference for the sake of this article. Remember, as stated above, stats are static and they simply measure events that occur and can't be argued.
Ratings, on the other hand, can be defined as a collection of stats that are weighed according to value. This means that ALL ratings are subjective. However, it does not mean that they do not matter or are inaccurate. What it means is that the precise value can't be viewed as a fact.
The purpose of ratings is to do what stats fail to do - demonstrate a high correlation between a good score and winning football games.
Let me repeat this with different wording.
Almost all stats have a low correlation to winning football games. This means they have almost no value. There are only a couple from the list above under "Stats Defined" that have a high correlation to winning football games.
Therefore, ratings are designed to measure all of the important aspects of football, with the proper weight for each category.
This leads us to the three forms of ratings that are used by major stat-based websites.
- Play-by-play grading for players and teams based on the fundamentals of football and techniques taught by coaches. Best website for this: Pro Football Focus (PFF).
- Play-by-play analysis for players and teams based on a weighted grading scale derived from perceived value from research. Best websites for this: Football Outsiders or Advanced NFL Stats.
- Efficiency Stats based on percentages and correlation to winning football games. Best website for this: Cold Hard Football Facts (CHFF).
Each of these groups measure each play and each game/box score differently. Its important to understand the methodology of the ratings that are used by each website. Before taking anything they use as gospel, each rating needs to not only make sense by match up with the idea that these numbers are supposed correlate with winning.
This is step where we carve out the role each stat or rating is going to have in our attempt to provide analysis for each team's offense and defense. Special teams is much more difficult to measure and can be very volatile.
Even though it is better to use more than one season as a baseline for win correlation data, I will use what I have for the 2012 season.
Here are some of the basic stats you will find in any box score. I tracked each stat and how often the team that won that stat in the game won the game. Here are the scores in order of highest correlation to lowest correlation:
- NFL passer rating: 209-45 (.823)
- Turnovers: 163-42 (.795)
- Yards Per Attempt (including sacks): 187-68 (.733)
- Rushing Yards: 180-74 (.709)
- Yards Per Play: 169-85 (.665)
- Total Yards: 160-94 (.630)
- Passing Yards: 139-115 (.547)
To find if passing yards and rushing yards were similar this season, I did a quick test for the some of the odd weeks during the 2013 regular season.
During weeks 1,5,9,13,17 (5 weeks and 75 games), teams went:
- Rushing Yards: 52-22 (.703)
- Passing Yards: 37-38 (.493)
Of note is that teams went 15-0 when gaining the most yards rushing in Week 17. The record for the other 4 weeks was 37-22 (.627). One game ended in a tie for total rushing yards during the final week.
These 75 games represent 29.3 percent of all games during the 2013 season. In general, a sample size of 15 is necessary to get a reasonable sample (which would only account for 5.9% of the season. So nearly 1/3 of the total games does the job just fine).
What This Means
There are many inferences that can be made about why teams win more with the running game:
- Teams that are able to run the ball tend to make fewer mistakes (fumbles are less likely than interceptions)
- You can gain yards on a short run, while an incomplete passes nets the offense nothing
- Its less difficult to run the ball. Even with all of the rule changes, its still more complicated to pass.
- There are a lot of young QB's in the NFL who are asked to do a lot but struggle with the responsibility.
- There are more dual-threat QBs in the NFL who add to the rushing total and pass fewer times
- Teams tend to run when they have a lead, while teams trailing need to pass more to "catch up"
I imagine there is some truth to the bullets above. The problem is that as good as the running game might seem, it still barely reaches a win correlation of 70%, and that is after a perfect 15-0 week. In reality, when compared to the chart above with the other stats as well, we see a pattern with stats involving yards. They are overrated and, for the most part, useless. The only yards total with a sufficient win correlation is passing yards per attempt (sacks included), which hovers around 73%.
When we look at TOs with a win correlation of 80% and the NFL passer rating with a win correlation of 82%, we are ake to see the huge difference between yards and the more important stats and ratings.
The verdict on yards: They are great for fantasy points and individual numbers, but offer very little insight into why a team won the game. Its best to just ignore them. Passing yards, rushing yards, total yards per game, etc...
WIN CORRELATION (WC)
Are there any other stats/ratings we can use besides NFL passer rating to measure team success?
Thankfully, the answer is yes. The box score stats are necessary, but they can't tell the whole picture. The reason for this is because there is a hidden element to football the most people don't want to hear - efficiency wins football games. Even if you do not like stats or ratings, the reality is there are only two ways to study football: through film or with numbers.
Since the box score provides little as to why a team won or lost besides turnovers and passer rating, we have to dig deeper. This explains the outbreak in advanced metrics over the past decade. People wanted to learn more and the box score couldn't provide it at face-value.
I created a set of stats that haven't been used anywhere else that take the raw data from the box score and translate it into efficiency stats based on percentages. The main factor is points scored, not yards.
Play-by-Play Weighted Grading Scales
The idea behind grading each play is sound, in theory. DVOA attempts to account for strength of opponent and a variety of other variables. I don't know their system and it doesn't seem like their formulas are floating around.
However, I have a problem with any system, whether by FO or AdvancedNFLStats that makes an attempt to quantify an immeasurable. I have tried it many times, and don't see a feasible way to quantify the game of football on a play-by-play basis.
Here are some of the imperfections:
- DVOA grades yards, which is an almost useless stat. Secondly, even if gaining 8 yards on 3rd and 10 isn't as important as gaining 8 yards on 3rd and 6th, it can still have a very positive effect, whether allowing the team to get into field goal range, going for it on 4th down or punting to back the other team in their own red zone. There are no numbers to measure whether those yards actually impacted the outcome of the following play.
- Weather, injuries, emotion and coaching decisions are intangibles for a reason. They can't be measured. We could argue all day long about what scores or grading system should be used to take these into account. FO and AdvancedNFLStats do indeed take all this "fluff" into account, its just debatable as to how accurate the model is....
- Which leads us to our final point...how do measure the accuracy of a rating system for intangibles? What kind of score does the offense and defense get after a game, whether good or bad? Even though the rating is applied to all teams, it is impossible to measure many of the factors the rating is trying to measure. There is no win correlation testing going on. Why and how is Denver's offense 33.7% better than the average, and when weighted only 27.1% better than the average, just 1.8 percentage points above PHI's offense, according to DVOA?
For these reasons, the best stats to use are efficiency ratings. Not because I do them or because I am friends with CHFF, but because we can prove the validity of each rating, while ignoring the "fluff" that is immeasurable.
In 2012, I used tracked 5 efficiency ratings (3 were my own). The yards per attempt from the list above is one of them. The other 4 all have a WC of at least 82%, which is far superior to anything the regular box score will give you.
- Plays/TD: 231-24 (.906). This rating takes number of plays and divides it by the net offensive points (offensive points - defensive points allowed). The defensive points allowed figure represents turnovers returned for touchdowns and safeties allowed by the offense.
- My QBR: 216-39 (.847). This rating accounts for points off turnovers and estimated points lost. It also measures rushing totals for a QB and pressures.
- Yards/TD: 214-41 (.839). This rating does the same as plays per touchdown, except instead of total plays, its total yards.
- NFL Passer Rating: 209-45 (.823). This rating was created by the NFL and measures TDs, INTs, completion % and YPA. It also has a max on each category.
- Yards Per Attempt: 187-68 (.733). This rating includes sacks as attempts and subtracts the sack yards lost from the passing yards total.
NOTE: I have created another rating that measures the whole offense and defense, but it takes too long for the average person to calculate it. My team QBR rating has a WC of around 90%.
While some of the advanced metrics attempt to account for everything, there is no way to prove its legitimacy. Efficiency stats, on the other hand, are simple and the most effective way to measure team success.
Because the plays/TD and yards/TD ratings produced numbers for offense and defense that would decrease/increase at different rates, the net scores would always favor the defense because a high number (good for a defense) would always be greater than a low number (good for the offense). By switching the numerator and denominator, I was able to get these numbers as a percentage, which balanced it out.
SUMMARY AND RECOMMENDATION
I have seen tons of articles and comments on which numbers are more accurate and how to interpret them. As I stated above, the box score is mostly useless on its own. The only three numbers that are of use by themselves are turnovers, the NFL passer rating, and the yards per attempt stat (as long as the sacks are included).
Using advanced metrics can be tricky because there is no way to prove the WC. Secondly, its only one rating. There is no such thing is a perfect stat or rating. Instead, many need to be used to counter each other, so that we can be sure about the success of a team through repetition. If a team is 1st in Pts/Play but 10th in Yards/Play and 8th in QBR, then all of those need to be taken into account, not just whichever one a fan chooses.
PFF does a great job measuring each play and grading them on a fundamental level; however, too much credit goes to plays that a spectacular in nature, even if they don't add a lot of value to the team (ex. 1 handed catch for 5 yards over a simple slant pass to a wide open receiver for 12 yards). Again, the flaw here is the same as the flaw with advanced metrics using a grading system. Its a slippery slope when you begin assigning credit/blame to players and teams.
So the best stats are the efficiency stats. Use them, in this order, and use these formulas for simple calculations:
- Pts/Play: [offensive points - defensive points allowed] / total plays * 100.
- Pts/Possession: [offensive points - defensive points allowed] / total possessions (use pro-football-reference and subtract the drives ending the half or game)
- Pts/Yards: [offensive points - defensive points allowed] / total yards * 100.
- QBR (NFL passer rating will do just fine)
- Yards/Att: Net passing yards / [attempts + sacks]. use this number instead of passing yards for/against. Remember, its never about the volume.
Other stats that are good to use: Net Points and Net TOs.
Ways to improve some stats if you want to get a more accurate total:
- Subtract the kneel downs from the total plays, because they don't matter when it comes to offensive efficiency. Also subtract the yards lost on the kneel downs from the total yards for the same reason.
- Get a weighted total for points and turnovers. If a team was +100 in 12 wins and -30 in 4 losses, it had a net of 70 for the season. If you give it weight, it will value the score for the outcome that happened more frequently. Therefore multiply 100 by 12/16 and multiply -30 by 4/16. You will get a +75 for the wins and -7.5 for the losses for a weighted net score of +67.5. This will adjust all teams, of course, according to how the performed in their wins and losses.The same concept applies for turnovers.
- Use a relativity index, which is similar to what CHFF uses. They compare the points scored by a teams' offense vs. TEAM B to how the rest of the league performed against TEAM A's defense. For example, if the rest of the league averaged 23 PPG against Oakland's defense, and Denver scored 35 points against OAK, then DEN was a +12 points relative to the league average. This helps accommodate the SOS issue many of these advanced metrics are attempting to account for.
I can't stress these three statements enough.
- THERE IS NO PERFECT STAT or RATING.
- USE EFFICIENCY RATINGS WITH A HIGH WC.
- USE AS MANY DIFFERENT STATS or RATINGS AS POSSIBLE TO MEASURE A TEAM or UNIT.
I know this was long, but I felt it was necessary. If you have any questions about what should or shouldn't be used, feel free to let me know. If you don't care, then enjoy cheering for the Broncos.
For those of you SEA fans who are ready to cry about me challenging your favorite rating, listen to this. Denver's weighted DVOA is 27.1, meaning it is 27.1% better than the average and only 1.8 percentage points above PHI.
According to a rating that has a WC of nearly 90 percent (Pts/Play), Denver had a score of 49.16 on offense. The NFL average in 2013 was 31.67. This means Denver's offense was 55.2% more efficient than the average offense. The 2nd ranked team on offense was Dallas with a score of 41.14, 39.4% more efficient that the average offense. The gap between the 1st and 2nd ranked offensive teams is far greater than DVOA suggests.
Meanwhile, Seattle's defense posted a score of 19.10, 39.7% below league average. The 2nd ranked defense was Carolina, with a score of 21.70 (31.5% below the league average).
According to the Pts/Yds rating, Denver had a score of 7.59. The NFL average in 2013 was 5.80. Denver's offense was 30.9% above the NFL average. The 2nd ranked offense was Dallas, with a score of 7.14 (23.1% above the NFL average).
Seattle's defense had a score of 4.27. The league average on defense was 5.82. This means Seattle's defense was 26.6% below the NFL average. The 2nd ranked defense was Carolina, with a score of 4.34 (25.4% below the NFL average).
In both cases, Denver's offense is significantly greater than the 2nd ranked offense (Dallas). The difference between Denver's #1 offense and Dallas was 40.1% in the Pts/Play rating. The difference was 33.8% in the Pts/Yard rating.
For Seattle, the difference between their #1 defense and Carolina was 26.0% in the Pts/Play rating. The difference was 4.7% in the Pts/Yard rating.
As always, Go Broncos!
Both of these totals are a percentage of the total number of plays or yards. If a team has a pts/play score of 40, then their net offensive points total was 40 percent of the total number of plays by the offense. The same goes for the pts/yard rating. That is why the score is so much smaller for the yards rating.
Net Points / Play %
|Team||Off Pts / Play||Def Pts / Play||Net Pts / Play|
|3||San Francisco 49ers||38.88||23.76||15.12|
|7||Kansas City Chiefs||34.99||24.67||10.32|
|8||New England Patriots||36.55||27.27||9.28|
|9||New Orleans Saints||37.84||30.95||6.89|
Net Points / Yard %
|Team||Off Pts/Yd %||Def Pts/Yd %||Net YPT|
|2||San Francisco 49ers||7.08||4.73||2.35|
|4||Kansas City Chiefs||6.54||4.44||2.10|
|5||New England Patriots||6.65||5.08||1.57|