Statistics are always, and have always been, extremely tricky. Often times in a debate the side with the most stats to support their theory is considered the winner by sheer volume (a strategy I often times use with my wife). It doesn't always matter exactly how relevant those particular stats are, the mere fact that you have them and your opponent does not often signals a gross lack of preparation on that sad party that attempted to engage you in debate.
Sports in general lend themselves so nicely to statistical analysis that it becomes entirely possibly to spend weeks of your life staring at box scores until you realize that you've officially reached the insane level of Maximillian Cohen in the movie Pi. Yet, all the stats in the world can't give you an entire picture, and like we've argued time and time again stats simply can't measure everything. Hence the use of the naked eye test and gut. The trick becomes, how do we combine the critical analysis that statistics offer with the more emotional naked eye test to fill in the gaps?
I won't ever claim to be a statistician. I didn't particularly enjoy stats class because word problems make my head hurt. The first time I ever heard the word sabermetrics I thought perhaps it was a new X-Men (X-Man?). What I have attempted to do in this post is combine my limited knowledge of stats with an idea that I have believed for a long time due to the naked eye test.
My theory was that:
A great team must own the endgame. They must be clutch.
To be a championship organization you must command the endgame. The 4th quarter. The 9th inning. The final 2-minutes. The bonus minutes (this was an attempt at a soccer reference that I am about 14% sure makes even a little sense). If you cannot win the endgame you will not be a championship team. The ability to play consistently clutch is what separates the good teams from the great teams. My question became, how clutch are the Broncos?
I went into this study with zero background research and committed myself to simply reporting everything I found here, even if it proved my naked eye test wrong.
I also decided if the stats proved my theory wrong I would simply choose to live in a glass case of emotion.
I've created two posts in the Case for Clutch. The first one, this post, will deal directly with my statistical analysis. In the second I will attempt to fill in the gaps with what I've gleaned from this research through my own thoughts and conclusions.
For those of you that can stomach the stats I would love to get some feedback as to how I can better present these numbers and/or refine my data to take account for more anomalies.
Note: With almost perfect timing, Maxwell released his post on the need for statistical evaluation as I wrote this one. It's definitely worth checking out.
1. If we take away the 4th quarter of every game the Broncos would have had a much better W-L record.
2. Teams that win the 4th quarter win the game most of the time.
3. Teams that win the 4th quarter win the game more of the time than if they won any other quarter.
4. Taking into account both offensive and defensive production together, we can better understand how good teams are in the 4th quarter.
I used only the teams in the AFC West to test my sample. I did this for two reasons. The results most directly reflect our Broncos and I am much too lazy to crunch all these numbers for 32 teams. The sample size is not large, but I felt it was most relevant to the Broncos as we play the teams in our division vastly more than any other teams. I recorded every game played by our division in 2010 and separated each game into 8 categories:
Won 1st Quarter/Won Game, Won 1st Quarter/Lost Game (etc. for each quarter). In other words, if a team that won the first quarter went on to win the game, it got a check in the W1/Win column but if it won the first quarter and lost the game then it got a check in the W1/Lost column. This gave me a simplistic mathematical number of how important each quarter was to providing a win. I considered a tie score in a quarter to offset and didn't consider it in this equation.
Note: There is at least one serious flaw in this particular test and it has to do with the emotional side of the game. If a team went up by 14 or more points in the first and/or second quarter, that team would rarely go on to win the 3rd and 4th quarter, arguably to not "run up the score" or to protect it's players and run clock. It's difficult to account for that scenario statistically.
What I did find is that teams that won the 4th quarter went on to win at a higher rate and lost at a lower rate than in any other quarter. It's not rock solid evidence, but it does support that idea that it is better to be a good team in the 4th quarter than in any other.
My first assumption was easy to test. I simply went down the Broncos schedule and if we were winning by the 3rd quarter I counted it as a win.
As it turned out my memory did not serve me well and the Broncos would have had exactly the same W-L record in 2010 if they never played the 4th quarter. There were a couple games we won in the 4th and they were offset exactly by a couple games we lost in the 4th.
The Broncos simply weren't competitive in the first 3 quarters of football this season. We were down by an average of 11.2 pts going into the 4th quarter this year, and we went into the 4th quarter behind 12 out of 16 games. Were these deficits even possible to overcome? I decided to do some 4th quarter breakdown.
My second assumption, that teams that win the 4th win most of the time, seems to be correct. Teams that won the 4th quarter won the game 75% of the time.
My third assumption, that teams that win the 4th quarter win games more of the time than if they won any other quarter, while correct, is not extremely significant. In order of win percentage importance the list goes:
1. 4th Quarter - 75%
2. 2nd Quarter - 72%
3. 1st Quarter - 71%
4. 3rd Quarter - 67%
Bottom line: The 4th quarter is the most critical quarter and while all quarters are important, winning the 4th seems to create the most wins (for the AFC West in 2010). I expected the disparity to be higher, but in the case of the 2010 AFC West the first 2 quarters, with the strange exception of the third, were almost as important to providing a W as the 4th.
I decided to pay special attention to just the 4th quarter. It was important to me to include offensive and defensive stats together. I believe the two always work in tandem. I was able to create a stat that helped include both offensive production and defensive production into one number.
I broke down every 4th quarter played in 2010 in the AFC West and created two categories for each team, Points For(PF) and Points Against (PA). I then totaled those up, got an average for each team, and then created a stat that I'm going to call FODE (Fourth Quarter Offensive/Defensive Efficiency)... or the Clutch Equation.
It simply takes the average PF total and divides it by the average PA total. A FODE of 1 would mean that in the 4th quarter the offense provided as many points as the defense gave up. A total greater than 1 means the offense scores more than the defense, and a total less than 1 would be the opposite.
Those stats looks like this:
The FODE Ranking is:
1. Raiders - 1.32
2. Chargers - 1.28
3. Chiefs - 0.94
4. Broncos - 0.86
I thought these results were really interesting. The Raiders were the most dominant in the 4th quarter, consistently putting up more points than they gave up. While finishing the season at only 8-8, if they could keep a game close heading into the 4th quarter odds were in their favor to win.
I thought it was interesting to see the Chiefs so low but as you take a closer look at their schedule you would notice they often won games in the first 3 quarters... or in overtime. If these stats are meant to be believed, one would guess that in a close game the Chiefs and Broncos would be odds on favorite to choke, unless of course the Chiefs played the Broncos in a close game and then the odds would suggest the Broncos would choke.
Also, the 4th quarter PA number exactly matched the FODE ranking which suggests that it is better to be good defensively than it is to be powerful offensively in the 4th quarter.
Note: These stats have a slight amount of self-fulling prophecy. For example, if the stats suggest the Broncos would choke against the Chiefs, to a certain extent the stat simply reflects the fact that they did choke against the Chiefs in that Week 13 game.
I know what some of your are thinking. What happened when we changed QB's?
I know that we've been raging on and on regarding Orton and Tebow. I wanted to qualify what I'm about to show you next by saying as small as this entire sample size was, the one I'm about to use next is even smaller. I don't think that we can bet the house on a 3 game sample BUT the results are significant enough that I thought it was worth sharing.
In the 4th quarter, with Tebow starting, the Broncos scored nearly 4 more points on average. Also, with Tebow starting the Broncos on average gave up fewer points by more than one in the 4th quarter. Again, the sample size is very small and could be changed dramatically had one or two plays gone down differently, but the overall improvement of the entire team is interesting to me.
Did I miss any big team change that could have also played into these numbers, particularly on the defensive side of the ball?
So that is my research. Stats are difficult. Even writing this post I can see certain holes that could only be filled by expanding the sample pool so any criticism that the sample size is too small is absolutely valid.
However, I think that the results are not so skewed that they can't be used to help address some of the Broncos needs in the upcoming season. We have got to become a more clutch organization, especially during division games. What I'm going to do in the second part of the The Case for Clutch is explain some of my takeaways from this project. I'm going to attempt to fill in the statistical holes with the proverbial gut check.