What does Trevor Siemian’s first year performance say about his future?

About a month ago, I got into a fun conversation in the comment section of a fanpost. The post was about Trevor Siemian and as is typical of posts about QBs, there was a fair amount of debate.

One of the big points of debate was how much could one infer about future potential based on one year of statistics. I have always held that the answer is: not much. Thus, I have typically been averse to statistical comparisons that infer a QB’s potential for greatness, or lack thereof, based on a small sample size.

However, I left that conversation curious about the correlation between first year performance and future potential, and decided to do some digging. While I still hold that raw statistical comparisons are often not very helpful, I found some interesting things using some advanced metrics.

NumberFire put out a great piece a few years ago attempting to answer the question: “Does a Quarterback's Rookie Season Predict His Future?” What they found was that there actually is some predictive value using the NEP metric, or Net Expected Points. Here’s a brief overview of how they describe it:

Every single situation on the football field has an expected point value; that is, how many points an average team would be expected to score in that situation (given down, distance-to-go, and yard line). For example, the Chiefs may be facing the Pittsburgh Steelers, with a third-and-two on the 50-yard line. That's a ton of variables, but luckily, numberFire has data from the past dozen years of every single play, so most situations have come up at least once. According to our data, an average team may be "expected" to score 1.23 (estimated number) points on that drive. However, Jamaal Charles reels off a 32-yard run to bring the Chiefs into the red zone, increasing the "expected" point value of the next play to 4.23 (still an estimated number) points. Jamaal Charles then gets credit for the difference, in this case 2.96 points, as his NEP total. That's Net Expected Points.

NumberFire took this metric and applied it across rookie QBs from 2000 up to 2014 (when the article was written), and from that, broke them down into tiers of performance.

The lowest tier (tier 4), they affectionately refer to as the “I have no business starting in the NFL” tier:

Tier 4

The names you would expect to be on this list are here, and there really aren’t any surprises. One interesting thing to note is that Mark Sanchez made it to the AFC Championship game his first year despite performing about on par with Geno Smith’s rookie year; which is why if anyone tries to cite QB wins and playoff trips as a reliable measure of a QB’s performance, tell them to shove it where the sun don’t shine (as Peyton Manning so eloquently put it).

Tier 3

The next tier has a few names that you may recognize:

This one has a few guys that have at times been considered in the top half of the league as starters (Palmer and Stafford), but a lot of scrubs as well.

Tier 2

Tier number 2 actually has less quality players than the previous list, in my opinion; which just further shows that those guys were outliers.

Tier 1

The last group is the money tier.

Here you have four QBs who I would consider near elite players at the quarterback position in Luck, Ryan, Roethlisberger, and Newton.

While there are a few outliers on this list, for the most part you can see the trend that those in tier 1, according to their NEP, are your long-term franchise guys while the rest of the tiers are essentially career backups or flame outs.

One notable exception is Raiders QB, Derek Carr. This article was written in January, 2014 so it didn’t have Carr included. Carr’s first season he put up a putrid -40.94 NEP, putting him squarely in tier 3, yet he has since had a turn around of historic proportions over the last two years.

Additional Data Points

Due to this particular study being constrained to QBs who started their first year, and being 3 years old, we’re missing a few key players relevant to this discussion; so I did some digging to fill in the gaps.

Tom Brady - 2001: NEP 19.43 | NEP per drop back 0.04

Brady sneaks into tier 1 ahead of Andy Dalton and Matt Leinhart. One additional note NumberFire adds on Brady is something I’ve pounded the table for every time his name comes up in comparison to young QBs: historical context.

Historical Context

Passing numbers are up across the board, and that has impacted NEP numbers overall. But even though Brady's numbers are diminutive compared to Wilson's, he was pretty good compared to his historical peers, except in 2002.

This was an excerpt from an article during the lead up to Super Bowl 49 comparing Brady to Russell Wilson. The other thing they do for comparison is also something I’ve mentioned before, comparing him to his historical peer set. Ok, I’m done tea sipping.

Dak Prescott - 2016: NEP 140.42 | NEP per drop back 0.29

I included Prescott just to show another recent example of someone not drafted in round 1. He clearly sits at the top of tier 1 with historically great numbers for a first year starting QB.

Where does Trevor Siemian land?

If you made it this far, congratulations (or perhaps you just skipped to this section for the payoff).

Trevor Siemian finished the 2016 season with a passing NEP of 29.79 and 0.06 NEP per drop back.

This puts him in tier 1 of the charts above, underneath Andrew Luck, and above Andy Dalton and Matt Leinart.

So what does this mean for his future in Denver? NumberFire’s article summed it up nicely:

I'm fully aware that a study like this could go a lot deeper. Team weapons, coaching – all of that goes into how well a quarterback performs over the course of his career as well. But at the same time, that’s the nature of the NFL. Quarterbacks often times can have poor careers because they weren’t placed in the right situation from the start.

The fact of the matter is that there seems to be a clear connection between a quarterback performing at a moderately high level during his rookie season versus the quarterback’s future success, as long as that quarterback does, indeed, play his rookie season. Just move your eyes from top to bottom on that list, and you’ll see just that.

This is by no means the end all to grading whether or not a signal-caller will succeed, but it certainly can’t be ignored.

So while the raw statistics don’t look that appealing, and there are several things that I personally have issues with about his tape, this data shows that Trevor Siemian has at least a shot to be a long-time starting QB based on what he showed his “rookie” year.

Again, making predictions and declarative statements are tricky, as he could just as easily turn into a Matt Leinart or Jason Campbell: career backups, but the numbers from this particular study are on his side, especially considering the overall situation of the offense last year.

Speaking of that, in a separate article, NumberFire compared Siemian to the other three QBs that have started for Denver over the last two years (Manning/Osweiler - 2015 and Paxton Lynch - 2016) in order to gain additional context based on the team around him.

While Manning’s poor 2015 likely pulled down this average, it is still interesting to see the improvement when Siemian was on the field, especially when it seemed at times that Denver’s offense/QB play was worse this year than last year.

This is certainly good news for the Denver Broncos as they continue to groom two young QBs. Having more than one potential long-term option at QB in a starving market is never a bad thing.