[BostonCard]

My apologies, I should have cited my source.  I was referring to ESPN's adjusted QB Ratings, which had Costello at 88.1 in 2017 and 80.1 in 2018.

Unlike NCAA Passer Efficiency, which uses only box score statistics, Total QBR accounts for what a quarterback does on a play-by-play level, meaning it accounts for down, distance, field position, as well as the clock and score. A 5-yard gain on third-and-4 is a good play, whereas a 5-yard gain on third-and-14 isn't. A 20-yard touchdown pass when tied in the second quarter means more than a 20-yard touchdown pass when down 30 points late in the fourth quarter. QBR accounts for those things using analysis that turns traditional productivity into points on the scoreboard and wins in the standings.

It also accounts for a quarterback's ability to scramble, his ability to run on designed rush plays, how well he avoids sacks, drawing and committing penalties, and all-important fumbles, which can be significant for quarterbacks.  Here's the link:

http://www.espn.com/college-football/pla....-costello

BTW, Costello is #11 in the NCAA with that metric.  If he'd had the same number as last year, he'd be #3 in the NCAA.

Thanks for the clarification.  Interesting that if you look at the breakdown:

2017: Pass 41.8, Run 5.8, Sack -5.6,  Penalty 3.7, Total 45.5, Plays 266, Raw QBR 81.6, Total QBR 88.1
2018: Pass 75.1, Run 0.5, Sack -16.3, Penalty 8.4, Total 67.8, Plays 517, Raw QBR 74.5, Total QBR 80.1

His "pass EPA" (expected points added) improved from 2017 to 2018, but is that adjusted for the number of plays run?  If it is adjusted, then the improvement in his pass EPA follows the improvement in his pass efficiency numbers, and the difference comes from his run and (especially) his sack numbers., which might reasonably be attributable to his injury and limited mobility.  If they are not adjusted for number of plays (and it looks like they are not given that "total EPA" is higher in 2018 than 2017 but raw QBR is not, then that seems a bit odd and isn't consistent with what either the raw numbers say, or my impression of the passing offense this year versus last.  That doesn't mean that some of the other factors that go into QBR don't play a role (for example, issues like down and distance, controlling for what other players do, etc.) can't account for the difference, but at least my cursory glance of situational stats suggests that Costello's situational statistics follow a favorable pattern (taking more risks on third and long and while behind than on early downs or in a close game).  Then there is the conversion from total EPA to raw QBR, which is not straightforward.  And, unfortunately, the way ESPN calculates QBR is opaque, so there is no way to drill down to its components and figure out what is driving the apparent decline.

I don't mean to dump on QBR.  Pass efficiency rating is a very flawed metric.  But it is a simple enough metric to be understandable, so it is possible to understand its limitations and see when you ran into a flukey reading.  QBR, based on ESPN's description, sounds like it probably represents reality better than PER, if you trust the way ESPN choices as to what is important for inclusion and what adjustments they make.  But in order to generate this "better" measure, you lose transparency.  So if there is a disconnect between QBR and other measures, or between QBR and the neubulous "eye test", it is harder to figure out what is driving that discrepancy.

BC

[BostonCard]

My apologies, I should have cited my source.  I was referring to ESPN's adjusted QB Ratings, which had Costello at 88.1 in 2017 and 80.1 in 2018.

Unlike NCAA Passer Efficiency, which uses only box score statistics, Total QBR accounts for what a quarterback does on a play-by-play level, meaning it accounts for down, distance, field position, as well as the clock and score. A 5-yard gain on third-and-4 is a good play, whereas a 5-yard gain on third-and-14 isn't. A 20-yard touchdown pass when tied in the second quarter means more than a 20-yard touchdown pass when down 30 points late in the fourth quarter. QBR accounts for those things using analysis that turns traditional productivity into points on the scoreboard and wins in the standings.

It also accounts for a quarterback's ability to scramble, his ability to run on designed rush plays, how well he avoids sacks, drawing and committing penalties, and all-important fumbles, which can be significant for quarterbacks.  Here's the link:

http://www.espn.com/college-football/pla....-costello

BTW, Costello is #11 in the NCAA with that metric.  If he'd had the same number as last year, he'd be #3 in the NCAA.

Thanks for the clarification.  Interesting that if you look at the breakdown:

2017: Pass 41.8, Run 5.8, Sack -5.6,  Penalty 3.7, Total 45.5, Plays 266, Raw QBR 81.6, Total QBR 88.1
2018: Pass 75.1, Run 0.5, Sack -16.3, Penalty 8.4, Total 67.8, Plays 517, Raw QBR 74.5, Total QBR 80.1

His "pass EPA" (expected points added) improved from 2017 to 2018, but is that adjusted for the number of plays run?  If it is adjusted, then the improvement in his pass EPA follows the improvement in his pass efficiency numbers, and the difference comes from his run and (especially) his sack numbers., which might reasonably be attributable to his injury and limited mobility.  If they are not adjusted for number of plays (and it looks like they are not given that "total EPA" is higher in 2018 than 2017 but raw QBR is not, then that seems a bit odd and isn't consistent with what either the raw numbers say, or my impression of the passing offense this year versus last.  That doesn't mean that some of the other factors that go into QBR don't play a role (for example, issues like down and distance, controlling for what other players do, etc.) can't account for the difference, but at least my cursory glance of situational stats suggests that Costello's situational statistics follow a favorable pattern (taking more risks on third and long and while behind than on early downs or in a close game).  Then there is the conversion from total EPA to raw QBR, which is not straightforward.  And, unfortunately, the way ESPN calculates QBR is opaque, so there is no way to drill down to its components and figure out what is driving the apparent decline.

I don't mean to dump on QBR.  Pass efficiency rating is a very flawed metric.  But it is a simple enough metric to be understandable, so it is possible to understand its limitations and see when you ran into a flukey reading.  QBR, based on ESPN's description, sounds like it probably represents reality better than PER, if you trust the way ESPN choices as to what is important for inclusion and what adjustments they make.  But in order to generate this "better" measure, you lose transparency.  So if there is a disconnect between QBR and other measures, or between QBR and the neubulous "eye test", it is harder to figure out what is driving that discrepancy.

BC