[BostonCard]

I think nothing upsets certain cardboarders more than when David Shaw calls a time out to avoid a delay of game penalty.  Fivethirtyeight did an analysis of these “operational” time outs (defined as “timeouts due to play clock mismanagement, the wrong personnel on the field, failed challenges, or simply the look the defense is giving the offense”) in the NFL:

https://fivethirtyeight.com/features/hey...-timeouts/

Only two teams — the Dolphins and the Ravens — managed to extract surplus value from their operational timeouts. Whatever benefit the other 30 teams received from stopping the clock and regrouping before the end of the half (or game) didn’t come close to covering the costs of the timeouts they spent to purchase those precious extra seconds.

The analysis refers to this earlier analysis (http://www.advancedfootballanalytics.com...art-2.html) showing that a time out is, on average, worth about 3.1 percentage points of win probability. There are some issues with the analysis, and it still seems Slightly hightr than I would expect, but I don’t have better numbers than that.

Crucially, that .03 WP is more than the wp change from taking a DoG, which in the most extreme example (Going from a 3rd and 1 to a 3rd and 6) is only worth about 0.02 WP.

The bottom line is that the wasted time outs are worth a lot more than I thought they were, and that the risk from trying to get the “right” play in is much higher than calling a generic run up the middle if time is running low. There is such a thing as decision paralysis and overthinking things.

That being said, the article notes that Sean McVey was one of the worst offenders of “operational time outs”, and the guy is only the youngest coach to ever win a Super Bowl.

BC

[BostonCard]

I think nothing upsets certain cardboarders more than when David Shaw calls a time out to avoid a delay of game penalty.  Fivethirtyeight did an analysis of these “operational” time outs (defined as “timeouts due to play clock mismanagement, the wrong personnel on the field, failed challenges, or simply the look the defense is giving the offense”) in the NFL:

https://fivethirtyeight.com/features/hey...-timeouts/

Only two teams — the Dolphins and the Ravens — managed to extract surplus value from their operational timeouts. Whatever benefit the other 30 teams received from stopping the clock and regrouping before the end of the half (or game) didn’t come close to covering the costs of the timeouts they spent to purchase those precious extra seconds.

The analysis refers to this earlier analysis (http://www.advancedfootballanalytics.com...art-2.html) showing that a time out is, on average, worth about 3.1 percentage points of win probability. There are some issues with the analysis, and it still seems Slightly hightr than I would expect, but I don’t have better numbers than that.

Crucially, that .03 WP is more than the wp change from taking a DoG, which in the most extreme example (Going from a 3rd and 1 to a 3rd and 6) is only worth about 0.02 WP.

The bottom line is that the wasted time outs are worth a lot more than I thought they were, and that the risk from trying to get the “right” play in is much higher than calling a generic run up the middle if time is running low. There is such a thing as decision paralysis and overthinking things.

That being said, the article notes that Sean McVey was one of the worst offenders of “operational time outs”, and the guy is only the youngest coach to ever win a Super Bowl.

BC

[BobK]

One of the things I hate about football announcers is their constant dribble about saving your timeouts for the end of the half or end of the game. No use the timeouts when you need them. That said if Shaw used 4 of the 6 to avoid delay of game penalties then yes I’d be upset but he doesn’t. Time outs are there to be used don’t be brain washed by announcers.
Sermon for a Sunday morning

[BobK]

One of the things I hate about football announcers is their constant dribble about saving your timeouts for the end of the half or end of the game. No use the timeouts when you need them. That said if Shaw used 4 of the 6 to avoid delay of game penalties then yes I’d be upset but he doesn’t. Time outs are there to be used don’t be brain washed by announcers.
Sermon for a Sunday morning

[OutsiderFan]

Anyone who watches a lot of football knows timeouts are extremely valuable, and the best teams almost never waste them. This data just provides empirical proof to back up what close followers of the sport and even those who play Madden always figure out.

One of my friends who plays it has always said head coaches need to play Madden to learn how to manage a game.

[OutsiderFan]

Anyone who watches a lot of football knows timeouts are extremely valuable, and the best teams almost never waste them. This data just provides empirical proof to back up what close followers of the sport and even those who play Madden always figure out.

One of my friends who plays it has always said head coaches need to play Madden to learn how to manage a game.

[Goose]

The bottom line is that the wasted time outs are worth a lot more than I thought they were, and that the risk from trying to get the “right” play in is much higher than calling a generic run up the middle if time is running low.  There is such a thing as decision paralysis and overthinking things.


That being said, the article notes that Sean McVey was one of the worst offenders of “operational time outs”, and the guy is only the youngest coach to ever win a Super Bowl.



BC

It's a model. Is it a useful model? I don't know, what is the evidence that it is? The model of timeout value compares the performance of teams that had N timeouts at a given time in the game to those that had N - 1 timeouts at the same point. They attempted to control for time remaining in the game etc. However, the elephant they ignore is that the number of "operational" time outs a team takes may be correlated but not causative to it's performance. If your team is playing poorly that day, making lots of mental mistakes, or the coaching staff is having an "off" day, or the opposing defense has you figured out, you may need those "operational" time outs. There is no way to control for that effect. If you don't need them, you don't take them, and if you do need them, you already have a problem compared to the teams that don't. The goal is to not need them. The control would be a team that needs them but doesn't take them. How do you create that control, since taking a time out isn't a random variable? Note that you still may be winning the game, because you are overall a better team than your opponent despite having an "off day" calling plays.

It is IMHO probable that this model simply reflects the fact that teams that call a lot of "operational" time outs have a problem already, not that calling the time out was a mistake. You have already made one more "significant" mistake than your opponent and have chosen to call an "operational" time out to remediate it. It isn't calling the time out that is the problem. It is the mistake. Some teams compensate better than others from needing the time out. There is no way to evaluate how many operational time outs SHOULD have been called (would have contributed to win probability) but were not.

The "value" of taking time out because the play-clock is running down instead of running a "bad" play clearly depends on the situation. If it is a critical third down in "no man's land" needed to keep a drive alive, it is probably a "good" call. If it is first and ten on your own 35, perhaps it is not. It undoubtedly also depends on how bad the call is against the current defense, and how probable a turnover is if your team appears confused. The alternative choice always available is a DOG. Is that universally better according to the model?

The "optimal" solution is to not require any "operational" time outs. On that, all can agree. However, in certain formations a run up the middle is impossible unless the QB carries the ball. Ideally the QB can compensate for the play being slow coming in, but bad personnel he cannot. Lots of "operational" time outs due to slow thinking and/or inadequate foresight are for sure something coaching staffs need to minimize. I am not so sure that time outs called due to "gotcha" situations where clearly the defense has the play defended and therefore an advantage are a bad thing. The QB needs to audible, but he isn't/didn't. If that happens a lot, there is a "tell" involved somewhere.

[Goose]

The bottom line is that the wasted time outs are worth a lot more than I thought they were, and that the risk from trying to get the “right” play in is much higher than calling a generic run up the middle if time is running low.  There is such a thing as decision paralysis and overthinking things.


That being said, the article notes that Sean McVey was one of the worst offenders of “operational time outs”, and the guy is only the youngest coach to ever win a Super Bowl.



BC

It's a model. Is it a useful model? I don't know, what is the evidence that it is? The model of timeout value compares the performance of teams that had N timeouts at a given time in the game to those that had N - 1 timeouts at the same point. They attempted to control for time remaining in the game etc. However, the elephant they ignore is that the number of "operational" time outs a team takes may be correlated but not causative to it's performance. If your team is playing poorly that day, making lots of mental mistakes, or the coaching staff is having an "off" day, or the opposing defense has you figured out, you may need those "operational" time outs. There is no way to control for that effect. If you don't need them, you don't take them, and if you do need them, you already have a problem compared to the teams that don't. The goal is to not need them. The control would be a team that needs them but doesn't take them. How do you create that control, since taking a time out isn't a random variable? Note that you still may be winning the game, because you are overall a better team than your opponent despite having an "off day" calling plays.

It is IMHO probable that this model simply reflects the fact that teams that call a lot of "operational" time outs have a problem already, not that calling the time out was a mistake. You have already made one more "significant" mistake than your opponent and have chosen to call an "operational" time out to remediate it. It isn't calling the time out that is the problem. It is the mistake. Some teams compensate better than others from needing the time out. There is no way to evaluate how many operational time outs SHOULD have been called (would have contributed to win probability) but were not.

The "value" of taking time out because the play-clock is running down instead of running a "bad" play clearly depends on the situation. If it is a critical third down in "no man's land" needed to keep a drive alive, it is probably a "good" call. If it is first and ten on your own 35, perhaps it is not. It undoubtedly also depends on how bad the call is against the current defense, and how probable a turnover is if your team appears confused. The alternative choice always available is a DOG. Is that universally better according to the model?

The "optimal" solution is to not require any "operational" time outs. On that, all can agree. However, in certain formations a run up the middle is impossible unless the QB carries the ball. Ideally the QB can compensate for the play being slow coming in, but bad personnel he cannot. Lots of "operational" time outs due to slow thinking and/or inadequate foresight are for sure something coaching staffs need to minimize. I am not so sure that time outs called due to "gotcha" situations where clearly the defense has the play defended and therefore an advantage are a bad thing. The QB needs to audible, but he isn't/didn't. If that happens a lot, there is a "tell" involved somewhere.

[WindyCityCard]

I don't find fivethirtyeight's analysis compelling.

As I understand it, their comparison is what happens after an operational timeout is called versus the AVERAGE outcome for all plays at that particular game situation. But presumably coaches don't call an operational timeout just to improve over average outcomes.  At least some of the time they do it because they think their current play call and personnel on the field are particularly bad given what the defense is doing.  So they're trying to improve from a below-average outcome to an above average one, not just from an average outcome to an above-average one.

Unless I'm missing something, fivethirtyeight's methodology doesn't properly consider that.  (And I'm not sure how they could, given that we don't observe what would have happened had the timeout not been called and we can't run a RCT on whether coaches call timeouts).  So it's not obvious to me that fivethirtyeight's analysis shows that at the moment coaches take operational timeouts they are making bad decisions.

A cleaner comparison is in the Advanced Football Analytics article that BostonCard linked and discussed: taking a timeout vs taking a delay of game penalty.  This sort of analysis of course does require assuming we have good estimates of the value of timeouts and the value of 5 yards given time, score, and field position.  But it does seem to make a pretty compelling case for taking a delay of game rather than a timeout.

One final note:  even if, at the moment that the play clock says 0:01, it's sometimes a good idea to take an operational timeout, that doesn't excuse coaches who frequently use them in situations where they took too long to send in their play & personnel.  My takeaway from the articles that BC links is that coaches who use a slow play calling system that leads to 1-2 operational timeouts per game are costing their team a nontrivial amount of win probability.  That is on the coach and his staff (much to the aggravation of many fans, myself included).

[WindyCityCard]

I don't find fivethirtyeight's analysis compelling.

As I understand it, their comparison is what happens after an operational timeout is called versus the AVERAGE outcome for all plays at that particular game situation. But presumably coaches don't call an operational timeout just to improve over average outcomes.  At least some of the time they do it because they think their current play call and personnel on the field are particularly bad given what the defense is doing.  So they're trying to improve from a below-average outcome to an above average one, not just from an average outcome to an above-average one.

Unless I'm missing something, fivethirtyeight's methodology doesn't properly consider that.  (And I'm not sure how they could, given that we don't observe what would have happened had the timeout not been called and we can't run a RCT on whether coaches call timeouts).  So it's not obvious to me that fivethirtyeight's analysis shows that at the moment coaches take operational timeouts they are making bad decisions.

A cleaner comparison is in the Advanced Football Analytics article that BostonCard linked and discussed: taking a timeout vs taking a delay of game penalty.  This sort of analysis of course does require assuming we have good estimates of the value of timeouts and the value of 5 yards given time, score, and field position.  But it does seem to make a pretty compelling case for taking a delay of game rather than a timeout.

One final note:  even if, at the moment that the play clock says 0:01, it's sometimes a good idea to take an operational timeout, that doesn't excuse coaches who frequently use them in situations where they took too long to send in their play & personnel.  My takeaway from the articles that BC links is that coaches who use a slow play calling system that leads to 1-2 operational timeouts per game are costing their team a nontrivial amount of win probability.  That is on the coach and his staff (much to the aggravation of many fans, myself included).

[BostonCard]

The point of the article is that time outs are much more valuable at the end of the half/game, and that taking the delay of game penalty is actually less of a penalty than using an operational time out, and almost no amount of genius playcalling overcomes that.  Yes, there is a good chance that you don’t need the time out at the end of the game.  But if you do, it can mean the difference between winning and losing.

Because of this, coaches would be better off with a fail safe play to which the QB defaults if time is running out.  Basically, in terms of value “generic play” > DoG penalty > operational time out.

BC

[BostonCard]

The point of the article is that time outs are much more valuable at the end of the half/game, and that taking the delay of game penalty is actually less of a penalty than using an operational time out, and almost no amount of genius playcalling overcomes that.  Yes, there is a good chance that you don’t need the time out at the end of the game.  But if you do, it can mean the difference between winning and losing.

Because of this, coaches would be better off with a fail safe play to which the QB defaults if time is running out.  Basically, in terms of value “generic play” > DoG penalty > operational time out.

BC

[lex24]

As with a lot of metrics, it’s “one size fits all” doesn’t cut it for me.  

Situation 1:  3d and 3 at midfield.

Situation 2.  3d and 5 on your own 20.

Situation 3 2nd and 2 on the 5O

Situation 4. 2nd and 7 on you own 20. 

Situation 5. 1st and 10 at 50

Situation 6 1st and 10 at your own 20 

Assume middle of second quarter for both. Tie score. You have 3 TOS left. Then same situations but you have 2 left

Same situations assume middle of 3d Tie score. 3 TOS left. Then tie score 2 left

Then same but 4th q. 10 minutes left. 3 tos. 2 tos.

Which ones do you can TOS for?

You get my point. It’s not just “operational” it’s situational. The ones that drive me crazy are the time outs versus just eating the DOG when it’s 3d and over 10. Chances of the first down are slim. The TO in that situation is almost certainly more valuable.

---

Anyone who watches a lot of football knows timeouts are extremely valuable, and the best teams almost never waste them. This data just provides empirical proof to back up what close followers of the sport and even those who play Madden always figure out.

One of my friends who plays it has always said head coaches need to play Madden to learn how to manage a game.

Right, cause guys sitting on their butts playing a video game know more  than NFL coaches. And the pressure is exactly the same as well.  Fans always think they know more.  They should try coaching.  Making decisions in real time that effect real people and real games with all the stuff flying around in a real game is a tad different….

[lex24]

As with a lot of metrics, it’s “one size fits all” doesn’t cut it for me.  

Situation 1:  3d and 3 at midfield.

Situation 2.  3d and 5 on your own 20.

Situation 3 2nd and 2 on the 5O

Situation 4. 2nd and 7 on you own 20. 

Situation 5. 1st and 10 at 50

Situation 6 1st and 10 at your own 20 

Assume middle of second quarter for both. Tie score. You have 3 TOS left. Then same situations but you have 2 left

Same situations assume middle of 3d Tie score. 3 TOS left. Then tie score 2 left

Then same but 4th q. 10 minutes left. 3 tos. 2 tos.

Which ones do you can TOS for?

You get my point. It’s not just “operational” it’s situational. The ones that drive me crazy are the time outs versus just eating the DOG when it’s 3d and over 10. Chances of the first down are slim. The TO in that situation is almost certainly more valuable.

---

Anyone who watches a lot of football knows timeouts are extremely valuable, and the best teams almost never waste them. This data just provides empirical proof to back up what close followers of the sport and even those who play Madden always figure out.

One of my friends who plays it has always said head coaches need to play Madden to learn how to manage a game.

Right, cause guys sitting on their butts playing a video game know more  than NFL coaches. And the pressure is exactly the same as well.  Fans always think they know more.  They should try coaching.  Making decisions in real time that effect real people and real games with all the stuff flying around in a real game is a tad different….

[Goose]

Because of this, coaches would be better off with a fail safe play to which the QB defaults if time is running out.  Basically, in terms of value “generic play” > DoG penalty > operational time out.



BC

The "generic" play can't be fully generic. You can't run a back up the middle when you are in an empty set. A QB sneak is also impossible if you are in the gun, it will have to be a QB run. You can get your QB hurt that way. If the QB is told to just throw the ball away, he may have a problem with intentional grounding etc. if the play was not originally a pass. If the original play was a pass, you don't need the generic play. Run the play as called and throw it away if necessary.

If the generic play requires an audible, you have to do so with enough time on the clock for said audible. The QB has to decide to go to the "default" play soon enough to do so. However, since many DOG/operational time outs occur when the QB is already trying to audible, it really isn't a very "generic" solution. It also won't work if the QB doesn't recognize the DOG is unavoidable and thinks he will get the play off. Many operational time outs are called by the coaches in that situation.

I am pretty sure the "default play" approach isn't practical for these reasons and is therefore not much used. The QB has audibles available in most cases. He needs to use them. If he is running out of time, he can't. In the "old days" there was always a tap signal to the center that it would be a sneak. Takes no time and nobody else is involved. Today, with the QB not under center, it's not available.

One problem with your three choice ranking is that it doesn't address one case that often causes operational time outs. The clock isn't going to expire and you are going start a play. However, that play is a real bad choice (for whatever reason), or that you are going to start a play after an audible that has clearly confused some of your players. You could end up with a turnover or worse.

I do agree with Lex24 that it is situational. All operational timeouts are not equally bad. If it is a 3rd and 2 in the opponents half of the field in an even quarter and I have all my timeouts, I would call the timeout for sure. I prefer the loss of the timeout to 3rd and 7. If it is 3rd and 10 on my own 35, I am not going to call the timeout to avoid a DOG or a bad play (except in very, very unusual circumstances).

[Goose]

Because of this, coaches would be better off with a fail safe play to which the QB defaults if time is running out.  Basically, in terms of value “generic play” > DoG penalty > operational time out.



BC

The "generic" play can't be fully generic. You can't run a back up the middle when you are in an empty set. A QB sneak is also impossible if you are in the gun, it will have to be a QB run. You can get your QB hurt that way. If the QB is told to just throw the ball away, he may have a problem with intentional grounding etc. if the play was not originally a pass. If the original play was a pass, you don't need the generic play. Run the play as called and throw it away if necessary.

If the generic play requires an audible, you have to do so with enough time on the clock for said audible. The QB has to decide to go to the "default" play soon enough to do so. However, since many DOG/operational time outs occur when the QB is already trying to audible, it really isn't a very "generic" solution. It also won't work if the QB doesn't recognize the DOG is unavoidable and thinks he will get the play off. Many operational time outs are called by the coaches in that situation.

I am pretty sure the "default play" approach isn't practical for these reasons and is therefore not much used. The QB has audibles available in most cases. He needs to use them. If he is running out of time, he can't. In the "old days" there was always a tap signal to the center that it would be a sneak. Takes no time and nobody else is involved. Today, with the QB not under center, it's not available.

One problem with your three choice ranking is that it doesn't address one case that often causes operational time outs. The clock isn't going to expire and you are going start a play. However, that play is a real bad choice (for whatever reason), or that you are going to start a play after an audible that has clearly confused some of your players. You could end up with a turnover or worse.

I do agree with Lex24 that it is situational. All operational timeouts are not equally bad. If it is a 3rd and 2 in the opponents half of the field in an even quarter and I have all my timeouts, I would call the timeout for sure. I prefer the loss of the timeout to 3rd and 7. If it is 3rd and 10 on my own 35, I am not going to call the timeout to avoid a DOG or a bad play (except in very, very unusual circumstances).

[BostonCard]

My gut tells me that you guys are both right and that the analysis overvalues time outs towards the end of games.  I think a lost time out is probably worth less than .03 WP, but I don’t have the analysis to back that up.

Circumstances definitely matter, but the point of comparison was a third and 1 at midfield with the offense up by 3 points, midway through the third quarter. While it is not the highest leverage situation you can think up, the difference between third and one and third and six is pretty pronounced, because if the offense gets the first down it can move into a position to pad its lead, while if they don’t get first down, they will have to punt. Under most circumstances, the DoG will be less damaging than that, so the fact that under those circumstances the DoG is only worth .02 WP suggests that in almost all situations, the time out will be more valuable than avoiding the penalty, of the value of the time out is correctly measured (which, as I said, I’m not sure).

BC

[BostonCard]

My gut tells me that you guys are both right and that the analysis overvalues time outs towards the end of games.  I think a lost time out is probably worth less than .03 WP, but I don’t have the analysis to back that up.

Circumstances definitely matter, but the point of comparison was a third and 1 at midfield with the offense up by 3 points, midway through the third quarter. While it is not the highest leverage situation you can think up, the difference between third and one and third and six is pretty pronounced, because if the offense gets the first down it can move into a position to pad its lead, while if they don’t get first down, they will have to punt. Under most circumstances, the DoG will be less damaging than that, so the fact that under those circumstances the DoG is only worth .02 WP suggests that in almost all situations, the time out will be more valuable than avoiding the penalty, of the value of the time out is correctly measured (which, as I said, I’m not sure).

BC

[winflop]

Shaw's problem with timeouts isn't just burning them, it's also about not using them. Anyone else remember the end of the UW game where by not using timeouts he wasted 30+ seconds off the game clock and basically cost us any real chance of winning? I can't recall other specifics but I can remember screaming about him NOT calling a timeout at the end of a half or game and ruining a potential score by doing so.

Shaw has a long-standing problem with timeout management that continues to be unaddressed. He's got a vast network of mentors. Why can't he reach out to one or more of them and ask them for help? Maybe he has, but what we've seen on the field doesn't indicate that it's happened

[winflop]

Shaw's problem with timeouts isn't just burning them, it's also about not using them. Anyone else remember the end of the UW game where by not using timeouts he wasted 30+ seconds off the game clock and basically cost us any real chance of winning? I can't recall other specifics but I can remember screaming about him NOT calling a timeout at the end of a half or game and ruining a potential score by doing so.

Shaw has a long-standing problem with timeout management that continues to be unaddressed. He's got a vast network of mentors. Why can't he reach out to one or more of them and ask them for help? Maybe he has, but what we've seen on the field doesn't indicate that it's happened

[fullmetal]

Last I heard, football situational playcalling had too much variance from which to draw statistical meaning.  Heard that from a friend who attended the MIT Sloan sports analytics conference some years ago.

[fullmetal]

Last I heard, football situational playcalling had too much variance from which to draw statistical meaning.  Heard that from a friend who attended the MIT Sloan sports analytics conference some years ago.

[old spanish trail]

I'm sorry. Based on years of watching both pro and college football, I think we all have a fairly well thought out view re this issue; Teejers on one end of the scale, Spanish Trail toward the other.

[old spanish trail]

I'm sorry. Based on years of watching both pro and college football, I think we all have a fairly well thought out view re this issue; Teejers on one end of the scale, Spanish Trail toward the other.

[slide]

BC -- any modeling the lost value of back-to-back operational time outs or a DOG following an operational time out?  :)

[slide]

BC -- any modeling the lost value of back-to-back operational time outs or a DOG following an operational time out?  :)