[ChicagoCard]

Al.com (The Birmingham News, The Huntsville Times and Mobile's Press-Register) just concluded a series of hypothetical Pac-12 vs SEC matchups with Stanford at Auburn. Not much to see here but I thought worth posting. Apparently they were impressed by Gaffney as they had him rushing for over 200 yards.

[ChicagoCard]

Al.com (The Birmingham News, The Huntsville Times and Mobile's Press-Register) just concluded a series of hypothetical Pac-12 vs SEC matchups with Stanford at Auburn. Not much to see here but I thought worth posting. Apparently they were impressed by Gaffney as they had him rushing for over 200 yards.

[81alum]

Interesting.  I just went to the simulator and ran the same matchup and Stanford won 35-28.  Furthermore, at the bottom of the simulator stats it states:

Matchup Simulation Stats

    This game has been simulated 47 times.
    Stanford (2014) has won 31 times (66.0%), won by > 20 pts 9 times (19.1%)
    Auburn (2014) has won 16 times (34.0%), won by > 20 pts 3 times (6.4%)
    The average score is Stanford (2014): 29.9 - Auburn (2014): 24.1, decided by < 5 pts 13 times (27.7%)
    The game has gone into overtime 1 times (2.1%)

[81alum]

Interesting.  I just went to the simulator and ran the same matchup and Stanford won 35-28.  Furthermore, at the bottom of the simulator stats it states:

Matchup Simulation Stats

    This game has been simulated 47 times.
    Stanford (2014) has won 31 times (66.0%), won by > 20 pts 9 times (19.1%)
    Auburn (2014) has won 16 times (34.0%), won by > 20 pts 3 times (6.4%)
    The average score is Stanford (2014): 29.9 - Auburn (2014): 24.1, decided by < 5 pts 13 times (27.7%)
    The game has gone into overtime 1 times (2.1%)

[PrinceLightfoot]

Auburn at home seems to have done slightly better but the simulator clearly does not favor Auburn over Stanford:

This game has been simulated 101 times.
Auburn (2014) has won 42 times (41.6%), won by > 20 pts 7 times (6.9%)
Stanford (2014) has won 59 times (58.4%), won by > 20 pts 10 times (9.9%)
The average score is Auburn (2014): 25.9 - Stanford (2014): 27.7, decided by < 5 pts 19 times (18.8%)
The game has gone into overtime 8 times (7.9%)

[PrinceLightfoot]

Auburn at home seems to have done slightly better but the simulator clearly does not favor Auburn over Stanford:

This game has been simulated 101 times.
Auburn (2014) has won 42 times (41.6%), won by > 20 pts 7 times (6.9%)
Stanford (2014) has won 59 times (58.4%), won by > 20 pts 10 times (9.9%)
The average score is Auburn (2014): 25.9 - Stanford (2014): 27.7, decided by < 5 pts 19 times (18.8%)
The game has gone into overtime 8 times (7.9%)

[fullmetal]

What did it say about Alabama vs. Stanford?  Or Michigan St. vs. Stanford?

[fullmetal]

What did it say about Alabama vs. Stanford?  Or Michigan St. vs. Stanford?

[81alum]

What did it say about Alabama vs. Stanford?  Or Michigan St. vs. Stanford?

FINAL SCORE -- Stanford (2014): 20 - ALABAMA (2014): 24

Matchup Simulation Stats

    This game has been simulated 27 times.
    Stanford (2014) has won 7 times (25.9%), won by > 20 pts 1 times (3.7%)
    ALABAMA (2014) has won 20 times (74.1%), won by > 20 pts 8 times (29.6%)
    The average score is Stanford (2014): 16.4 - ALABAMA (2014): 27.7, decided by < 5 pts 6 times (22.2%)
    The game has gone into overtime 0 times (0.0%)

FINAL SCORE -- Stanford (2014): 28 - Michigan State (2014): 20

Matchup Simulation Stats

    This game has been simulated 12 times.
    Stanford (2014) has won 7 times (58.3%), won by > 20 pts 1 times (8.3%)
    Michigan State (2014) has won 5 times (41.7%), won by > 20 pts 1 times (8.3%)
    The average score is Stanford (2014): 19.4 - Michigan State (2014): 19.0, decided by < 5 pts 2 times (16.7%)
    The game has gone into overtime 1 times (8.3%)

[81alum]

What did it say about Alabama vs. Stanford?  Or Michigan St. vs. Stanford?

FINAL SCORE -- Stanford (2014): 20 - ALABAMA (2014): 24

Matchup Simulation Stats

    This game has been simulated 27 times.
    Stanford (2014) has won 7 times (25.9%), won by > 20 pts 1 times (3.7%)
    ALABAMA (2014) has won 20 times (74.1%), won by > 20 pts 8 times (29.6%)
    The average score is Stanford (2014): 16.4 - ALABAMA (2014): 27.7, decided by < 5 pts 6 times (22.2%)
    The game has gone into overtime 0 times (0.0%)

FINAL SCORE -- Stanford (2014): 28 - Michigan State (2014): 20

Matchup Simulation Stats

    This game has been simulated 12 times.
    Stanford (2014) has won 7 times (58.3%), won by > 20 pts 1 times (8.3%)
    Michigan State (2014) has won 5 times (41.7%), won by > 20 pts 1 times (8.3%)
    The average score is Stanford (2014): 19.4 - Michigan State (2014): 19.0, decided by < 5 pts 2 times (16.7%)
    The game has gone into overtime 1 times (8.3%)

[washingtonismoney]

Obviously wrong so irrelevent.

Interesting new epistemological ground to explore. How can a simulation be wrong? (Does it *exist*?)

[washingtonismoney]

Obviously wrong so irrelevent.

Interesting new epistemological ground to explore. How can a simulation be wrong? (Does it *exist*?)

[washingtonismoney]

I was astroplaning last night and saw a tree fall in the woods.  There being no one around there was no sound.  However, the fallen tree root unearthed an ancient stone, which I saw from above.  It was written in a Minoan version of ancient (is there any other kind? ) Sumerian but, thanks to my Stanford anthropology training, I was able, just before drifting back to my bed.  It read, " Michigan State won.  Behold the words of mighty empires."

So there.

I'm at least 55% serious with my post! How would one declare a simulation obviously wrong? Applied to the Rose Bowl, for example: it's pretty easy to imagine a Stanford win, even with trivial changes -- say, catching one of the three or four dropped first downs, or one of the multitude of dropped picks? That's not to say MSU didn't *deserve* to win, just that a simulation showing a Stanford win isn't, like, crazy or anything. (Assuming we think simulations/hypotheticals have any truth-value at all!)

[washingtonismoney]

I was astroplaning last night and saw a tree fall in the woods.  There being no one around there was no sound.  However, the fallen tree root unearthed an ancient stone, which I saw from above.  It was written in a Minoan version of ancient (is there any other kind? ) Sumerian but, thanks to my Stanford anthropology training, I was able, just before drifting back to my bed.  It read, " Michigan State won.  Behold the words of mighty empires."

So there.

I'm at least 55% serious with my post! How would one declare a simulation obviously wrong? Applied to the Rose Bowl, for example: it's pretty easy to imagine a Stanford win, even with trivial changes -- say, catching one of the three or four dropped first downs, or one of the multitude of dropped picks? That's not to say MSU didn't *deserve* to win, just that a simulation showing a Stanford win isn't, like, crazy or anything. (Assuming we think simulations/hypotheticals have any truth-value at all!)

[stupac2]

I'm at least 55% serious with my post! How would one declare a simulation obviously wrong? Applied to the Rose Bowl, for example: it's pretty easy to imagine a Stanford win, even with trivial changes -- say, catching one of the three or four dropped first downs, or one of the multitude of dropped picks? That's not to say MSU didn't *deserve* to win, just that a simulation showing a Stanford win isn't, like, crazy or anything. (Assuming we think simulations/hypotheticals have any truth-value at all!)

That bullshit endzone PI call, for one.

But your point is completely right, it's actually impossible to say that a simulation is wrong from a sample size of 1. A simulation is telling you a range of possibilities (something they directly say in the results!) and as such any given actuality fits within the framework of the simulation. The question is how it performs over many trials, which is obviously something we can't actually know for many, many years.

[stupac2]

I'm at least 55% serious with my post! How would one declare a simulation obviously wrong? Applied to the Rose Bowl, for example: it's pretty easy to imagine a Stanford win, even with trivial changes -- say, catching one of the three or four dropped first downs, or one of the multitude of dropped picks? That's not to say MSU didn't *deserve* to win, just that a simulation showing a Stanford win isn't, like, crazy or anything. (Assuming we think simulations/hypotheticals have any truth-value at all!)

That bullshit endzone PI call, for one.

But your point is completely right, it's actually impossible to say that a simulation is wrong from a sample size of 1. A simulation is telling you a range of possibilities (something they directly say in the results!) and as such any given actuality fits within the framework of the simulation. The question is how it performs over many trials, which is obviously something we can't actually know for many, many years.

[Hank 91]

Just for fun...

FINAL SCORE -- Stanford (2012): 24 - Stanford (2014): 14

Matchup Simulation Stats

This game has been simulated 7 times.
Stanford (2012) has won 5 times (71.4%), won by > 20 pts 1 times (14.3%)
Stanford (2014) has won 2 times (28.6%), won by > 20 pts 0 times (0.0%)
The average score is Stanford (2012): 22.6 - Stanford (2014): 16.3, decided by < 5 pts 2 times (28.6%)
The game has gone into overtime 0 times (0.0%)

[Hank 91]

Just for fun...

FINAL SCORE -- Stanford (2012): 24 - Stanford (2014): 14

Matchup Simulation Stats

This game has been simulated 7 times.
Stanford (2012) has won 5 times (71.4%), won by > 20 pts 1 times (14.3%)
Stanford (2014) has won 2 times (28.6%), won by > 20 pts 0 times (0.0%)
The average score is Stanford (2012): 22.6 - Stanford (2014): 16.3, decided by < 5 pts 2 times (28.6%)
The game has gone into overtime 0 times (0.0%)

[French Rage]

So we're not getting the offense injection we were hoping for?

[French Rage]

So we're not getting the offense injection we were hoping for?

[Treebound]

I think the offensive injection is the "Luck" factor.  Injections like that are rare occurrences and I'm sure the simulator and the astralplane take that into account!  Now if it came down to a field goal kick....

[Treebound]

I think the offensive injection is the "Luck" factor.  Injections like that are rare occurrences and I'm sure the simulator and the astralplane take that into account!  Now if it came down to a field goal kick....

[CTcard]

So we're not getting the offense injection we were hoping for?

It may be worth noting that that web site ends up listing the team of the 2013-14 season as Stanford 2014.
Thus that particular simulation was the 2011-12 team (quarterbacked by Andrew Luck) vs the 2013-14 team (quarterbacked by Kevin Hogan - all year). Thus it doesn't say anything about the offense-injection we may get this coming year, and I guess we already know what level of injection we had last year.

[CTcard]

So we're not getting the offense injection we were hoping for?

It may be worth noting that that web site ends up listing the team of the 2013-14 season as Stanford 2014.
Thus that particular simulation was the 2011-12 team (quarterbacked by Andrew Luck) vs the 2013-14 team (quarterbacked by Kevin Hogan - all year). Thus it doesn't say anything about the offense-injection we may get this coming year, and I guess we already know what level of injection we had last year.

[stupac2]

Simulated results depend, wholly, on the input and therefor are inherently wrong, unless the inputter is omniscient which, since this is from an SEC blog is unlikely.

I think you need to familiarize yourself with the concept of error bars. Regardless, that standard is a completely ridiculous way to judge anything, you're basically saying that all simulations are a waste of time. This is trivially false. It's true that a simulation is only going to be so good (though it doesn't depend entirely on inputs, the way the inputs are handled is probably more important), but it doesn't mean that the simulation is inherently wrong. You're basically saying that entire industries shouldn't exist, development of modern computer components depends utterly on simulations, and the inputs (things like the laws of physics and materials behaviors) and the processes (things like grid sizes) are all imperfect. Does that mean it's a waste of time?

Simulations depend on simulating (see a trend here?) actual outcomes given certain random events.  A mild Russian winter in 1943; US carriers amassed in Pearl Harbor on Dec. 7th; Pickett not charging; the Chinese not coming to North Korea's aid; etc.  But a programmer has to set the probabilities of those events happening.  Simulate that bball game as often as you like. How often does the freshman with two HRs all season hit the walk-off?

This convinces me that you don't understand the point of simulations. It's not to say with certainty what will happen. It's to gauge probabilities. Vegas odds are based on simulations too (of a sort, anyway), over the long run they do pretty well for themselves.

[stupac2]

Simulated results depend, wholly, on the input and therefor are inherently wrong, unless the inputter is omniscient which, since this is from an SEC blog is unlikely.

I think you need to familiarize yourself with the concept of error bars. Regardless, that standard is a completely ridiculous way to judge anything, you're basically saying that all simulations are a waste of time. This is trivially false. It's true that a simulation is only going to be so good (though it doesn't depend entirely on inputs, the way the inputs are handled is probably more important), but it doesn't mean that the simulation is inherently wrong. You're basically saying that entire industries shouldn't exist, development of modern computer components depends utterly on simulations, and the inputs (things like the laws of physics and materials behaviors) and the processes (things like grid sizes) are all imperfect. Does that mean it's a waste of time?

Simulations depend on simulating (see a trend here?) actual outcomes given certain random events.  A mild Russian winter in 1943; US carriers amassed in Pearl Harbor on Dec. 7th; Pickett not charging; the Chinese not coming to North Korea's aid; etc.  But a programmer has to set the probabilities of those events happening.  Simulate that bball game as often as you like. How often does the freshman with two HRs all season hit the walk-off?

This convinces me that you don't understand the point of simulations. It's not to say with certainty what will happen. It's to gauge probabilities. Vegas odds are based on simulations too (of a sort, anyway), over the long run they do pretty well for themselves.

[Stymie]

Can we do Stanford 2013-4 vs. Stanford 2014-5 using predictive modelling?

[Stymie]

Can we do Stanford 2013-4 vs. Stanford 2014-5 using predictive modelling?

[CTcard]

Can we do Stanford 2013-4 vs. Stanford 2014-5 using predictive modelling?

Depends what you mean by "we".

Certainly somebody could do it. They'd have to figure out what they are going to use to predict 2014-15 performance. The people who made the game simulator featured here don't say in detail what they do, but do say "Game Sim takes the team's statistical profile and plays out each snap." So, presumably you need actual season statistics to simulate a game there.

[CTcard]

Can we do Stanford 2013-4 vs. Stanford 2014-5 using predictive modelling?

Depends what you mean by "we".

Certainly somebody could do it. They'd have to figure out what they are going to use to predict 2014-15 performance. The people who made the game simulator featured here don't say in detail what they do, but do say "Game Sim takes the team's statistical profile and plays out each snap." So, presumably you need actual season statistics to simulate a game there.

[cardcoug]

Relax, Stupac.  Just having a little fun.

What could possibly be more fun than a philosophical argument about something with direct application to football? As long as everything remains civil, of course. :) Since I can never get my colleagues to care about football, and I can never get my football friends to care about philosophy, I'm going to take this opportunity to avoid grading papers and babble a bit.

This is a really thorny epistemological issue. People have rightly pointed out above that assessing the accuracy of a simulation is not like assessing the truth or falsity of a standard proposition, because insofar as a simulation purports to make any claim about the real world, it does so in terms of probabilities. You need to observe many instances of an event to gauge the accuracy of the simulation, and a simulation is not proven wrong (or even faulty) by the singular occurrence of the less probable outcome. And you also can never get a true/false result, but rather a higher or lower degree of confidence. So I think Stupac's point is exactly right. Put even more strongly, it doesn't even make sense to say that a simulation is wrong or false, only that it is not very good.

But there is an additional complication for the more fantastical sorts of simulations. There is the basic fact that, for football at least, there is never a large enough sample size to conclude anything of any real significance whatsoever regarding a particular simulation. So if you're going to assess these simulations, you can only do so indirectly by assessing the model's performance on a larger scale (e.g. pickfire did better than accuscore over the course of the season, so you should place higher confidence in the former's simulation of a particular game). But here's the problem with the fantastical ones: there is no way to do this. There doesn't seem to be any possible evidence that could distinguish between different models for, e.g., making predictions across eras. In these cases, I think ferrari's basic point is right, and as stupac correctly pointed out, that means that these simulations are completely arbitrary and worthless.

But Stanford vs. Auburn in 2013? Yeah, I'll take that.

[cardcoug]

Relax, Stupac.  Just having a little fun.

What could possibly be more fun than a philosophical argument about something with direct application to football? As long as everything remains civil, of course. :) Since I can never get my colleagues to care about football, and I can never get my football friends to care about philosophy, I'm going to take this opportunity to avoid grading papers and babble a bit.

This is a really thorny epistemological issue. People have rightly pointed out above that assessing the accuracy of a simulation is not like assessing the truth or falsity of a standard proposition, because insofar as a simulation purports to make any claim about the real world, it does so in terms of probabilities. You need to observe many instances of an event to gauge the accuracy of the simulation, and a simulation is not proven wrong (or even faulty) by the singular occurrence of the less probable outcome. And you also can never get a true/false result, but rather a higher or lower degree of confidence. So I think Stupac's point is exactly right. Put even more strongly, it doesn't even make sense to say that a simulation is wrong or false, only that it is not very good.

But there is an additional complication for the more fantastical sorts of simulations. There is the basic fact that, for football at least, there is never a large enough sample size to conclude anything of any real significance whatsoever regarding a particular simulation. So if you're going to assess these simulations, you can only do so indirectly by assessing the model's performance on a larger scale (e.g. pickfire did better than accuscore over the course of the season, so you should place higher confidence in the former's simulation of a particular game). But here's the problem with the fantastical ones: there is no way to do this. There doesn't seem to be any possible evidence that could distinguish between different models for, e.g., making predictions across eras. In these cases, I think ferrari's basic point is right, and as stupac correctly pointed out, that means that these simulations are completely arbitrary and worthless.

But Stanford vs. Auburn in 2013? Yeah, I'll take that.

[Extra Point]

The validity of any computer simulation is dependent upon the number of variables that are accounted for in the simulation. When it comes to football, the number of factors that could affect the outcome of a game is enormous. There is no way to include sore thumbs and family problems in these simulations.

[Extra Point]

The validity of any computer simulation is dependent upon the number of variables that are accounted for in the simulation. When it comes to football, the number of factors that could affect the outcome of a game is enormous. There is no way to include sore thumbs and family problems in these simulations.