Recruiting and team statistics - Printable Version
+- The CardBoard (https://thecardboard.org/board)
+-- Forum: C-House! (https://thecardboard.org/board/forum-4.html)
+--- Forum: The CardBoard (https://thecardboard.org/board/forum-5.html)
+--- Thread: Recruiting and team statistics (/thread-11513.html)
+- The CardBoard (https://thecardboard.org/board)
+-- Forum: C-House! (https://thecardboard.org/board/forum-4.html)
+--- Forum: The CardBoard (https://thecardboard.org/board/forum-5.html)
+--- Thread: Recruiting and team statistics (/thread-11513.html)
Recruiting and team statistics - BostonCard - 02-01-2016
http://www.sbnation.com/college-football-recruiting/2016/2/1/10867816/ncaa-football-recruting-rankings-impact
Recruiting matters. And it matters more for defense than offense. And it matters more for pass defense than rush defense.
More specifically, 30% of the variability in offensive quality (as measured by S&P) can be accounted for by recruiting and 40% of defense (as measured by S&P) can be accounted for by recruiting.
BC
Re: Recruiting and team statistics - CTcard - 02-01-2016
This article is better than the more typical "analysis" that covers the obvious, and meaningless, conclusion that there is a correlation between recruiting rankings and winning percentage.
In this case they've looked at a little more detail and the basic result that recruiting is more correlated to defense and pass-defense than anything else does seem to be in the numbers beyond random variation. And at the very least, they've stopped to think if these two basic results make sense.
Among the items that give me pause are the relative squared correlations between the 2-year recruiting rankings and the 5- year recruiting rankings. Unfortunately, despite a link, I cannot find a definition of those terms. The obvious meaning would be that the 5-year recruiting would be some kind of average of the past 5 years of recruiting and the 2-year would be over the past two years. However, very few teams play a large fraction of their snaps with true frosh and sophs. One would expect that on field talent would mostly be determined by seniors, 5th year seniors and juniors. So, a higher correlation between on-field performance and 2 year recruiting over 5 year recruiting would seem to indicate that recruiting well is more likely a result of winning than a cause of winning.
Of course in reality it is both a cause and a result. It also seems likely both that raw talent is more important on defense than offense - and in defending the pass more than the run. But I feel stronger about that conclusion by analyzing what it takes to play these parts of the game than complicated statistical analysis.
-----------
It occurs to me that two bits of analysis that might give more insight into the relative importance of recruiting (in the sense of recruiting rankings) versus other aspects of coaching to the plight of a program would be to study what happens in the years preceding a significant rise or fall of a program.
For Stanford, for example, during the 2008-2010 rise the majority of the players on the field were from recruiting classes that were not highly regarded. From that, it would appear that either player development, talent evaluation, or the like was more important than persuading acknowledged star-HS players to come to Stanford (the essence of strong service rankings). After the on-field results improved, recruiting picked up as a result.
Similarly, in the other direction, my memory is that the most recent fall in the fortunes of Texas came on the backs of classes that were very highly rated. Another example indicating that recruiting ranking wasn't the critical issue.
Well, that's now an unrepresentative sample of two.
Re: Recruiting and team statistics - BostonCard - 02-01-2016
(02-01-2016, 09:18 PM)CTcard link Wrote:This article is better than the more typical "analysis" that covers the obvious, and meaningless, conclusion that there is a correlation between recruiting rankings and winning percentage.
In this case they've looked at a little more detail and the basic result that recruiting is more correlated to defense and pass-defense than anything else does seem to be in the numbers beyond random variation. And at the very least, they've stopped to think if these two basic results make sense.
Among the items that give me pause are the relative squared correlations between the 2-year recruiting rankings and the 5- year recruiting rankings. Unfortunately, despite a link, I cannot find a definition of those terms. The obvious meaning would be that the 5-year recruiting would be some kind of average of the past 5 years of recruiting and the 2-year would be over the past two years. However, very few teams play a large fraction of their snaps with true frosh and sophs. One would expect that on field talent would mostly be determined by seniors, 5th year seniors and juniors. So, a higher correlation between on-field performance and 2 year recruiting over 5 year recruiting would seem to indicate that recruiting well is more likely a result of winning than a cause of winning.
Of course in reality it is both a cause and a result. It also seems likely both that raw talent is more important on defense than offense - and in defending the pass more than the run. But I feel stronger about that conclusion by analyzing what it takes to play these parts of the game than complicated statistical analysis.
-----------
It occurs to me that two bits of analysis that might give more insight into the relative importance of recruiting (in the sense of recruiting rankings) versus other aspects of coaching to the plight of a program would be to study what happens in the years preceding a significant rise or fall of a program.
For Stanford, for example, during the 2008-2010 rise the majority of the players on the field were from recruiting classes that were not highly regarded. From that, it would appear that either player development, talent evaluation, or the like was more important than persuading acknowledged star-HS players to come to Stanford (the essence of strong service rankings). After the on-field results improved, recruiting picked up as a result.
Similarly, in the other direction, my memory is that the most recent fall in the fortunes of Texas came on the backs of classes that were very highly rated. Another example indicating that recruiting ranking wasn't the critical issue.
Well, that's now an unrepresentative sample of two.
Some good thoughts, as always, CTCard.
One thing I liked about the article is that the correlations opened a new line of thinking. Prior to seeing the correlations, it didn't occur to me to even think about whether recruiting rankings would influence the offense or defense more. Yes, there narrative is more convincing than the statistics, but without having seen the statistics, would the narrative evaluation even have been thought up?
I agree that a big part of the problem is autocorrelation. Alabama will (at least while Nick Saban is the coach) always field very good teams. And it will always recruit well. Kansas will always struggle in recruiting and on the field. Thus, there will always be an association between recruiting quality and outcomes, but I bet there is also a correlation if you looked at how a team does, and 2-year recruiting in the NEXT season.
There are statistical methods to deal with this, but they are complex. Some sort of mixed effects model (allowing for team-level effects) or an autocorrelation time series model would be needed. In theory, though, if recruiting mattered, some weighted average of recruiting quality in the prior 5 years (probably weighted maximally to 4 years back) would be correlated with on-field performance within an individual team AND between teams, once you adjust for the overall team quality. I think it could be done, but I don't have a ton of expertise running these types of analyses. My gut feeling is that the results would probably show a statistically significant correlation, but the strength of that correlation would be weakened by the team effects.
I think building upon what you mentioned, I think another interesting analysis would be to use what is called a natural experiment, where some external factor is expected to affect recruiting and on-field results. In the first case, what we are probably thinking of is a coaching change, and you can examine whether after the coaching change on-field results changed immediately or as a result of improved recruiting (in the Harbaugh transition, both are probably true; there was a very large improvement the first year, before any sort of change in recruiting could have taken place, but there continued to be improvement over the next three years as guys that Harbaugh recruited, such as Andrew Luck, came to dominate the roster).
Another place to look would be the effect of scholarship restrictions from NCAA sanctions, which would be an external factor affected recruiting (technically, this would be what is called an instrumental variable). If recruiting affected on-field performance then the NCAA sanctions should have an effect on-field performance (with a lag). USC's experience here probably supports that assertion, but it is very confounded by other factors, such as the coaching changes.
BC
Re: Recruiting and team statistics - CTcard - 02-01-2016
(02-01-2016, 09:52 PM)Boston Card link Wrote:One thing I liked about the article is that the correlations opened a new line of thinking. Prior to seeing the correlations, it didn't occur to me to even think about whether recruiting rankings would influence the offense or defense more. Yes, there narrative is more convincing than the statistics, but without having seen the statistics, would the narrative evaluation even have been thought up?
It's long been stated among football folks that raw athletic talent is more important on defense than offense. The reason is pretty intuitive: the offense determines what is going to happen on a play, and can scheme to it's own strengths and the defense's weakness whereas the defense has to react to what the offense tries. Of course, positing that doesn't make it true - and if one hadn't thought about it then such an analysis might make one hunt this stuff down.
Quote:... In theory, though, if recruiting mattered, some weighted average of recruiting quality in the prior 5 years (probably weighted maximally to 4 years back) would be correlated with on-field performance within an individual team AND between teams, once you adjust for the overall team quality.
... If recruiting affected on-field performance then the NCAA sanctions should have an effect on-field performance (with a lag).
I don't see that there is any point in trying to find an effect of recruiting on performance. Of course it matters. There's a reason Group of Five programs rarely beat Power 5 teams and Division II teams won't beat Division I.
The questions that I think are interesting are something more like:
What is the relevant importance of recruiting versus other aspects of coaching?
For that matter, what matters more in recruiting - evaluation or convincing obviously good kids to choose your school (i.e. recruting rankings)?Â
When a team changes their fortunes substantially, is it more likely because of recruiting or other issues?
Quote:In the first case, what we are probably thinking of is a coaching change, and you can examine whether after the coaching change on-field results changed immediately or as a result of improved recruiting (in the Harbaugh transition, both are probably true; there was a very large improvement the first year, before any sort of change in recruiting could have taken place, but there continued to be improvement over the next three years as guys that Harbaugh recruited, such as Andrew Luck, came to dominate the roster).
I looked into this a bit as a result of a thread on the Bootleg.
I used Scout since it was easy.
Here are the recruiting rankings by year:
Scout Class rankings:
2002 #30Â (HC = Buddy)
2003 #26
2004 #46
2005 #38Â (HC = Wlat)
2006 #38Â [R. Shaw Dean of Admissions]
2007 #43Â (HC = Harbs)
2008 #44
2009 #16
2010 #24
2011 #23Â (HC = Shaw)
2012 #7
2013 #57
2014 #16
2015 #25
If you put together the main classes that contribute towards each team (sophomores through 5th year seniors), here are the ranked classes that contributed:
2006: 1-11. Recruits: #30 (5th yrs), #26 (SRs), #46 (JRs), #38 (SOs)
2007:Â 4-8. Recruits: #26 (5th yrs), #46 (SRs), #38 (JRs), #38 (SOs)
2008:Â 5-7. Recruits: #46 (5th yrs), #38 (SRs), #38 (JRs), #43 (SOs)
2009:Â 8-5. Recruits: #38 (5th yrs), #38 (SRs), #43 (JRs), #44 (SOs)
2010: 12-1. Recruits: #38 (5th yrs), #43 (SRs), #44 (JRs), #16(SOs)
2011: 11-2. Recruits: #43 (5th yrs), #44 (SRs), #16 (JRs), #24 (SOs)
2012: 12-2. Recruits: #44 (5th yrs), #16 (SRs), #24 (JRs), #23 (SOs)
2013: 11-3. Recruits: #16 (5th yrs), #24 (SRs), #23 (JRs), #7 (SOs)
2014:Â 8-5. Recruits: #24 (5th yrs), #23 (SRs), #7 (JRs), #57 (SOs)
2015: 12-2. Recruits: #23 (5th yrs), #7 (SRs), #57 (JRs), #16 (SOs)
----------
A few things stand out.
- Our recruiting didn't suddenly take off either with the hiring of Rick Shaw as Dean of Admissions or Harbs as coach. The break (upwards) in our recruiting classes came in 2009 which started a run that probably began due to force of Harbs personality and continued probably mostly due to winning.Â
- The breakthrough 2009 team won without much contribution from a high ranked class. Thast was still largely, but less completely, true for the first great team in 2010.
- The 2013 ranking seems very out of place, but the reason was almost entirely that the class was tiny (12 players, 11 who came to campus that fall). The avg stars/rating across the boards was roughly the same as this year.
Re: Recruiting and team statistics - BostonCard - 02-02-2016
Quote:For that matter, what matters more in recruiting - evaluation or convincing obviously good kids to choose your school (i.e. recruting rankings)?Â
Well, recruiting rankings can be wrong, of course. At best, it is a proxy for recruit quality. To the extent that it matters, one might consider Hulk's offer theory as a different proxy and a better way of determining how good a coaching staff is at getting kids to commit to its program, though I know what you meant.
Quote:Here are the recruiting rankings by year:
...
Interesting that Harbaugh's first two years were no better from a recruiting perspective (and arguably worse) than Walt's two years and Buddy's three. In fact, the only year in the preceding five that was worse than Harbaugh's first two recruiting years was Buddy's last. That being said, it's hard to think of any class that included Andrew Luck in it as the 44th best in the country.
Quote:What is the relevant importance of recruiting versus other aspects of coaching?
To some extent, what you are looking for is what proportion of the variance in football outcomes can be explained by recruiting (versus other, harder to measure factors). The problem with this analysis is that it depends a lot on the total variance examined; the greater the total variance, the more recruiting will explain. You say so yourself:
Quote:There's a reason Group of Five programs rarely beat Power 5 teams and Division II teams won't beat Division I.
The gap in recruiting between between G5 and P5 programs is large, as is the gap in play quality. On the other hand, if you examine something where there is less variation in overall quality (like say, just P5 programs), you will likely find that recruiting rankings explain less of the variance. But by the time you get to the elite levels, recruiting rankings probably explains a small proportion. Otherwise, USC and UCLA would consistently be winning the Pac12, Texas would consistently be winning the Big12, and Tennessee and Georgia would be rivaling Alabama for the SEC title.
BC
Re: Recruiting and team statistics - OutsiderFan - 02-02-2016
Sometimes things are a lot more simple than people want to make them out to be.Â
It can all be boiled down the Bill Walsh's theory on football: You can score points by scheming on offense with mediocre athletes. You have to have great athletes to be good on defense.
Re: Recruiting and team statistics - CTcard - 02-02-2016
(02-02-2016, 12:00 AM)Boston Card link Wrote:... it's hard to think of any class that included Andrew Luck in it as the 44th best in the country.
Andrew Luck is an interesting case study (though not a particular outlier).
His ratings out of HS:
Scout #47, Pro Style QB #4, 5 star
Rivals #68, Pro Style QB #4, 4 star
247 comp #45, pro Style QB #3, 4 star
ESPN #61, QB #7, 4 star
And rather famously was supposedly rated #3QB by Harbaugh behind Crist and Landry Jones.
Obviously, your basic statement that any class containing Luck had to be better than #44 seems correct. However, given his actual ranking by the services, he didn't boost the class all that much. Stanford recruits ranked higher by Scout have been Kaden Smith (2016), Solomon Thomas (2014), Casey Tucker (2014), Kyle Murphy (2012), Joshua Garnett (2012), Shayne Skov (2009), Alex Fletcher (2004), Trent Edwards (2002), Julian Jenkins (2002).
Whether you believe the services got this right depends upon the fineness of your comb.
Re: Recruiting and team statistics - AthleticsDynasty - 02-02-2016
If Luck was 3rd was RG3 4th?
Re: Recruiting and team statistics - washingtonismoney - 02-02-2016
The one weakness from CTCard's method is that attrition from classes is so variable. The 2012 class that constitutes fifth-year seniors may have been ranked in the top five or so, but virtually every player that made the class so impressive from a rankings perspective will have left by 2016. In other words we'll probably use 247's composite team rankings for a bit more precision (though unfortunately it's got a short history at the moment).