Stanford study estimates 48,000+ infected in Santa Clara County - Printable Version
+- The CardBoard (https://thecardboard.org/board)
+-- Forum: Emergency (https://thecardboard.org/board/forum-11.html)
+--- Forum: Covid-19 (https://thecardboard.org/board/forum-12.html)
+--- Thread: Stanford study estimates 48,000+ infected in Santa Clara County (/thread-19445.html)
+- The CardBoard (https://thecardboard.org/board)
+-- Forum: Emergency (https://thecardboard.org/board/forum-11.html)
+--- Forum: Covid-19 (https://thecardboard.org/board/forum-12.html)
+--- Thread: Stanford study estimates 48,000+ infected in Santa Clara County (/thread-19445.html)
Stanford study estimates 48,000+ infected in Santa Clara County - burger - 04-17-2020
48,000-81,000 positive, 50-85 times the number of confirmed cases. That ratio of positive to confirmed is off the charts compared to other similar studies from Iceland, Germany, etc.
https://www.medrxiv.org/content/10.1101/2020.04.14.20062463v1
They recruited by targeted ads on Facebook. I would guess that they got a disproportionate number of people who had had symptoms and wanted to be tested. They said they recorded prior symptoms of participants, but I didn't see that reported in the manuscript.
Quote:Abstract
Background Addressing COVID-19 is a pressing health and social concern. To date, many epidemic projections and policies addressing COVID-19 have been designed without seroprevalence data to inform epidemic parameters. We measured the seroprevalence of antibodies to SARS-CoV-2 in Santa Clara County. Methods On 4/3-4/4, 2020, we tested county residents for antibodies to SARS-CoV-2 using a lateral flow immunoassay. Participants were recruited using Facebook ads targeting a representative sample of the county by demographic and geographic characteristics. We report the prevalence of antibodies to SARS-CoV-2 in a sample of 3,330 people, adjusting for zip code, sex, and race/ethnicity. We also adjust for test performance characteristics using 3 different estimates: (i) the test manufacturer's data, (ii) a sample of 37 positive and 30 negative controls tested at Stanford, and (iii) a combination of both. Results The unadjusted prevalence of antibodies to SARS-CoV-2 in Santa Clara County was 1.5% (exact binomial 95CI 1.11-1.97%), and the population-weighted prevalence was 2.81% (95CI 2.24-3.37%). Under the three scenarios for test performance characteristics, the population prevalence of COVID-19 in Santa Clara ranged from 2.49% (95CI 1.80-3.17%) to 4.16% (2.58-5.70%). These prevalence estimates represent a range between 48,000 and 81,000 people infected in Santa Clara County by early April, 50-85-fold more than the number of confirmed cases. Conclusions The population prevalence of SARS-CoV-2 antibodies in Santa Clara County implies that the infection is much more widespread than indicated by the number of confirmed cases. Population prevalence estimates can now be used to calibrate epidemic and mortality projections.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - 2006alum - 04-17-2020
Presumably this would be great if true, as it would imply a much lower IFR. A few things give me pause:
1. They estimate that their test has a sensitivity of 80.3% and specificity of 99.5%. I'm not an expert, but doesn't the claimed specificity seem really high for basically ANY test? And doesn't a test that has a 99.5% specificity and an 80.3% sensitivity seem kind of wonky?
2. The 95% Confidence Intervals for the sensitivity was 72.1-87%, for specificity 98.3-99.9%. Aren't those fairly high to be extrapolating so much from this data?
3. For whatever it's worth, John Ioannidis is identified as one of the co-authors.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - dabigv13 - 04-17-2020
(04-17-2020, 08:46 AM)2006alum Wrote: Presumably this would be great if true, as it would imply a much lower IFR. A few things give me pause:
1. They estimate that their test has a sensitivity of 80.3% and specificity of 99.5%. I'm not an expert, but doesn't the claimed specificity seem really high for basically ANY test? And doesn't a test that has a 99.5% specificity and an 80.3% sensitivity seem kind of wonky?
2. The 95% Confidence Intervals for the sensitivity was 72.1-87%, for specificity 98.3-99.9%. Aren't those fairly high to be extrapolating so much from this data?
3. For whatever it's worth, John Ioannidis is identified as one of the co-authors.
The claimed specificity is not off base for a good antibody test, and in fact is basically what we need if we want to use these tests the way government leaders and ths media talk about these tests. If they really have a 99.5% specific test, that is fantastic and means we could actually use this for population studies for a low prevalence disease like this without having too many false positives. It would not be a surprise for sensitivity to be 80% either considering the different immune responses this virus seems to generate in people. If these numbers are accurate, that is great news, though it does mean missing out on 20% of patients who had the disease and don't know.
The URL is not loading for me though.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - 2006alum - 04-17-2020
(04-17-2020, 08:50 AM)dabigv13 Wrote:(04-17-2020, 08:46 AM)2006alum Wrote: Presumably this would be great if true, as it would imply a much lower IFR. A few things give me pause:
1. They estimate that their test has a sensitivity of 80.3% and specificity of 99.5%. I'm not an expert, but doesn't the claimed specificity seem really high for basically ANY test? And doesn't a test that has a 99.5% specificity and an 80.3% sensitivity seem kind of wonky?
2. The 95% Confidence Intervals for the sensitivity was 72.1-87%, for specificity 98.3-99.9%. Aren't those fairly high to be extrapolating so much from this data?
3. For whatever it's worth, John Ioannidis is identified as one of the co-authors.
The claimed specificity is not off base for a good antibody test, and in fact is basically what we need if we want to use these tests the way government leaders and ths media talk about these tests. If they really have a 99.5% specific test, that is fantastic and means we could actually use this for population studies for a low prevalence disease like this without having too many false positives. It would not be a surprise for sensitivity to be 80% either considering the different immune responses this virus seems to generate in people. If these numbers are accurate, that is great news, though it does mean missing out on 20% of patients who had the disease and don't know.
The URL is not loading for me though.
Got it - so just so I understand, if they are presuming ~80% sensitivity, for every 5 negative results they get, they code one as a false negative and therefore treat it as a presumed positive, is that right?
RE: Stanford study estimates 48,000+ infected in Santa Clara County - dabigv13 - 04-17-2020
A very sensitive test is one that will capture all the people who had the disease. An 80% sensitive test means that for every 5 negatives, one of those patients actually had the disease but has a negative result, though you can't say which of those 5 results is incorrect.
A very specific test is one that will not be positive if you did not have the disease. So a 99% specific test means that 1 out of a hundred people who were told they had the disease, actually did not have it.
Hope this answers your question. I would like to read the paper to see how they arrived at those numbers.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - OutsiderFan - 04-17-2020
My biggest question: If so many people are positive virus carriers, and the numbers of hospitalizations and deaths keep going up, what does this tell us about exposure equalling immunity or not, or just incubation times before showing symptoms?
Also, China is now implementing new lockdowns to prevent another major outbreak after thinking they had things under control.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - burger - 04-17-2020
(04-17-2020, 08:57 AM)2006alum Wrote:More or less. It's more complicated than that because they have to account for specificity (false positives), too, though that rate was low. They used Bayes rule to come up with a formula that included both. Details here: https://www.medrxiv.org/content/medrxiv/suppl/2020/04/17/2020.04.14.20062463.DC1/2020.04.14.20062463-1.pdf(04-17-2020, 08:50 AM)dabigv13 Wrote:Got it - so just so I understand, if they are presuming ~80% sensitivity, for every 5 negative results they get, they code one as a false negative and therefore treat it as a presumed positive, is that right?(04-17-2020, 08:46 AM)2006alum Wrote: Presumably this would be great if true, as it would imply a much lower IFR. A few things give me pause:The claimed specificity is not off base for a good antibody test, and in fact is basically what we need if we want to use these tests the way government leaders and ths media talk about these tests. If they really have a 99.5% specific test, that is fantastic and means we could actually use this for population studies for a low prevalence disease like this without having too many false positives. It would not be a surprise for sensitivity to be 80% either considering the different immune responses this virus seems to generate in people. If these numbers are accurate, that is great news, though it does mean missing out on 20% of patients who had the disease and don't know.
1. They estimate that their test has a sensitivity of 80.3% and specificity of 99.5%. I'm not an expert, but doesn't the claimed specificity seem really high for basically ANY test? And doesn't a test that has a 99.5% specificity and an 80.3% sensitivity seem kind of wonky?
2. The 95% Confidence Intervals for the sensitivity was 72.1-87%, for specificity 98.3-99.9%. Aren't those fairly high to be extrapolating so much from this data?
3. For whatever it's worth, John Ioannidis is identified as one of the co-authors.
The URL is not loading for me though.
I still think the sampling issue makes this pretty much useless. It would be interesting (and, imo, required if I was a reviewer on this paper) to divide the sample into those with and without prior symptoms and see how the % positive differs.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - dabigv13 - 04-17-2020
This study doesnt answer any questions about immunity. It would mean the fatality rate is lower and we may be closer to herd immunity than otherwise, but there is no difference between 1% and 5% for herd immunity. Just that we'd get there sooner.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - 2006alum - 04-17-2020
I guess what I'm slightly confused about with this study is that they got 50 positive results out of 3,330 tests, for a crude prevalence rate of 1.5%. But nothing I can see in their study parameters excluded people who had previously tested positive for COVID-19, so it's not clear to me that their study is only undetected community spread. If there have been over 1,000 confirmed cases, do we know that none of those individuals participated in that test?
And the study design also may be somewhat unrepresentative. As they say themselves, "bias favoring those with prior COVID-19-like illnesses seeking antibody confirmation are also possible. The overall effect of such biases is hard to ascertain." And thus, they didn't try. So based on their estimated sensitivity, coupled with their demographic weighting, they drew from 50 positive results a prevalence rate as high as 4.16%.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - burger - 04-17-2020
(04-17-2020, 09:16 AM)2006alum Wrote: I guess what I'm slightly confused about with this study is that they got 50 positive results out of 3,330 tests, for a crude prevalence rate of 1.5%.
And the study design also may be somewhat unrepresentative. As they say themselves, "bias favoring those with prior COVID-19-like illnesses seeking antibody confirmation are also possible. The overall effect of such biases is hard to ascertain." And thus, they didn't try. So based on their estimated sensitivity, coupled with their demographic weighting, they drew from 50 positive results a prevalence rate as high as 4.16%.
I'd bet a large sum of money that they got an unrepresentative sample. It just makes sense that people who would go out of their way to get blood drawn are curious about their exposure. So their sample probably had a disproportionate number of people who either had symptoms or were exposed to the virus through family, coworkers, etc. With only 50 positives, it wouldn't take many such people to produce a highly biased estimate.
Reports from Wuhan today are saying that 2.5% of hospital employees there were seropositive. I have trouble imagining that there's a higher prevalence in an entire US county that has far fewer deaths per capita.
I hope this paper doesn't get published without brobdingnagian alterations.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - 2006alum - 04-17-2020
(04-17-2020, 09:46 AM)burger Wrote: I'd bet a large sum of money that they got an unrepresentative sample. It just makes sense that people who would go out of their way to get blood drawn are curious about their exposure. So their sample probably had a disproportionate number of people who either had symptoms or were exposed to the virus through family, coworkers, etc. With only 50 positives, it wouldn't take many such people to produce a highly biased estimate.
Reports from Wuhan today are saying that 2.5% of hospital employees there were seropositive. I have trouble imagining that there's a higher prevalence in an entire US county that has far fewer deaths per capita.
I hope this paper doesn't get published without of immense proportions alterations.
Indeed, from what I can tell, it didn't even exclude people who tested positive for COVID-19.
But publication won't matter, what matters is how it gets reported. I bet dollars to donuts one of the Fox News anchors features this story before evening is up.
Ok, now I'm pretty infuriated by how obviously irresponsible their estimates are. They estimate based on their data that IFR is between .12% and .2% If so, NYC has 8.4m people, so at 100% prevalence, that'd be 10,000 deaths at IFR of .12% and 16,700 deaths at IFT of .2%. NYC has already reported well over 11,000 deaths. So unless we think NYC is totally, utterly unrepresentative, it's already blown past the low end of their estimated IFR and will likely exceed the high end within a week.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - dabigv13 - 04-17-2020
I read the paper, it is interesting, but definitely has flaws. The self selection bias is a real issue particularly with the small numbers involved.
On the other hand, I bet the overrepresentation of white people with access to Facebook and time to drive in for testing in their sample pool would favor true prevalence on higher end of estimates, as minority populations and lower class populations likely have a higher prevalence of this disease at this time. Also it seems their sensitivity estimate based on their own samples was much lower (67%) than the manufacturer sensitivity estimate (92%). If the local data is more accurate, then true prevalence higher.
Bottom line, this is preprint, not peer reviewed, and probably shouldn't be in public domain at this time.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - BostonCard - 04-17-2020
(04-17-2020, 09:05 AM)dabigv13 Wrote: A very sensitive test is one that will capture all the people who had the disease. An 80% sensitive test means that for every 5 negatives, one of those patients actually had the disease but has a negative result, though you can't say which of those 5 results is incorrect.
A very specific test is one that will not be positive if you did not have the disease. So a 99% specific test means that 1 out of a hundred people who were told they had the disease, actually did not have it.
Hope this answers your question. I would like to read the paper to see how they arrived at those numbers.
This is not quite right; you are talking about an 80% negative predictive value, not an 80% sensitivity. How to adjust for false negatives depends a bit on the prevalence. For example, in the event that you had a disease with 100% prevalence, then all of your negative tests, by definition would be false negatives (since everyone should be positive). In the event that you had a disease with 0% prevalence, then you would not have any false negatives by definition, since every negative test would reflect a true negative.
You can plug in the numbers in this spreadsheet:
https://docs.google.com/spreadsheets/d/1vAWzdBxR0jnovjiNRZqUGG7D0jZxpTeVJMcFgr1XUCg/edit#gid=0
It turns out that negative predictive value is about 99.4%, driven by the relatively low prevalence. So for every 1000 negative tests, 994 will be correct (true negatives) and 6 will be false negatives. The positive predictive value is 95% which means that for every 100 positive test results, 5 will be false positives, and 95 will be true positives.
Given the test performance characteristics, out of the 3330 people they tested, about 94 of them would have had COVID-19. Of these, 75 would have tested positive and 18 would have tested negative (numbers don't add up to 100 due to rounding). The vast majority of the patients (3236 would not have had COVID-19, and of those, because of the high specificity, 3239 would have correctly had a negative test and 16 would have had a positive test, so in all they would have seen about 91 positive tests (95% of which would have been true positives) and 3239 negative tests (of which 99.4% would have been true negatives).
BC
RE: Stanford study estimates 48,000+ infected in Santa Clara County - Snorlax94 - 04-17-2020
This is wonderful, really exciting news. Even if there are mistakes, I'm excited that there is an indication the ifr is much lower than feared.
This means the cfr in SCC could end up being less than 1%? Also, recall SCC has one of the longest life expectancies in the US due to a relatively lower rate of obesity, a lower rate of smoking, access to good healthcare, and a very high median household salary.
I saw the ad on Facebook, and I started filling out the form, but then paused when it said you'd have to wait in line, and it did not mention I'd be waiting in line in my car, so I did not complete the sign-up form. A few thoughts on bias:
1) To its credit, it said it wanted a cross-section of different races and different symptoms. I had never had any symptoms, so it felt like the best opportunity to for me to get a test because most other places seem to require some symptoms. Or maybe people felt like they'd be more likely to qualify if they downplayed their symptoms, because people want a free test conducted by Stanford.
2) I saw the ad around 2-3pm on a Wednesday or a Thursday and it reportedly filled up within hours. Who is Facebooking at that time, and has the schedule flexibility to jump on a test that might be on either Friday or Saturday? Probably not a grocery store clerk or a healthcare worker.
3) One of the testing locations was at Vasona Park in Los Gatos, it is a very nice park in a somewhat wealthy area, adjacent to other wealthy areas like Saratoga. If you look at the SCC dashboard and normalize # of cases / 10,000 people by city, you'll see Saratoga, Los Gatos , Cupertino and Mountain View have much fewer positive cases per 10,000 people. Saratoga and Cupertino (home of Apple) have less than 1/3 the cases per 10,000 people than San Jose (and sadly, Palo ALto has the second highest # of cases per 10,000, possibly due to all the healthcare workers?). Two of the three testing locations were in Mountain View (home of Google) and Los Gatos (home of Netflix). The third location was Hellyer Park in San Jose, but I am less familiar with that area.
If you look at demographic data on the SCC Dashboard, whites have death rates roughly in-line with their share of the population, and Asians are under-represented by almost a third, perhaps because more of them work for tech companies and have been working from home even before the Shelter-in-Place. Hispanics are over-represented by about a third, which I think is reflective of taking a larger share of jobs with a high degree of exposure. My guess is that the study under-represented this demographic -- for example, the ad I saw was written in English and there was no indication they'd accommodate people in other languages.
So yes, my guess is that future studies may come up with a lower multiple. I hope there are many follow-up studies soon. Maybe they can analyze where they may have had biases and try to have a broader, more reflective study the next time.
But let's say they're off by 3x (which is the ratio in cases per 10,000 people in San Jose:Cupertino), that would still be a multiplier of 17-28, which would still be very good news!
Self correction: on second thought, some of the sampling biases mechanisms would work in the opposite direction I initially thought, but I am still very excited by this possibility, even if the multiple changes a bit. In any event, picking parks in Los Gatos and Mountain View could give you a very non-representative sample.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - oldalum - 04-17-2020
(04-17-2020, 09:05 AM)dabigv13 Wrote: A very sensitive test is one that will capture all the people who had the disease. Yes. An 80% sensitive test means that for every 5 negatives, one of those patients actually had the disease but has a negative result, though you can't say which of those 5 results is incorrect. No, this is the negative predictive value (of a negative test), not the sensitivity.I think you got yourself confused. Your two first sentences are right but the second ones aren't. It is easiest for me to understand 80% sensitivity as 80% of people with the disease will have a positive test; the other 20% with the disease are false negatives. I.e., the test will correctly identify 80% of the people with the disease. And 99% specificity means that of 100 people who do not have the disease, 99 will have (true) negative tests and 1 will have a (false) positive test. I.e., the test will correctly identify 99% of people who do not have the disease as not having the disease.
A very specific test is one that will not be positive if you did not have the disease. Yes. So a 99% specific test means that 1 out of a hundred people who were told they had the disease, actually did not have it. This is the positive predictive value (of a positive test), not the specificity.
This diagram may help: diagram
RE: Stanford study estimates 48,000+ infected in Santa Clara County - dabigv13 - 04-17-2020
Yes, you and BC are correct, thank you for the correction.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - Snorlax94 - 04-17-2020
In related news -- there was a NYC study of random pregnant women giving birth being tested.
"Of the 211 women without symptoms, all were afebrile on admission. Nasopharyngeal swabs were obtained from 210 of the 211 women (99.5%) who did not have symptoms of Covid-19; of these women, 29 (13.7%) were positive for SARS-CoV-2. Thus, 29 of the 33 patients who were positive for SARS-CoV-2 at admission (87.9%) had no symptoms of Covid-19 at presentation."
https://www.nejm.org/doi/full/10.1056/NEJMc2009316
RE: Stanford study estimates 48,000+ infected in Santa Clara County - BostonCard - 04-17-2020
By the way, with 80 deaths in Santa Clara to date, that suggests an infection fatality rate of 0.1 to 0.2%, though it might end up being higher because not all the people infected around the time of the study who will eventually die have died yet. I would put broad confidence intervals around that, though because of the concerns raised in this thread.
Although the points brought up by folks on this thread are very valid concerns, I do think that this and other evidence should push us to revise our estimates of the IFR downwards. Maybe it won't quite be 0.1%, but it is increasingly unlikely that it is above 1%.
BC
RE: Stanford study estimates 48,000+ infected in Santa Clara County - g1313 - 04-17-2020
How does an IFR of 0.1-0.2% compare to the seasonal flu? I am not asking out of some agenda, I just want to have my facts straight. Even if the IFR is similar, we are obviously seeing much higher impact on the healthcare system than a normal flu season.
RE: Stanford study estimates 48,000+ infected in Santa Clara County - BostonCard - 04-17-2020
(04-17-2020, 02:25 PM)g1313 Wrote: How does an IFR of 0.1-0.2% compare to the seasonal flu? I am not asking out of some agenda, I just want to have my facts straight. Even if the IFR is similar, we are obviously seeing much higher impact on the healthcare system than a normal flu season.
The 1918/19 pandemic flu had an CFR of 2.5% (though keep in mind that there was no molecular diagnostic that could diagnose everyone). A typical pandemic flu has a CFR of <0.1%. I'm also not sure of the ability of serology for influenza to differentiate between strains, so I don't know if the prevalence has ever been found that way.
https://wwwnc.cdc.gov/eid/article/12/1/05-0979_article
BC