[burger]

By the way, with 80 deaths in Santa Clara to date, that suggests an infection fatality rate of 0.1 to 0.2%, though it might end up being higher because not all the people infected around the time of the study who will eventually die have died yet. I would put broad confidence intervals around that, though because of the concerns raised in this thread.

Although the points brought up by folks on this thread are very valid concerns, I do think that this and other evidence should push us to revise our estimates of the IFR downwards.  Maybe it won't quite be 0.1%, but it is increasingly unlikely that it is above 1%.

BC

A caveat here: the difference between 0.1% and 1% feels small, but 0.1% implies 10 times as many asymptomatic people as the 1% does.  And I think that this study, with its many flaws, should have no bearing our beliefs about the IFR.

[burger]

By the way, with 80 deaths in Santa Clara to date, that suggests an infection fatality rate of 0.1 to 0.2%, though it might end up being higher because not all the people infected around the time of the study who will eventually die have died yet. I would put broad confidence intervals around that, though because of the concerns raised in this thread.

Although the points brought up by folks on this thread are very valid concerns, I do think that this and other evidence should push us to revise our estimates of the IFR downwards.  Maybe it won't quite be 0.1%, but it is increasingly unlikely that it is above 1%.

BC

A caveat here: the difference between 0.1% and 1% feels small, but 0.1% implies 10 times as many asymptomatic people as the 1% does.  And I think that this study, with its many flaws, should have no bearing our beliefs about the IFR.

[dabigv13]

I think we'll end up around 0.5, or 5 times that of a really bad flu year. But it's different for a couple reasons. 1), this virus has no immunity in the population, as it is totally novel, and there is no vaccine, 2) this virus seems to be much more contagious than the flu, so it spreads on an exponentially steeper curve, even though it takes more time to start with symptoms and likely longer to kill them than the flu. So that means it will more rapidly overwhelm your healthcare system, and will keep it occupied for a while even if it doesn't get fully overwhelmed.

The actual fatality ratio doesn't tell the whole story. Also, the fatality ratio rapidly goes up when healthcare systems are overwhelmed like this virus has demonstrated.

[dabigv13]

I think we'll end up around 0.5, or 5 times that of a really bad flu year. But it's different for a couple reasons. 1), this virus has no immunity in the population, as it is totally novel, and there is no vaccine, 2) this virus seems to be much more contagious than the flu, so it spreads on an exponentially steeper curve, even though it takes more time to start with symptoms and likely longer to kill them than the flu. So that means it will more rapidly overwhelm your healthcare system, and will keep it occupied for a while even if it doesn't get fully overwhelmed.

The actual fatality ratio doesn't tell the whole story. Also, the fatality ratio rapidly goes up when healthcare systems are overwhelmed like this virus has demonstrated.

[magnus]

Thoughts from Trevor Bedford(who noted the connection between the nursing home in WA and the first US case)

[Tweet]https://twitter.com/trvrb/status/1251332447691628545?s=20[/tweet]

Points out that if the specificity is just a tad worse at 98.5%, the estimated prevalence would be 0.
(I'll leave it to you all to explain if that makes sense)

There's also a more in depth breakdown of reasons why the numbers may be off on that tweet thread.

[magnus]

Thoughts from Trevor Bedford(who noted the connection between the nursing home in WA and the first US case)

[Tweet]https://twitter.com/trvrb/status/1251332447691628545?s=20[/tweet]

Points out that if the specificity is just a tad worse at 98.5%, the estimated prevalence would be 0.
(I'll leave it to you all to explain if that makes sense)

There's also a more in depth breakdown of reasons why the numbers may be off on that tweet thread.

[BostonCard]

It means that if the specificity were 98.5%, you would generate enough false positives to account for all the positive tests they saw.

BC

[BostonCard]

It means that if the specificity were 98.5%, you would generate enough false positives to account for all the positive tests they saw.

BC

[burger]

It means that if the specificity were 98.5%, you would generate enough false positives to account for all the positive tests they saw.

BC

You don't need 98.5% specificity.  Even with 99.5% specificity, the confidence interval on that estimate easily accounts for all of the positives in the real test.

Bottom line: everyone should ignore this study.  I'm embarrassed that Stanford is associated with this.

[burger]

It means that if the specificity were 98.5%, you would generate enough false positives to account for all the positive tests they saw.

BC

You don't need 98.5% specificity.  Even with 99.5% specificity, the confidence interval on that estimate easily accounts for all of the positives in the real test.

Bottom line: everyone should ignore this study.  I'm embarrassed that Stanford is associated with this.

[Mick]

Graduated to clickbait:

https://spectator.us/stanford-study-sugg...-realized/

[Mick]

Graduated to clickbait:

https://spectator.us/stanford-study-sugg...-realized/

[burger]

Graduated to clickbait:

https://spectator.us/stanford-study-sugg...-realized/

Here's one sentence from that article.  I count 4 things wrong in that sentence. I didn't think was actually possible to have so many separate things wrong in a single sentence.

But if SARS-Cov-2 is already endemic in the population there is nothing we can do to stop it but no great reason to try to stop it, either: it has already ripped its way through the population with only a small proportion showing any symptoms.

Edit: actually 5 things wrong since I think he means "epidemic" instead of "endemic."

[burger]

Graduated to clickbait:

https://spectator.us/stanford-study-sugg...-realized/

Here's one sentence from that article.  I count 4 things wrong in that sentence. I didn't think was actually possible to have so many separate things wrong in a single sentence.

But if SARS-Cov-2 is already endemic in the population there is nothing we can do to stop it but no great reason to try to stop it, either: it has already ripped its way through the population with only a small proportion showing any symptoms.

Edit: actually 5 things wrong since I think he means "epidemic" instead of "endemic."

[Mick]

Not scientific, but this article indicates that Mass. General tested 200 asymptomatic people at random for coronavirus using the fingerprick blood test in Chelsea, which is in Suffolk County, Massachusetts.  64 of the 200 (32%) tested positive.
https://www.foxnews.com/science/third-bl...oronavirus

Chelsea's a hot spot.  About 2% of the population had otherwise tested positive (712 cases, 39 deaths).

[Mick]

Not scientific, but this article indicates that Mass. General tested 200 asymptomatic people at random for coronavirus using the fingerprick blood test in Chelsea, which is in Suffolk County, Massachusetts.  64 of the 200 (32%) tested positive.
https://www.foxnews.com/science/third-bl...oronavirus

Chelsea's a hot spot.  About 2% of the population had otherwise tested positive (712 cases, 39 deaths).

[M T]

More bad news for the Stanford testing in Santa Clara County.  The Stanford Daily indicated they used the Premier Biotech test.

EDIT: THIS NBCREPORT HAS BEEN UPDATED. See Premier Biotech's parent company Biotest Biotech's response.

https://www.nbcnews.com/health/health-ne...s-n1185131

Two U.S. companies — Premier Biotech of Minneapolis and Aytu Bioscience of Colorado — have been distributing the tests from unapproved Chinese manufacturers, according to health officials, FDA filings and a spokesman for one of the Chinese manufacturers. Many of the unapproved tests appear to have been shipped to the U.S. after the FDA relaxed its guidelines for tests in mid-March and before the Chinese government banned their export just over two weeks later.
.

By the way, I believe at least one person here participated in the Santa Clara County testing.   I'd be curious as to whether they did ask whether you had symptoms, and whether you had taken a regular test.   If they asked that, and didn't reveal that in their results, I wonder why.

[M T]

More bad news for the Stanford testing in Santa Clara County.  The Stanford Daily indicated they used the Premier Biotech test.

EDIT: THIS NBCREPORT HAS BEEN UPDATED. See Premier Biotech's parent company Biotest Biotech's response.

https://www.nbcnews.com/health/health-ne...s-n1185131

Two U.S. companies — Premier Biotech of Minneapolis and Aytu Bioscience of Colorado — have been distributing the tests from unapproved Chinese manufacturers, according to health officials, FDA filings and a spokesman for one of the Chinese manufacturers. Many of the unapproved tests appear to have been shipped to the U.S. after the FDA relaxed its guidelines for tests in mid-March and before the Chinese government banned their export just over two weeks later.
.

By the way, I believe at least one person here participated in the Santa Clara County testing.   I'd be curious as to whether they did ask whether you had symptoms, and whether you had taken a regular test.   If they asked that, and didn't reveal that in their results, I wonder why.

[g1313]

Edit: I misremembered, they do ask if you had symptoms. From the paper:

The
survey asked for six data elements: zip code of residence, age, sex, race/ethnicity, underlying comorbidities, and prior clinical symptoms.

No, the only info they asked for in the recruiting questionnaire were age, gender, and zip code.

They did their primary recruit through Facebook ads (unknown targeting criteria) but your friends and family could send you the recruitment link. I know it was circulating in healthcare worker circles so certainly potential for bias there.

[g1313]

Edit: I misremembered, they do ask if you had symptoms. From the paper:

The
survey asked for six data elements: zip code of residence, age, sex, race/ethnicity, underlying comorbidities, and prior clinical symptoms.

No, the only info they asked for in the recruiting questionnaire were age, gender, and zip code.

They did their primary recruit through Facebook ads (unknown targeting criteria) but your friends and family could send you the recruitment link. I know it was circulating in healthcare worker circles so certainly potential for bias there.

[Snorlax94]

No, the only info they asked for in the recruiting questionnaire were age, gender, and zip code. 

They did their primary recruit through Facebook ads (unknown targeting criteria) but your friends and family could send you the recruitment link. I know it was circulating in healthcare worker circles so certainly potential for bias there.

I started to fill out the questionnaire but did not complete it. It was longer than that.

I did not get to a part about symptoms, but I recall it saying I may or may not qualify based on answers about reported symptoms.

[Snorlax94]

No, the only info they asked for in the recruiting questionnaire were age, gender, and zip code. 

They did their primary recruit through Facebook ads (unknown targeting criteria) but your friends and family could send you the recruitment link. I know it was circulating in healthcare worker circles so certainly potential for bias there.

I started to fill out the questionnaire but did not complete it. It was longer than that.

I did not get to a part about symptoms, but I recall it saying I may or may not qualify based on answers about reported symptoms.

[burger]

Another good takedown of the Stanford study from Andrew Gelman, who is maybe the most famous statistician (oxymoron, I know) working today. 

This part is absolutely brutal and totally justified:

I think the authors of the above-linked paper owe us all an apology. We wasted time and effort discussing this paper whose main selling point was some numbers that were essentially the product of a statistical error.

I’m serious about the apology. Everyone makes mistakes. I don’t think they authors need to apologize just because they screwed up. I think they need to apologize because these were avoidable screw-ups. They’re the kind of screw-ups that happen if you want to leap out with an exciting finding and you don’t look too carefully at what you might have done wrong.

Ouch. There are some other hits at Stanford there, including a gratuitous Theranos mention. But I agree with everything that Gelman wrote about this study.

https://statmodeling.stat.columbia.edu/2...revalence/

[burger]

Another good takedown of the Stanford study from Andrew Gelman, who is maybe the most famous statistician (oxymoron, I know) working today. 

This part is absolutely brutal and totally justified:

I think the authors of the above-linked paper owe us all an apology. We wasted time and effort discussing this paper whose main selling point was some numbers that were essentially the product of a statistical error.

I’m serious about the apology. Everyone makes mistakes. I don’t think they authors need to apologize just because they screwed up. I think they need to apologize because these were avoidable screw-ups. They’re the kind of screw-ups that happen if you want to leap out with an exciting finding and you don’t look too carefully at what you might have done wrong.

Ouch. There are some other hits at Stanford there, including a gratuitous Theranos mention. But I agree with everything that Gelman wrote about this study.

https://statmodeling.stat.columbia.edu/2...revalence/

[g1313]

It looks like you are right and my memories are faulty. In the results writeup, they say:

The survey asked for six data elements: zip code of residence, age, sex, race/ethnicity, underlying comorbidities, and prior clinical symptoms.

No, the only info they asked for in the recruiting questionnaire were age, gender, and zip code. 

They did their primary recruit through Facebook ads (unknown targeting criteria) but your friends and family could send you the recruitment link. I know it was circulating in healthcare worker circles so certainly potential for bias there.

I started to fill out the questionnaire but did not complete it. It was longer than that.

I did not get to a part about symptoms, but I recall it saying I may or may not qualify based on answers about reported symptoms.

[g1313]

It looks like you are right and my memories are faulty. In the results writeup, they say:

The survey asked for six data elements: zip code of residence, age, sex, race/ethnicity, underlying comorbidities, and prior clinical symptoms.

No, the only info they asked for in the recruiting questionnaire were age, gender, and zip code. 

They did their primary recruit through Facebook ads (unknown targeting criteria) but your friends and family could send you the recruitment link. I know it was circulating in healthcare worker circles so certainly potential for bias there.

I started to fill out the questionnaire but did not complete it. It was longer than that.

I did not get to a part about symptoms, but I recall it saying I may or may not qualify based on answers about reported symptoms.

[Griffins78]

NY Times: Antibody tests, seen as key to reopening country, are raising alarms

The tests, many made in China without F.D.A. approval, are often inaccurate. Some doctors are misusing them. The rollout is nowhere close to the demand.

https://www.nytimes.com/2020/04/19/us/co...ticleShare

Others have referenced the quality of the antibody testing. This is an article that talks about some of the testing that is failing.

Are there antibody tests that are more accurate and how scalable are they?

[Griffins78]

NY Times: Antibody tests, seen as key to reopening country, are raising alarms

The tests, many made in China without F.D.A. approval, are often inaccurate. Some doctors are misusing them. The rollout is nowhere close to the demand.

https://www.nytimes.com/2020/04/19/us/co...ticleShare

Others have referenced the quality of the antibody testing. This is an article that talks about some of the testing that is failing.

Are there antibody tests that are more accurate and how scalable are they?

[needle]

Very good skeptical points raised in this thread and elsewhere, imo ... and yet here's John Ioannidis, Stanford prof and provocateur, eagerly overstating what conclusions can be drawn from the study: "Immediately that means the infection fatality rate, the chance of dying, the probability of dying if you have been infected, diminishes by 50 to 85-fold because the denominator in the population becomes 50 to 85-fold bigger."

https://www.youtube.com/watch?v=jGUgrEfS...e=youtu.be

The vid publishes today and has 44k views. I'd put the over-under on views at 2 million a week from today.

++++

UPDATE: Turns out, virality never emerged for this vid. Has about 166k views as of April 27.

[needle]

Very good skeptical points raised in this thread and elsewhere, imo ... and yet here's John Ioannidis, Stanford prof and provocateur, eagerly overstating what conclusions can be drawn from the study: "Immediately that means the infection fatality rate, the chance of dying, the probability of dying if you have been infected, diminishes by 50 to 85-fold because the denominator in the population becomes 50 to 85-fold bigger."

https://www.youtube.com/watch?v=jGUgrEfS...e=youtu.be

The vid publishes today and has 44k views. I'd put the over-under on views at 2 million a week from today.

++++

UPDATE: Turns out, virality never emerged for this vid. Has about 166k views as of April 27.

[stupac2]

One thing that's been strange to me about this ordeal is that Ioannidis is known for his skepticism of most published research, and for reasons that are along the lines of the apparent flaws in this paper. Did he just turn off his skepticism here, or something?

[stupac2]

One thing that's been strange to me about this ordeal is that Ioannidis is known for his skepticism of most published research, and for reasons that are along the lines of the apparent flaws in this paper. Did he just turn off his skepticism here, or something?

[oldalum]

Did he just turn off his skepticism here, or something?

May be just good ol' confirmation bias at work.

[oldalum]

Did he just turn off his skepticism here, or something?

May be just good ol' confirmation bias at work.

[BostonCard]

Piece in the Merc about the study:

https://www.mercurynews.com/2020/04/20/f...n-apology/

The good news is that apparently more detail about the methods will be forthcoming.  And also, the paper notes that a bunch of other groups are doing similar studies, so we will have a better sense of what is happening.

In the end, no single study is going to answer the question of how prevalent COVID-19 is in our communities, scientists said.  More studies with different technologies and analytic approaches are needed.
That’s coming. A UC Berkeley project, which will begin in May, will test a large and representative swath of 5,000 East Bay residents. Scientists will take saliva, swab and blood samples from volunteers between the ages of 18 and 60 around the region.
Starting Monday, UC San Francisco and a privately-funded operation will test all 1,680 residents of rural Bolinas for evidence of the virus. UCSF will launch a similar effort Saturday in San Francisco’s densely populated and largely Latino Mission District, where it hopes to test 5,700 people.
Results are expected soon from seroprevalence surveys run by other groups around the world, including teams in China, Australia, Iceland, Italy and Germany.

BC

[BostonCard]

Piece in the Merc about the study:

https://www.mercurynews.com/2020/04/20/f...n-apology/

The good news is that apparently more detail about the methods will be forthcoming.  And also, the paper notes that a bunch of other groups are doing similar studies, so we will have a better sense of what is happening.

In the end, no single study is going to answer the question of how prevalent COVID-19 is in our communities, scientists said.  More studies with different technologies and analytic approaches are needed.
That’s coming. A UC Berkeley project, which will begin in May, will test a large and representative swath of 5,000 East Bay residents. Scientists will take saliva, swab and blood samples from volunteers between the ages of 18 and 60 around the region.
Starting Monday, UC San Francisco and a privately-funded operation will test all 1,680 residents of rural Bolinas for evidence of the virus. UCSF will launch a similar effort Saturday in San Francisco’s densely populated and largely Latino Mission District, where it hopes to test 5,700 people.
Results are expected soon from seroprevalence surveys run by other groups around the world, including teams in China, Australia, Iceland, Italy and Germany.

BC

[burger]

It's good to see more studies being done, but are any of these worth a damn if the tests are bad?  That NY Times article mentioned upthread casts a lot of doubt on these tests. 

And another issue not being addressed is whether or not a true positive test means you're immune to the disease.  I doubt that can even be answered without a challenge test (where the person is exposed to live virus), right?  And no one is going to be doing those tests because of the ethical issues.

I'm starting to think that figuring out who's had the disease in the past is a distraction--an interesting distraction but not really important to combating the spread of the virus and reopening things.  Given that time, money, and resources in the fight are limited, I'd rather see efforts focused on how to implement mass testing to find and isolate currently infected people.  Prevalence is probably in the low single digits range or <1% almost everywhere outside NYC in the US.  Does getting that down to an exact number really help anything going forward?  Color me skeptical.

[burger]

It's good to see more studies being done, but are any of these worth a damn if the tests are bad?  That NY Times article mentioned upthread casts a lot of doubt on these tests. 

And another issue not being addressed is whether or not a true positive test means you're immune to the disease.  I doubt that can even be answered without a challenge test (where the person is exposed to live virus), right?  And no one is going to be doing those tests because of the ethical issues.

I'm starting to think that figuring out who's had the disease in the past is a distraction--an interesting distraction but not really important to combating the spread of the virus and reopening things.  Given that time, money, and resources in the fight are limited, I'd rather see efforts focused on how to implement mass testing to find and isolate currently infected people.  Prevalence is probably in the low single digits range or <1% almost everywhere outside NYC in the US.  Does getting that down to an exact number really help anything going forward?  Color me skeptical.

[Snorlax94]

I think it’s helpful to know, and that these efforts are by no means mutually exclusive: we should simultaneously be trying to find a vaccine, increasing testing, increase tracking and tracing, improving our quarantining, increasing our ppe supplies (ideally eventually getting to the point where everyone can get an n95 mask), increasing access to hand washing, improving our social distancing (including the spit guards protecting cashiers), investigating treatments, tracking how many people have antibodies while also learning whether that confers immunity.

If antibodies confer immunity, those people could be directed to help the elderly, to work in healthcare settings, to work in customer-facing jobs. 

Even if the Stanford Seroprevalence study is deeply flawed, we will be getting a lot more data, and even if antibodies *don’t* confer immunity, because Antibody tests are cheaper, faster and easier to roll out on a large scale, they could be helpful to understand who is getting infected, which could inform our future measures.


(* assuming we can get some that are accurate)

[Snorlax94]

I think it’s helpful to know, and that these efforts are by no means mutually exclusive: we should simultaneously be trying to find a vaccine, increasing testing, increase tracking and tracing, improving our quarantining, increasing our ppe supplies (ideally eventually getting to the point where everyone can get an n95 mask), increasing access to hand washing, improving our social distancing (including the spit guards protecting cashiers), investigating treatments, tracking how many people have antibodies while also learning whether that confers immunity.

If antibodies confer immunity, those people could be directed to help the elderly, to work in healthcare settings, to work in customer-facing jobs. 

Even if the Stanford Seroprevalence study is deeply flawed, we will be getting a lot more data, and even if antibodies *don’t* confer immunity, because Antibody tests are cheaper, faster and easier to roll out on a large scale, they could be helpful to understand who is getting infected, which could inform our future measures.


(* assuming we can get some that are accurate)