Good explanation of false positive rate

[Brendan Burgess]

There is a good article in today's Irish Times by

Dr David Robert Grimes is a physicist, cancer researcher and author of The Irrational Ape: Why Flawed Logic Puts Us All at Risk and How Critical Thinking Can Save the World

Why there is some good news in the latest Covid-19 data

Analysis of infection rates gives reason to avoid nihilistic apathy and draw strength

www.irishtimes.com

Sensitivity refers to how frequently a test correctly identifies a positive result. International experience suggests that PCR-testing for Covid-19 has a sensitivity of 98 per cent,

Conversely, specificity is a measure of whether a negative result is correctly labelled. [The PCR testing has a ]specificity upwards of 99.99 per cent.

Dublin at the time of writing was averaging roughly double the national incidence of Covid-19 at 140 cases per 100,000 people.

A sensitivity of 98 per cent means that if you tested 100,000 people in Dublin, you would correctly identity 137 positive cases (98 per cent of 140),

while a specificity of 99.99 per cent means the testing would incorrectly flag a further 10 cases as positive (0.01 per cent of 100,000). This gives a “true-positive” rate’ of 137/147, or over 93 per cent.

So there are 140 cases per 100,000 in Dublin.

If we test all 100,000, 137 of these will be picked up. 3 will be given the all clear although they have Covid.

However, 10 people who do not have Covid will be told that they do have it. ( So what? They will just have to self-isolate and will be surprised at how quickly they recover.)

Brendan

 

[Duke of Marmalade]

Yes, an excellent article.
It's conclusions are very heavily dependent on the assumed rate of false positives which he states at .01%, which seems to me a scarcely credible figure. For example, false positives for Cervical Cancer screening runs at about 10%. So if we had that level of false positives we would be showing 10,000 new cases per 100,000 even if COVID had vanished.
It would need only a false positive rate of 0.14% to fully explain Dublin's 140 per 100,000 rate.
What really perplexes me is that whilst cases are way up deaths seem to be almost eliminated. It seems to me there must be a significant amount of false positives, certainly more than a scarcely credible .01%.

 

[losttheplot]

Are the deaths lower because most infections are amongst younger people who have a higher recovery rate.

Is cervical screening the same type of test as a covid-19 test? Different types of tests will have different performance characteristics. Some diagnostic tests may even be designed to give more false positives than negatives, depending on the impact of the result.


The test would only be part of the diagnosis. I'd imagine someone displaying symptoms and with a confirmed close contact, if they tested negative, a retest may be made to confirm. The probability of two false negatives would be tiny.

 

[Duke of Marmalade]

Are the deaths lower because most infections are amongst younger people who have a higher recovery rate.

There must be an element of that, but still the deaths/cases is running at way below 1%. I feel that a part of the explanation must be false positives.

Is cervical screening the same type of test as a covid-19 test? Different types of tests will have different performance characteristics. Some diagnostic tests may even be designed to give more false positives than negatives, depending on the impact of the result.

Agreed and maybe I shouldn't have invoked the cervical screening but it was the one I Googled. There is a huge difference between 10% and .01%. Talked to a doctor contact. His understanding is that the COVID test is DNA based and these can be very accurate. All the same .01% could well be .05% and still be very accurate but greatly affect our interpretation of the results.

 

[losttheplot]

According to the Covid app, there were 88,000 tests this week and 2.8% positive. So more than 97% of those tested were negative. Could the criteria for testing be overly sensitive.

 

[Duke of Marmalade]

According to the Covid app, there were 88,000 tests this week and 2.8% positive. So more than 97% of those tested were negative. Could the criteria for testing be overly sensitive.

I suppose that is the point.

 

[Duke of Marmalade]

It would need only a false positive rate of 0.14% to fully explain Dublin's 140 per 100,000 rate.

Wrong
The key stat is the percentage of tests proving positive. This is 2.8% over the last 7 days. So even if false positives were a very high 0.5% of the population that would lead to an overestimation in the ratio by 2.8/2.3 i.e. that the true figure would be 115 (2.3/2.8x140) per 100,000.
So the article is substantially correct, false positives would not change the picture much.

 

[Purple]

The number of tests we are carrying out has increased, 0.68 per thousand on the first of August to 2.63 per thousand now. That is nearly a four fold increase and must be a significant part of the reason we are seeing a big increase in cases. The rolling average death rate and the numbers in hospital and ICU are more significant statistics for me.

 

[Sunny]

The number of tests we are carrying out has increased, 0.68 per thousand on the first of August to 2.63 per thousand now. That is nearly a four fold increase and must be a significant part of the reason we are seeing a big increase in cases. The rolling average death rate and the numbers in hospital and ICU are more significant statistics for me.

To be honest, it is really hard to get a clear picture on most of the figures. For example on the numbers of people in hospital, we are told they are increasing every day. Yet according to the HSE, in the last two weeks 47 hospital patients got Covid. So are they are included in the hospital figures? If so, it distorts the idea that we are seeing more and more people being admitted every day with Covid from the general population.

 

[XMarks]

The daily Covid admission and discharge numbers reported do not stack up with the hospital numbers so I can only assume that it is people who are already in hospital testing positive. Dr. Durkan from Beaumont Hospital said people are coming to hospital for other reasons with no Covid symptoms subsequently testing positive.

I don’t understand why they don’t include the % asymptomatic in their daily reports. If we take the Diamond Princess as a control group over 50% who tested positive had no symptoms.

 

[bradyassoc]

It would need only a false positive rate of 0.14% to fully explain Dublin's 140 per 100,000 rate.
What really perplexes me is that whilst cases are way up deaths seem to be almost eliminated. It seems to me there must be a significant amount of false positives, certainly more than a scarcely credible .01%.

I think that to understand that question we have to look at the word cases and what it means? It appears the word is being used to describe as if someone is seriously ill when what we are actually recording is positive tests, the same positive tests which are being taken because of an increased effort to make people take the tests. I don't understand why people can't make the simple deduction that the more tests you have the more positive number of tests you will have.

TLDR: The increase in cases means we are testing a lot more and largely among people who are healthy.

 

[christopher..]

GP's and hospitals are not accepting patients without first having to get the test for covid 19. Could this be one of the reasons for the high numbers of positive results? I have heard that alot of people are refusing to get tested as this would prevent them from working and earning money. They are even stopping their children from being tested as the household would have to isolate.

Would false negatives results be more of a worry in the fight against this virus? I read an article on the burn out of medical lab scientists who are performing this testing everyday and the instruments they are using. These instruments have been operating at full capacity for the last 6 months.
Do the HSE need to look at additional methods or types of testing? The WHO have given emergency authorisation for 2 qualitative rapid diagnostic covid kits to be used in third world countries. Could these kits be used for schools and hospitals in Ireland? They are roughly 98% accurate. They look like a pregnancy test and do not need expensive testing equipment. They can be performed by a non medical person and they are cheap. They would be a good screening test to allow people to enter a work place, school, construction site or obtain a covid pass to go about their normal tasks.
I was listening to Ciara Kelly on news talk radio and she said that we need to think outside the box. Can we improve on the testing regime or tests that we use? PCR testing is only one type of test.

 

[Purple]

The increase in cases means we are testing a lot more and largely among people who are healthy.

Testing has increased 4 fold since July. Infection rates have increased 10 fold but yes, the key numbers are the positivity percentage and the hospital admissions, ICU admissions and deaths.
The real question here is will this and the inevitable winder flu and general sickness increase break our bloated, inefficient and dysfunctional healthcare system.

 

[Leo]

GP's and hospitals are not accepting patients without first having to get the test for covid 19.

Across the board? My wife was with her GP in recent days, no COVID test required.

 

[EmmDee]

Across the board? My wife was with her GP in recent days, no COVID test required.

I know people who booked GP appointments. They are asked about any symptoms or potential exposure and if symptoms are similar to any COVID ones or a risk of exposure, you are asked to test first.

And before someone points it out - yes you could lie in which case you should have a look at yourself

 

[Leo]

I know people who booked GP appointments. They are asked about any symptoms or potential exposure and if symptoms are similar to any COVID ones or a risk of exposure, you are asked to test first.

They should all be asking those questions, but Christopher said no one was being accepted without a test.

And before someone points it out - yes you could lie in which case you should have a look at yourself

Agreed!!

 

[christopher..]

Leo, You cannot get a walk-in appointment here. Phone only unfortunately. Leo, you must live in a low area?

 

[Leo]

Leo, You cannot get a walk-in appointment here. Phone only unfortunately. Leo, you must live in a low area?

No I live in Dublin, and not one of the lower rate areas. No walk-in appointments here either, they were never an option in most of the local medical practices. But you only have to answer the standard questions to attend an appointment, not present a negative test result.

Are you now suggesting you just need to make a phone call to get an appointment or are you still saying you must get an actual test?

 

[Sunny]

Leo, You cannot get a walk-in appointment here. Phone only unfortunately. Leo, you must live in a low area?

As far as I know, every GP is doing phone consults before having people turn up in the surgery and will only see you if he deems it necessary. Certainly not the case that you have to have a test before seeing the doctor. Obviously they will ask you questions and if there is doubt, they won't see you.

 

[Purple]

GP's and hospitals are not accepting patients without first having to get the test for covid 19.

My daughter has been to the doctor twice since March and I've been once and we weren't asked anything about tests. I got more questions at the dentist.