> What do you make of the claim that based on 5 photos they achieved a 90% success rate?

I'm reminded of Gerd Gigerenzer's book "Reckoning with risk". It's about medical testing, so forgive the language used:

> A condition exists. There is a test for that condition. The test is good, but not perfect. If someone has the condition there is a 90% chance they'll return positive. If someone does not have the condition there is a 1% chance they'll return positive. About 1% of the population have the condition. Bob has the test, and it comes back positive. What's the probability Bob has the condition?

Most people, even doctors who give these types of tests, get this wrong. The answer is about 50%, but most people put it much higher at 99% or 90%.

I think the Guardian reporting is irresponsible because the general public do not understand percentages, and any reporter using percentages is misleading (albeit inadvertantly) the public.

Here's a link to the book: https://www.amazon.co.uk/Reckoning-Risk-Learning-Live-Uncert...

No one in the general population can do that calculation. Try it, go and talk to people.

Or read Gerd Gigerenzer's book Reckoning with Risk http://www.amazon.com/Reckoning-Risk-Learning-Live-Uncertain...

https://plus.maths.org/content/reckoning-risk

And, as Temporal says, without knowing the base rate (am I raising my risk from 1 in 100,000 or from 1 in 1,000?) it provides very little useful information.

Most people have no idea how to use percentages. Most of the population has no hope of understanding what's actually going on with this article.

That becomes scary when you're talking to a doctor about a test that returned a positive result. Does that doctor know how likely it is you actually have what you tested positive for? Are you going to get treatment for this thing?

Gerd Gigerenzer has a nice book about this: http://www.amazon.com/Reckoning-Risk-Learning-Live-Uncertain...

Here's one example about breast cancer testing: http://imgur.com/zO4zkl4

> From a $100 test? I hope not

Look at Morgellon's; Mercury Chelation; Anti-vaccination; etc etc etc.

There are very many people willing to sell tests, and very many people happy to sell quack cures based on those tests. (I'm not saying that 23andMe are quacks!)

> "Man, is there anything worse than being told you have something terrible wrong with you, living with that, then finding out it was a false alarm". Uhm, yeah. How about finding out it wasn't?

That's happened to a few people. You get told you're HIV+ (in the late 90s, when this means it's a death sentence.) You lose your job (because people are arseholes), you stop showing your 8 year old son affection (because you're scared of the infection), you have unprotected sex with people with HIV (you're already +, so what does it matter?) and then you get told that there was a mistake with the original test and you're actually negative.

http://www.amazon.com/Reckoning-Risk-Learning-Live-Uncertain...

An excellent post by Schneier.

> The problem isn't just that such a system is wrong, it's that the mathematics of testing makes this sort of thing pretty ineffective in practice. It's called the "base rate fallacy." Suppose you have a test that's 90% accurate in identifying both sociopaths and non-sociopaths. If you assume that 4% of people are sociopaths, then the chance of someone who tests positive actually being a sociopath is 26%. (For every thousand people tested, 90% of the 40 sociopaths will test positive, but so will 10% of the 960 non-sociopaths.) You have postulate a test with an amazing 99% accuracy -- only a 1% false positive rate -- even to have an 80% chance of someone testing positive actually being a sociopath.

Interestingly here he uses percentages to describe base rates and risk. Gerd Gigerenzer has a nice book, Reckoning with Risk, where he explains with many examples the problems of this approach. Gerd asks people to use real numbers instead, which are much easier to understand for most people.

Thus, Schneier's example becomes:

> Out of 1,000 people about 40 of will be sociopaths. You have a test that will tell you if someone is, or is not, a sociopath. The test will be correct 9 times out of 10. Bob has taken the test, and has been identified as a possible sociopath. The chance that Bob is actually a sociopath are actually about 1 in 4. This is because the test will tell you that 36 of the 40 sociopaths are sociopaths, but it will also incorrectly tell you that 96 non-sociopaths are sociopaths.

My writing is lousy, and other people will be able to clean this up, but even with my poor writing style it's easier for most people to follow and understand than the percentages.

This is alarmingly important when you're making a health decision - "Should I remove my breasts to reduce my risk of breast cancer?" for example.

(http://www.amazon.com/Reckoning-Risk-Learning-Live-Uncertain...)

EDIT: I use "sociopath" because it's in the source article. I agree with NNQ that it's very troubling to bandy around diagnostic labels like this, and deem people to be dangerous, just because of a tentative probabilistic diagnosis.