Wow this post got popular. I got called into work and didnt see the replies, sorry ladies and gentlemen! Trying to catch up tonight.
Wow this post got popular. I got called into work and didnt see the replies, sorry ladies and gentlemen! Trying to catch up tonight.
That study seems to state a conclusion precisely the opposite of what the experimental results were. Based on a small sample set, there’s a high degree of match, far more accurate than random chance, between the observations and the genetic findings.
So, at intake, 18 dogs were identified as pit bulls but only 2/3rds were at least 12% pit bull.
During the study, 56 dogs were identified as being pit bulls, but only about 1/3rd were in fact at least 12% pit bull.
This is the classic 'base rate fallacy'. The false positive rate isn't that high, and the false negative rate isn't that high either. But because the true positive rate is pretty low, the ratio of true positives to false positives is much worse than you'd intuitively think.
Tests for rare diseases and attempts to behaviorally profile terrorists at airports runs into the same problem. Sometimes, a 99.9% accurate test just moves you from searching for a needle on a farm to a needle in only a single haystack.
And yet still wrong two thirds of the time.