Thursday, October 6, 2011

Why Your Gaydar is Horribly Inaccurate, Your Kids Are Safe with Strangers, and Medical Tests Are Often Wrong

Hmmm, I think it's broken.

Much of why humans get probabilities wrong, make bad decisions, and do dumb things can be illustrated by the situations highlighted in the title of this post - why is your gaydar is horribly inaccurate, why are your kids safe with strangers, and why are medical tests often wrong?

The reason is that, when evaluating situations, estimating probabilities, and the like, people often (actually, almost always) fail to take into account prior probabilities - i.e. they only look at the particular situation they're concerned about, and they fail to take into account the world in which the situation exists.

These insights come from the application of Bayes' theorem - we'll have a short demonstration of how this works in the real world, and how it can help you from making stupid decisions or incorrect assessments.

The gaydar example comes from The Hardest Science:
In a typical study, half of the targets are gay/lesbian and half are straight, so a purely random guesser (i.e., someone with no gaydar) would be around 50%. The reported accuracy rates in the articles . . . say that people guess correctly about 65% of the time. . . . Let’s assume that the 65% accuracy rate is symmetric — that guessers are just as good at correctly identifying gays/lesbians as they are in identifying straight people. Let’s also assume that 5% of people are actually gay/lesbian. From those numbers, a quick calculation tells us that for a randomly-selected member of the population, if your gaydar says “GAY” there is a 9% chance that you are right. Eerily accurate? Not so much. If you rely too much on your gaydar, you are going to make a lot of dumb mistakes. . . .
So, if you're an average person, you have a 9% chance, on average, of actually being right when you think someone is gay. But let's say that you have AMAZINGLY AWESOME gaydar, and that you guess correctly 90% of the time. Even then, when you look across the street at that guy/girl and are SURE that that person is gay, they still only have a 45% chance of being gay.

A more serious example of Bayes' theorem is in the realm of child sexual abuse - this is my own example, inspired by this old blog post of mine. In 2009, there were about 250,000 cases of child sexual abuse reported in the US. As I state in my old post, about 90% of child sexual abuse is perpetuated by people who know the child or are related to the child; only about 10% is perpetuated by strangers.

So, right away, we see that people you know are a much bigger threat to your child than people you don't know - a statement accentuated by the fact that you don't know far more people than you know. But what are the odds that a given stranger is going to sexually abuse your child? There are approximately 84.7 million kids in the US under the age of twenty, so the odds that your kid will be sexually abused by a stranger in a given year is about 0.0295 percent, or 1 in 3388. Given this prior probability, the odds that any one particular random adult in the US is going to sexually abuse your kid is about 1 in 753,152,400,000. So, no, your kids don't really need to be wary of strangers - let them go out, play, and explore on their own!

This final example comes from Dr. Zeckhauser's class over at HKS (one of the best courses I took while at HKS, by the way) - if you get a positive result from a medical test for a particular disease, what is the actual chance that you have that disease?

Let's assume that the disease rate for this particular disease in the general population is in the USA is about 0.3% (this is the actual HIV infection rate in the US). Furthermore, let's assume we have a test that is 99% accurate, which is a high level of accuracy. So, if you're a relatively average US citizen and get a positive result on this highly accurate test, what is the actual probability that you have the disease? It turns out that the probability that you actually have the disease is only about 23%, because the rate of infection in the overall population is so low - that's why many medial tests are performed more than once, especially if the test gives you a result you don't want.

So, go learn a little about Bayesian inference - and drastically improve your evaluation of probabilities and your capability to make good decisions!

2 comments:

  1. Andy,

    Hmmm, I guess the point of the post is that I have pretty poor odds of knowing myself - are you? ;)

    -The Angry Bureaucrat

    ReplyDelete