Car Talk was a popular show on NPR. One of the features on the show was the Puzzler where the hosts would read a riddle and invite listeners to send in solutions for the chance to win a prize. Here is an example of a Puzzler that the show made a complete hash out of (the original audio starts at 19:12 here):
Puzzler: There's a rare disease that's sweeping through your town. Of all the people who are exposed to it, 0.1% of the people actually contract the disease. There are no symptoms until the disease actually occurs. However, there's a diagnostic test that can detect the presence of the disease up to a year before it strikes. You go to your doctor, and he administers the test. It comes out positive. You say, "I'm done for!" Then you get a little bit encouraged. You say, "Wait a minute, doc, is this test 100% accurate?" Your doctor responds, "Well, not really. It's 95% accurate. In other words, 5% of the people who take the test will test positive but they don't really have the disease."
Here's the question: What are the chances that you actually have the disease?
Solution: Let's say 1,000 people take the test. Fifty people will test positive and yet they will not have it. One will test positive and have it. So your chances of actually having it, even though you tested positive, are one in 51, or a little less than 2%.
Uh, what? Before proceeding, let us fix some notation. Let D denote your true disease status and T denote the result of your test. Then:
p = P(D=+) = f = P(T=+) | D=+ f = P(T=+) | D=- 1 - f = P(T=+) | D=+ 1 - f = P(T=+) | D=-
(prevalence) (false negative rate) (false positive rate) (sensitivity) (specificity).
(a) Suffice it to say, the show did not get this one right. But what did they even do? Explain how the show interpreted the information given in the Puzzler and explain.
(b) Explain the correct way to solve this problem. Exactly what probability are we asked to compute, and what formula should we apply to do it? Does the prompt even provide enough information to actually answer the question?
(c) Under what conditions is the formula that the show used actually an upper bound on the correct answer? Explain.
(d) Explain why an upper bound on the true probability might still be a useful thing to calculate.
