A new test is proposed that may be better or worse than the...

A new test is proposed that may be better or worse than the standard test. To assess their relative efficacy, a pilot study is conducted where both tests are used on the same subjects and compared with the true diagnosis. Let $+$ $=$ correct assessment, $-=$ incorrect assessment. The results are given in Table 10.44 Thus, for 82 infants, both tests make the correct classification, for 3 infants both tests make the wrong classification, for 5 infants the standard test makes the correct diagnosis while the new test does not, and for 10 infants the new test makes the correct diagnosis while the standard test does not. Perform a hypothesis test to compare the accuracy of the two tests, and report a two-sided $p$ -value.

A new test is proposed that may be better or worse than the standard test. To assess their relative efficacy, a pilot study is conducted where both tests are used on the same subjects and compared with the true diagnosis. Let $+$
$=$ correct assessment, $-=$ incorrect assessment. The results are given in Table 10.44
Thus, for 82 infants, both tests make the correct classification, for 3 infants both tests make the wrong classification, for 5 infants the standard test makes the correct diagnosis while the new test does not, and for 10 infants the new test makes the correct diagnosis while the standard test does not.
Perform a hypothesis test to compare the accuracy of the two tests, and report a two-sided $p$ -value.

Key Concepts

McNemar's Test

McNemar's Test is a statistical method used to analyze paired nominal data where each subject is measured on a dichotomous outcome under two different conditions. The test specifically evaluates the hypothesis of marginal homogeneity, meaning it checks whether the probabilities of success are the same under both conditions, by focusing on the discordant pairs where the two tests give different outcomes.

Paired Categorical Data

Paired categorical data refer to observations where each subject provides a pair of responses related to two conditions, tests, or time points, and each response is categorical (such as a positive or negative outcome). This study design is particularly useful for comparing two diagnostic tests or treatments on the same subjects since it controls for individual variability.

Discordant Pairs

Discordant pairs are the instances in paired data where the outcomes for the two conditions differ between the two measurements. In the context of McNemar's Test, only these pairs (where one test is positive and the other negative) contribute information about the differences between the two conditions. The asymmetry in the count of discordant pairs is used to assess whether there is a statistically significant difference.

Hypothesis Testing Procedure

Hypothesis testing in this context involves formulating a null hypothesis that the two tests have equivalent performance (i.e., the probability of a correct classification is the same for both tests) and an alternative hypothesis that they differ. The analysis is based on the observed frequency of discordant outcomes, comparing them against the expected distribution if the null hypothesis were true.

Two-Sided p-value Calculation

The two-sided p-value in a paired test such as McNemar's is calculated by assessing the probability of observing a result as extreme or more extreme than the one obtained, under the null hypothesis. This involves using the binomial distribution to model the number of outcomes favoring one test over the other from the discordant pairs, and then multiplying the one-sided probability by two to account for the two-sided nature of the alternative hypothesis.

Transcript

00:01 The following is a solution to number 19, and this looks at two versions of the same test.

00:04 One is arranged, version a is arranged easy to difficult in terms of problems.

00:08 So it start out with easy and then it gets progressively more difficult.

00:12 And then version b, it basically flip -flops it.

00:14 So it starts out very, very difficult, and then it gets easier from there.

00:17 So here you have the data.

00:19 The sample sizes are both bigger than 30, so we can use the two samples, z interval, and z test, since the sample sizes are greater than 30.

00:25 So you can see here the sample means, so 83 and 78, and then the sample standard deviations of 4 .6 and 4 .3.

00:32 So first we need to do a 90 % confidence interval.

00:34 And i'm going to use a tid4 to show this.

00:36 So if we go to stat and test, it's going to be this ninth option down here, the two samples the interval.

00:42 And make sure stats is highlighted there.

00:44 So for sigma 1 and sigma 2, those are just going to be where your s1 and s2 go.

00:48 Like i said, we're going to use the z just because the sample says is r graded in 30.

00:53 So they're big enough.

00:54 So here i punched in the data.

00:56 The c level is 0 .9.

00:57 That stands for the 90%.

01:00 And we go ahead and calculate, and that gives us 3 .1534 and 6 .846.

01:06 Okay, so let's go and write that down.

01:08 So we have a difference, 90 % sure that, or confident that the difference is between 3 .1534 and 6 .846.

01:20 So that's the confidence interval...

Please give Ace some feedback