A study was conducted of a diagnostic test (the FAIR test,...

A study was conducted of a diagnostic test (the FAIR test, i.e., hip flexion, adduction, and internal rotation) used to identify people with piriformis syndrome (PS), a pelvic condition that involves malfunction of the piriformis muscle (a deep buttock muscle), which often causes lumbar and buttock pain with sciatica (pain radiating down the leg) [7]. The FAIR test is based on nerve-conduction velocity and is expressed as a difference score (nerve-conduction velocity in an aggravating posture minus nerve-conduction velocity in a neutral posture). It is felt that the larger the FAIR test score, the more likely a participant will be to have PS. Data are given in the Data Set PIRIFORM.DAT for 142 participants without $P S$ (piriform $=1$ ) and 489 participants with PS (piriform $=2$ ) for whom the diagnosis of PS was based on clinical criteria. The FAlR test value is called MAXCHG and is in milliseconds (ms). A cutoff point of $\geq 1.86 \mathrm{ms}$ on the FAIR test is proposed to define a positive test. The criterion of $\geq 1.86$ ms to define a positive test is arbitrary. Using different cutoff points to define positivity, obtain the ROC curve for the FAIR test. What is the area under the ROC curve? What does it mean in this context?

A study was conducted of a diagnostic test (the FAIR test, i.e., hip flexion, adduction, and internal rotation) used to identify people with piriformis syndrome (PS), a pelvic condition that involves malfunction of the piriformis muscle (a deep buttock muscle), which often causes lumbar and buttock pain with sciatica (pain radiating down the leg) [7]. The FAIR test is based on nerve-conduction velocity and is expressed as a difference score (nerve-conduction velocity in an aggravating posture minus nerve-conduction velocity in a neutral posture). It is felt that the larger the FAIR test score, the more likely a participant will be to have PS. Data are given in the Data Set PIRIFORM.DAT for 142 participants without $P S$ (piriform $=1$ ) and 489 participants with PS (piriform $=2$ ) for whom the diagnosis of PS was based on clinical criteria. The FAlR test value is called MAXCHG and is in milliseconds (ms). A cutoff point of $\geq 1.86 \mathrm{ms}$ on the FAIR test is proposed to define a positive test.
The criterion of $\geq 1.86$ ms to define a positive test is arbitrary. Using different cutoff points to define positivity, obtain the ROC curve for the FAIR test. What is the area under the ROC curve? What does it mean in this context?

Key Concepts

Diagnostic Test Evaluation

This concept involves assessing a diagnostic tool's ability to correctly identify the presence or absence of a condition. It typically focuses on determining how well the test discriminates between diseased and non-diseased subjects, often by measuring metrics such as sensitivity, specificity, and predictive values.

Receiver Operating Characteristic (ROC) Curve

The ROC curve is a graphical plot used to illustrate the performance of a diagnostic test across a range of cutoff values. It displays the relationship between the test's sensitivity (true positive rate) and 1-specificity (false positive rate), allowing evaluation of the trade-offs involved in choosing different thresholds for a positive result.

Area Under the ROC Curve (AUC)

The AUC quantifies the overall ability of a diagnostic test to discriminate between those with and without the condition, regardless of the cutoff chosen. An AUC of 1.0 represents perfect discrimination, while an AUC of 0.5 indicates no discriminatory power. In practical terms, a higher AUC shows that the test is more accurate at distinguishing between the two groups.

Cutoff Value and its Arbitrary Selection

A cutoff value is a threshold used to decide whether the test result should be considered positive or negative. The choice of cutoff is often arbitrary and might not be optimal, as it can influence the calculated sensitivity and specificity, reflecting the inherent trade-off between these measures in diagnostic decision-making.

Sensitivity and Specificity

Sensitivity and specificity are key performance metrics in evaluating a diagnostic test. Sensitivity measures the proportion of true positives correctly identified by the test, whereas specificity measures the proportion of true negatives correctly identified. Balancing these measures is essential when selecting an optimal threshold and understanding the test's practical utility in clinical settings.

Transcript

00:01 For this problem, we are told that after exercise, massage is often used to relieve pain.

00:05 And a recent study shows that it also may relieve inflammation and help muscles heal.

00:10 We're told that in the study, 11 male participants who had just strenuously exercised had 10 minutes of massage on one quadricep, and no treatment on the other, with treatment randomly assigned.

00:22 We're told that after 2 .5 hours muscle biopsies were taken and the production of the inflammatory cytokine interleukine 6, was measured relative to the resting level.

00:33 The differences, control minus massage, are given in table 4 .11.

00:38 In part a, we are asked, is this an experiment or an observational study, and why is it not double -blind? so the reason why this is not, or actually, i'll back up.

00:48 So first of all, this is an experiment because the researchers are randomly assigning the variable.

00:56 And let's see here.

01:00 So the reason why it's not double blind is because the participants know, obviously, whether they have received the treatment or not.

01:10 There's no placebo.

01:12 There are no, excuse me, there are no participants who would be receiving a placebo, so to speak.

01:18 In part b, we are asked, what does the sample mean difference in inflammation between no massage and massage? so i'm actually going to jump over into stat key at this point, because we want to, we'll be doing that soon anyways, where in this case we would want, oh, excuse me, we don't want proportions, we want test for a difference in means, and we'll go to edit data, delete all of this, and we have, we'll just put in our data points, 0 .6, 4 .7, 3 .8, 0 .4, 1 .5, negative 1 .2, 2 .8, 2 .8, negative 1 .2, 2 .8, negative 0 .4, 1 .4, 3 .5, and negative 2 .8.

02:08 And we do not have a header row.

02:13 Uh -oh.

02:17 Okay, it says, okay, one moment here.

02:21 Okay, excuse me, that was silly.

02:23 So, we'll find that the sample mean difference is 1 .3.

02:29 In part c, we are told that we want to test to see of the population mean difference, mu -d.

02:36 Is greater than zero, meaning muscle with no treatment has more inflammation than muscle that has been massaged.

02:42 This case, we are to state the null and alternative hypotheses.

02:46 So with the way that this is phrased, the null hypothesis will be that mu -d equals zero, and the alternative hypothesis will be that mu -d is greater than zero.

03:01 For part d, we are asked to use stat key or other technology to find the p value from a randomization distribution...

Please give Ace some feedback