(a) In testing hypotheses, if the consequences of rejecting the null hypothesis are very serious, we should A. use a very small level of significance. B. insist that the level of significance be smaller than the P-value. C. insist that the P-value be smaller than the level of significance. D. use a very large level of significance.
(b) A medical researcher is working on a new treatment for a certain type of cancer. The average survival time after diagnosis on the standard treatment is two years. In an early trial, she tries the new treatment on three subjects who have an average survival time after diagnosis of four years. Although the survival time has doubled, the results are not statistically significant even at the 0.10 significance level. The explanation is A. the placebo effect is present, which limits statistical significance. B. the sample size is small. C. the calculation was in error. The researchers forgot to include the sample size. D. that although the survival time has doubled, in reality the actual increase is really two years.
(c) The Survey of Study Habits and Attitudes (SSHA) is a psychological test that measures the motivation, attitude, and study habits of college students. Scores range from 0 to 200 and follow (approximately) a normal distribution with mean 115 and standard deviation σ=25. You suspect that incoming freshman have a mean μ which is different from 115 because they are often excited yet anxious about entering college. To test your suspicion, you test the hypotheses H0: μ=115, Ha: μ≠115. You give the SSHA to 25 students who are incoming freshman and find their mean score. Based on this, you reject H0 at significance level α=0.01. Which of the following would be most helpful in assessing the practical significance of your results? A. Report the P-value of your test. B. Construct a 99% confidence interval for μ in order to see the magnitude of the difference between 115 and your sample results. C. Take another sample and retest just to make sure the results are not due to chance. D. Test the hypotheses again, this time using significance level α=0.001.