When is a hypothesis test most likely to be statistically significant?
Added by Jeffery J.
Step 1
Statistical significance typically refers to the likelihood that a result or relationship is caused by something other than mere random chance. It is often determined using a p-value threshold (commonly set at 0.05). Show more…
Show all steps
Close
Your feedback will help us improve your experience
Sri K and 55 other Intro Stats / AP Statistics educators are ready to help you.
Ask a new question
Labs
Want to see this concept in action?
Explore this concept interactively to see how it behaves as you change inputs.
Key Concepts
Recommended Videos
True or False 5. A high p-value indicates the null hypothesis must be true. 6. A small p-value indicates strong evidence against the null hypothesis. 7. The formulas for confidence intervals and hypothesis tests work even if the data was not from a SRS. 8. The result of a hypothesis test can be statistically significant without being practically significant. Multiple Choice 9. A medical researcher is working on a new treatment for a certain type of cancer. The average survival time after diagnosis on the standard treatment is two years. In an early trial, she tries the new treatment on three subjects who have an average survival time after diagnosis of four years. Although the survival time has doubled, the results are not statistically significant even at the 0.10 significance level. The most likely explanation is: A) the placebo effect is present, which limits statistical significance. B) the sample size is small. C) that although the survival time has doubled, in reality the actual increase is really two years. D) the calculation was in error. The researchers forgot to include the sample size. 10. An engineer designs an improved light bulb. The previous design had an average lifetime of 1,200 hours. The new bulb had a lifetime of 1,200.2 hours, using a sample of 40,000 bulbs. Although the difference is quite small, the effect was statistically significant. The most likely explanation is: A) that new designs typically have more variability than standard designs. B) that the sample size is very large. C) that the mean of 1,200 is large. D) all of the above are likely. 11. Suppose that the population of the scores of all high school seniors that took the Math SAT test this year follows a normal distribution with mean of 480 and standard deviation of 90. You read a report that claims that 10,000 students who took part in a national program for improving one's Math SAT score had significantly better scores (at the 0.05 level of significance) than the population as a whole. In order to determine if the improvement is of practical significance one should: A) find out the actual P-value. B) run the national program a second time to see if similar results are obtained. C) use a two-sided test rather than the one-sided test implied by the report. D) find out the actual mean score and compute a confidence interval for the 10,000 students.
Sri K.
The standard error of most sample estimators approaches zero as the sample size increases, so almost any difference between the sample statistic and the hypothesized parameter value, no matter how tiny, will be significant if the sample size is large enough. Researchers who deal with large samples must expect "significant" effects, even when an effect is too slight to have any practical importance. Is an improvement of 0.2 mpg in fuel economy important to Toyota buyers? Is a 0.5 percent loss of market share important to Hertz? Is a laptop battery life increase of 15 minutes important to Dell customers? Such questions depend not so much on statistics as on a cost/benefit calculation. Since resources are always scarce, a dollar spent on a quality improvement always has an opportunity cost (the foregone alternative). If we spend money to make a certain product improvement, then some other project may have to be shelved. Since we can't do everything, we must ask whether a proposed product improvement is the best use of our scarce resources. These are questions that must be answered by experts in medicine, marketing, product safety, or engineering, rather than by statisticians. As an example, consider the following question. Suppose we want to compare the IQ test scores of high school students from two different states. In Nevada, we sample 12,000 students and find a mean of 100.15 with a standard deviation of 15. In New Jersey, we sample 188,000 students and find a mean of 99.85 with a standard deviation of 14.75. a) Test if there is a difference in the average IQ of students between the two states. Show your implementation in Minitab. b) Discuss practical versus statistical significance for this example and provide relevant citations.
Sheryl E.
What is the practical significance of a hypothesis test for a population mean?
Adi S.
Recommended Textbooks
Elementary Statistics a Step by Step Approach
The Practice of Statistics for AP
Introductory Statistics
Transcript
18,000,000+
Students on Numerade
Trusted by students at 8,000+ universities
Watch the video solution with this free unlock.
EMAIL
PASSWORD