Statistical Significance versus Practical Significance A...

Statistical Significance versus Practical Significance A math teacher claims that she has developed a review course that increases the scores of students on the math portion of the SAT exam. Based on data from the College Board, SAT scores are normally distributed with $\mu=514$ and $\sigma=113$ The teacher obtains a random sample of 1800 students, puts them through the review class, and finds that the mean SAT math score of the 1800 students is 518 . (a) State the null and alternative hypotheses. (b) Test the hypothesis at the $\alpha=0.10$ level of significance. Is a mean SAT math score of 518 significantly higher than $514 ?$ (c) Do you think that a mean SAT math score of 518 versus 514 will affect the decision of a school admissions administrator? In other words, does the increase in the score have any practical significance? (d) Test the hypothesis at the $\alpha=0.10$ level of significance with $n=400$ students. Assume that the sample mean is still $518 .$ Is a sample mean of 518 significantly more than $514 ?$ Conclude that large sample sizes cause $P$ -values to shrink substantially, all other things being the same.

Statistical Significance versus Practical Significance A math teacher claims that she has developed a review course that increases the scores of students on the math portion of the SAT exam. Based on data from the College Board, SAT scores are normally distributed with $\mu=514$ and $\sigma=113$ The teacher obtains a random sample of 1800 students, puts them through the review class, and finds that the mean SAT math score of the 1800 students is 518 .
(a) State the null and alternative hypotheses.
(b) Test the hypothesis at the $\alpha=0.10$ level of significance. Is a mean SAT math score of 518 significantly higher than $514 ?$
(c) Do you think that a mean SAT math score of 518 versus 514 will affect the decision of a school admissions administrator? In other words, does the increase in the score have any practical significance?
(d) Test the hypothesis at the $\alpha=0.10$ level of significance with $n=400$ students. Assume that the sample mean is still $518 .$ Is a sample mean of 518 significantly more than $514 ?$ Conclude that large sample sizes cause $P$ -values to shrink substantially, all other things being the same.

Key Concepts

Null Hypothesis

The null hypothesis represents the assumption that there is no effect or difference from a known or claimed value. In tests comparing means, it typically states that the population mean remains unchanged, serving as the baseline against which the alternative is tested.

Alternative Hypothesis

The alternative hypothesis is a statement that contradicts the null hypothesis, proposing that there is an effect or a difference. It is the claim that the researcher aims to support by showing evidence that the true parameter differs from the null value.

Significance Level

The significance level, often denoted by alpha (?), is the threshold for deciding whether an observed effect is statistically significant. It represents the probability of making a Type I error, which is rejecting the null hypothesis when it is actually true.

p-value and Statistical Significance

A p-value indicates the probability of obtaining results at least as extreme as those observed, assuming the null hypothesis is true. When the p-value is lower than the chosen significance level, the result is deemed statistically significant, suggesting strong evidence against the null hypothesis.

Practical Significance

Practical significance evaluates whether a statistically significant difference is large or important enough to matter in real-world applications. It concerns the magnitude and impact of the effect rather than just its statistical reliability.

Effect of Sample Size in Hypothesis Testing

The sample size affects the precision of estimates and the power of tests. Larger samples tend to produce smaller p-values, potentially leading to statistically significant results even for minimal differences. This underscores the need to consider practical significance alongside statistical significance, as large samples can detect trivial effects that may not be meaningful in real contexts.

Transcript

00:01 The following is a solution for number 37, and it's actually data we've used before.

00:04 The sat math scores.

00:06 The mean is 514 and the standard deviation for the population is 113.

00:11 And there is a product, i guess, that says that they can raise the math score.

00:15 So they sampled 1 ,800 people that used the product and took the sat, and they found that the mean math score was 518, which appears to be a little bigger.

00:24 And we're testing at the 10 % level of significance if, in fact, they did improve their.

00:30 Statistically improved their math test scores.

00:34 So the first part is to state what our null hypotheses are, and there's always equality with the mean, or with the null hypothesis, so mu equals 14, and then the alternative would be mu is greater than 514, because it's increasing the score.

00:49 So we do that, and let's go ahead and do it in the calculator, because it'll be a little quicker this way.

00:54 We're going to do a z test, since we actually know what the population standard deviation is.

00:58 The hypothesized value is 514, population standard deviation is 113, the x bar, the sample mean was 518, the sample size is quite large, 1800, and then our alternative is that it increased, so greater than.

01:10 And we calculate, and we get a test statistic of 1 .502 with the p value of about 0 .067.

01:18 Okay, so let's write those down, and let's analyze them.

01:20 So 1 .502 and 0 .067, which is less than alpha...

Please give Ace some feedback