Significance Testing Things you need to consider when choosing a test 1. Differences/correlation 2. Between/within participant design 3. Criteria to carry out a more powerful test e.g. type of data, normality Differences/Correlation Differences - A difference in performance or some psychological measure between two groups or conditions Correlations - Do people with a score on variable 1 get a similar score on variable 2? (positive correlation) OR Do they get a high score on one variable and a low score on the other? (negative correlation) Between/within participants design Between/within for a test of difference Only within for a correlation Data type Strong data requires mean and SD Normality You only report the K-S result in your results 1. Plot a histogram to see if it is symmetrical 2. Check mean and median are similar 3. Check Kurtosis and Skew figures (less than double) 4. Kolmogorov-Smirnoff (Explore-Plots-Normality Plots) Compares your data set with normally distributed set. We need p>.05. 5. Can exclude outliers and see if this improves the distribution (must report) P When we want to find a significant difference, we want to be less than 0.05. When we don't want a significant difference, e.g. seeing if our data is similar to a normal distribution we want more than 0.05. Choosing a test Differences parametric between subjects - unrelated t-test Differences parametric within subjects - related t-test Differences non-parametric between subjects - Mann Whitney Differences non-parametric within subjects - Wilcoxon
Correlations parametric - Pearson's Correlations non-parametric - Spearman's Justification of test choice . The data is interval · A Kolmogorov-Smirnov test was used to test for normality and although the sleep condition was normally distribution D (19) = . 132, p=200, the no sleep condition was not. One outlier was excluded and the Kolmogorov-Smirnov test re-run where the no sleep condition was now also normally distributed D (19) = . 132, p= . 200. · The design was within subjects so a within subjects t-test was carried out Within subjects t-test Ignore paired samples correlation tables, we are looking for differences Use a sentence to state means for each condition Report t, df and p (one or two tailed) include confidence interval (CI) Always report the figure p actually equals but if p= . 000 you must report as p< . 001 Formal notation on how to write this on slide 19 Finish with a sentence to explain what results mean in terms of the hypothesis Formal notion for within t-test Faster reaction times were recorded for the sleep condition (mean=34.1) than the no sleep (mean=39.0). A paired t-test showed there was a significant differences, t (18) = 4.41, p< . 001, two tailed. The effect was large d= 1.1, 95%Cl (-7.23, -2.56). APA rules for results · Report means for each condition · Ignore +/- for the t-statistic . Round decimals to two places · Report p without leading 0 . Only true for p. Probability from 0 to 1 so can't be greater than 1 · Always report