12 June 2021 18:47 Sample size and power If your study is too small, you will not be able to identify an effect as confidence intervals will be wide e.g. you will see a significant overlap between treated and untreated individuals in terms of variable measured If your study is too large, you will find effects with a high degree of certainty, but their size will be minimal e.g. you may be highly certain a new drug reduced blood pressure by 0.01mmHg, but this is not clinically useful You therefore need to estimate your sample size to be just large enough to detect an effect of scientific importance! 5 step plan: 1. Specify your hypothesis in advance and DO NOT CHANGE THIS TO SUIT YOUR RESULTS LATER ON! 2. Specify the significance level of the test (usually p=0.05, but not always, especially in genetics) 3. Specify the smallest effect size of scientific interest (hard! e.g. how long should a drug prolong life for?) 4. Estimate the standard deviation of your study (ideally from pilot data, or from another study) 5. Set your power level - how certain you want to be that you can detect an effect if it true 6. typically set at 0.8 (80% certain to detect an effect with your sample size if there is an effect to be found) You can think of statistical power as the probability of detecting an effect, if there is a true effect present to detect In this way, you can also give some certainty about a treatment/intervention NOT having an effect so that others do not repeat the same experiment when the outcome is likely to be negative Fortunately for us, there are online calculators to do sample size calculations for you: https://clincalc.com/stats/samplesize.aspx Allows you to specify your study type, vary power, significant level, effect size, end point, incidence and more: Sample size estimation Study Group Design & vs. & Two independent study groups & vs. One study group vs population Two study groups will each receive different treatments Primary Endpoint Incidence dropping from 23% to 22% Sample Size Group 1 27372 Group 2 27372 Total 54744 Study Parameters Incidence, group 1 23% Incidence, group 2 22% Alpha 0.05 Bet 0.2 Power 0.8 Dichotomous (yes/no) Continuous (means) The primary endpoint is binomial - only two possible outcomes Eg, mortality (dead/not dead), pregnant (pregnant/not) Incidence dropping from 23% to 13% Statistical Parameters Sample Size Anticipated Incidence Group 1 @ 23 Group 2 (? 22 % Type I/II Error Rate Alpha ® 0.05 Power ( 00% Group 1 231 Group 2 231 Total 462 Study Parameters Incidence, group 1 23% Incidence, group 2 13% Alpha 0.05 Beta 0.2 Power 0.8 Incidence Reset Calculate Enrollment ratio (?) 1 Relative Risks and Odds Ratios
You covered clinical trials extensively in a previous lecture, and we are not going to cover that again here. We do need to cover outcome from clinical trials or experiments however and particularly relative risk and odds ratios. Relative risk