Understanding Analysis of Variance (ANOVA) and Post Hoc
Analyses
Statistical Technique in Review
Analysis of variance (ANOVA) statistical
technique is conducted to examine differences between two or more
groups. There are different types of ANOVAs, with the most basic
being the one-way ANOVA, which is used to
analyze data in studies with one independent and one dependent
variable. More details on the types of ANOVAs can be found in your
research textbook and statistical texts (Grove, Burns, & Gray,
2013; Plichta & Kelvin, 2013). The outcome of ANOVA is a
numerical value for the F statistic. The
calculated F-ratio from ANOVA indicates the extent to
which group means differ, taking into account the variability
within the groups. Assuming the null hypothesis of no differences
among the groups studied is true; the probability of obtaining
an F-ratio as large as the obtained value in a given
sample is determined by the calculated p value.
If the p value is greater than the level of
significance, or alpha (α), = 0.05 set for the study, then the
study results are nonsignificant and the F-ratio will
be less than the critical values for F in the
statistical table. With nonsignificant results, researchers will accept
the null hypothesis of no significant differences between the
groups. In a study, if p = 0.01, this value is
less than α = 0.05, which indicates the groups are significantly
different and the null hypothesis is rejected. However, there is
always a possibility that this decision is in error, and the
probability of committing this Type I error is determined by α.
When α = 0.05, there are 5 chances in 100 the results are a Type I
error, or saying something is significant when it is not.
ANOVA is similar to the t-test because the null
hypothesis (no differences between groups) is rejected when the
analysis yields a smaller p value, such
as p ≤ 0.05, than the α set for the study.
Assumptions for the ANOVA statistical technique include the
following:
1. The populations from which the
samples were drawn or the random samples are normally
distributed.
2. The groups should be mutually
exclusive.
3. The groups should have equal
variance, also known as homogeneity of variance.
4. The observations are
independent.
5. The dependent variable is
measured at the interval or ratio level (Plichta & Kelvin,
2013; Zar, 2010).
Researchers who perform ANOVA on their data record their results
in an ANOVA summary table or in the text of a research article. An
example of how an ANOVA result is commonly expressed is as
follows:
where:
• F is the
statistic.
• 2 is the group degrees of
freedom (df) calculated by k − 1,
where k = number of groups in the study. In this
example, k − 1 = 3 − 1 = 2.
• 120 is the error degrees of
freedom (df) that is calculated based upon the number of
participants, or N − k. In this
example, 123 subjects − 3 groups = 120 error df.
• 4.79 is
the F-ratio or value.
• p indicates the
significance of the F-ratio in this study
or p = 0.01.
The simplest ANOVA is the one-way ANOVA, but many of the studies
in the literature include more complex ANOVA statistical
techniques. A commonly used ANOVA technique is
the repeated-measures analysis of variance,
which is used to analyze data from studies where the same
variable(s) is(are) repeatedly measured over time on a group or
groups of subjects. The intent is to determine the change that
occurs over time in the dependent variable(s) with exposure to the
independent or intervention variable(s).
Post Hoc Analyses Following ANOVA
When a significant F value is obtained from
the conduct of ANOVA, additional analyses are needed to determine
the specific location of the differences in a study with more than
two groups. Post hoc analyses were
developed to determine where the differences lie, because some of
the groups might be different and others might be similar. For
example, a study might include three groups, an experimental group
(receiving an intervention), placebo group (receiving a pseudo or
false treatment such as a sugar pill in a drug study), and a
comparison group (receiving standard care). The ANOVA resulted in a
significant F-ratio or value, but post hoc analyses
are needed to determine the exact location of the differences. With
post hoc analyses, researchers might find that the experimental
group is significantly different from both the placebo and
comparison groups but that the placebo and comparison groups were
not significantly different from each other. One could conduct
three t-tests to determine differences among the
three groups, but that would inflate the Type I error. A Type I
error occurs when the results indicate that two groups are
significantly different when, in actuality, the groups are not
different. Thus post hoc analyses were developed to detect the
differences following ANOVA in studies with more than two groups to
prevent an inflation of a Type I error. The frequently used post
hoc analyses include the Newman-Keuls test, the Tukey Honestly
Significant Difference (HSD) test, the Scheffé test, and the
Dunnett test (Plichta & Kelvin, 2013).
With post hoc analyses, the α level is reduced in proportion to
the number of additional tests required to locate the statistically
significant differences. As the α level is decreased, reaching the
level of significance becomes increasingly more difficult. The
Newman-Keuls test compares all possible pairs of means and is the
most liberal of the post hoc tests discussed here. “Liberal”
indicates that the α is not as severely decreased. The Tukey HSD
test computes one value with which all means within the data set
are compared. It is considered more stringent than the Newman-Keuls
test and requires approximately equal sample sizes in each group.
The most conservative test is the Scheffé, but with the decrease in
Type I error there is an increase in Type II error, which is saying
something is not significant when it is. The Dunnett test
requires a control group, and the experimental groups are compared
with the control group without a decrease in α.
Research Article
Source
Mayland, C. R., Williams, E. M., Addington-Hall, J., Cox, T. F.,
& Ellershaw, J. E. (2014). Assessing the quality of care for
dying patients from the bereaved relatives' perspective: Further
validation of “Evaluating Care and Health Outcomes—for the
Dying.” Journal of Pain and Symptom Management,
47(4), 687–696.
Introduction
The Liverpool Care Pathway (LCP) for the Dying Patient was
created to address the need for better end of life care for both
patients and families, which had been identified as an issue in the
United Kingdom at the national level. “LCP is an integrated care
pathway used in the last days and hours of life that aims to
transfer the hospice principles of best practice into the acute
hospital and other settings” (Mayland et al., 2014, p. 688).
“Evaluating Care and Health Outcomes—for the Dying (ECHO-D) is a
post-bereavement questionnaire that assesses quality of care for
the dying and is linked with the Liverpool Care Pathway for the
Dying Patient (LCP)” (Mayland et al., 2014, p. 687).
The purpose of this comparative descriptive study was to assess
the internal consistency reliability, test-retest reliability, and
construct validity of the key composite subscales of the ECHO-D
scale. The study's convenience sample consisted of 255 next-of-kin
or close family members of the patients with an anticipated death
from cancer at either the selected hospice or hospital in
Liverpool, United Kingdom. The sample consisted of three groups of
family members based on where the patients received end of life
care; the hospice, which used LCP; the hospital group that also
used LCP; and another group from the same hospital that did not use
LCP. The ECHO-D questionnaire was completed by all 255 study
participants and a subset of self-selected participants completed a
second ECHO-D 1 month after the completion of the first
ECHO-D. Mayland and colleagues (2014) concluded their
study provided additional evidence of reliability and validity for
ECHO-D in the assessment of end of life care.
Relevant Study Results
“Overall, hospice participants had the highest scores for all
composite scales, and ‘hospital without LCP’ participants had the
lowest scores (Tables 2 and 3). The scores for the
‘hospital with LCP’ participants were between these two levels”
(Mayland et al., 2014, p. 693). The level of significance was
set at 0.05 for the study. One-way analysis of variance was
calculated to assess differences among the hospice, hospital with
LCP, and hospital without LCP groups. Post hoc testing was
conducted with the Tukey HSD test. ANOVA and post hoc results are
displayed in Tables 2 and 3.
COMPARISON OF HOSPICE AND HOSPITAL PARTICIPANTS' SCORES
FOR COMPOSITE SCALES WITHIN THE ECHO-D QUESTIONNAIRE
Table 2
Composite Scale
Mean (SD) Range
ANOVA
(p)a
Post Hoc Comparisons Using Tukey HSD
Testb
All Participants (n = 255)
Hospice (n = 109)
Hospital with LCP (n = 78)
Hospital without LCP (n = 68)
Hospice vs. Hospital with LCP
Hospice vs. Hospital without LCP
Hospital with LCP vs. Hospital without LCP
Ward environment
7.3 (2.7) 0–10
9.1 (1.2) 5–10
6.4 (2.6) 0–10
5.4 (2.7) 0–10
60.4 (<0.0001)
<0.0001
<0.0001
0.01
Facilities
7.3 (4.8) 0–18
10.5 (4.0) 2–18
4.5 (3.8) 0–18
4.1 (2.7) 0–18
76.7 (<0.0001)
<0.0001
<0.0001
0.85
Care
18.4 (6.4) 0–25
22.0 (3.75) 7–25
16.8 (0.66) 3–25
14.6 (7.33) 0–25
35.9 (<0.0001)
<0.0001
<0.0001
0.05
Communication
9.8 (3.7) 0–14
11.2 (3.2) 0–14
9.4 (3.5) 0–14
8.2 (3.8) 0–14
16.6 (<0.0001)
0.002
<0.0001
0.86
a One-way ANOVA (between-groups ANOVA with
planned comparisons).
b Post hoc comparisons allow further exploration
of the differences between individual groups using the Tukey HSD
test, which assumes equal variances for the groups.
ECHO-D = Evaluating Care and Health Outcomes for
the Dying; ANOVA = analysis of
variance; HSD = honestly significant
difference; LCP = Liverpool Care Pathway for the
Dying Patient.
TABLE 3
COMPARISON OF HOSPICE AND HOSPITAL PARTICIPANTS' SCORES
FOR COMPOSITE VARIABLES WITHIN THE ECHO-D
QUESTIONNAIRE
Composite Variable
Mean (Range)
ANOVA (p)
Post Hoc Comparisons Using Tukey HSD Test
All Participants (n = 255)
Hospice (n= 109)
Hospital with LCP (n = 78)
Hospital without LCP (n = 68)
Hospice vs. Hospital with LCP
Hospice vs. Hospital without LCP
Hospital with LCP vs. Hospital without LCP
Symptom Control
Degree of affliction from symptoms commonly associated with dying
patients: pain, restlessness, respiratory tract secretions, nausea
and/or vomiting, and breathlessness.
Scores range from 0 (all five symptoms present all of the time) to
10 (no symptoms present).
6.8 (0–10)
7.0 (0–10)
7.0 (2–10)
6.1 (1–10)
4.4 (0.01)
0.99
0.02
0.03
Symptom Management
Reflecting whether more should have been done by staff to control
symptoms.
Scores range from 0 (not enough done by staff to control symptoms)
to 6 (staff did all they could to control symptoms).
4.8 (0–6)
5.2 (2–6)
4.8 (0–6)
4.2 (0–6)
10.6 (<0.0001)
0.17
<0.0001
0.02
Spiritual Need—Patient
Reflecting whether patients' spiritual and religious needs were
met.
Scores range from 0 (where need was not met at all) to 6 (where
needs were extremely well met).
3.0 (1.9)
3.9 (0–6)
2.9 (0–6)
1.6 (0–6)
38.1 (<0.0001)
0.0001
0.0001
0.0001
Spiritual Need—Next-of-Kin
Reflecting whether relatives' religious and spiritual needs were
met.
Scores range from 0 (where need was not met at all) to 7 (where
needs were extremely well met).
2.7 (0–7)
3.5 (0–7)
2.6 (0–7)
1.5 (0–7)
22.6 (<0.0001)
0.006
0.0001
0.002
ECHO-D = Evaluating Care and Health Outcomes for
the Dying; ANOVA = analysis of
variance; HSD = Honestly Significant
Difference; LCP = Liverpool Care Pathway for the
Dying Patient.
Study Questions
1. What type of analysis was
conducted in this study to examine group differences? What three
groups were analyzed for differences?
2. What did the researcher set
the level of significance, or alpha (α), at for this study?
3. State the null hypothesis for
communication for the three groups. Should this null hypothesis be
accepted or rejected? Provide a rationale for your answer.
4. What is the purpose of
conducting post hoc analysis?
5. Identify the post hoc results
for communication on Table 2. Which results are statistically
significant? What do these results mean?