NUR 304 Chapter005

Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins
Chapter 5: Critically Appraising
Quantitative Evidence for
Clinical Decision Making

Critically Appraising Quantitative Studies
• The process focuses on three questions
1. Are the results of the study valid? (Validity)
2. What are the results? (Reliability)
3. Will the results help me in caring for my patients?
(Applicability)
• Interpretation of results requires consideration of the
clinical significance of the study findings and the
statistical significance of the results

The Question of Validity
• Bias - anything that distorts study findings in a
systematic way and arises from the study methodology
− Selection bias
− Knowledge of who is or is not receiving an
intervention
− Measurement bias
− Recall bias
− Contamination

Question
• Tell whether the following statement is true or false.
• The best way to prevent selection bias is to randomly
assign study participants to groups.

Answer
• True
• Rationale: Selection bias in quantitative studies is best
controlled by assigning participants to groups on a
random basis. Other systematic and deliberate methods
of assignment normally increase the chance of selection
bias.

The Question of Validity (cont’d)
• Confounded Study Results
− A study’s results may be confounded when a
relationship between two variables is actually due to
a third, either known or unknown variable
− Often encountered in studies about lifestyle and
health
− Frequently a result of participant history

Reliability
• Evaluate whether the sum of all n values equals the
original N
• Magnitude of the effect
− How strong is the difference between groups?
− Tables
− Statistical tests

Reliability (cont’d)
• Strength of association
− Absolute risk reduction (ARR)
− Absolute risk increase (ARI)
− Relative risk (RR)
− Relative risk reduction (RRR)
− Odds ratio (OR)
− Number needed to treat/harm (NNT/NNH)

Question
• The findings of a quantitative study testing a high
school-based sexual health program reveal that for every
140 female students who take the program, one
pregnancy is prevented. This conclusion indicates the:
a. OR
b. NNT
c. NNH
d. ARR

Answer
• b. NNT
• Rationale: The number needed to treat (NNT) represents
the number of people who would need to receive the
therapy or intervention (the educational program) to
prevent one bad outcome (teenage pregnancy).

• Random error
− Variations that occur purely by chance
− The extent to which random error may influence a
measurement can be reported using statistical
significance (or p values) or by confidence intervals.

• Statistical significance
− The aim of statistical analysis is to determine
whether an observed effect arises from the study
intervention or has occurred by chance
− Study hypothesis and null hypothesis
− The smaller the p value, the less likely the null
hypothesis is true
− Confidence interval - describes the range in which
the true effect lies with a given degree of certainty

Confidence Intervals

Questions to Ask in a Critical Appraisal
• Why was the study done?
• What is the sample size?
• Are measurements reliable and valid?
• How were the data analyzed?
• Did any untoward event happen during the study?
• How do the findings fit with previous research?
• What does this mean for practice?

Appraising Case Studies
• Historically ranked lower in the hierarchy of evidence
• Must be used with caution to inform practice, and any
application requires careful evaluation of the outcomes

Appraising Case Control Studies
• These investigate why certain people develop a specific
illness, have an adverse event with a particular
treatment, or behave in a particular way
• Questions to ask
− How were the cases obtained?
− Were appropriate controls selected?
− Were data collection methods the same for the cases
and controls?

Appraising Cohort Studies
• These are used for investigating the course of a disease
or the unintended consequences of a treatment
• Questions to ask
− Was there a representative and well-defined sample
of individuals at a similar point in the course of the
disease?
− Was follow up sufficiently long and complete?
− Were objective and unbiased outcome criteria used?

Appraising Cohort Studies (cont’d)
• Did the analysis adjust for important prognostic risk
factors and confounding variables?
• What is the magnitude of the relationship between
predictors (i.e., prognostic indicators) and targeted
outcome?
• How likely is the outcome event(s) in a specified period
of time?
• How precise are the study estimates?

Question
• A team of researchers have received a grant to
investigate the potential links between diet and the
development of stomach cancer. What methodology is
most likely to answer the researchers’ clinical question?
a. Case control
b. Case study
c. Randomized controlled trial (RCT)
d. Qualitative study

Answer
• a. Case control
• Rationale: A case control study often selects individuals
with a particular disease (e.g., stomach cancer) and
looks back to identify factors that may underlie that
disease (e.g., diet). Neither a case study nor a qualitative
study would inform this relationship and an RCT would be
unethical and impractical.

Appraising Randomized Controlled Trials
(RCTs)
• These can accurately establish cause and effect and can
inform the efficacy and effectiveness of interventions
• Questions related to validity of RCTs
− Were the subjects randomly assigned to the
experimental and control groups?
− Was random assignment concealed from the
individuals who were first enrolling subjects into the
study?
− Were the subjects and providers kept blind to study
group?

Appraising the Validity of RCTs
• Were reasons given to explain why subjects did not
complete the study?
• Were the follow-up assessments long enough to fully
study the effects of the intervention?
• Were the subjects analyzed in the group to which they
were randomly assigned?
• Was the control group appropriate?
• Were instruments valid and reliable?
• Were the subjects in each of the groups similar on
demographic and baseline clinical variables?

Appraising Systematic Reviews
• Systematic reviews are a compilation of similar studies
that address a specific clinical question
• Not the same as a literature review or narrative review
• The process the was used to conduce the review should
be explicit
• Look for a detailed description of the databases accessed,
the search strategies, and the search terms
• Clinicians should be able to clearly see which studies
were included and which were excluded and how selected
studies were assessed

Appraising Systematic Reviews (cont’d)
• The report should inform clinicians about how data were
extracted from the individual studies
• It should provide an overview of the evaluation of the
included studies
• Meta-analysis
− The statistical approach to synthesizing the results of
two or more studies
− A relatively new methodology that has become a
hallmark of EBP
• As with all methodologies, applicability must be
considered

Evaluation and Synthesis of Quantitative
Studies
• Once studies have been selected, they should be melded
together into a synthesis upon which to base practice and
standards
• Individual studies should be evaluated using an
evaluation table - see Table 5-8
• Synthesis is not reporting the findings of study after
study; it is combining, contrasting, and interpreting a
body of evidence to reach a conclusion

Question
• Tell whether the following statement is true or false.
• Meta-analysis results in evidence that is applicable to a
larger patient population than individual studies.

Answer
• False
• Rationale: In meta-analyses, combining the results of
several studies produces a larger sample size and thus
greater power to accurately determine the magnitude of
the effect. This does not, however, increase the size of
the relevant patient population.

NUR 304 Chapter005

More Related Content

What's hot

Viewers also liked

Similar to NUR 304 Chapter005

More from stanbridge

Recently uploaded

NUR 304 Chapter005