Module Components

• Learning Activity

Topics

• Hierarchy of Evidence
• Primary Sources of Evidence
• Secondary Sources Evidence
• Evidence Appraisal

NRSG 790-Methods for Research and Evidence-Based Practice

Module 5: Overview: Critically Appraising Evidence

Evidence Appraisal

Once you have located evidence of interest, the next step in the evidence-based practice process is critical appraisal. Critical appraisal is the process of examining a research article to determine its validity and applicability to your clinical question. Critical appraisal is important because is ensures that you have a holistic view of the research article - including any strengths, weaknesses, or biases. Having a comprehensive understanding of the research articles you are reviewing will make you better able to apply this information in your practice.

Be prepared - critical appraisal can be time intensive. It requires that you read the entire article, not just skim, so that you are able to properly evaluate it. The University of Warwick's Evidence-Based Medicine Tutorial suggests that you begin by asking yourself the following questions for each article:

1. Who wrote the article. Are they credentialed, a reputable individual on the topic?
2. Why was the article written? What were the aims and objectives of the author?
3. Has the right research methodology been used?
4. Is it recent or seminal research?
5. Has it been published in a recognized or peer reviewed journal?
6. Is there any obvious bias or conflict of interests?
7. Do the conclusions match the results/findings?

When appraising any research study – experimental or observational, get into the habit of organizing your thoughts into the following three categories:

1. Are the Results Valid? Were the results obtained through a valid scientific method. Bias, or confounding variables may “threaten” or compromise the validity of the findings. Bias is anything that causes the results of a study to be what they would have been otherwise, had it not been for the bias. More biases, less validity – this is a judgement call made based on the totality of your review
• Did experimental and control groups begin the study with a similar characteristic?
• Were patients randomized?
• Was randomization concealed?
• Aside from the experimental intervention, were groups treated equally?
  
  Review these concepts associated with validity.
2. What are the Results? Validity must be established before making a determination as the reliability of the results. Regardless of what a study finds, if the data is not valid, the study is not useful for further consideration. Given valid results, now interpret those results.
   • Was there an effect of the intervention and how large was the treatment effect? This goes beyond the p value (differences between groups) and considers the actual size of the difference and is calculated through statistical formulas.
   • Answering this question should inform your thoughts regarding the difference between statistical significance and clinical significance
   • How precise was the treatment effect?
     A confidence interval (CI) represents the precision with which we are able to report the effect size, or our level of certainty. A CI describes a range of values within which we can be reasonably sure that the true effect actually lies.
   • If the confidence interval is relatively narrow (e.g. 0.70 to 0.80), the effect size is known precisely.
   • If the interval is wider (e.g. 0.60 to 0.93) the uncertainty is greater. Consider the risk, as there may still be enough precision to make decisions about the utility of the intervention.
   • Intervals that are very wide (e.g. 0.50 to 1.10) indicate that we have little knowledge about the effect, and that further information is needed.
3. How can I apply the results to my patient care?
   • Were the study patients similar to my patient?
   • Were all patient- important outcomes considered?
   • Are the likely benefits worth the potential harms and costs?