Skip to Main Content

Systematic review

A step by step guide to doing a systematic review or other forms of Evidence Synthesis

Quality assessment: critical appraisal

Critical appraisal is an essential step of evidence synthesis. You may see it also being referred to as quality assessment or risk of bias assessment. It is focused on the methods of evidence rather than simply the results. Critical appraisal seeks to understand the internal and external validity of a study, as explained below in Gosall and Gosall, The Doctor’s Guide to Critical Appraisal (2020).

"Critical appraisal assesses the validity of the research and statistical techniques employed in studies, and generates clinically useful information from them. It seeks to answer two major questions:​

  • Does the research have internal validity - to what extent does the study measure what it sets out to measure?
  • Does the research have external validity - to what extent can the results from the study be generalised to a wider population?" (pp3-4)
     

A systematic approach

Systematic reviews require a formal, systematic and uniform approach to appraisal of the quality (or risk of bias) of all studies included in the data synthesis. If your review only includes Randomised Controlled Trials, then all the RCTs should undergo the same appraisal process and the decisions on quality recorded in the same way e.g. by using an appropriate tool such as the Cochrane Risk of Bias Tool, which has been specifically designed for evaluating RCTs.
 

Not the same as certainty of evidence

While similar, the assessment of certainty of evidence is not the same as critical appraisal. This independent stage of the systematic review process is explained in this guide's section on summarising evidence certainty with GRADE. You will need to complete both the quality assessment and data extraction activities to conduct an assessment on evidence certainty.

Critical appraisal tools and checklists

Critical Appraisal in Covidence

Covidence can help manage critical appraisal as part of it's Data Extraction workflow. You can choose to use the inbuilt template for the Cochrane Risk of BIas tool (although note this is version 1 not RoB2) or a customizable template where you can build a form to suit the tool you are using.

When your critical appraisal of all included studies has been completed you can export the Quality Assessment consenus data (or the consensus with individual data as well) via a CSV file using the Export option on the main review screen.

Last Updated: Nov 17, 2026 11:58 AM