Steps in a systematic review
Systematic reviews and other evidence syntheses are performed according to a pre-planned, systematic, and documented process.
The image above illustrates the steps involved in systematic reviews. Critical appraisal of the methodological quality of the included studies should be done in all systematic reviews, but this is not a requirement for scoping reviews. GRADE (Grading of Recommendations Assessment, Development and Evaluation), which is a method to grade the quality (or certainty) of evidence and strength of recommendations, is often used in systematic reviews on effects, diagnosis, and prognosis.
Protocol
If you are embarking on a systematic review or other evidence synthesis you should start by writing a protocol with clear descriptions of what you are planning to do and how you are going to do it. The methodological handbooks mentioned below and PRISMA for systematic review protocols (PRISMA-P) include guidance on how to write the protocol. It is also smart to publish or register the protocol so that others can find information about planned and onging systematic reviwes. This may reduce research waste.
Protocols for systematic reviews, rapid reviews and umbrella reviews of health-related outcomes can be registered in the PROSPERO database. Protocols for other types of evidence syntheses can be registered in the Open Science Framework or in journals which publish protocols (check with journals in your subject area). Note that many journals only publish systematic reviews and other evidence syntheses if a protocol has been registered or published in advance.
Much of the text from the protocol can be reused in the method section of the systematic review, which saves time. Whether or not such reuse could be considered as self-plagiarism is highlighted in this article:
Pieper, D., Ge, L. & Abou-Setta, A. Is reusing text from a protocol in the completed systematic review acceptable?. Syst Rev 10, 131 (2021). https://doi.org/10.1186/s13643-021-01675-9
Cochrane Handbook
Cochrane Handbook for Systematic Reviews of Interventions describes the methods used to develop Cochrane Reviews.
JBI Manual for Evidence Synthesis
JBI Manual for Evidence Synthesis provides guidance on how to conduct systematic reviews and other evidence syntheses, such as umbrella reviews and scoping reviews.
PRISMA
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) is a standard for reporting in systematic reviews. In recent years, several extensions, such as PRISMA for Protocols, PRISMA for Scoping Reviews, and PRISMA for Searching have been developed.
GRADE
GRADE (Grading of Recommendations Assessment, Development and Evaluation) is a method to grade the quality (or certainty) of evidence and strength of recommendations.
Critical appraisal of individual studies
Critical appraisal of the methodological quality, also called or risk of bias assessment, in the included studies should be done for all systematic reviews. There are checklists for critical appraisal of different study designs. Examples of such are the Cochrane Risk of Bias 2 (RoB 2) tool for randomized controlled trials, the ROBINS-I tool for non-randomized trials and the QUADAS-2 tool for diagnostic studies. JBI has critical appraisal tools for a wide range of study designs.
Note that the checklists from the Critical Appraisal Skills Program (CASP) are primarily educational tools for critical appraisal training. They are not regarded as sufficient for appraisal of included studies in a systematic review.
Critical appraisal of systematic reviews
Like all other research, you should critically appraise the methodological quality of systematic reviews to assess the trustworthiness of the results. The ROBIS tool (Risk Of Bias In Systematic Reviews) and AMSTAR (A Measurement Tool to Assess Systematic Reviews) are good tools for critical appraisal of systematic reviews.