Your browser does not support the NLM PubReader view.

Go to this page to see a list of supporting browsers.

Tools for assessing quality and risk of bias by levels of evidence

Article information

J Korean Med Assoc. 2011;54(4):419-429
Publication date (electronic) : 2011 April 08
doi : https://doi.org/10.5124/jkma.2011.54.4.419
Sun Mi Lim, PhD1, Ein Soon Shin, PhD2, Sun Hee Lee, MD2, Kyung Hwa Seo, MBA1, Yu Min Jung, RN2, Ji Eun Jang, BS2
1Research Institute for Healthcare Policy, Korean Medical Association, Korea.
2Department of Preventive Medicine, Ewha Womans University School of Medicine, Seoul, Korea.
Corresponding author: Ein Soon Shin, shin1121@ewha.ac.kr
Received 2011 January 30; Accepted 2011 February 17.

Abstract

Tools for assessing methodological quality or risk of bias in randomized controlled trials (RCTs) and non-randomized studies (NRS) were reviewed. The van Tulder scale and Cochrane's assessment of risk of bias are the two most useful methodological quality evaluation tools for RCTs. Cochrane's tool includes sequence generation, allocation of sequence concealment, blinding, incomplete outcome data, selective outcome reporting, and other potential sources of bias. The Cochrane Collaboration Group recommends the Downs and Black instrument and the Newcastle-Ottawa Scale for evaluating the quality of NRS. In conclusion, this study offers useful information to physicians about tools for assessing the quality of evidence in clinical guidelines. Further research is needed to provide an essential core for evidence-based decision making regarding levels and/or grades of recommendations.

Keywords: Jadad's scale; van Tulder scale; Cochrane's tool; Downs and Black instrument; Newcastle-Ottawa Scale

References

1. Higgins JP, Green S, eds. Cochrane handbook for systematic reviews of interventions. Ver. 5.1.0. The Cochrane Collaboration 2011. cited 2011 Jan 7. Available from: http://www.cochrane-handbook.org.
2. Chalmers TC, Smith H Jr, Blackburn B, Silverman B, Schroeder B, Reitman D, Ambroz A. A method for assessing the quality of a randomized control trial. Control Clin Trials 1981. 231–49.
3. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 1995. 1662–73.
4. Barratt A. Evidence based medicine and shared decision making: the challenge of getting both evidence and preferences into health care. Patient Educ Couns 2008. 73407–412.
5. Kranke P. Evidence-based practice: how to perform and use systematic reviews for clinical decision-making. Eur J Anaesthesiol 2010. 27763–772.
6. Parrilla-Castellar ER, Almeyda R, Nogales E, Velez M, Ramos M, Rivera JE, Da Vila B, Torres V, Capriles J, Adamsons K. Evidence-based medicine as a tool for clinical decision-making in Puerto Rico. P R Health Sci J 2008. 27135–140.
7. Tiburi MF. Evidence-based medicine as viewed by key decision-makers of health plans in southern Brazil. Health Serv Manage Res 2008. 21185–191.
8. Stolba N, Nguyen TM, Tjoa AM. Towards sustainable decision-support system facilitating EBM. Conf Proc IEEE Eng Med Biol Soc 2007. 20074355–4358.
9. Wyer PC, Silva SA. Where is the wisdom? I. a conceptual history of evidence-based medicine. J Eval Clin Pract 2009. 15891–898.
10. Collins J. Evidence-based medicine. J Am Coll Radiol 2007. 4551–554.
11. Anastasiu M, Strâmbu V, Popa F. Evidence-based medicine, conceptual challenge or the future of daily practice? Chirurgia (Bucur) 2007. 102527–530.
12. Carter MJ. Evidence-based medicine: an overview of key concepts. Ostomy Wound Manage 2010. 5668–85.
13. Maluf-Filho F. The importance of evidence-based medicine concepts for the clinical practitioner. Arq Gastroenterol 2009. 4687–89.
14. Borgerson K. Valuing evidence: bias and the evidence hierarchy of evidence-based medicine. Perspect Biol Med 2009. 52218–233.
15. Isaac CA, Franceschi A. EBM: evidence to practice and practice to evidence. J Eval Clin Pract 2008. 14656–659.
16. Kruer MC, Steiner RD. The role of evidence-based medicine and clinical trials in rare genetic disorders. Clin Genet 2008. 74197–207.
17. Rogers W, Ballantyne A. Justice in health research: what is the role of evidence-based medicine? Perspect Biol Med 2009. 52188–202.
18. Kitto S, Petrovic A, Gruen RL, Smith JA. Evidence-based medicine training and implementation in surgery: the role of surgical cultures. J Eval Clin Pract 2010. 08. 04. [Epub]. DOI: 10.1111/j.1365-2753.2010.01526.x.
19. Carretier J, Bataillard A, Fervers B. The patient's role in evidence-based medicine. J Chir (Paris) 2009. 146537–544.
20. Soll RF. Evaluating the medical evidence for quality improvement. Clin Perinatol 2010. 3711–28.
21. Deeks JJ, Dinnes J, D'Amico R, Sowden AJ, Sakarovitch C, Song F, Petticrew M, Altman DG. International Stroke Trial Collaborative Group. European Carotid Surgery Trial Collaborative Group. Evaluating non-randomised intervention studies. Health Technol Assess 2003. 7iii–x. 1–173.
22. MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, Black AM. A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technol Assess 2000. 41–154.
23. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996. 171–12.
24. van Tulder M, Furlan A, Bombardier C, Bouter L. Editorial Board of the Cochrane Collaboration Back Review Group. Updated method guidelines for systematic reviews in the cochrane collaboration back review group. Spine (Phila Pa 1976) 2003. 281290–1299.
25. Newell SA, Sanson-Fisher RW, Savolainen NJ. Systematic review of psychological therapies for cancer patients: overview and recommendations for future research. J Natl Cancer Inst 2002. 94558–584.
26. Scottish Intercollegiate Guideline Network. SIGN 50: a guideline developer's handbook 2008 [Internet] 2008. cited 2011 Jan 28. Edinburgh: Scottish Intercollegiate Guidelines Network; Available from: http://www.sign.ac.uk/pdf/sign50.pdf.
27. National Institute for Health and Clinical Excellence. The guidelines manual 2009 [Internet] 2009. cited 2011 Jan 27. London: National Institute for Health and Clinical Excellence; Available from: http://www.nice.org.uk/media/5F2/44/The_guidelines_manual_2009_-_All_chapters.pdf.
28. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health 1998. 52377–384.
29. Wells GA, Shea B, O'Connell D, Peterson J, Welch V, Losos M, Tugwell P. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses [Internet] c1996-2010. cited 2011 Mar 28. Ottawa: Ottwa Hospital Research Institute; Available from: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.
30. Reisch JS, Tyson JE, Mize SG. Aid to the evaluation of therapeutic studies. Pediatrics 1989. 84815–827.
31. Thomas H. Quality assessment tool for quantitative studies: effective public health practice project 2003. Hamilton: McMaster University;
32. Zaza S, Wright-De Agüero LK, Briss PA, Truman BI, Hopkins DP, Hennessy MH, Sosin DM, Anderson L, Carande-Kulis VG, Teutsch SM, Pappaioanou M. Data collection instrument and procedure for systematic reviews in the Guide to Community Preventive Services. Task Force on Community Preventive Services. Am J Prev Med 2000. 181 Suppl. 44–74.

Article information Continued

Copyright © 2011 Korean Medical Association

This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Table 1

Quality assessment tools of randomized controlled trials

Table 1

CONSORT, consolidated standards of reporting trials; SIGN, Scottish Intercollegiate Guidelines Network; NICE, National Institute for Health and Clinical Excellence.

Table 2

Jadad scale

Table 2

From Jadad AR, et al. Control Clin Trials 1996;17:1-12, Appendix with permission from Elsevier [23].

Table 3

van Tulder scale [24]

Table 3

Table 4

Cochrane's assessment of risk of bias [1]

Table 4

*Minimization may be implemented without a random element, and this is considered to be equivalent to being random.

Table 5

Quality assessment tools of non-randomized studies

Table 5

Table 6

Downs and Black scale: checklist for measuring study quality [28]

Table 6

Table 7A

Newcastle-Ottawa quality assessment scale: case-control studies [29]

Table 7A

Table 7B

Newcastle-Ottawa quality assessment scale: cohort studies [29]

Table 7B