Trend analysis of grading systems for level of evidence and strength of recommendation

Kyung Hwa Seo; Sun Hee Lee; Ein Soon Shin; Sun Mi Lim; Ji Eun Jang; Yu Min Jung; Yoon Hyung Park

doi:10.5124/jkma.2011.54.7.758

J Korean Med Assoc > Volume 54(7); 2011 > Article

Seo, Lee, Shin, Lim, Jang, Jung, and Park: Trend analysis of grading systems for level of evidence and strength of recommendation

Healthcare Policy

Journal of the Korean Medical Association

Journal of the Korean Medical Association 2011; 54(7): 758-768.

Published online: July 13, 2011

DOI: http://dx.doi.org/10.5124/jkma.2011.54.7.758

Trend analysis of grading systems for level of evidence and strength of recommendation

Kyung Hwa Seo, MBA¹, Sun Hee Lee, MD², Ein Soon Shin, PhD², Sun Mi Lim, PhD¹, Ji Eun Jang, BS², Yu Min Jung, RN², Yoon Hyung Park, MD^1,³

¹Research Institute for Healthcare Policy, Korean Medical Association, Seoul, Korea.

²Department of Preventive Medicine, EWHA Womans University School of Medicine, Seoul, Korea.

³Department of Preventive Medicine, Soonchunhyang University College of Medicine, Cheonan, Korea.

Corresponding author: Sun Hee Lee, lsh0270@ewha.ac.kr

Received May 17, 2011 Accepted May 24, 2011

This is an Open-Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

When clinicians or healthcare professionals are to make decisions, they can judge the quality of evidence and reliability of recommendations by 'Level of evidence' and 'Grade of recommendation'. Because of this, the step of grading evidence and recommendations is very important in developing clinical practice guidelines. The objective of this study was to identify the various grading systems and criteria of the clinical practice guidelines. We reviewed 101 guidelines from the National Guideline Clearinghouse Database and chose 66 guidelines to analyze in terms of the grading systems for level of evidence and strength of recommendation. The grading systems for 'Level of evidence' were classified into 4 types by criteria such as study design, study quality, consistency, limitations, strength of evidence, and validity. Type II was the most common evidence grading system applied by 12 organizations (37.5%) and 30 guidelines (45.5%). The grading systems for 'Grade of recommendation' were classified into 4 types by criteria such as level/quality of evidence, strength of recommendations, study quality, consistency, applicability, balance between benefit and harm, and effectiveness/usefulness. Type I was the most common recommendation grading system applied by 9 organizations (33.4%) and 23 guidelines (40.4%). A formal grading system based on consistent and clear approaches is needed because the process of grading work can be subjective when clinical practice guideline users are making decisions. It is necessary for clinical practice guideline developers to have a common criterion so that they can judge the grade of evidence and recommendations objectively in the development of clinical practice guidelines.

Grading system; Level of evidence; Quality of evidence; Strength of recommendation; Clinical practice guidelines

References

1. Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A, Schunemann HJ. GRADE Working Group. Going from evidence to recommendations. BMJ 2008;336:1049-1051.

2. The GRADE working group. GRADE handbook 2009;The Cochrane Collaboration.

3. Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, Guyatt GH, Harbour RT, Haugh MC, Henry D, Hill S, Jaeschke R, Leng G, Liberati A, Magrini N, Mason J, Middleton P, Mrukowicz J, O'Connell D, Oxman AD, Phillips B, Schunemann HJ, Edejer TT, Varonen H, Vist GE, Williams JW Jr, Zaza S. GRADE Working Group. Grading quality of evidence and strength of recommendations. BMJ 2004;328:1490.

4. Timmermans S, Mauck A. The promises and pitfalls of evidence-based medicine. Health Aff (Millwood) 2005;24:18-28.

5. Owens DK, Lohr KN, Atkins D, Treadwell JR, Reston JT, Bass EB, Chang S, Helfand M. Grading the strength of a body of evidence when comparing medical interventions 2009;Rockville, MD: Agency for Healthcare Research and Quality.

6. National Guideline Clearinghouse [Internet]. Agency for Healthcare Research and Quality cited 2011 Feb 21. Rockville, MD: Agency for Healthcare Research and Quality. Available from: http://guideline.gov/index.aspx

7. National Guideline Clearinghouse: Guideline Synthesis [Internet]. Agency for Healthcare Research and Quality cited 2011 Feb 21. Rockville, MD: Agency for Healthcare Research and Quality. Available from: http://guideline.gov/syntheses/index.aspx

8. Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, Liberati A, O'Connell D, Oxman AD, Phillips B, Schünemann H, Edejer TT, Vist GE, Williams JW Jr. GRADE Working Group. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC Health Serv Res 2004;4:38.

9. Guyatt G, Vist G, Falck-Ytter Y, Kunz R, Magrini N, Schunemann H. An emerging consensus on grading recommendations? Evid Based Med 2006;11:2-4.

10. Balshem H, Helfand M, Schünemann HJ, Oxman AD, Kunz R, Brozek J, Vist GE, Falck-Ytter Y, Meerpohl J, Norris S, Guyatt GH. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 2011;64:401-406.

11. Ansari MT, Tsertsvadze A, Moher D. Grading quality of evidence and strength of recommendations: a perspective. PLoS Med 2009;6:e1000151.

Table 1

Classification by criteria of evidence grading system

See the Appendix for abbreviations of organizations.

Table 2

Comparison of the level of evidence grading system by organization: Type I^a)

RCT, randomized controlled trial. See the Appendix for abbreviations of organizations.

^a)The main grading criteria was study design and study quality (consistency).

Table 3

Comparison of the level of evidence grading system by organization: Type II^a)

RCT, randomized controlled trial. See the Appendix for abbreviations of organizations.

^a)The main grading criteria was study design.

Table 4

Comparison of the level of evidence grading system by organization: Type III^a)

RCT, randomized controlled trial. See the Appendix for abbreviations of organizations.

^a)The main grading criteria was study design and limitation.

Table 5

Comparison of the level of evidence grading system by organization: Type IV^a)

See the Appendix for abbreviations of organizations.

^a)The main grading criteria was strength of study design.

Table 6

Classification by criteria of recommendation grading system

See the Appendix for abbreviations of organizations.

Table 7

Comparison grading system of recommendation by organization: Type I^a)

See the Appendix for abbreviations of organizations.

^a)The main grading criteria was study quality and level of evidence (consistency/applicability).

Table 8

Comparison grading system of recommendation by organization: Type II^a)

See the Appendix for abbreviations of organizations.

^a)The main grading criteria was level of evidence and strength of recommendations.

Table 9

Comparison grading system of recommendation by organization: Type III^a)

See the Appendix for abbreviations of organizations.

^a)The main grading criteria was level of evidence.

Table 10

Comparison grading system of recommendation by organization: Type IV^a)

See the Appendix for abbreviations of organizations.

^a)The main grading criteria was level of evidence and balance between benefit and harm.