Association between research topics and disease burden in health technology assessment

Article information

J Korean Med Assoc. 2016;59(7):536-546

Publication date (electronic) : 2016 July 20

doi : https://doi.org/10.5124/jkma.2016.59.7.536

Hee Sun Kim, PhD ¹

, Jisu Lee, KMD ²

, Bit-Na Yoo, MPH ¹

¹National Evidence-based Healthcare Collaborating Agency, Seoul, Korea.

²School of Medicine, Seoul National University, Seoul, Korea.

Corresponding author: Hee Sun Kim. hskim7336@neca.re.kr

Received 2016 April 15; Accepted 2016 May 06.

Abstract

The National Evidence-based Healthcare Collaborating Agency (NECA), an institution for health technology assessment in Korea, has used public solicitation of research topics since its establishment in 2009. This creates a necessity for examining whether a given research topic was selected to be considered when prioritizing healthcare technology assessment and distributing healthcare resources. This study aimed to investigate the relationship between the research topics suggested to NECA and the disease burden in Korea. To find the correlation between disease burden and 1,112 suggested topics and 91 performed topics that were classified by Human Research Classification System a linear auxiliary trend line and scatter plot were constructed using disability-adjusted life years (DALYs), and Pearson's correlation coefficients were calculated. The results suggested that cancer was most common, followed by cardiovascular diseases, among suggested research topics and research topics performed by NECA, as well as in terms of the ratio of performed to suggested topics. The correlation between research topic and disease burden index indicated a strong correlation with DALYs and years of life lost (YLLs). However, years lived with disability and research topic had no relationship. Suggested topics showed a greater correlation with YLLs than DALYs, whereas performed topics showed a greater correlation with DALYs than YLLs, showing that despite the fact that the diseases with a high burden from morbidity were appropriately considered with respect to selecting research topics, a statistically significant difference was not found. As the first Korean study to assess the correlation between research topics and disease burden, our results will be used as base data for prioritizing the allocation of healthcare resources in the future.

Keywords: Research topic; Cost of illness; Disability-adjusted life years

Introduction

With development of medicine and health technology and increased social benefits in Korea, medical resource is becoming more and more scarce [1]. This is also becoming a problem in developed countries, such as the USA and the UK, where public health assessment institutes are being operated to solve this problem. In Korea, the National Evidence-based Healthcare Collaborating Agency (NECA) was established in 2009 to support reasonable decision-making in the nation's social insurance system, which lacked evidence-based decision-making system at the time [2].

Since its establishment for this purpose, NECA has conducted research which reflected opinions of various groups on the healthcare priorities by way of open topic suggestions. Being one of the methods to investigate research topics by health technology assessment institutes around the world, the open topic suggestion is capable of reflecting public opinions [3] but not of deciding whether or not a selected topic is a high priority. This creates a necessity to examine whether or not a research topic was selected properly based on the priority list, which would be possible by exploring correlation between a research topic and disease burden [4 5 6], an important factor to be considered when prioritizing healthcare technology assessment and distributing healthcare resources.

NECA currently prioritizes the effect of burden of disease over those of other factors when deliberating the appropriateness of a selected research topic. Still, there is a need to develop an approach that allows experts to make quantitative comparisons between disease burdens, thereby allowing a more systematic selection of research topics.

The UK Clinical Research Collaboration (UK-CRC) have conducted studies to find gaps and opportunities between research associated with World Health Organization between 2006 and 2007 and worldwide disease burden [7 8 9]. Other studies have also explored the correlation between research fund and disease burden in the health and medical research and development [10 11 12 13] or analyzed research trends among published health technology assessment articles [14]. However, none of the previous studies have conducted a study of topics suggested or researched by government or institute for health technology assessment.

This study was aimed at exploring topics suggested and researched between 2009 and 2014 by NECA, an institute for health technology assessment in Korea, classifying them according to the Human Research Classification System (HRCS) of UK-CRC, and examining their correlation with disability-adjusted life years (DALYs), YLLs and YLDs in the global burden of diseases (GBD) 2010. Health Technology Assessment (HTA) research in this study refers to studies conducted by NECA.

Materials and Methods

1. Data collection

1) Suggested topics and performed topics

NECA has been receiving suggestions on research topics both online (email and website) and offline (fax, mailing, meeting, and workshop) from the government, related organizations, citizens, healthcare workers, scholars and inside staff. Annual selection process of research topics consists of topic suggestion, topic review such as similarity to existing research, prioritization by quick review of topics, and topic selection by review of research proposal. This study involved 1,534 topics that were suggested and 158 topics that were researched between 2009 and 2014.

Research topics were classified by using the HRCS, a system developed by the UK-CRC in 2006 to promote strategic research funding in the field of healthcare. The HRCS is comprised of Health Categories and Research Activity Code. It encompasses every type of biomedical and health research from health and diseases-related principles to their applications, allowing research to be systematically classified into research topics; other European research centers have taken this system into interest and have also been recommending it to others [15].

In our study, we used the HRCS to classify research topics according to disease so that we could evaluate the relationship between research topic and disease burden. The classification was conducted by at least two individual investigators. The first classification was conducted individually based on the research proposal after which the investigators reached an agreement. If not, a third party has intervened to discuss and reach a conclusion by majority rule. Multiple codes were presented in the result if a topic involves multiple disease categories.

2) Global burden of diseases

To estimate the disease burden in Korea, this study used years of life lost (YLLs), years lived with disability (YLDs), and DALYs for Korea in GBD 2010 [16]. YLLs are the time lost due to early death and are used to measure the burden of premature mortality. YLDs are the time lived with a disease and measures the burden of living with a disability. DALYs are calculated as the sum of YLLs and YLDs and represent both the burden of premature death and disability by a disease [17 18]. Thus, these indicators have different magnitude depending on the characteristics of a disease. An acute and fatal disease may have YLLs bigger than YLDs, while a chronic and non-fatal disease may have YLLs smaller than YLDs. Thus DALYs were used as an indicator of general burden of a disease, and analysis was carried out using YLLs and YLDs in order to find out which kinds of diseases got more attention.

The two health categories general health relevance (suggested topics=356, performed topics=65) and others (suggested topics=41, performed topics=0), which did not overlap with those of the GBD data, were excluded from our study. This is because the difference in the health categories of HRCS and that of the GBD data meant the data from these two databases could not be interchangeably used nor be compatible pairs for comparison. In total, our study analyzed 1,112 suggested topics and 91 performed topics by employing a classification system used by Kinge et al. [10] to investigate their relationship with disease burden. As 11 suggested topics and two performed topics under 'cardiovascular' and 'stroke' in the HRCS health categories were recategorized into 'cardiovascular diseases', three suggested topics under 'inflammatory and immune system (IIS)' and 'musculoskeletal' were recategorized into 'musculoskeletal', three suggested topics under 'IIS' and 'respiratory' were re-categorized into 'respiratoryd', and eight suggested topics of unspecified allergic disease under 'IIS' were excluded (Appendix 1).

2. Analytical methods

Descriptive analysis was used to compare disease categories under the HRCS of suggested topics and performed topics. To see the correlation between disease burden and disease was classified according to the new category, 422 suggested topics and 33 performed topics were excluded. To find correlation between disease burden and 1,112 suggested topics and 91 performed topics that were categorized according to the new disease categories, linear auxiliary trend line and scatter plot were made using DALYs, YLLs and YLDs, and Person's correlation coefficients were calculated.

Results

1. Comparison of suggested and performed topics by UK-CRC health categories (Table 1)

Table 1

Association of research topics and global burden of disease by UK Clinical Research Collaboration health categories

Among the suggested topics, cancer was most common (n=176, 11.60%), followed by cardiovascular diseases (n=148, 9.76%), musculoskeletal (n=120, 7.91%), oral and gastrointestinal disease (n=107, 7.05%). The area with the most topics selected as projects was cancer (n=23, 14.74%), followed by cardiovascular disease (n=15, 9.62%), mental health disease (n=11, 7.05%), and musculoskeletal disease (n=10, 6.41%). Beginning from the least, topics that were among the least suggested were injuries (n=20, 1.32%), blood/urogential diseases (n=32, 2.11%), and neurological disorders (n=43, 2.83%). Likewise, those that were among the least performed were, injuries (n=0, 0.00%), blood/urogenital diseases (n=1, 0.64%), and reproductive health and childbirth (n=1, 0.64%).

2. Correlation between research topic category and DALYs, YLLs, and YLDs (Table 1, Figure 1)

Figure 1

Scatter plots with suggested topics, performed topics, Korea disability-adjusted life years (DALYs; A,B), years of life lost (YLLs; C,D), and years lived with disability (YLDs; E,F).

Correlation between research topic category and DALYs was 0.558 (P=0.047) for suggested topics and 0.746 (P=0.003) for performed topics, indicating a higher correlation between disease burden and performed topics rather than suggested topics. The cancer with the highest DALYs was found in the suggested topics and in the performed topics most often. Cardiovascular diseases, which have the second-highest YLLs and the fourth-highest DALYs, were often suggested and performed only second to cancer. Examples of diseases with DALYs of upper ranks but which are not often suggested as topics include injuries. Whereas, those with a low ratio of suggested to performed topics were injuries and reproductive health and childbirth.

Correlation between research topic category and YLLs was 0.566 (P=0.044) for suggested topics and 0.665 (P=0.013) for performed topics, indicating a higher correlation between disease burden and performed topics rather than suggested topics. Cancer showed the highest proportions of YLLs and a high incidence of suggested and performed topics, whereas, injuries and oral and GI oral and gastrointestinal diseases showed high proportions of YLLs but a low incidence of suggested and performed topics.

The correlation between research topics and YLDs was 0.013 for suggested topics (P=0.967) and 0.124 for performed topics (P=0.686), not showing a statistical significance. Specifically, cancers with relatively lower YLDs were most frequently found in both suggested and performed topics. And musculoskeletal diseases, which had the highest YLDs, was found to be the third most frequently listed in the suggested topics and fourth most frequently, in the performed topics. Mental health which had the second to highest YLDs was found 8th on the list of suggested topics and 3rd on the performed topics. Thus, YLDs did not show a prominent association with suggested research topics.

Discussion

HTA studies often reflect the narrow interests of individual scholars and studies are sometimes initiated and supported by commercial sponsors, so it is necessary to ensure that HTA studies focus on major health problems [19]. To determine whether an institute with a specific purpose of establishment and scope of research has selected relevant research topics and allocated research funds properly, it would be necessary to analyze research associated with the institute so far. The significance of such analysis is that it studies whether or not national health needs are well reflected in suggested topics and the process of suggestion is efficient. This study was conducted to categorize suggested research topics and research topics performed by NECA in the past 6 years and to find correlation between these topics and disease burden index.

The results suggested that cancer was most common, followed by cardiovascular diseases, among suggested research topics and research topics performed by NECA and in terms of the ratio of performed to suggested topics as well. We found injuries, blood/urogenital diseases, and neurological disorders were less suggested than other disease and that injuries, blood/urogenital diseases, and reproductive health and childbirth were less performed than others. Cancer and cardiovascular disease, both a disease with a high DALYs and YLLs, were the most common diseases among both suggested topics and performed topics, suggesting that they are highly concerned diseases in terms of social policy in Korea.

Correlation between research topic and disease burden index indicated a strong correlation with DALYs and YLLs. But YLDs and research topic was not association.

When the suggested research topics were considered, we found that they showed a significant correlation with DALYs (0.558, P=0.047) and YLLs (0.566, P=0.044) but not with YLDs (0.013, P=0.967). We can deduce from this that, in general, the suggested research topics were selected with their range of burden of disease (DALYs) homogeneously considered. Also given that YLLs, which is the burden of disease resulting from mortalities such as death, is greater than YLDs, the burden of disease resulting from morbidities, we can also deduce that the public's interest lies with burden of disease resulting from the former than the latter.

When we assessed the correlation between research topics and disease burden with respect to all three markers of disease burden, we found that the performed topics showed a stronger correlation with disease burden than suggested topics, and statistically, that it showed a significant association with only DALYs and YLLs, which was similar to what was seen with the suggested topics. Suggested topics showed a greater correlation with YLLs than with DALYs, and performed topics showed a greater correlation with DALYs than with YLLs. Yet a statistically significant difference was not found even though both mortality and morbidity were appropriately considered with respect to selecting research topics.

Specifically, within the performed topics, we can interpret that the reason why the 5 diseases showed higher YLLs than YLDs (i.e., those with a greater burden from morbidity than mortality) is because only a few individuals suggested the topics relating to these diseases relative to the size of their disease burden. These 5 diseases are as follows: musculoskeletal, mental health, neurological disorders, other non-communicable disorders, and blood/urogenital diseases.

From our findings, we can conclude that the performed research topics are more indicative of the burden of disease carried by the Korean population than in the suggested topics. Interestingly, we can also infer that the public's interest lie more in diseases with a greater burden from mortality than morbidity. In addition, with regards to selecting the research topics, we found that a degree of alteration occurs during the process of selection as the disease burden is taken account, but we found that this does not make a statistically significant impact on the selection outcome. Because of this preference for diseases with mortality-induced burdens, we found that those with relatively less burdens from mortality but more from morbidity such as musculoskeletal or mental health diseases were not sufficiently studied. But since demographic changes in the populations is anticipated to increase the size of the burden of these diseases and the social impact brought about by these diseases, we suggest that these diseases should be equally considered during the topic suggestion and selection process.

Especially, we found that the diseases with high DALYs and YLDs but low YLLs such as mental health and musculoskeletal diseases were included in the suggested topics at a greater prevalence than in the performed topics, an observation that reflects the population's greater interest in diseases that cause mortalities. We suggest that this bias towards the former type of diseases be taken into consideration during allocation of research topics in prospective studies. Conversely, those diseases, such as Injuries, with DALYs, YLLs, and YLDs in the lower estimates that are neither suggested nor performed as topics should be investigated not through a topic-suggestion approach, as there is the potential that they will be overlooked, but through a top-down approach. For instance, the HTA appoints preeminent policy makers or public health individuals to select research topics that are related to public welfare or uses horizontal scanning as a means to include research topics in dire need of assessment. As such, a more diverse array of channels to select research topic should be considered by us. Using a standardized marker of disease burden, we suggest that a re-assessment of the selection process and of the selection standards is made to create an objective, consistent, and burden-adjusted approach that prevents diseases with high burdens to be over-suggested or over-performed in topics.

This study is significant in that it was an empirical study that explored correlation between disease burden and health technology assessment research, but there are several limitations in this study. First, being a study investigating correlation between disease burden and health technology assessment research in Korea, there were only 91 studies included in the analysis, which should be considered when interpreting the results.

Second, this study is significant in that it has investigated whether or not disease burden was well reflected in the course of topic suggestion and selection, but also has several limitations from using DALYs and mortality in GBD 2010 as indicators of disease burden. There is still a controversy as to whether actual health needs, such as difference in disability weights between countries, are well reflected in DALYs [10 20 21 22], which raises the possibility that the disease burden extracted from GBD might be different from the actual level of disease burden among Koreans.

Third, since the level of disease burden in a single year (2010) was used in this study, the disease burden in this study might not reflect the change in disease burden between 2009 and 2014 when the research topics were performed. Therefore, there is a limit in stating that resource allocation was properly done, even if the research topics were selected based on disease burden, which should be considered when interpreting the results.

NECA is an institution that promotes the responsible and effective use of medical findings robustly supported by scientific research and aims to enhance the health and welfare of the general population. To this end, NECA believes elucidating the research topics which patients and medical consumers value as important to be integral to carrying out scientific research. In addition, NECA is a research institution that stages the National Strategic Coordinating Center which aims to set up an evidence-based public health system to facilitate formation of solutions to public health questions by providing a resource bank of scientific data for policy makers, clinicians, and patients. In this regard, we have investigated the research topics that were suggested or performed at NECA in light of their disease burden. We believe that our findings may be used as the basis of future studies that investigate public health questions and novel approaches to selecting research topics. As the first Korean study to assess the correlation between research topics and disease burden, we anticipate that our results will be used as base data for prioritizing the allocation of healthcare resources in the future.

Conclusion

To find correlation between disease burden and 1,112 suggested topics and 91 performed topics that were calculated Person's correlation coefficients of the disease burden (DALYs, YLLs, and YLDs) and research topics. This study was aimed to investigate the relationship between the research topics suggested to NECA and the disease burden in Korea. The results suggested that cancer was most common, followed by cardiovascular diseases, among suggested research topics and research topics performed by NECA. Correlation between research topic and disease burden index indicated a strong correlation with DALYs and YLLs. But years lived with disability and research topic was not association. Suggested topics showed a greater correlation with YLLs than DALYs whereas performed topics showed a greater correlation with DALYs than YLLs. From our findings, we can conclude that the performed research topics are more indicative of the burden of disease carried by the Korean population than in the suggested topics. Interestingly, we can also infer that the public's interest lie more in diseases with a greater burden from mortality than morbidity. As the first Korean study to assess the correlation between research topics and disease burden, our results will be used as base data for prioritizing the allocation of healthcare resources in the future.

Acknowledgement

This research was supported by National Evidence-based Healthcare Collaborating Agency (NECA-P-14-002).

Appendices

Appendix 1

The new health disease category for the correlation analysis of the disease burden [10]

References

1. Kim CM. Health economics and outcomes research. Korean J Fam Med 2009;30:577–587.

2. Heo DS. Economic analysis of health technology and development of outcome research system Seoul: Seoul National University; 2008.

3. Yoon HS. Issues of health care decision-making process in protection-type expansion policy. KDI Policy Forum 2007;(178):2007–2006.

4. Gross CP, Anderson GF, Powe NR. The relation between funding by the National Institutes of Health and the burden of disease. N Engl J Med 1999;340:1881–1887.

5. Husereau D, Boucher M, Noorani H. Priority setting for health technology assessment at CADTH. Int J Technol Assess Health Care 2010;26:341–347.

6. Noorani HZ, Husereau DR, Boudreau R, Skidmore B. Priority setting for health technology assessments: a systematic review of current practical approaches. Int J Technol Assess Health Care 2007;23:310–315.

7. Terry RF, van der Rijt T. Overview of research activities associated with the World Health Organization: results of a survey covering 2006/07. Health Res Policy Syst 2010;8:25.

8. UK Clinical Research Collaboration. UK health research analysis 2004/05 London: UK Clinical Research Collaboration; 2006.

9. UK Clinical Research Collaboration. UK health research analysis 2009/10 London: UK Clinical Research Collaboration; 2012.

10. Kinge JM, Roxrud I, Vollset SE, Skirbekk V, Rottingen JA. Are the Norwegian health research investments in line with the disease burden? Health Res Policy Syst 2014;12:64.

11. Carter AJ, Nguyen CN. A comparison of cancer burden and research spending reveals discrepancies in the distribution of research funding. BMC Public Health 2012;12:526.

12. Stuckler D, King L, Robinson H, McKee M. WHO's budgetary allocations and burden of disease: a comparative analysis. Lancet 2008;372:1563–1569.

13. Gillum LA, Gouveia C, Dorsey ER, Pletcher M, Mathers CD, McCulloch CE, Johnston SC. NIH disease funding levels and burden of disease. PLoS One 2011;6:e16837.

14. Nielsen CP, Funch TM, Kristensen FB. Health technology assessment: research trends and future priorities in Europe. J Health Serv Res Policy 2011;16Suppl 2. :6–15.

15. European Science Foundation. Health research classification systems: current approaches and future recommendations [Internet] Strasbourg: European Science Foundation; 2011. cited 2016 Jun 22. Available from: http://www.esf.org/fileadmin/Public_documents/Publications/spb43_Health_Research_Classification.pdf.

16. Institute of Health Metrics and Evaluation. GBD 2010 data [Internet] Seattle: Institute of Health Metrics and Evaluation; cited 2016 Jun 22. Available from: http://www.healthdata.org/gbd/data.

17. Murray CJ. Quantifying the burden of disease: the technical basis for disability-adjusted life years. Bull World Health Organ 1994;72:429–445.

18. Fenwick A. The global burden of neglected tropical diseases. Public Health 2012;126:233–236.

19. Teerawattananon Y, Russell S, Mugford M. A systematic review of economic evaluation literature in Thailand: are the data good enough to be used by policy-makers? Pharmacoeconomics 2007;25:467–479.

20. Nord E. Disability weights in the global burden of disease 2010: unclear meaning and overstatement of international agreement. Health Policy 2013;111:99–104.

21. Ustun TB, Rehm J, Chatterji S, Saxena S, Trotter R, Room R, Bickenbach J. Multiple-informant ranking of the disabling effects of different health conditions in 14 countries: WHO/NIH Joint Project CAR Study Group. Lancet 1999;354:111–115.

22. Salomon JA, Vos T, Hogan DR, Gagnon M, Naghavi M, Mokdad A, Begum N, Shah R, Karyana M, Kosen S, Farje MR, Moncada G, Dutta A, Sazawal S, Dyer A, Seiler J, Aboyans V, Baker L, Baxter A, Benjamin EJ, Bhalla K, Bin Abdulhak A, Blyth F, Bourne R, Braithwaite T, Brooks P, Brugha TS, Bryan-Hancock C, Buchbinder R, Burney P, Calabria B, Chen H, Chugh SS, Cooley R, Criqui MH, Cross M, Dabhadkar KC, Dahodwala N, Davis A, Degenhardt L, Diaz-Torne C, Dorsey ER, Driscoll T, Edmond K, Elbaz A, Ezzati M, Feigin V, Ferri CP, Flaxman AD, Flood L, Fransen M, Fuse K, Gabbe BJ, Gillum RF, Haagsma J, Harrison JE, Havmoeller R, Hay RJ, Hel-Baqui A, Hoek HW, Hoffman H, Hogeland E, Hoy D, Jarvis D, Karthikeyan G, Knowlton LM, Lathlean T, Leasher JL, Lim SS, Lipshultz SE, Lopez AD, Lozano R, Lyons R, Malekzadeh R, Marcenes W, March L, Margolis DJ, McGill N, McGrath J, Mensah GA, Meyer AC, Michaud C, Moran A, Mori R, Murdoch ME, Naldi L, Newton CR, Norman R, Omer SB, Osborne R, Pearce N, Perez-Ruiz F, Perico N, Pesudovs K, Phillips D, Pourmalek F, Prince M, Rehm JT, Remuzzi G, Richardson K, Room R, Saha S, Sampson U, Sanchez-Riera L, Segui-Gomez M, Shahraz S, Shibuya K, Singh D, Sliwa K, Smith E, Soerjomataram I, Steiner T, Stolk WA, Stovner LJ, Sudfeld C, Taylor HR, Tleyjeh IM, van der Werf MJ, Watson WL, Weatherall DJ, Weintraub R, Weisskopf MG, Whiteford H, Wilkinson JD, Woolf AD, Zheng ZJ, Murray CJ, Jonas JB. Common values in assessing health outcomes from disease and injury: disability weights measurement study for the Global Burden of Disease Study 2010. Lancet 2012;380:2129–2143.

Article information Continued

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Funded by : National Evidence-based Healthcare Collaborating Agencyhttp://dx.doi.org/10.13039/501100003670

Award ID : NECA-P-14-002