- Research article
- Open Access
Factors related to the use of opioids as early treatment in patients with knee osteoarthritis
Arthritis Research & Therapy volume 21, Article number: 222 (2019)
To examine factors related to the use of opioids as an early treatment option for knee OA patients
Using the Korean nationwide claim database, we selected knee OA patients between 2013 and 2015. Among them, patients without any claim of knee OA for 2 years before the index date were included as our study population. We analyzed their first claim for prescriptions, including tramadol and stronger opioids, at the index date of each patient. Using a multinomial model, we identified factors associated with the early use of tramadol and stronger opioids in knee OA patients.
Among a total of 2,857,999 knee OA patients, 12.2% (n = 348,516) were treated with opioids as their first treatment. However, the prevalence of stronger opioid use was only 0.07% (n = 1972). Male sex (OR 1.28 in tramadol, OR 1.13 in stronger opioids) and comorbidities with depression (OR 1.05, 1.46), low back pain (OR 1.13, 1.30), intervertebral disc disorder (OR 1.11, 1.40), and spinal stenosis (OR 1.27, 1.55) were the factors for the early use of tramadol or stronger opioids in knee OA patients. Patients in a tertiary referral hospital tended to use tramadol or stronger opioids than those in clinics (OR 1.04, 56.63, respectively).
In Korea, 12.2% of knee OA patients were treated with opioids as an early treatment, and tramadol was used more commonly than stronger opioids. Male sex and having comorbidities such as depression or musculoskeletal disease are patient factors associated with the early use of opioids in knee OA patients.
Osteoarthritis (OA) is a highly prevalent, chronic condition affecting nearly 25% of adults aged 55 years and older in the United States (US) , and its prevalence continues to rise . Knee OA is one of the most frequent diseases around the world, with more than 250 million sufferers . In Korea, a recent report described the number of knee OA patients aged more than 50 years in males and females as 21.1% and 43.8%, respectively .
Current guidelines for knee OA treatment recommend a range of non-pharmacological and pharmacological interventions to alleviate pain and improve function and quality of life. Most guidelines do not recommend opioids for knee OA treatment as an early treatment option [5, 6], but some guidelines suggest using opioids on a restricted basis for short-term use in patients with refractory symptoms . However, a recent Cochrane systematic review of 22 randomized controlled trials showed that opioids yielded a small clinical benefit, but with a high risk of side effects, including opioid dependence in patients with hip and knee OA . Furthermore, a recent study suggested that preoperative opioid use in knee OA patients results in less pain relief from knee joint surgery .
Over the past two decades, the increasing use of opioids for chronic musculoskeletal pain in the US  and Europe  has been a serious public health problem. A study using the Medicare Current Beneficiary Survey in the US showed that there was a significant increase in opioid prescribing between 2003 and 2009, resulting in 40% of knee OA patients receiving an opioid in 2009 . However, a more recent study from 2007 to 2014 showed that 15.9% of patients with knee OA were prescribed opioids for their condition and that yearly rates of prescription were fairly stable over this period . Factors related to opioid use have previously been suggested as age, sex, functional limitation, poor self-reported health status, comorbidities, and healthcare environment [12,13,14,15,16,17,18]. However, several previous studies examined mixed populations, such as patients with any site of OA or patients with non-cancer pain, and obtained inconsistent results for several factors. In addition, a recent study in the US revealed substantial statewide variation in the prescription of opioids in OA patients .
Until now, the prevalence of opioid use as an early treatment option for knee OA has not been much studied. Early opioid use to alleviate pain in knee OA may be related to a long-term use of opioids, because knee OA can cause persistent and chronic pain. Therefore, we conducted a retrospective observational study using Korean national claim data to investigate how many opioid prescriptions were given as early treatment for knee OA and to examine factors related to opioid use for knee OA patients. We hypothesized that knee OA patient’s demographics and comorbidities would be related to early opioid use.
The Health Insurance Review and Assessment (HIRA) database of Korea, which covers 50 million people, or virtually the whole population of Korea, is a national claim database. The Korean national health insurance system offers universal access for most residents to healthcare through the National Health Insurance (NHI) system and through the Medical Aid program for lower income groups . This database contains individual beneficiary information, as well as healthcare service information, including diagnoses, procedures, prescriptions, and tests. We used the full dataset of HIRA between the years 2011 to 2015.
Population and study design
Identification of knee OA patients
Knee OA patients between 2013 and 2015 were identified using the operational definition reported in our previous study . Among them, patients with inflammatory arthritis codes such as rheumatoid arthritis, ankylosing spondylitis, and psoriatic arthritis were excluded. We then identified the date of the first claim for knee OA for each patient between 2013 and 2015 and defined it as the index date. We further selected patients without any claim for knee OA for 2 years before the index date (Fig. 1).
Definition of opioids and early opioid users
To identify opioids used for early treatment of knee OA, we determined opioid use at the index date and classified patients according to tramadol users and stronger opioid users (hydromorphone, morphine, hydrocodone, oxycodone, pethidine, fentanyl, sufentanil, remifentanil, pentazocine, buprenorphine, butorphanol, nalbuphine, tapentadol, codeine, and dihydrocodeine) (Fig. 1).
Factors for the early use of opioids
Demographic characteristics such as age, sex, and type of institution or department related to the opioid prescription were determined at the index date. Medications combined with opioids were also investigated at the index date. Comorbidities were detected based on the Elixhauser comorbidity index , which was calculated using ICD 10 codes within 1 year preceding the knee OA index date, because the existing comorbidities could be considered when a physician makes a prescription for opioids in knee OA patients (Additional file 1: Figure S1). In addition, common musculoskeletal diseases related to chronic pain such as low back pain, intervertebral disc disorder, spinal stenosis, and fibromyalgia were also determined based on ICD 10 codes. We performed a sensitivity analysis involving patients without malignancies, because the opioid use to control malignancy pain can influence opioid use for knee OA. In addition, we performed a subgroup analysis of opioid use for 1 year by regular users, defined as patients with medication possession ratios (MPRs) ≥ 75%.
We assessed baseline characteristics of three groups: opioid non-users, tramadol users, or stronger opioid users. Categorical variables are presented as the frequency with percentage, and continuous variables are presented as mean with standard deviation (SD). A multinomial model was constructed to examine factors related to tramadol users or stronger opioid users compared to opioid non-users. All demographic factors and comorbidities that were significant in the crude model, as prespecified in the previous paragraph, were included in the multinomial model. All factors were controlled in the same model. All analyses were performed using SAS 9.2 (SAS Institute, Cary, NC).
This study was determined to be exempt from IRB review by our university hospital IRB (IRB file No. HYUN 2016-06-008-002) because we used the existing, publicly available data, and subject information could not be identified directly through identifiers linked to the subjects.
Prevalence of opioid users in knee OA patients and their characteristics
Among all knee OA patients, 348,516 patients (12.2%) were treated with opioids. Most (n = 346,544, 12.1%) were tramadol users and only 0.07% (n = 1972) were stronger opioid users. Opioids for knee OA patients were prescribed mainly by orthopedic surgeons (70.1%) followed by internists (9.7%), including rheumatologists.
When we compared patient characteristics according to the early use of opioids, opioid users were slightly older (64.2 vs. 64.0 years old, p < 0.01) and more likely to be male (44.0% vs. 37.8%, p < 0.01) than opioid non-users. The Elixhauser comorbidity index score was higher in opioid users than opioid non-users (4.51 ± 6.36 vs. 4.43 ± 6.23, p < 0.01). Differences in comorbidities and concomitant medications between the two groups are presented in Table 1.
Factors for early use of opioids in knee OA patients
After classifying opioid users as tramadol or stronger opioid users, we generated a multinomial model that compared the two groups with opioid non-users (Table 2). Several similar factors were associated with early tramadol and stronger opioid use, including male sex (OR 1.28 for tramadol, OR 1.13 for stronger opioids), tertiary hospital (OR 1.04 for tramadol, OR 56.63 for stronger opioids), and depression (OR 1.05 for tramadol, OR 1.46 for stronger opioids), low back pain (OR 1.13 for tramadol, OR 1.30 for stronger opioids), intervertebral disc disorder (OR 1.11 for tramadol, OR 1.40 for stronger opioids), and spinal stenosis (OR 1.27 for tramadol, OR 1.55 for stronger opioids). Patients supported by national insurance (OR 0.94 for tramadol, OR 0.73 for stronger opioids) and rehabilitation visits compared to internal medicine (OR 0.81 for tramadol, OR 0.30 for stronger opioids) were associated with lower opioid use. However, the most recent year of use (2015) was associated with less tramadol use (OR 0.98, 95% CI 0.97–0.99) and more use of stronger opioids (OR 1.40 in 2014, OR 1.14 in 2015). This finding showed the possibility of an increasing tendency to use stronger opioids over tramadol. Patients in their 60s were less likely to receive tramadol than those in their 50s (OR 0.97, 95% CI 0.96–0.98), but those over 70 years old were more likely to take stronger opioids compared to those in their 50s (OR 1.45, 95% CI 1.29–1.63). In terms of comorbidities, myocardial infarction (OR 0.96, 95% CI 0.93–0.99), peptic ulcer disease (PUP) (OR 1.04, 95% CI 1.03–1.05), and fibromyalgia (OR 1.13, 95% CI 1.10–1.16) were the factors associated only with tramadol and not stronger opioids. Diabetes (OR 0.97, 95% CI 0.96–0.98) and malignancy (OR 0.87, 95% CI 0.85–0.88) were associated with less tramadol, but stronger opioids (diabetes [OR 1.21, 95% CI 1.10–1.33] and malignancy [OR 2.91, 95% CI 2.60–3.25]). Dementia (OR 1.24, 95% CI 1.07–1.43) and alcohol abuse (OR 1.39, 95% CI 1.10–1.75) were significantly more associated with stronger opioids. The outcome of the sensitivity analysis in patients without malignancies was similar to that of the main analysis (Additional file 1: Table S2). In the subgroup analysis, we evaluated factors related to regular opioid use MPR of ≥ 75% for knee OA: those in the 60s and 70s were less likely to be regular opioid users than those in the 50s (OR 0.82, 95% CI 0.80–0.83 for the 60s, OR 0.76, 95% CI 0.74–0.77 for the over 70s), and patients with musculoskeletal disease were less likely to use opioids regularly, while patients supported by the national insurance were more likely to use opioids regularly than those receiving medical aid or veterans (OR 1.16, 95% CI 1.13–1.19) (Additional file 1: Table S3).
In Korea, 12.2% of knee OA patients were treated with opioids as an early treatment option, and tramadol was more commonly used than stronger opioids. Male sex as well as having comorbidities such as depression or musculoskeletal diseases was a patient factor associated with the early use of opioids in knee OA patients. In the most recent year of use (2015), patients were less likely to take tramadol and more likely to take stronger opioids. Compared with age in the 50s, age in the 60s was associated with less tramadol use, while age over 70 years was more associated with stronger opioid use.
The opioid use in Korean patients with knee OA was less than the overall opioid use in other Western countries. The prevalence of opioid use was 40% in the US patients with knee OA  and nearly a third of Canadian pre-surgical patients with end-stage knee OA . Opioid use as an early treatment in Korea was similar to that of 12% in Swedish patients with knee or hip OA, but the type of opioids differed from those in Sweden. In Sweden, the most commonly used opioids are stronger, amounting to 17% of OA patients overall, followed by weak opioids, amounting to 9% of OA patients . It is difficult to draw a direct comparison between the results from different countries, because drug utilization may vary depending on the severity of knee OA and the time frame of the investigation. However, it is obvious that the use of stronger opioids was low in Korean patients with knee OA. The low frequency of stronger opioids in Korea may be related to physician reluctance to prescribe stronger opioids for knee OA patients or high accessibility of medical care in Korea. On the other hand, tramadol can be suggested for patients who do not respond to other medications or cannot take other medication, according to a clinical practice guideline for the treatment of knee OA approved by the Korean Knee Society . In addition, formulations of tramadol with acetaminophen and its various generic drugs may have also contributed to the observed high frequency of tramadol. However, this speculation needs further investigation.
Previous studies about the association of age and sex with opioid use show inconsistent results. One study in the US found that older age and female sex were associated with opioid use , while another US study found an increased use among younger patients and women . Meanwhile, a Canadian survey in older patients with hip and knee OA observed no association between opioid use and age or sex . These differences are likely attributable to the differential perception of the safety of opioid treatment in different geographic regions. In our study, males were more likely to be treated with opioids, patients in their 60s were less likely to use tramadol, and patients in their 70s were more likely to use stronger opioids compared to patients in their 50s. The frequent use of stronger opioids in elderly males more than 70 years old and the increase in stronger opioid use depending on periods of use are noteworthy. Elderly males may be more likely to experience musculoskeletal diseases or alcohol abuse, and these combined diseases may result in association with stronger opioids. However, careful monitoring should be considered during the opioid use in elderly patients, because of greater exposure to cognitive impairment and fall injuries when they are on opioids. In addition, for men, opioid-related hypogonadism was associated with lower bone mineral density and adverse cardiovascular outcomes .
In addition, patients supported by national insurance, compared with medical aid or veterans, were less likely to use opioids. This finding was consistent with previous studies showing that low economic status is associated with opioid use [25, 26]. Treatment at tertiary hospitals was also a factor related to opioid use, possibly implying that patients who visit a tertiary hospital suffer from more intractable stages of knee OA compared to patients who visit hospitals/clinics. The department of rehabilitation was less likely to prescribe tramadol or stronger opioids, and orthopedics was less likely to prescribe stronger opioids. These differences might be explained by patient preference: patients who want to find non-pharmacological treatment options would more likely visit the department of rehabilitation or orthopedics, rather than the department of rheumatology.
The observed increased risk of opioid use for patients with comorbidities of musculoskeletal disease , COPD , and depression [12, 14, 17, 27] was consistent with previous studies. Alcohol abuse and dementia were also significant factors associated with the use of stronger opioids. It is unclear whether patients with such mental comorbidities have a lower pain threshold and diminished responsiveness to opioids [28, 29] or perhaps lower patient empowerment. However, a previous study showed that mental health conditions, substance dependence and abuse, and preexisting pain disorders increase the risk for prolonged opioid use among opioid-naïve patients . Therefore, physicians need to prescribe opioids carefully in patients with mental comorbidities or preexisting pain disorders, because the risk of long-term use can increase in such patients.
The strength of the present study was that it was a large population-based study using a Korean nationwide database that covers all drugs prescribed to the overall population. The Korean insurance system is a single-payer, universal, and compulsory healthcare system. The reimbursement standard of the national Health Insurance Review and Assessment Service applies to all health providers. This use of comprehensive and longitudinal data avoided selection bias and included all opioid use in knee OA patients. In addition, we selected early treatment for knee OA based on a validated definition. Therefore, this study presented information about early opioid treatment in knee OA patients that has been rarely presented until now.
There were several limitations to this population-based study. First, we did not score the disease stage, pain-intensity, or functional disability due to limitations of the data source. In the opioid users, the prevalence of NSAIDs and SYSADOA prescriptions was higher than in the opioid non-users. Opioid non-users may not use medication for pain because their pain is mild, or they may use oriental medicine or alternative treatments that are not covered by our national insurance. Studies are needed using appropriate data sources to evaluate the use of health functional foods and acupuncture by knee OA patients that do not take pain medication. In addition, we were unable to determine the reason for opioid use. Therefore, we considered several chronic pain-related conditions to evaluate factors related to opioid use. Second, we did not evaluate the safety or outcomes of each opioid, particularly tramadol. Although tramadol is a weak opioid, further study is needed to evaluate long-term safety and outcomes. Third, we could not examine incident knee OA patients, because it was difficult to identify early-stage knee OA patients in the claim database since non-symptomatic early-stage knee OA patients might not need medication. Instead of trying to extract incident patients with knee OA, we restricted our cohort to knee OA patients who had not accessed health services over the 2 previous years to identify patients without prior treatment for knee OA and for whom opioids were an early treatment option. However, some patients with advanced knee OA who had no prior treatment or had received alternative treatments may have been included. Therefore, it is possible that we overestimated the prevalence of the early use of opioids. Fourth, even though we performed a subgroup analysis of patients with regular opioid use, defined by an MPR ≥ 75%, we could not evaluate treatment patterns after the index date, which may vary. Future work on the prescription trajectories of opioids will be needed to determine the actual regular use of opioids. Lastly, factors with odds ratios close to 1 in our study using a large population database should be interpreted carefully.
There is currently a lack of consensus from professional medical organizations around the appropriate use of opioid medications in OA. Our findings about current patterns of opioid use as an early treatment option for knee OA and related factors provide useful information for making appropriate recommendations.
In Korea, 12.2% of knee OA patients were treated with opioids as an early treatment option, and tramadol was more commonly used than stronger opioids. Male sex as well as having comorbidities such as depression or musculoskeletal diseases was a patient factor associated with the early use of opioids in knee OA patients.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Health Insurance Review and Assessment
Peat G, McCarney R, Croft P. Knee pain and osteoarthritis in older adults: a review of community burden and current use of primary health care. Ann Rheum Dis. 2001;60(2):91–7.
Nguyen US, Zhang Y, Zhu Y, Niu J, Zhang B, Felson DT. Increasing prevalence of knee pain and symptomatic knee osteoarthritis: survey and cohort data. Ann Intern Med. 2011;155(11):725–32.
Vos T, Flaxman AD, Naghavi M, Lozano R, Michaud C, Ezzati M, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380(9859):2163–96.
Lee S, Kim SJ. Prevalence of knee osteoarthritis, risk factors, and quality of life: the Fifth Korean National Health and Nutrition Examination Survey. Int J Rheum Dis. 2017;20(7):809–17.
McAlindon TE, Bannuru RR, Sullivan MC, Arden NK, Berenbaum F, Bierma-Zeinstra SM, et al. OARSI guidelines for the non-surgical management of knee osteoarthritis. Osteoarthr Cartil. 2014;22(3):363–88.
Jevsevar DS. Treatment of osteoarthritis of the knee: evidence-based guideline, 2nd edition. J Am Acad Orthop Surg. 2013;21(9):571–6.
Hochberg MC, Altman RD, April KT, Benkhalti M, Guyatt G, McGowan J, et al. American College of Rheumatology 2012 recommendations for the use of nonpharmacologic and pharmacologic therapies in osteoarthritis of the hand, hip, and knee. Arthritis Care Res (Hoboken). 2012;64(4):465–74.
Da Costa B, Nüesch E, Kasteler R, Husni E, Welch V, Rutjes A, et al. Oral or transdermal opioids for osteoarthritis of the knee or hip. Cochrane Database Syst Rev. 2014;9(9):CD003115.
Smith SR, Bido J, Collins JE, Yang H, Katz JN, Losina E. Impact of preoperative opioid use on total knee arthroplasty outcomes. J Bone Joint Surg Am. 2017;99(10):803–8.
Larochelle MR, Zhang F, Ross-Degnan D, Wharam JF. Trends in opioid prescribing and co-prescribing of sedative hypnotics for acute and chronic musculoskeletal pain: 2001–2010. Pharmacoepidemiol Drug Saf. 2015;24(8):885–92.
Birke H, Kurita G, Sjøgren P, Højsted J, Simonsen M, Juel K, et al. Chronic non-cancer pain and the epidemic prescription of opioids in the Danish population: trends from 2000 to 2013. Acta Anaesthesiol Scand. 2016;60(5):623–33.
Wright EA, Katz JN, Abrams S, Solomon DH, Losina E. Trends in prescription of opioids from 2003–2009 in persons with knee osteoarthritis. Arthritis Care Res (Hoboken). 2014;66(10):1489–95.
DeMik DE, Bedard NA, Dowdle SB, Burnett RA, McHugh MA, Callaghan JJ. Are we still prescribing opioids for osteoarthritis? J Arthroplasty. 2017;32(12):3578–82 e1.
Braden JB, Sullivan MD, Ray GT, Saunders K, Merrill J, Silverberg MJ, et al. Trends in long-term opioid therapy for noncancer pain among persons with a history of depression. Gen Hosp Psychiatry. 2009;31(6):564–70.
Campbell CI, Weisner C, LeResche L, Ray GT, Saunders K, Sullivan MD, et al. Age and gender trends in long-term opioid analgesic use for noncancer pain. Am J Public Health. 2010;100(12):2541–7.
Fisher JE, Ballantyne PJ, Hawker GA. Older adults living with osteoarthritis: examining the relationship of age and gender to medicine use. Can J Aging. 2012;31(3):323–33.
Power JD, Perruccio AV, Gandhi R, Veillette C, Davey JR, Lewis SJ, et al. Factors associated with opioid use in pre-surgical knee, hip and spine osteoarthritis patients. Arthritis Care Res (Hoboken). 2019;71(9):1178–85.
Thorlund JB, Turkiewicz A, Prieto-Alhambra D, Englund M. Opioid use in knee or hip osteoarthritis: a region-wide population-based cohort study. Osteoarthr Cartil. 2019;27(6):871–77.
Desai RJ, Jin Y, Franklin PD, Lee YC, Bateman BT, Lii J, et al. Association of geography and access to healthcare providers with long term prescription opioid use in Medicare patients with severe osteoarthritis: a cohort study. Arthritis Rheumatol. 2019;71(5):712–21.
Kim JA, Yoon S, Kim LY, Kim DS. Towards actualizing the value potential of Korea Health Insurance Review and Assessment (HIRA) data as a resource for health research: strengths, limitations, applications, and strategies for optimal use of HIRA data. J Korean Med Sci. 2017;32(5):718–28.
Park HR, Im S, Kim H, Jung SY, Kim D, Jang EJ, et al. Validation of algorithms to identify knee osteoarthritis patients in the claims database. Int J Rheum Dis. 2019;22(5):890–96.
Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8–27.
Guidelines KKSSoO. Guidelines for the treatment of osteoarthritis of the knee. Knee Surg Relat Res 2010;22(1):69–74.
Duarte RV, Raphael JH, Southall JL, Labib MH, Whallett AJ, Ashford RL. Hypogonadism and low bone mineral density in patients on long-term intrathecal opioid delivery therapy. BMJ Open. 2013;20;3(6):e002856.
Svendsen K, Fredheim OM, Romundstad P, Borchgrevink PC, Skurtveit S. Persistent opioid use and socio-economic factors: a population-based study in Norway. Acta Anaesthesiol Scand. 2014;58(4):437–45.
Friedman J, Kim D, Schneberk T, Bourgois P, Shin M, Celious A, et al. Assessment of racial/ethnic and income disparities in the prescription of opioids and other controlled medications in California. JAMA Intern Med. 2019;179(4):469–76.
Vina ER, Hausmann LRM, Obrosky DS, Youk A, Ibrahim SA, Weiner DK, et al. Social & psychological factors associated with oral analgesic use in knee osteoarthritis management. Osteoarthritis Cartilage. 2019.
Wasan AD, Davar G, Jamison R. The association between negative affect and opioid analgesia in patients with discogenic low back pain. Pain. 2005;117(3):450–61.
Rhudy JL, Dubbert PM, Parker JD, Burke RS, Williams AE. Affective modulation of pain in substance-dependent veterans. Pain Med. 2006;7(6):483–500.
Gil JA, Gunaseelan V, DeFroda SF, Brummett CM, Bedi A, Waljee JF. Risk of prolonged opioid use among opioid-naive patients after common shoulder arthroscopy procedures. Am J Sports Med. 2019;47(5):1043–50.
Ethics approval and consent to participate
This study was approved by the Institutional Review Board (IRB) of Hanyang University Medical Center and was exempt from an IRB review because we used existing, publicly available data, and the information of the subjects could not be identified directly or through identifiers linked to the subjects (IRB file No. HYUN 2016-06-008-002). Informed consent was waived because we utilized a de-identified database which was opened to the public.
Consent for publication
YKS has received research grants from Bristol-Myers Squibb, Eisai, Pfizer, and JW Pharmaceutical. The other authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Additional file 1: Figure S1. Study design. Table S1. Factors for early opioid use compared with opioid non-use in knee OA patients, Table S2. Factors associated with early opioid use compared with opioid non-use in knee OA patients without malignancies*, Table S3. Factors associated with regular opioid use (MPR ≥75%) compared with opioid non-use in knee OA patients*.
About this article
Cite this article
Cho, S., Jung, S., Choi, S. et al. Factors related to the use of opioids as early treatment in patients with knee osteoarthritis. Arthritis Res Ther 21, 222 (2019). https://doi.org/10.1186/s13075-019-2004-x