Veterans Affairs databases are accurate for gout-related health care utilization: a validation study
© Singh; licensee BioMed Central Ltd. 2013
Received: 5 August 2013
Accepted: 12 December 2013
Published: 31 December 2013
The aim of this study was to assess the accuracy of Veterans Affairs (VA) databases for gout-related health care utilization.
This retrospective study utilized VA administrative and clinical databases. A random sample of gout patients with visits (outpatient, inpatient or emergent/urgent care) with or without the diagnosis of gout (International Classification of Diseases, ninth revision, common modification ICD-9-CM code of 274.x or 274.xx) at the Birmingham VA hospital was selected. A blinded abstractor performed a review of VA electronic health records for the documentation of gout or gout-related terms (gouty arthritis, tophaceous gout, tophus/tophi, acute gout, chronic gout, podagra, urate stones, urate or uric acid crystals and so on) in the chief complaint, history of present illness or assessment and plan for the visit; this constituted the gold standard for gout-related utilization. The accuracy of database-derived gout-related claims was assessed by calculating sensitivity, specificity, and positive and negative predictive values (PPV and NPV).
Of 108 potential visits, 85 outpatient, inpatient or urgent care/emergency room visits to a health care provider (85 patients: 84 men and 1 woman with a mean age of 63 years) and retrievable data from medical records constituted the analyzed dataset. Administrative claims for gout-related utilization with ICD-9 code for gout were accurate with a PPV of 86%, specificity of 95%, sensitivity of 86% and NPV of 95%.
VA databases are accurate for gout-related visits. These findings support their use for studies of health services and outcome studies. It remains to be seen if these findings are generalizable to other settings and databases.
Gout is the most common inflammatory arthritis in adults . It has a significant effect on patients’ quality of life, productivity, health care costs and utilization [2–4]. In the era of comparative effectiveness research (CER) , comparisons of various treatments with regard to health care utilization and costs are very important from patient, provider and policy maker perspectives. As new and more expensive gout treatments become available and the cost of gout treatment increases, a critical question is whether the higher costs of treatment are counterbalanced by more benefits to the patient, in terms of reduced morbidity and reduced gout-related health care utilization. Thus, it is important to assess if large administrative databases can provide valid health care utilization and outcomes data for CER in gout.
Two studies assessed the accuracy of administrative claims/codes for gout [6, 7]. A study in a managed care plan found that an International Classification of Disease ninth revision, common modification (ICD-9-CM) code for gout had a positive predictive value of 61% compared to rheumatologist adjudication of medical record abstractions . The second study found that the concordance of ICD-9-CM code from databases with gout classification criteria was low at 30% to 36% . The wide variation of accuracy of administrative claims/codes may be attributable to differences in patient populations and the gold standards between these studies, and raises the question whether the use of administrative databases is a valid approach for health services research in gout.
It is critical to establish the accuracy of administrative database definitions for a diagnosis of gout (including ICD-9-CM codes) and for gout-related utilization before we can rely on the findings from health services and outcomes studies in gout. Several studies of gout-related health care utilization from large databases have been published recently [9–11]. In contrast, to the best of our knowledge there are no published validation studies assessing the accuracy of gout-related health care utilization. Thus, it is unknown whether the administrative-derived data for gout-related visits are accurate and specific for gout-related utilization. The Veterans Affairs (VA) is the largest integrated health care system in the US. It serves more than five million veterans annually . The objective of this study was to assess the accuracy of gout-related visits in the VA administrative and claims databases.
For this validation study, a retrospective cohort of veterans with a known prior diagnosis of gout with any health care encounter at Birmingham VA Medical Center in the fiscal year 2006 (October 2005 to September 2006) was selected. Fiscal year 2006 was chosen for this study for the following reasons: (1) it allowed an adequate follow-up duration for the included patients; (2) there were no ongoing or new quality initiatives in gout or other arthritic conditions at Birmingham VA medical center; (3) no new treatment guidelines were published; and (4) no new gout drugs were launched. A data programmer created a dataset of two samples that were mixed randomly: VA health care visits with or without gout (based on the presence of ICD-9-CM code of 274.x or 274.xx) as the primary or secondary diagnosis for the visit other diseases included hyperuricemia without gout. The Institutional Review Board at Birmingham VA Medical Center approved the study and waived the requirement for patient consent for this retrospective database study.
The random sample was drawn from the administrative data available from the Austin Automation Center, a national VA data warehouse for the Birmingham VA cohort. These national computerized datasets have detailed data on VA health care provided to enrolled veterans, allowing the assessment of demographics and health care utilization. Data included the date of the visit, type of visit (outpatient, inpatient and emergent/urgent care), primary diagnosis (the main reason for health care visit), up to 15 secondary diagnoses and patient demographics including patient age and gender.
A standardized data extraction form was used to abstract data for each visit identified from the administrative databases from the computerized medical records of the patients. Data were abstracted related to the presence or absence of gout or gout-related complaints in the various parts of the clinical note (chief complaint, history of present illness and assessment and plan), gout-specific medications from the pharmacy medication list in the note (allopurinol, probenecid, colchicine) and results of imaging studies, including X-rays, and laboratory tests (serum uric acid, crystal analyses in synovial fluid or tophus).
Definitions of gold standard and test standard
The gold standard for this study was whether the visit was gout-related as determined by the review of the medical records. Gout-related visit was defined a priori as the mention of gout or gout-related terms (gouty arthritis, tophaceous gout, tophus/tophi, acute gout, chronic gout, podagra, urate stones, urate or uric acid crystals) in the chief complaint, history of the present illness or assessment and plan for the index visit in the medical records. This indicated that gout was the main reason or one of the main reasons for the index visit. The test standard was the presence of an ICD-9-CM code for gout (274.x or 274.xx) in VA inpatient or outpatient databases for the index patient visit. The author, a senior epidemiologist, experienced in data abstraction [13, 14] and blinded to the test standard (that is, gout diagnosis for the visit from VA databases), abstracted the data from the VA electronic health records. He considered limiting the study cohort to only those gout patients who have demonstration of urate crystals in joint/bursa fluid or mass, but decided to look for all patients with gout, given that <5% of patients with gout have crystal-proven gout . When a visit listed in the databases was missing from the medical records, the abstractor reviewed notes in the immediate period (before and after) to assess if the visit date was miscoded.
The administrative data definition of gout-related visit, that is, gout as the primary or secondary diagnosis in outpatient, inpatient or emergent/urgent care setting in the VA administrative database (test standard), was compared to the gold standard of medical record documentation that the patient’s visit was related to gout. Sensitivity, specificity, positive and negative predictive value (PPV and NPV) and kappa statistic for administrative data were calculated, in a similar manner as that previously reported for rheumatoid arthritis and spondylarthritides [14, 15]. Sensitivity was the fraction of those with a gout-related visit according to the gold standard that were correctly identified as positive by the VA database definition. Specificity was the fraction of visits not for gout according to the gold standard that were correctly identified as negative by the database definition. PPV was the proportion of those with gout-related visits in the VA database who met the gold standard definition of medical chart documentation. NPV was the proportion of visits not for gout in the VA database that did not meet the gold standard definition. The kappa coefficient was used to describe agreement (beyond chance) between the medical record documentation of gout (gold standard) and VA database documentation (test standard).
The Institutional Review Board (IRB) at the Birmingham VA Medical Center approved this study. The need for patient consent was waived by the IRB, since this was a retrospective database study. All investigations were conducted in conformity with ethical principles of research.
Of the 108 potential visits of gout patients, 85 visits (85 patients: 84 men, 1 woman; mean age, 63 years) with a medical record note constituted the main analyzed dataset. The 23 visits excluded from the main analysis were either laboratory or telephone encounters only (n = 15) or visits recorded in VA databases for which no documentation of a visit (n = 8) was found in medical records; they were included in sensitivity analyses.
Accuracy of VA database for gout-related visits
Main analyses (number = 85), including only patients with outpatient, inpatient, ER or Urgent care visits
All visits (number = 85)
Outpatient visits only (number = 76)
Inpatient, emergent or urgent care visits only (number = 9)
Sensitivity analyses (number = 108), including all visits in database a
All visits (number = 108)
Outpatient visits only (number = 99)
Inpatient, emergent or urgent care visits only (number = 9)
The VA database claims for gout-related visits had moderate to high specificity of 95.3%, PPV of 85.7% and sensitivity of 85.7% (Table 1). When the analyses were limited to outpatient claims only, specificity was still high at 94.8%, PPV was moderate at 84.2% and sensitivity was high at 88.9%. For non-outpatient claims (inpatient, emergent or urgent care), specificity and PPV were 100% each and sensitivity was 66.7%.
Sensitivity analyses that included the entire cohort (n = 108; including telephone and laboratory visits and those with no note) revealed that specificity of VA claims was high at 92.9%, PPV was moderate at 76% and sensitivity was 82.6%. Sensitivity analyses for the outpatient and non-outpatient claims for the entire cohort showed minimal differences compared to the main analysis (Table 1).
In this validation study, VA database claims for gout-related visits based on ICD-9-CM codes have moderate to high specificity and PPV meaning that most gout-related visits identified by VA administrative databases were, in fact, related to gout. Specificity and PPV for non-outpatient claims were numerically higher and sensitivity lower, compared to outpatient claims. In the absence of previous studies, these findings are novel and have important implications for health services studies using administrative databases. This study shows that if one were to use a broad approach of using any VA administrative claims, 76% of all visits coded as related to gout in VA administrative and clinical databases would truly be related to gout (PPV). If one takes a more sophisticated approach of excluding telephone encounters or visits with laboratory test only as non-specific, which can be easily done in most administrative databases including the VA, the specificity exceeded 95% and PPV exceeded 85%. Thus, VA databases were accurate for identifying gout-related utilization. The accuracy further improved by excluding visits for laboratory tests only and telephone encounters.
These predictive values are acceptable for health services studies, since the diagnostic accuracy is similar to some of the most commonly used laboratory/diagnostic tests in medicine. Our accuracy statistics cannot be directly compared to validation studies by Harrold and Malik et al. due to the differences in the gold standard and the study question (accuracy of gout diagnosis compared to accuracy of gout-related health care visit) [6, 7]. Similar to our previous validation studies in other types of arthritis [13, 14], this study is an important step in developing validated outcomes for health services and outcomes research that rely heavily on databases. Health services and CER studies often rely on large databases to address policy-level questions. Administrative databases such as Medicare, health plans, VA and so on are often used for the comparison of the effectiveness of various medications, treatment approaches or systems interventions to improve disease outcomes and reduce health care utilization. An inherent assumption made in these studies is that databases are accurate for health services utilization, which may not always be true. This study validates this assumption for gout-related utilization in the largest integrated health care system in the US, that is, the VA, that serves more than five million US veterans and has state-of-the-art sophisticated electronic health care record systems and national datasets designed for research . One must consider that, although VA is a national health care system with uniform access, benefits and staffing, and there are no obvious known systematic differences in patients or patient care compared to other facilities, this was not a random sample of all VA patients. Therefore, a further validation of these findings in a national sample will strengthen these conclusions.
Specificity and PPV of non-outpatient visits for gout (inpatient, emergent or urgent care) was even higher than that noted for outpatient visits. In fact, the databases are very accurate for non-outpatient gout visits with 100% specificity and 100% PPV, although sensitivity was lower. This has important implications. A PPV of 100% indicates that one can use gout-related inpatient, emergent or urgent care visits in VA databases with great confidence in health services and outcomes research. This easier approach is a more valid alternative to the unvalidated administrative definitions for gout flares used previously [16–19]. No validation was done except in one study, that mentioned performing a chart review on 100 patients, but did not provide any accuracy statistics . A better accuracy of databases for non-outpatient visits may indicate the focused nature of non-outpatient visits that leads to more accurate coding, as opposed to routine care for multiple medical conditions during scheduled outpatient visits.
This study has several limitations. The findings may not be generalizable to other health care settings (health maintenance organizations) and datasets (Medicare) and/or to other musculoskeletal conditions. It remains to be seen whether this approach can be validated in other large databases; studies are needed to examine the validity of this approach. Under-documentation in medical records is well known and, therefore, some visits deemed as not related to gout based on the medical record gold standard may have been labeled incorrectly. Considering this possibility, broad criteria for a visit to be determined as related to gout were made, namely, the presence of gout or gout-related complaints in any part of the provider’s note. A limitation to bear in mind is that while common gout-related terms including the presence of urate crystals were searched, accuracy in patients with crystal-proven gout was not tested, since a very small proportion of patients with gout have crystal-proven gout. It is possible that a greater health care provider understanding or awareness of gout may influence the extent of medical record documentation of gout, an important determinant that cannot be assessed with the current study. This is a question of interest for future studies and will be particularly relevant where the accuracy statistics are low or not acceptable.
In conclusion, the VA databases were found to be accurate for gout-related visits. The accuracy was higher for inpatient, emergent or urgent care visits, compared to outpatient visits related to gout. This implies that VA administrative databases can be used for health services outcomes research and CER in gout.
Comparative effectiveness research
International Classification of Disease ninth revision common modification
Negative predictive value
Positive predictive value
I thank Michael Conner, BS from Birmingham VA Medical Center, for creating a random sample of gout and non-gout visits. Takeda Pharmaceuticals USA, Inc. funded this investigator-initiated study, but had no contribution to the study design, analytic plan, data collection, analyses or the decision to submit the study. The study author (Dr. Singh) developed study protocol, performed data collection and data analyses, prepared the manuscript and made the decision to submit the manuscript. An internal review committee at Takeda Pharmaceuticals reviewed a draft of the manuscript and provided comments to the author. The decision to or not to incorporate these comments was made by the author (Dr. Singh). Dr. Singh is also supported by research grants from the National Institutes of Arthritis, Musculoskeletal and Skin Diseases (NIAMS), National Institutes of Aging (NIA), National Cancer Institute (NCI) and the Agency for Health Quality and Research Center for Education and Research on Therapeutics (CERTs), and the resources and use of facilities at the Birmingham VA Medical Center.
‘The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.’
- Zhu Y, Pandya BJ, Choi HK: Prevalence of gout and hyperuricemia in the US general population: the National Health and Nutrition Examination Survey 2007–2008. Arthritis Rheum. 2011, 63: 3136-3141. 10.1002/art.30520.View ArticlePubMedGoogle Scholar
- Roddy E, Zhang W, Doherty M: Is gout associated with reduced quality of life? A case–control study. Rheumatology (Oxford). 2007, 46: 1441-1444. 10.1093/rheumatology/kem150.View ArticleGoogle Scholar
- Singh JA, Strand V: Gout is associated with more comorbidities, poorer health-related quality of life and higher healthcare utilisation in US veterans. Ann Rheum Dis. 2008, 67: 1310-1316.View ArticlePubMedGoogle Scholar
- Kleinman NL, Brook RA, Patel PA, Melkonian AK, Brizee TJ, Smeeding JE, Joseph-Ridge N: The impact of gout on work absence and productivity. Value Health. 2007, 10: 231-237. 10.1111/j.1524-4733.2007.00173.x.View ArticlePubMedGoogle Scholar
- 100 Initial Priority Topics for Comparative Effectiveness Research. Available at: http://www.iom.edu/~/media/Files/Report%20Files/2009/ComparativeEffectivenessResearchPriorities/Stand%20Alone%20List%20of%20100%20CER%20Priorities%20-%20for%20web.ashx
- Malik A, Dinnella JE, Kwoh CK, Schumacher HR: Poor validation of medical record ICD-9 diagnoses of gout in a veterans affairs database. J Rheumatol. 2009, 36: 1283-1286. 10.3899/jrheum.081195.View ArticlePubMedGoogle Scholar
- Harrold LR, Saag KG, Yood RA, Mikuls TR, Andrade SE, Fouayzi H, Davis J, Chan KA, Raebel MA, Von Worley A, Platt R: Validity of gout diagnoses in administrative data. Arthritis Rheum. 2007, 57: 103-108. 10.1002/art.22474.View ArticlePubMedGoogle Scholar
- Malik A, Schumacher HR, Dinnella JE, Clayburne GM: Clinical diagnostic criteria for gout: comparison with the gold standard of synovial fluid crystal analysis. J Clin Rheumatol. 2009, 15: 22-24. 10.1097/RHU.0b013e3181945b79.View ArticlePubMedGoogle Scholar
- Saseen JJ, Agashivala N, Allen RR, Ghushchyan V, Yadao AM, Nair KV: Comparison of patient characteristics and gout-related health-care resource utilization and costs in patients with frequent versus infrequent gouty arthritis attacks. Rheumatology (Oxford). 2012, 51: 2004-2012. 10.1093/rheumatology/kes183.View ArticleGoogle Scholar
- Park H, Rascati KL, Prasla K, McBayne T: Evaluation of health care costs and utilization patterns for patients with gout. Clin Ther. 2012, 34: 640-652. 10.1016/j.clinthera.2012.01.020.View ArticlePubMedGoogle Scholar
- Wu EQ, Forsythe A, Guerin A, Yu AP, Latremouille-Viau D, Tsaneva M: Comorbidity burden, healthcare resource utilization, and costs in chronic gout patients refractory to conventional urate-lowering therapy. Am J Ther. 2012, 19: e157-e166. 10.1097/MJT.0b013e31820543c5.View ArticlePubMedGoogle Scholar
- Perlin JB, Kolodner RM, Roswell RH: The Veterans Health Administration: quality, value, accountability, and information as transforming strategies for patient-centered care. Am J Manag Care. 2004, 10: 828-836.PubMedGoogle Scholar
- Singh JA, Holmgren AR, Krug H, Noorbaloochi S: Accuracy of the diagnoses of spondylarthritides in veterans affairs medical center databases. Arthritis Rheum. 2007, 57: 648-655. 10.1002/art.22682.View ArticlePubMedGoogle Scholar
- Singh JA, Holmgren AR, Noorbaloochi S: Accuracy of Veterans Administration databases for a diagnosis of rheumatoid arthritis. Arthritis Rheum. 2004, 51: 952-957. 10.1002/art.20827.View ArticlePubMedGoogle Scholar
- Harrold LR, Yood RA, Mikuls TR, Andrade SE, Davis J, Fuller J, Chan KA, Roblin D, Raebel MA, Von Worley A, Platt R, Saag KG: Sex differences in gout epidemiology: evaluation and treatment. Ann Rheum Dis. 2006, 65: 1368-1372. 10.1136/ard.2006.051649.PubMed CentralView ArticlePubMedGoogle Scholar
- Halpern R, Fuldeore MJ, Mody RR, Patel PA, Mikuls TR: The effect of serum urate on gout flares and their associated costs: an administrative claims analysis. J Clin Rheumatol. 2009, 15: 3-7. 10.1097/RHU.0b013e3181945d2c.View ArticlePubMedGoogle Scholar
- Primatesta P, Plana E, Rothenbacher D: Gout treatment and comorbidities: a retrospective cohort study in a large US managed care population. BMC Musculoskelet Disord. 2011, 12: 103-10.1186/1471-2474-12-103.PubMed CentralView ArticlePubMedGoogle Scholar
- Rothenbacher D, Primatesta P, Ferreira A, Cea-Soriano L, Rodriguez LA: Frequency and risk factors of gout flares in a large population-based cohort of incident gout. Rheumatology (Oxford). 2011, 50: 973-981. 10.1093/rheumatology/keq363.View ArticleGoogle Scholar
- Sarawate CA, Brewer KK, Yang W, Patel PA, Schumacher HR, Saag KG, Bakst AW: Gout medication treatment patterns and adherence to standards of care from a managed care perspective. Mayo Clin Proc. 2006, 81: 925-934. 10.4065/81.7.925.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.