Skip to main content

Derivation and internal validation of a multi-biomarker-based cardiovascular disease risk prediction score for rheumatoid arthritis patients

Abstract

Background

Rheumatoid arthritis (RA) patients have increased risk for cardiovascular disease (CVD). Accurate CVD risk prediction could improve care for RA patients. Our goal is to develop and validate a biomarker-based model for predicting CVD risk in RA patients.

Methods

Medicare claims data were linked to multi-biomarker disease activity (MBDA) test results to create an RA patient cohort with age ≥ 40 years that was split 2:1 for training and internal validation. Clinical and RA-related variables, MBDA score, and its 12 biomarkers were evaluated as predictors of a composite CVD outcome: myocardial infarction (MI), stroke, or fatal CVD within 3 years. Model building used Cox proportional hazard regression with backward elimination. The final MBDA-based CVD risk score was internally validated and compared to four clinical CVD risk prediction models.

Results

30,751 RA patients (904 CVD events) were analyzed. Covariates in the final MBDA-based CVD risk score were age, diabetes, hypertension, tobacco use, history of CVD (excluding MI/stroke), MBDA score, leptin, MMP-3 and TNF-R1. In internal validation, the MBDA-based CVD risk score was a strong predictor of 3-year risk for a CVD event, with hazard ratio (95% CI) of 2.89 (2.46–3.41). The predicted 3-year CVD risk was low for 9.4% of patients, borderline for 10.2%, intermediate for 52.2%, and high for 28.2%.

Model fit was good, with mean predicted versus observed 3-year CVD risks of 4.5% versus 4.4%. The MBDA-based CVD risk score significantly improved risk discrimination by the likelihood ratio test, compared to four clinical models. The risk score also improved prediction, reclassifying 42% of patients versus the simplest clinical model (age + sex), with a net reclassification index (NRI) (95% CI) of 0.19 (0.10–0.27); and 28% of patients versus the most comprehensive clinical model (age + sex + diabetes + hypertension + tobacco use + history of CVD + CRP), with an NRI of 0.07 (0.001–0.13). C-index was 0.715 versus 0.661 to 0.696 for the four clinical models.

Conclusion

A prognostic score has been developed to predict 3-year CVD risk for RA patients by using clinical data, three serum biomarkers and the MBDA score. In internal validation, it had good accuracy and outperformed clinical models with and without CRP. The MBDA-based CVD risk prediction score may improve RA patient care by offering a risk stratification tool that incorporates the effect of RA inflammation.

Background

Cardiovascular disease (CVD) is the leading cause of mortality for patients with rheumatoid arthritis (RA), accounting for 30–40% of deaths [1]. Patients with RA have approximately 50% greater risk for cardiovascular disease (CVD) compared to the general population [2]. Traditional CVD risk factors such as diabetes, hypertension, and hyperlipidemia are important in RA patients and are not difficult to assess. However, the time constraints of a busy office practice often preclude making CVD risk stratification a routine part of RA patient care. Indeed, 79% of rheumatologists cite a lack of time as a major barrier [3]. Even so, rheumatologists are well positioned to help manage CVD risk in RA patients because 30% of CVD risk in RA patients is attributable to systemic inflammation and other RA-related factors [4, 5].

CVD risk predictors developed for the general population tend to underestimate CVD risk in RA patients [6,7,8]. European League Against Rheumatism (EULAR) guidelines recommend that CVD risk predicted by tools such as the Framingham Risk Score (FRS) or the American College of Cardiology and American Heart Association (ACC/AHA) pooled cohort risk equation [9] be multiplied by 1.5 to account for the effect of RA on CVD risk [6, 10]. A limitation of this approach is that it treats all RA patients the same, regardless of the level of disease activity.

ACC/AHA guidelines recommend preventive strategies for all patients with high predicted risk of CVD. Current recommendations support managing hyperlipidemia by “treating to risk” rather than a targeted LDL [11,12,13]. It is well established that vascular inflammation has a central role in atherosclerosis and CVD, but evidence that reducing systemic inflammation has potential to lower CVD risk is more recent. Proof of principle comes from the CANTOS trial, which showed that canakinumab, an anti-IL-1β biologic drug, reduced the CVD event rate in non-RA patients with a high risk of CVD and elevated high-sensitivity C-reactive protein (CRP) [14]. Patients with greater reduction in inflammation, measured by CRP, benefited the most [15].

Synovial and systemic inflammation in RA patients contribute to CVD risk independently of traditional risk factors [4]. In observational studies, the risk for CVD events was greatest in RA patients with high disease activity [16,17,18,19,20] and effective RA treatment appeared to reduce the risk for atherosclerosis [21] and CVD events [22, 23]. Traditional CVD risk factors, such as diabetes, may be exacerbated by RA-related mechanisms [24, 25]. Thus, it may be possible to reduce the CVD risk elevation attributable to RA by treating RA inflammatory pathways.

High sensitivity CRP has prognostic value for CVD events in non-RA populations, but its role for CVD risk prediction in RA patients is less clear because CRP may be a marker for systemic inflammation in RA rather than a surrogate for the extent of vascular involvement [26]. Moreover, CRP is not elevated in some RA patients with active disease [27]. CVD risk prediction models that combine measures of RA disease activity with traditional risk factors [19, 28, 29] are not yet the standard of care. Molecular markers of inflammation other than CRP have not been incorporated into validated CVD risk predictors for RA patients. Their inclusion would be novel and may have potential to improve CVD preventive care for RA patients by making CVD risk stratification more accurate and accessible.

The multi-biomarker disease activity (MBDA) test assesses RA disease activity by measuring 12 serum protein biomarkers to provide a validated score on a scale of 1–100 that correlates with the Disease Activity Score in 28 joints with CRP (DAS28-CRP) [30]. In 2019, the American College of Rheumatology disease activity measures working group concluded that the MBDA score was one of 11 measures of RA disease activity that met the minimum standard for regular use [31]. The MBDA score is predictive of future radiographic damage, independently of other measures [32, 33]. In a large, cross-sectional observational study, the MBDA score was found to be associated with risk for CVD, suggesting that the MBDA score and at least some of its biomarkers detect inflammation that is relevant to cardiovascular pathology [16].

Building on this evidence, we now describe the development and internal validation of an RA-specific CVD risk prediction score that uses routine clinical assessments plus RA-related biomarkers to predict CVD risk. The goal of this approach was to improve preventive CVD care in RA patients by developing a prognostic score that uses biomarkers to incorporate the contribution of RA-related inflammation to individual CVD risk. The intended end result of this endeavor is to create a validated CVD risk score that will enable rheumatologists to risk stratify their RA patients efficiently in an office setting, with components associated with RA disease activity directly represented in the CV risk estimate.

Methods

Data source

A retrospective RA cohort was created for this study by linking claims data in the Medicare database with data in the MBDA test commercial database (Vectra®, formerly Crescendo Bioscience, Inc., South San Francisco, CA, USA, currently Myriad Genetics Laboratories, Salt Lake City, UT, USA), using all fee-for-service Medicare data from 2006 to 2016 for all individuals who underwent MBDA testing. Data were linked on patient date of birth, sex, MBDA test date, MBDA testing codes (defined by Current Procedural Terminology codes 81479, 83520, 84999, 86140, and 81490, submitted by Crescendo Bioscience or Myriad Genetics Laboratories), and the National Provider Identifier of the treating rheumatologist. Data were linked deterministically, using established methods [16, 34]. The University of Alabama at Birmingham institutional review board approved the study.

Participant and MBDA test eligibility criteria

The patient cohort and MBDA test results included in this study were selected by applying a series of criteria to the patients and MBDA tests in the linked database described above (Supplemental Table 1). To be eligible for inclusion in the study, patients were required to (1) be ≥ 40 years old, (2) have at least one RA diagnosis code from a rheumatologist (ICD9 714.0; ICD10 M05.*, M06.*, excluding M06.4 and M06.1, with * representing any number of digits or characters), (3) have received an RA-specific treatment (TNF-inhibitor, abatacept, rituximab, anti-IL-6R, Janus kinase inhibitor, conventional synthetic disease-modifying anti-rheumatic drug including methotrexate, sulfasalazine, leflunomide and hydroxychloroquine) anytime up to and including the date of the first MBDA test, and (4) have at least one linked MBDA test result. The accuracy of this claim-based method of identifying RA patients exceeds 85% [35] and is likely made greater here by the linkage with data from MBDA testing, which is only for patients diagnosed with RA.

The baseline period for a patient was defined as the interval preceding the date of the first MBDA test in the linked database. It included all available preceding Medicare data and was required to span at least 1 year, with patients being required to have had at least 365 days of continuous coverage with Medicare parts A (hospital coverage), B (outpatient coverage), and D (pharmacy coverage). Patients were excluded if they had any diagnosis code in the baseline period for malignancy (except non-melanoma skin cancer), myocardial infarction (MI), or stroke. MBDA test results (i.e., the MBDA score and 12 biomarker measurements) were used from the earliest MBDA test performed after the above requirements had been met, unless (1) it was performed within 14 days following any hospital discharge or (2) the patient had used anti-IL-6R treatment in the preceding 90 days (because tocilizumab treatment may affect the MBDA score in a way that might confound CVD risk prediction) [36]; in these cases, the next MBDA test meeting the above requirements was used and the baseline period was anchored to that test. The follow-up period for ascertaining CVD outcomes (see below) began on the date of the first qualifying MBDA test. The follow-up period ended at the earliest of (1) a CVD outcome, (2) diagnosis of malignancy, (3) non-CVD death, or (4) the end of study (December 31, 2016).

CVD outcome

The CVD outcome we used for the prognostic test was a composite, defined as the occurrence of hospitalized MI, stroke, or fatal CVD. This outcome definition is consistent with the outcome used in the guidelines of the ACC/AHA [9]. MI was defined as ICD-9 diagnosis code 410.x1 or ICD-10 diagnosis code I21.* from an inpatient hospitalization lasting ≥ 1 night or where the patient died. Stroke was identified using ICD-9 diagnosis codes 430.*, 431.*, 433.x1, 434.x1, 436.* or ICD-10 diagnosis codes I60.*, I61.*, I63.* or I67.89 from hospital discharge. This approach has been described previously [37,38,39]. Fatal CVD was identified using a validated algorithm that identifies fatal MIs and fatal strokes from Medicare data at a threshold yielding a positive predictive value > 80%, with greater accuracy than is obtained using hospital discharge diagnoses [40].

Biomarkers and other predictors

MBDA score

All biomarker data in this study came from the MBDA test, which measures the serum concentrations of 12 biomarkers and uses an algorithm to produce a disease activity score on a scale of 1 to 100. The MBDA score has been validated against DAS28-CRP in patients treated with a variety of RA therapies, with AUROC values of 0.77 and 0.70 observed in seropositive and seronegative RA patients, respectively [30, 41]. The MBDA score is used to assess and monitor inflammatory disease activity in RA patients and is complementary to clinical assessment. It is a stronger predictor of risk for radiographic progression than DAS28-CRP [32, 33]. The MBDA score is not intended for the diagnosis of RA but rather is for use in assessing disease activity in patients with already-diagnosed RA. The MBDA score has been available for use in clinical practice in the US since 2010. Its cost has been covered in the US by Medicare since 2013 and is also covered by some private insurers.

The biomarkers in the MBDA test reflect the biology of RA and comprise cytokine-related proteins (IL-6, TNF-R1), acute phase reactants (CRP, serum amyloid A), an adhesion molecule (VCAM-1), a skeletal-related protein (bone glycoprotein 39 [YKL-40]), growth factors (EGF, VEGF-A), matrix metalloproteinases (MMP-1, MMP-3), and adipokines (leptin, resistin). All MBDA scores analyzed here were from tests that had been ordered by practitioners in the US as part of routine patient care. All MBDA testing was performed in a Clinical Laboratory Improvements Amendment-certified commercial laboratory in South San Francisco, CA (Crescendo Bioscience), where MBDA scores were calculated and stored with related data in a secure database.

Prior to and independently of the present study, an algorithm was developed and validated to adjust the MBDA score for the effects of age, sex, and leptin (as a surrogate for adiposity) [42]. This adjustment acts on the original MBDA score without affecting the individual contributions of the 12 biomarkers. Thus, the original MBDA score is calculated as previously, then adjusted to produce a score that, like the original score, has a scale of 1–100 and RA disease activity categories of low (< 30), moderate (30–44), and high (> 44) [30, 42]. The adjusted MBDA score has been in routine use since December 2017. Original MBDA scores were converted to adjusted MBDA scores for this study. In the remainder of this report, the term “MBDA score” means the adjusted MBDA score.

Variables considered for inclusion in model building

Variables considered for use in model building that came from the MBDA database included the MBDA score and the serum concentrations of its 12 component biomarkers. This approach was non-redundant because the algorithm for the MBDA score is a non-linear combination of its component biomarkers, which were neither selected nor weighted for CVD prediction [30, 41].

Demographic and clinical predictors were obtained from the Medicare database and were considered for inclusion in model building based upon their expected association with CVD risk, informed by subject matter expertise and the medical literature. Other considerations were face validity, data quality in the Medicare database, and feasibility of collecting a variable accurately in clinical practice. These predictors included age, sex, race, tobacco use (past or present), history of CVD other than MI or stroke, diagnoses of and medications for diabetes, hypertension and hyperlipidemia, RA medications as described above, glucocorticoids, and non-steroidal anti-inflammatory drugs. A diagnosis was counted as present if any of its diagnostic codes was found for the patient. Diagnostic codes for the candidate predictors, i.e., the subset of variables that were included in the final model-building exercise, and the prevalences of CVD-related conditions, appear in Supplemental Table 2.

Clinical measurements (e.g., blood pressure or lipid levels) were not available in either database and were not considered for inclusion in model building. Current use of CV-related medications (e.g., lipid-lowering therapies) and RA medications was initially considered and was evaluated as part of baseline data assessment. However, a decision was made to not include any medications as variables in model building for two reasons: (1) without being able to account for disease-related clinical measurements, the estimated effect of medications may be counterintuitive or inaccurate and (2) suboptimal medication adherence could result in meaningful misclassification of the CV risk associated with these treatments. Race was excluded because of uncertainties related to racial heterogeneity and the reporting of race.

Statistical analysis

A principled, pre-specified approach to model building and selection was conducted that followed Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines [43]. First, the cohort was randomly split 2:1 into separate datasets for training and testing (i.e., internal validation).

Prior to model building, the independent association of the MBDA score with the CVD risk was evaluated in the training dataset with a multivariable analysis that included all non-biomarker candidate predictors [16]. Separately, the form of the relationship between MBDA score and CVD risk, on the logarithmic scale of hazard, was examined and found to be linear up to MBDA scores of approximately 60 and non-linear thereafter—a relationship that can be described with a hyperbolic tangent function (see below), which is commonly used in other fields, e.g., in models of neural networks [44].

Training: evaluation of variables and model building

Model development was conducted in the training dataset, to achieve the goal of estimating individual risk for the composite CVD outcome as a function of the candidate predictors. Individual biomarker concentrations in ng/ml were natural log transformed. MBDA scores (integers on a scale of 1 to 100) were hyperbolic tangent-transformed, as f(x) = tanh(ax), where a is a constant parameter that was based on maximum likelihood estimation and updated in each step of model building. Age in years was treated as a continuous variable. A separate age-squared term was initially included to account for possible nonlinearity between age and the composite CVD outcome, but it added no additional value to model building and was dropped. Other candidate predictors were treated as binary variables.

Association with 3-year CVD risk was assessed for each candidate variable with a hazard ratio (HR) and determined by univariable analysis in the training dataset. A 3-year time frame was chosen based on the availability of MBDA biomarker data from testing performed as part of routine care. Model building used Cox proportional hazards regression with backward elimination in the training dataset. In the first step, a model was fit by including every candidate predictor variable; in each subsequent step, the least significant variable (i.e., with the highest p value) was removed, and the model was refit with the remaining variables. This process was repeated until all remaining variables had p < 0.05.

Clinical models developed for comparison

Four prespecified models for predicting CVD risk were built in the training dataset for comparison with the MBDA-based model: (1) age + sex, (2) age + sex + CRP, (3) a clinical model (age + sex + tobacco use + diabetes + hypertension + history of CVD [excluding MI and stroke]), and (4) the clinical model + CRP. These models were chosen for the availability of their variables in routine clinical practice and in our linked database.

Derivation of categories of 3-year risk for CVD events

The thresholds for 3-year CVD risk categories that would be equivalent to the thresholds for 10-year risk categories of other CVD risk prediction equations were derived in a cohort with 10 years of longitudinal data. To create a dataset in which CVD event rates at 3 and 10 years could be bridged, a cohort of 533,139 Medicare RA patients with data available from 2006 to 2016 was selected with the same requirements as for the main cohort of this study but without requiring MBDA testing. An age + sex model was developed in this cohort to establish 10-year rates of CVD events, and 3-year cutpoints corresponding to the 10-year ACC/AHA risk thresholds of 5% (± 0.1%), 7.5% (± 0.1%), and 20% (± 0.1%) [11] were obtained by bootstrapping. The derived cutpoints were 1.3%, 1.8%, and 5.2%, defining 3-year CVD risk categories of low (0 to < 1.3%), borderline (≥ 1.3 to < 1.8%), intermediate (≥ 1.8 to < 5.2%), and high (≥ 5.2%) risk.

Internal validation

The primary analysis for establishing internal validation was to estimate the risk of a composite CVD event at 3 years (i.e., the probability of a patient having an MI, a stroke, or CVD death in the next 3 years), by using the MBDA-based CVD risk score as the only variable in a Cox proportional hazard regression model. HR (with 95% confidence interval [CI]; p value by partial likelihood ratio test [LRT]) was determined for the MBDA-based CVD risk score [45,46,47]. A risk curve was constructed to illustrate this relationship, using methods described in Supplementary Text. These and all other validation analyses were performed in the validation dataset.

To assess accuracy of the MBDA-based CVD risk score, a secondary analysis for internal validation examined goodness of fit with plots that compared observed risk (based on Kaplan-Meier estimates with 95% CI) with predicted risk across CVD event-based deciles. P values were determined using the Greenwood-Nam-D’Agostino test [48], with higher (i.e., non-significant) p values indicating better fit. Goodness of fit was also assessed among patient subgroups, based on age, sex, diagnosis of diabetes, hypertension, tobacco use (past or present), and hyperlipidemia, as well as history of CVD, statin use, oral glucocorticoid use, initiation or change of a biologic agent during follow-up, and MBDA score category. Bonferroni correction was used to adjust for multiple testing. CVD event quintiles, rather than deciles, were used for patient subgroups with fewer than 110 CVD events to avoid data sparsity. In addition, Kaplan-Meier plots of CVD event-free status over time were constructed for patients grouped into CVD risk categories by the MBDA-based CVD risk score, using the Mantel-Haenszel test [45, 46].

Validation included comparisons of the predictive abilities of the MBDA-based CVD risk score and four clinical models described above. HR (95% CI) and p value (using the partial LRT) were calculated from Cox proportional hazards models in single-score (i.e., univariable) analyses of the MBDA-based CVD risk score and each of the four clinical models. To determine the incremental contribution of the MBDA-based model to each clinical model for predicting CVD risk (and vice versa), change in model deviance was determined using the likelihood ratio statistic in sequential (i.e., bivariable) analyses for each model pair.

The MBDA-based CVD risk model was also compared to the four clinical models with reclassification tables and the Net Reclassification Index (NRI) [49, 50]. The five models were each evaluated for discrimination based on the C-index (similar to AUROC) for predicting risk at 3 years, with times weighted by the square inverse of the censoring distribution [51].

Statistical software

SAS 9.4 was used for data preparation. R version 3.4 and R packages survival, nricens, and pec were used for evaluating model performance, calculating NRIs and C-indices, and generating plots [52].

Results

Cohort selection

30,751 RA patients with 904 CVD events (480 MI, 362 stroke, 62 CVD death) were eligible for the total cohort (Supplemental Table 1). Total follow-up from the index date was 56,684 patient-years (PY) with median (interquartile range [IQR]) follow-up duration of 1.7 (0.8–2.7) years. The overall CVD event rate (95% CI) was 15.9 (14.9–17.0) events per 1000 PY.

At baseline, the mean age was 69 years, 23% of patients were under age 65 years, 18% were men, and 8% were Black (Table 1). The prevalence of CVD-related comorbidities, such as diabetes (40%) and hypertension (79%), was high. Statin use was found in 42%. Sixty percent of patients were receiving methotrexate, 33% a TNF inhibitor (TNFi), and 15% a non-TNFi biologic. Median (IQR) CRP value was 4.5 (1.6–12.0) mg/L (or 1.5 [0.5–2.5] μg/ml natural log transformed). Median (IQR) MBDA score was 40 [32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49], which is in the moderate MBDA category (range, 30–44) (Table 1).

Table 1 Patient characteristics at baseline*

Confirming the MBDA score as an independent predictor of CVD risk

In the training dataset (N = 20,476 patients with 611 CVD events), the MBDA score, untransformed, was significantly prognostic of CVD events in a multivariable analysis with age, sex, diabetes, hypertension, tobacco use, CVD history, and hyperlipidemia, but with no individual biomarker variables (HR = 1.023; 95% CI 1.017–1.029).

Training of the MBDA-based model

In univariable analyses in the training dataset, all candidate predictors except EGF and MMP-1 were individually predictive of CVD events (Table 2). In the final MBDA-based model, derived from backward elimination, the variables of age, diabetes, history of CVD, hypertension, tobacco use, MBDA score, and three biomarkers (leptin, MMP-3, TNF-R1) were significant predictors in multivariable analyses; sex, hyperlipidemia, and nine biomarkers were not. HRs were significantly > 1.0 for all predictor variables in the final MBDA-based model except leptin, for which HR was 0.84, indicating a negative relationship between leptin concentration and CVD risk (Table 2).

Table 2 Hazard ratios (HR) of predictor variables used in CVD risk models (training dataset, N = 20,476)

The equation for the final MBDA-based CVD risk score was:

$$ 0.0314\times \boldsymbol{age}\kern0.3em +\kern0.3em 0.2691\times \boldsymbol{tobacco}\;\boldsymbol{use}+\kern0.3em 0.2732\times \boldsymbol{diabetes}+0.2694\times \boldsymbol{hypertension}+0.3378\times \boldsymbol{history}\kern0.17em \boldsymbol{of}\;\boldsymbol{CVD}-0.1711\times \mathrm{In}\left(\boldsymbol{Leptin}\right)+0.1454\times \ln \left(\boldsymbol{MMP}\mathbf{3}\right)+0.5724\times \ln \left(\boldsymbol{TNFR1}\right)+1.6076\times \tanh \left(\boldsymbol{MBDA}\kern0.17em \boldsymbol{score}/33.0807\right), $$

where the age is in years, clinical variables are scored as 1 when present and zero when absent, Leptin, MMP-3, and TNF-R1 represent serum concentrations in ng/mL, the term “ln” means natural logarithm, and “tanh” means hyperbolic tangent transformation. The output of this algorithm is the MBDA-based CVD risk score. This score is used in a separate formula to calculate the predicted 3-year risk for a CVD event as a percentage value (see Supplemental Text).

In the four multivariable clinical models that were generated for comparison—i.e., an age + sex model and an age + sex + diabetes + hypertension + history of CVD + tobacco use model, each one with and without CRP—all variables in each model were significant CVD predictors (Table 2).

Internal validation of the MBDA-based model

The MBDA-based CVD risk score was a strong predictor of 3-year risk for a CVD event in the validation dataset (N = 10,275 patients with 293 CVD events), with an HR (95% CI) of 2.89 (2.46–3.41, p = 4.67 × 10− 38). The relationship between the MBDA-based CVD risk score and predicted 3-year CVD risk is shown in Fig. 1a. The proportions of patients in the low, borderline, intermediate, and high categories of predicted 3-year CVD risk in the validation dataset were 9.4%, 10.2%, 52.2%, and 28.2%, respectively (Fig. 1b).

Fig. 1
figure1

Characterization of the MBDA CVD risk score in the validation dataset (N = 10,275). a Relationship between MBDA-based CVD risk score and predicted 3-year risk of a CVD event, with 95% confidence interval. b Distribution of predicted 3-year risks. Dotted lines, horizontal in a and vertical in b, indicate thresholds at 1.3%, 1.8%, and 5.2% separating the categories of low, borderline, intermediate, and high risk, which contained 9.4%, 10.2%, 52.2%, and 28.2% of patients, respectively. CVD event is myocardial infarction, stroke, or CVD death. CVD cardiovascular disease, MBDA multi-biomarker disease activity

Assessment of accuracy with goodness of fit

The 3-year CVD risk predictions made by the MBDA-based model were similar to the observed CVD event rates across deciles based on observed CVD events (Fig. 2). The goodness of fit test statistic indicated good fit (p = 0.39). The confidence intervals for observed risk contained the average predicted risk for all but one decile group. Overall, the mean predicted 3-year CVD risk in the validation dataset was 4.5%, compared with the observed 3-year CVD risk of 4.4%. Subanalyses showed that the MBDA-based model performed well in subgroups of interest: males and females, with/without diagnosis of diabetes, with/without diagnosis of hypertension, with/without tobacco use, with/without history of CVD, with/without hyperlipidemia, taking/not taking statins, < 65 years old, < 75 years old, and patients who had or had not used oral glucocorticoids in the baseline period, or initiated or changed a biologic drug during the follow-up period, or had low, moderate, or high disease activity (MBDA score) (Supplemental Fig. 1).

Fig. 2
figure2

Goodness of fit: Predicted CVD risk versus observed 3-year CVD event rates. The observed 3-year CVD event rate was determined for each event-based decile and is shown vs. the average predicted 3-year risk in each decile. Analysis used the validation dataset (N = 10,275). Observed event rates were determined as Kaplan-Meier (95% log-log CI) estimates. P = 0.39 by the Greenwood-Nam-D’Agostino test, indicating good fit. CVD event is myocardial infarction, stroke, or CV death. 3-year CVD risk categories (low, borderline, intermediate, high) were derived from the 10-year risk categories of the 2018 Guidelines of the American College of Cardiology/American Heart Association [8]. Threshold between low and borderline risk categories is 1.3% (not shown). CI confidence interval, CVD cardiovascular disease, MBDA multi-biomarker disease activity

Loss of CVD outcome-free status by category of predicted risk

A Kaplan-Meier plot depicting loss of CVD outcome-free status in the validation dataset showed statistically significant separation of the low, borderline, intermediate and high predicted CVD risk groups over time (p = 1.7 × 10−32) (Fig. 3).

Fig. 3
figure3

Kaplan-Meier plot of CVD event-free survival. Occurrence of CVD events by Kaplan-Meier survival analysis is shown for patients in the validation dataset (N = 10,275) grouped by a 3-year CVD risk category predicted by the MBDA-based CVD risk score at baseline. P = 1.7 × 10−32 by the Mantel-Haenszel test. CVD event is myocardial infarction, stroke, or CVD death. See Fig. 2 for explanation of CVD risk categories. CVD cardiovascular disease, MBDA multi-biomarker disease activity

Model evaluation and comparison by likelihood test

When analyzed alone, each of the four clinical models made statistically significant contributions to the prediction of CVD risk in terms of the likelihood ratio, which represents how well the model fits the data (Fig. 4). However, these models made smaller contributions than the MBDA-based CVD risk score (Fig. 4). Moreover, the addition of these clinical models to the MBDA-based CVD risk score in paired analyses did not improve CVD risk prediction, as indicated by the respective increments in LRT statistic (0.4–3.0), which were small and non-significant (Table 3). In contrast, the MBDA-based CVD risk score provided additional information to improve the prediction of CVD risk when it was added to each clinical model, with the increments in LRT statistic being large (35.4–83.3) and statistically significant (all p < 3 × 10− 9) (Table 3).

Fig. 4
figure4

Contribution to CVD risk prediction by MBDA-based CVD risk score and clinical models. Likelihood ratio test statistics are shown for univariable (i.e., single-score) analyses of a CVD risk prediction by the MBDA-based CVD risk score and four comparison models, using the validation dataset (N = 10,275) (see also Table 3). P values are by the likelihood ratio test. The clinical model includes age, sex, tobacco use, diabetes, hypertension, and history of CVD. CRP C-reactive protein, CVD cardiovascular disease, MBDA multi-biomarker disease activity

Table 3 Contribution of MBDA-based CVD risk score and other models to prediction of 3-year CVD risk

Reclassification

Compared to the simplest of the clinical models, the age + sex model, the MBDA-based model reclassified the CVD risk for 42% of patients overall and as many as 75% of patients, depending on the age + sex model risk category (Table 4A). Compared to the most comprehensive clinical model, the clinical + CRP model, the MBDA-based model reclassified the CVD risk for 28% of patients overall and as many as 64% of patients, depending on the clinical + CRP model risk category (Table 4B). Reclassification results for the age + sex + CRP model and the clinical model (without CRP) were generally intermediate to those of the other two models (Supplemental Tables 3A and 3B).

Table 4 Reclassification of patients by the MBDA-based CVD risk score versus: A, age + sex model and B, clinical + CRP model

NRI test statistics demonstrated that the MBDA-based model significantly improved classification versus all four clinical models, with NRI test statistics (95% CI) of 0.19 (0.10–0.27) versus the age + sex model, 0.16 (0.08–0.23) versus the age + sex + CRP model, 0.10 (0.04–0.17) versus the clinical model, and 0.07 (0.001–0.13) versus the clinical + CRP model.

Discrimination

The C-index (95% CI) for the prediction of CVD risk at 3 years by the MBDA-based CVD risk score in the validation dataset was 0.715 (0.683–0.747), which was numerically greater than the C-index for each clinical model. The difference was greatest versus the simplest clinical model and least versus the most comprehensive clinical model, with C-indices (95% CI) of 0.661 (0.628–0.695) for the age + sex model, 0.674 (0.642–0.707) for the age + sex + CRP model, 0.688 (0.656–0.721) for the clinical model, and 0.696 (0.664–0.729) for the clinical + CRP model.

Relationship between individual biomarkers and MBDA-based CVD risk score

Scatterplots derived from the validation dataset demonstrate the positive relationships between 3-year risk predicted by the MBDA-based CVD risk score and MBDA score (r = 0.438), MMP-3 (r = 0.437), and TNF-R1 (r = 0.632); and the negative relationship with leptin (r = − 0.179). For the MBDA score and for each biomarker, at most levels a range of CVD risks was observed, consistent with variation among the other variables of the MBDA-based CVD risk score (Fig. 5).

Fig. 5
figure5

Relationship between predicted CVD risk and molecular variables. The predicted 3-year risk for a CVD event (myocardial infarction, stroke, or fatal CVD) is shown versus (a) the MBDA score and (bd) serum concentrations (ng/ml, natural log transformed) of the three biomarker variables in the MBDA-based CVD risk score, using the validation dataset (N = 10,275). R values are Spearman correlation coefficients. CVD cardiovascular disease, MBDA multi-biomarker disease activity

Discussion

We have used a cohort of over 30,000 RA patients to derive and internally validate an MBDA-based CVD risk score for use in patients with RA. This score reflects the contribution of systemic inflammation to CVD risk by including the MBDA score and three individual biomarkers, while also incorporating age and four clinical risk factors. The MBDA-based risk score accurately predicted CVD risk in terms of goodness of fit analyses in the internal validation cohort and in clinically relevant subgroups, including patients who did or did not have prior CVD, who were already taking statins, or had different levels of RA disease activity. The MBDA-based risk score discriminated CVD risk better than clinical models, assigning some patients to higher or lower risk categories compared with clinical assessment alone.

This test is unique because it uses biomarker-based measurements to incorporate the contribution of RA inflammation to CVD risk in a more personalized way than by multiplying by a fixed value, such as 1.5 [6]. The MBDA score is a measure of RA disease activity that is also predictive of risk for radiographic progression. It was shown here and previously to be associated with the CVD risk [16], even though it was not originally developed for that purpose. MMP-3 and TNF-R1 were included in the final MBDA-based CVD risk score because in model building, they were positively associated with CVD risk independently of the MBDA score and other variables, which is consistent with previous reports of their role in cardiovascular risk [53,54,55].

The other individual biomarker in the CVD risk score was leptin. In our cohort, patients with a CVD event had less obesity and a numerically lower median leptin concentration than patients without a CVD event (Table 1). Leptin had a negative coefficient in the multivariable CVD risk prediction model. These results are consistent with evidence that leptin correlates strongly with body mass index (BMI) and that BMI has been negatively associated with CVD risk in RA patients [56], even though it is positively associated with CVD risk in the general population [57]. Our findings may reflect a contribution of RA inflammation to both weight loss and mortality, rather than a biologically protective effect of obesity [58]. They may also be a reflection of index case bias, which can lower the effect estimate for a risk factor, such as leptin, if it is associated with both the sequela of a disease and the disease itself, as with CVD events and RA [59]. IL-6, CRP, and other MBDA biomarkers were not included in the MBDA-based CVD risk score despite being individually associated with the CVD risk because none added significant information to leptin, MMP-3, TNF-R1, and the MBDA score for predicting CVD risk.

Clinical covariates that might have been expected in the final MBDA-based model, such as sex and hyperlipidemia, were associated with the CVD risk in univariable analyses but were not included because they made small incremental contributions to the multivariable model and did not survive the model building process. Sex was less significant as a univariable predictor of CVD risk than any of the variables that were included in the model (Table 2). It may have been excluded due to co-linearity with other variables, such as tobacco use, which is less common in women with RA than men with RA [4], and leptin, the levels of which tend to be greater in women [60]. It is unlikely that the MBDA score caused sex to be excluded from the model because adjustment of the MBDA score (for age, sex, and leptin) should have reduced its co-linearity with sex. The failure of hyperlipidemia to survive backward elimination may relate to it also having been a less significant univariable predictor of CVD risk than any of the predictors that survived. In addition, the “lipid paradox” [61] may make it difficult to interpret lipid values in RA patients, as they can be lower during active RA and increase with effective treatment. A practical consideration is that many RA patients have not had lipids tested recently, and co-management with primary care physicians may be needed to improve rates of screening for hyperlipidemia [62].

The cohort we used included patients with diabetes or a history of CVD and patients who were receiving statin treatment. Excluding such patients, as some CVD risk calculators do, would have greatly narrowed the utility of the score and reduced the power to see differences in the risk due to other variables. Instead, diabetes and a history of CVD were entered into model building as predictor variables and they were included in the score. Subanalyses demonstrated good fit between predicted and observed CVD events for patients with or without diabetes or a history of CVD. Statin use is not in the MBDA-based CVD risk score because we excluded drug-related variables from model building. However, the risk score demonstrated good fit in subanalyses of patients who were and were not receiving statins. The MBDA-based CVD risk score accounts for the level of inflammation, the treatment of which has potential to reduce CVD risk in RA patients [21,22,23]. The score may have utility for RA patients who are receiving statins because the statin dose may not yet have been optimized and because the non-statin treatment options for elevated CVD risk in RA patients may include DMARDs.

Other RA-specific CVD risk prediction models have been created. The expanded risk scored for CVD in RA (ERS-RA) was derived from a large RA cohort in the USA [19] and has been externally validated [28]. It quantifies RA disease activity categorically with the clinical disease activity index (CDAI) and also includes the Health Assessment Questionnaire (HAQ). A Trans-Atlantic Cardiovascular Risk Consortium for Rheumatoid Arthritis (ATACC-RA) developed two predictors that include serum lipid levels and account for RA disease activity with the 28-joint Disease Activity Score with erythrocyte sedimentation rate (DAS28-ESR) or HAQ, respectively [29]. The MBDA-based CVD risk score requires no clinical measurements and no laboratory data except results from the MBDA test. Rheumatologist preference among these predictors may depend on convenience and on which RA disease activity measures they use most routinely [63, 64]. CVD risk prediction for RA patients could be facilitated in a practical way if a risk score were to be automatically calculated—within an electronic medical record or, in the case of the MBDA-based CVD risk score, when the MBDA score is calculated by the testing laboratory—and provided to the ordering rheumatologist.

The large size of this study was made possible by linking administrative data from the Medicare database to a database of existing MBDA test results. The approach we used to capture CVD endpoint components in the Medicare database has a positive predictive value of approximately ≥ 93% for MI and 80–85% for stroke [37,38,39]. Fatal CVD events were identified using algorithms with positive predictive values ≥ 80% [40]. This study was restricted to patients ≥ 40 years old, to be aligned with the ACC/AHA guidelines [9]. A limitation of having used the Medicare cohort is that it contained predominantly older patients with high rates of CVD risk factors, and most of the 23% of patients < 65 years old were eligible for Medicare because they were disabled. In subanalyses of the patients who were < 65 years old and of patients who had or lacked each of the four clinical risk factors in the model, the MBDA-based CVD risk score had good fit with observed CVD events. In a previous report, CVD risk was relatively similar in younger disabled vs. younger non-disabled RA patients after accounting for the lower prevalence of CVD risk factors [65], suggesting that the MBDA-based CVD risk score may be applicable to patients < 65 years old who are not disabled. However, further validation of the CVD risk score in younger RA patients is needed.

Another limitation of our linked cohort is that clinical practice measurements, such as the blood pressure or lipid levels, were not available and the reasons for ordering MBDA tests were not known. Nevertheless, the MBDA-based CVD risk score demonstrated good fit with observed CVD events in patients with hypertension, hyperlipidemia, history of CVD or statin use, and in patients grouped by level of biomarker-based disease activity or according to whether a biologic DMARD treatment had been initiated or changed during follow-up. Because we lacked clinical measurements, the MBDA-based CVD risk score could not be compared with CVD risk predictors that require them, such as the ACC/AHA Pooled Cohort Equation or the Framingham Risk Score. As an alternative, the MBDA-based CVD risk score was compared to four clinical models of increasing complexity, from an age + sex model to a model that included age, sex, four traditional clinical risk factors available in the Medicare database, and CRP. The MBDA-based CVD risk score showed better fit than all four models, based on LRT. It also demonstrated statistically significantly better NRI and a numerically greater C-index. Because likelihood has been considered the most powerful means for comparing CVD risk prediction tests [66], and C-indices can fail to reflect meaningful incremental contributions of CVD-related biomarkers [67], these results suggest that the MBDA-based CVD risk score may be at least comparable to existing CVD risk calculators and potentially more practical for routine use. Direct comparison with other RA-specific calculators and general population CVD risk calculators adjusted for RA would be of interest.

The 3-year horizon used here for the composite CVD outcome reflects a constraint from the availability and uptake of the MBDA test for routine clinical practice in the US. Of more scientific relevance, however, is that RA is a dynamic disease and disease activity for many patients will fluctuate, such that a single measurement of disease activity may become less associated with true CVD risk over time. Thus, our shorter, 3-year time horizon may be preferable for predicting CVD risk in patients with RA, in that it is less subject to misclassification of RA disease activity than with the 10-year time horizon used by many existing CVD risk calculators. Indeed, the dynamic nature of RA disease activity and other factors that may be important to assessing CVD risk in RA patients is reflected in the ACC/AHA recommendation that, for adult patients with RA, “it can be useful to recheck lipid values and other major ASCVD (atherosclerotic CVD) risk factors 2 to 4 months after the patient’s inflammatory disease has been controlled [11].” Among all specialists, rheumatologists are likely in the best position to assess treatment response and systemic inflammatory burden in RA patients. The MBDA-based CVD risk score may assist rheumatologists by reminding them of the need for CVD risk management in RA patients—which some may wish to co-manage with a primary care physician or cardiologist—and of the unique role rheumatologists have in treating the inflammatory disease component of CVD risk [13].

Conclusions

In conclusion, we have developed and internally validated an MBDA-based CVD risk score that predicts risk for MI, stroke, or fatal CVD in the next 3 years for RA patients. It is novel because it accounts for the contribution of RA inflammatory disease activity by including the MBDA score and three biomarkers that are independently associated with CVD. It performed better than prediction models that used only clinical data. The MBDA-based CVD risk prediction score provides rheumatologists with a feasible tool for assessing CVD risk to inform the management of traditional CVD risk factors and RA inflammation. Further validation with more extended time frames and more heterogeneous cohorts of RA patients will be helpful to assure its robustness as a prediction model.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

ACC/AHA:

American College of Cardiology and American Heart Association

CDAI:

Clinical Disease Activity Index

CI:

Confidence interval

CRP:

C-reactive protein

CVD:

Cardiovascular disease

DAS28-CRP:

Disease Activity Score in 28 joints with CRP

DAS28-ESR:

Disease Activity Score in 28 joints with erythrocyte sedimentation rate

EULAR:

European League Against Rheumatism

FRS:

Framingham Risk Score

LRT:

Likelihood ratio test

MBDA:

Multi-biomarker disease activity

MI:

Myocardial infarction

NRI:

Net Reclassification Index

PY:

Patient-years

RA:

Rheumatoid arthritis

TNFi:

TNF inhibitor

TRIPOD:

Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis

References

  1. 1.

    DeMizio DJ, Geraldino-Pardilla LB. Autoimmunity and inflammation link to cardiovascular disease risk in rheumatoid arthritis. Rheumatol Ther. 2020;7(1):19–33.

    PubMed  Article  Google Scholar 

  2. 2.

    Aviña-Zubieta JA, Choi HK, Sadatsafavi M, Etminan M, Esdaile JM, Lacaille D. Risk of cardiovascular mortality in patients with rheumatoid arthritis: a meta-analysis of observational studies. Arthritis Rheum. 2008;59(12):1690–7.

    PubMed  Article  Google Scholar 

  3. 3.

    Ladak K, Hashim J, Clifford-Rashotte M, Tandon V, Matsos M, Patel A. Cardiovascular risk management in rheumatoid arthritis: a large gap to close. Musculoskeletal Care. 2018;16(1):152–7.

    PubMed  Article  Google Scholar 

  4. 4.

    Crowson CS, Rollefstad S, Ikdahl E, Kitas GD, van Riel P, Gabriel SE, et al. Impact of risk factors associated with cardiovascular outcomes in patients with rheumatoid arthritis. Ann Rheum Dis. 2018;77(1):48–54.

    CAS  PubMed  Article  Google Scholar 

  5. 5.

    Solomon DH, Kremer J, Curtis JR, Hochberg MC, Reed G, Tsao P, et al. Explaining the cardiovascular risk associated with rheumatoid arthritis: traditional risk factors versus markers of rheumatoid arthritis severity. Ann Rheum Dis. 2010;69:1920–5.

    PubMed  PubMed Central  Article  Google Scholar 

  6. 6.

    Agca R, Heslinga SC, Rollefstad S, Heslinga M, McInnes IB, Peters MJ, et al. EULAR recommendations for cardiovascular disease risk management in patients with rheumatoid arthritis and other forms of inflammatory joint disorders: 2015/2016 update. Ann Rheum Dis. 2017;76(1):17–28.

    CAS  PubMed  Article  Google Scholar 

  7. 7.

    Arts EE, Popa C, Den Broeder AA, Semb AG, Toms T, Kitas GD, et al. Performance of four current risk algorithms in predicting cardiovascular events in patients with early rheumatoid arthritis. Ann Rheum Dis. 2015;74(4):668–74.

    CAS  PubMed  Article  Google Scholar 

  8. 8.

    Alemao E, Cawston H, Bourhis F, Al M, Rutten-van Molken M, Liao KP, et al. Comparison of cardiovascular risk algorithms in patients with vs without rheumatoid arthritis and the role of C-reactive protein in predicting cardiovascular outcomes in rheumatoid arthritis. Rheumatology (Oxford). 2017;56(5):777–86.

    Google Scholar 

  9. 9.

    Stone NJ, Robinson JG, Lichtenstein AH, Bairey Merz CN, Blum CB, Eckel RH, et al. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014;63(25 Pt B):2889-2934.

  10. 10.

    Crowson CS, Liao KP, Davis JM 3rd, Solomon DH, Matteson EL, Knutson KL, et al. Rheumatoid arthritis and cardiovascular disease. Am Heart J. 2013;166(4):622–8 e1.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  11. 11.

    Grundy SM, Stone NJ, Bailey AL, Beam C, Birtcher KK, Blumenthal RS, et al. 2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA Guideline on the Management of Blood Cholesterol: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation. 2019;139(25):e1082–e143.

    PubMed  Google Scholar 

  12. 12.

    Goff DC, Jr., Lloyd-Jones DM, Bennett G, Coady S, D'Agostino RB, Sr., Gibbons R, et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014;63(25 Pt B):2935-2959.

  13. 13.

    Arnett DK, Blumenthal RS, Albert MA, Buroker AB, Goldberger ZD, Hahn EJ, et al. 2019 ACC/AHA guideline on the primary prevention of cardiovascular disease: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. Circulation. 2019;140(11):e596–646.

    PubMed  Google Scholar 

  14. 14.

    Ridker PM, Everett BM, Thuren T, MacFadyen JG, Chang WH, Ballantyne C, et al. Antiinflammatory therapy with Canakinumab for atherosclerotic disease. N Engl J Med. 2017;377(12):1119–31.

    CAS  PubMed  Article  Google Scholar 

  15. 15.

    Ridker PM, MacFadyen JG, Everett BM, Libby P, Thuren T, Glynn RJ, et al. Relationship of C-reactive protein reduction to cardiovascular event reduction following treatment with canakinumab: a secondary analysis from the CANTOS randomised controlled trial. Lancet. 2018;391(10118):319–28.

    CAS  PubMed  Article  Google Scholar 

  16. 16.

    Curtis JR, Xie F, Chen L, Saag KG, Yun H, Muntner P. Biomarker-related risk for myocardial infarction and serious infections in patients with rheumatoid arthritis: a population-based study. Ann Rheum Dis. 2018;77(3):386–92.

    CAS  PubMed  Article  Google Scholar 

  17. 17.

    Arts EE, Fransen J, den Broeder AA, Popa CD, van Riel PL. The effect of disease duration and disease activity on the risk of cardiovascular disease in rheumatoid arthritis patients. Ann Rheum Dis. 2015;74(6):998–1003.

    CAS  PubMed  Article  Google Scholar 

  18. 18.

    Mantel A, Holmqvist M, Nyberg F, Tornling G, Frisell T, Alfredsson L, et al. Risk factors for the rapid increase in risk of acute coronary events in patients with new-onset rheumatoid arthritis: a nested case-control study. Arthritis Rheumatol. 2015;67(11):2845–54.

    CAS  PubMed  Article  Google Scholar 

  19. 19.

    Solomon DH, Greenberg J, Curtis JR, Liu M, Farkouh ME, Tsao P, et al. Derivation and internal validation of an expanded cardiovascular risk prediction score for rheumatoid arthritis: a consortium of rheumatology researchers of North America Registry Study. Arthritis Rheumatol. 2015;67(8):1995–2003.

    CAS  PubMed  Article  Google Scholar 

  20. 20.

    Solomon DH, Reed GW, Kremer JM, Curtis JR, Farkouh ME, Harrold LR, et al. Disease activity in rheumatoid arthritis and the risk of cardiovascular events. Arthritis Rheumatol. 2015;67(6):1449–55.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  21. 21.

    Karpouzas GA, Ormseth SR, Hernandez E, Budoff MJ. Biologics may prevent cardiovascular events in rheumatoid arthritis by inhibiting coronary plaque formation and stabilizing high-risk lesions. Arthritis Rheumatol.n/a(n/a).

  22. 22.

    Micha R, Imamura F, Wyler von Ballmoos M, Solomon DH, Hernan MA, Ridker PM, et al. Systematic review and meta-analysis of methotrexate use and risk of cardiovascular disease. Am J Cardiol 2011;108(9):1362–1370.

  23. 23.

    Roubille C, Richer V, Starnino T, McCourt C, McFarlane A, Fleming P, et al. The effects of tumour necrosis factor inhibitors, methotrexate, non-steroidal anti-inflammatory drugs and corticosteroids on cardiovascular events in rheumatoid arthritis, psoriasis and psoriatic arthritis: a systematic review and meta-analysis. Ann Rheum Dis. 2015;74(3):480–9.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  24. 24.

    England BR, Thiele GM, Anderson DR, Mikuls TR. Increased cardiovascular risk in rheumatoid arthritis: mechanisms and implications. BMJ. 2018;361:k1036.

    PubMed  PubMed Central  Article  Google Scholar 

  25. 25.

    Su CC, Chen Ie C, Young FN, Lian IB. Risk of diabetes in patients with rheumatoid arthritis: a 12-year retrospective cohort study. J Rheumatol. 2013;40(9):1513–8.

    PubMed  Article  Google Scholar 

  26. 26.

    Kozera L, Andrews J, Morgan AW. Cardiovascular risk and rheumatoid arthritis--the next step: differentiating true soluble biomarkers of cardiovascular risk from surrogate measures of inflammation. Rheumatology (Oxford). 2011;50(11):1944–54.

    CAS  Article  Google Scholar 

  27. 27.

    Kay J, Morgacheva O, Messing SP, Kremer JM, Greenberg JD, Reed GW, et al. Clinical disease activity and acute phase reactant levels are discordant among patients with active rheumatoid arthritis: acute phase reactant levels contribute separately to predicting outcome at one year. Arthritis Res Therapy. 2014;16(1):R40.

    Article  Google Scholar 

  28. 28.

    Ljung L, Ueda P, Liao KP, Greenberg JD, Etzel CJ, Solomon DH, et al. Performance of the expanded cardiovascular risk prediction score for rheumatoid arthritis in a geographically distant national register-based cohort: an external validation. RMD Open. 2018;4(2):e000771.

    PubMed  PubMed Central  Article  Google Scholar 

  29. 29.

    Crowson CS, Rollefstad S, Kitas GD, van Riel PL, Gabriel SE, Semb AG, et al. Challenges of developing a cardiovascular risk calculator for patients with rheumatoid arthritis. PLoS One. 2017;12(3):e0174656.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  30. 30.

    Curtis JR, van der Helm-van Mil AH, Knevel R, Huizinga TW, Haney DJ, Shen Y, et al. Validation of a novel multibiomarker test to assess rheumatoid arthritis disease activity. Arthritis Care Res (Hoboken). 2012;64(12):1794–803.

    Article  Google Scholar 

  31. 31.

    England BR, Tiong BK, Bergman MJ, Curtis JR, Kazi S, Mikuls TR, et al. 2019 Update of the American College of Rheumatology recommended rheumatoid arthritis disease activity measures. Arthritis Care Res (Hoboken). 2019;71(12):1540–55.

    PubMed  PubMed Central  Article  Google Scholar 

  32. 32.

    Curtis JR, Brahe CH, Ostergaard M, Lund Hetland M, Hambardzumyan K, Saevarsdottir S, et al. Predicting risk for radiographic damage in rheumatoid arthritis: comparative analysis of the multi-biomarker disease activity score and conventional measures of disease activity in multiple studies. Curr Med Res Opin. 2019;35(9):1483–93.

    CAS  PubMed  Article  Google Scholar 

  33. 33.

    van der Helm-van Mil AH, Knevel R, Cavet G, Huizinga TW, Haney DJ. An evaluation of molecular and clinical remission in rheumatoid arthritis by assessing radiographic progression. Rheumatology (Oxford). 2013;52(5):839–46.

  34. 34.

    Curtis JR, Chen L, Bharat A, Delzell E, Greenberg JD, Harrold L, et al. Linkage of a de-identified United States rheumatoid arthritis registry with administrative data to facilitate comparative effectiveness research. Arthritis Care Res (Hoboken). 2014;66(12):1790–8.

    PubMed  PubMed Central  Article  Google Scholar 

  35. 35.

    Kim SY, Servi A, Polinski JM, Mogun H, Weinblatt ME, Katz JN, et al. Validation of rheumatoid arthritis diagnoses in health care utilization data. Arthritis Res Ther. 2011;13(1):R32.

    PubMed  PubMed Central  Article  Google Scholar 

  36. 36.

    Reiss WG, Devenport JN, Low JM, Wu G, Sasso EH. Interpreting the multi-biomarker disease activity score in the context of tocilizumab treatment for patients with rheumatoid arthritis. Rheumatol Int. 2016;36(2):295–300.

    CAS  PubMed  Article  Google Scholar 

  37. 37.

    McCormick N, Lacaille D, Bhole V, Avina-Zubieta JA. Validity of myocardial infarction diagnoses in administrative databases: a systematic review. PLoS One. 2014;9(3):e92286.

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  38. 38.

    Birman-Deych E, Waterman AD, Yan Y, Nilasena DS, Radford MJ, Gage BF. Accuracy of ICD-9-CM codes for identifying cardiovascular and stroke risk factors. Med Care. 2005;43(5):480–5.

    PubMed  Article  Google Scholar 

  39. 39.

    Tirschwell DL, Longstreth WT Jr. Validating administrative data in stroke research. Stroke. 2002;33(10):2465–70.

    PubMed  Article  PubMed Central  Google Scholar 

  40. 40.

    Xie F, Colantonio LD, Curtis JR, Kilgore ML, Levitan EB, Monda KL, et al. Development of algorithms for identifying fatal cardiovascular disease in Medicare claims. Pharmacoepidemiol Drug Saf. 2018;27(7):740–50.

    PubMed  PubMed Central  Article  Google Scholar 

  41. 41.

    Centola M, Cavet G, Shen Y, Ramanujan S, Knowlton N, Swan KA, et al. Development of a multi-biomarker disease activity test for rheumatoid arthritis. PLoS One. 2013;8(4):e60635.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  42. 42.

    Curtis JR, Flake DD, Weinblatt ME, Shadick NA, Ostergaard M, Hetland ML, et al. Adjustment of the multi-biomarker disease activity score to account for age, sex and adiposity in patients with rheumatoid arthritis. Rheumatology (Oxford). 2019;58(5):874–83.

    CAS  Article  Google Scholar 

  43. 43.

    Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD statement. Br J Surg. 2015;102(3):148–58.

    CAS  PubMed  Article  Google Scholar 

  44. 44.

    Karlik B, Olgak A. Performance analysis of various activation functions in generalized MLP architectures of neural networks. Int J Artificial Intelligence Expert Systems. 2010;4:111–22.

    Google Scholar 

  45. 45.

    Therneau TM, Grambsch PM. Modeling survival data: extending the Cox model. Gail M, Samet JM, editors: Springer; 2000.

  46. 46.

    Therneau TM, Lumley T, Atkinson E, Cynthia C. A package for survival analysis in R. R package version 3.1–11.: https://CRAN.R-project.org/package=survival. Accessed 25 Mar 2020; 2020.

  47. 47.

    Breslow N. Discussion on professor Cox’s paper. J R Stat Soc Ser B Methodol. 1972;34(2):216–7.

    Google Scholar 

  48. 48.

    Demler OV, Paynter NP, Cook NR. Tests of calibration and goodness-of-fit in the survival setting. Stat Med. 2015;34(10):1659–80.

    PubMed  PubMed Central  Article  Google Scholar 

  49. 49.

    Pencina MJ, D'Agostino RB Sr, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. 2011;30(1):11–21.

    PubMed  Article  Google Scholar 

  50. 50.

    Leening MJ, Vedder MM, Witteman JC, Pencina MJ, Steyerberg EW. Net reclassification improvement: computation, interpretation, and controversies: a literature review and clinician’s guide. Ann Intern Med. 2014;160(2):122–31.

    PubMed  Article  Google Scholar 

  51. 51.

    Uno H, Cai T, Pencina MJ, D'Agostino RB, Wei LJ. On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Stat Med. 2011;30(10):1105–17.

    PubMed  PubMed Central  Article  Google Scholar 

  52. 52.

    Mogensen UB, Ishwaran H, Gerds TA. Evaluating random forests for survival analysis using prediction error curves. J Stat Softw. 2012;50(11):1–23.

    PubMed  PubMed Central  Article  Google Scholar 

  53. 53.

    Guizani I, Zidi W, Zayani Y, Boudiche S, Hadj-Taieb S, Sanhaji H, et al. Matrix metalloproteinase-3 predicts clinical cardiovascular outcomes in patients with coronary artery disease: a 5 years cohort study. Mol Biol Rep. 2019;46(5):4699–707.

    CAS  PubMed  Article  Google Scholar 

  54. 54.

    Li X, Zhang F, Zhou H, Hu Y, Guo D, Fang X, et al. Interplay of TNF-alpha, soluble TNF receptors and oxidative stress in coronary chronic total occlusion of the oldest patients with coronary heart disease. Cytokine. 2020;125:154836.

    CAS  PubMed  Article  Google Scholar 

  55. 55.

    Valgimigli M, Ceconi C, Malagutti P, Merli E, Soukhomovskaia O, Francolini G, et al. Tumor necrosis factor-ALPHA receptor 1 is a major predictor of mortality and new-onset heart failure in patients with acute myocardial infarction: the Cytokine-Activation and Long-Term Prognosis in Myocardial Infarction (C-ALPHA) study. Circulation. 2005;111(7):863–70.

    CAS  PubMed  Article  Google Scholar 

  56. 56.

    Escalante A, Haas RW, del Rincon I. Paradoxical effect of body mass index on survival in rheumatoid arthritis: role of comorbidity and systemic inflammation. Arch Intern Med. 2005;165(14):1624–9.

    PubMed  Article  Google Scholar 

  57. 57.

    Khan SS, Ning H, Wilkins JT, Allen N, Carnethon M, Berry JD, et al. Association of body mass index with lifetime risk of cardiovascular disease and compression of morbidity. JAMA Cardiol. 2018;3(4):280–7.

    PubMed  PubMed Central  Article  Google Scholar 

  58. 58.

    Baker JF, Billig E, Michaud K, Ibrahim S, Caplan L, Cannon GW, et al. Weight loss, the obesity paradox, and the risk of death in rheumatoid arthritis. Arthritis Rheumatol. 2015;67(7):1711–7.

    PubMed  PubMed Central  Article  Google Scholar 

  59. 59.

    Choi HK, Nguyen U-S, Niu J, Danaei G, Zhang Y. Selection bias in rheumatic disease research. Nat Rev Rheumatol. 2014;10(7):403–12.

    PubMed  PubMed Central  Article  Google Scholar 

  60. 60.

    Saad MF, Damani S, Gingerich RL, Riad-Gabriel MG, Khan A, Boyadjian R, et al. Sexual dimorphism in plasma leptin concentration*. J Clin Endocrinol Metabolism. 1997;82(2):579–84.

    CAS  Google Scholar 

  61. 61.

    Myasoedova E, Crowson CS, Kremers HM, Roger VL, Fitz-Gibbon PD, Therneau TM, et al. Lipid paradox in rheumatoid arthritis: the impact of serum lipid measures and systemic inflammation on the risk of cardiovascular disease. Ann Rheum Dis. 2011;70(3):482–7.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  62. 62.

    Navarro-Millan I, Yang S, Chen L, Yun H, Jagpal A, Bartels CM, et al. Screening of hyperlipidemia among patients with rheumatoid arthritis in the United States. Arthritis Care Res (Hoboken). 2019;71(12):1593–9.

    PubMed  PubMed Central  Article  Google Scholar 

  63. 63.

    Yun H, Chen L, Xie F, Patel H, Boytsov N, Zhang X, et al. Do patients with moderate or high disease activity escalate rheumatoid arthritis therapy according to treat-to-target principles? Results from the rheumatology informatics system for effectiveness registry of the American College of Rheumatology. Arthritis Care Res (Hoboken). 2020;72(2):166–75.

    PubMed  Article  Google Scholar 

  64. 64.

    Curtis JR, Xie F, Yang S, Danila MI, Owensby JK, Chen L. Uptake and clinical utility of multibiomarker disease activity testing in the United States. J Rheumatol. 2019;46(3):237–44.

    PubMed  Article  Google Scholar 

  65. 65.

    Xie F, Crowson C, Navarro-Millan I, Safford M, Curtis JR. Comparing the generalizability of cardiovascular risk in different rheumatoid arthritis cohorts [abstract]. 2019 ACR/ARP annual meeting; November 10, 2019; Atlanta: Arthritis Rheumatol; 2019.

  66. 66.

    Cook NR. Quantifying the added value of new biomarkers: how and how not. Diagnostic and Prognostic Research. 2018;2(1):14.

    PubMed  PubMed Central  Article  Google Scholar 

  67. 67.

    Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115(7):928–35.

    PubMed  Article  Google Scholar 

Download references

Acknowledgements

The authors thank Brooke Hullinger, JD, for her assistance in preparing the figures and tables and editing the manuscript.

Funding

This work was supported by the Myriad Genetics, Inc.

Author information

Affiliations

Authors

Contributions

JC and FX had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study conception and design: JC, FX, CSC, ES, RB-S, AG, DF, BM, and JL. Acquisition of data: JC, FX, AG, DF, and BM. Analysis and interpretation of data: JC, FX, CSC, ES, EH, CLC, RB, RB-S, AG, DF, BM, and JL. All authors were involved drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published.

Authors’ information

Eric Sasso, M.D., is an Affiliate Professor of Medicine (Rheumatology) at the University of Washington, Seattle, WA., USA.

Corresponding author

Correspondence to Jeffrey R. Curtis.

Ethics declarations

Ethics approval and consent to participate

The University of Alabama at Birmingham institutional review board approved the study.

Consent for publication

Not applicable

Competing interests

JC received grants and personal fees from the Abbvie, Amgen, BMS, Corrona, Eli Lilly, Jannsen, Myriad Genetics, Inc., Pfizer, Regeneron, Roche, and UCB during the conduct of the study. FX and CSC received research funding from the Myriad Genetics, Inc., during the conduct of the study. ES, EH, CLC, RB, RB-S, AG, DF, BM, and JL are employed by the Myriad Genetics, Inc., and receive salaries and stock options as compensation.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1: Supplemental Figure 1.

Goodness of fit in patient subgroups (validation dataset, total N=10,275). Supplemental Table 1. Cohort Derivation. Supplemental Table 2. A, Diagnostic codes for candidate variables used to build the MBDA-based CVD risk score and B, Frequencies of CVD-related conditions comprising the History of CVD variable. Supplemental Table 3. Reclassification of patients based on CVD risk predicted by the MBDA-based CVD risk score versus: A, the Age + Sex + CRP model and B, the Clinical model. Supplemental Text. Conversion of the MBDA-based CVD risk score into 3-year percentage risk of a CVD event.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Curtis, J.R., Xie, F., Crowson, C.S. et al. Derivation and internal validation of a multi-biomarker-based cardiovascular disease risk prediction score for rheumatoid arthritis patients. Arthritis Res Ther 22, 282 (2020). https://doi.org/10.1186/s13075-020-02355-0

Download citation

Keywords

  • Biomarker
  • Cardiovascular risk
  • MBDA score
  • Multi-biomarker
  • Myocardial infarction
  • Rheumatoid arthritis
  • Stroke
\