Population, data source, and study design
The study population included US Veterans enrolled in the VARA registry [7,8,9,10], a prospective, observational registry involving 11 Veterans Affairs (VA) medical centers. DAM components are recorded during routine visits using templated notes. The DAMs are extracted from medical notes stored in the VA Corporate Data Warehouse (CDW) [11] using validated extraction algorithms or data entered manually into the VARA registry database.
Patient data extracted from the CDW include pharmacy, laboratory, outpatient diagnoses, and electronic medical notes. Patient demographics, disease history, and duration of RA were collected from VARA enrollment data. Serologic samples collected to assess rheumatoid factor (RF) and anti-cyclic citrullinated peptide antibodies (ACPA) were assayed at a central laboratory on enrollment into the VARA registry. An additional chart review was performed to collect any data not identified in the CDW or VARA database.
A historical cohort design was used to compare the clinical response between patient visits with and without MTC. The unit of observation was an eligible patient visit to a rheumatology clinic during the study period (January 1, 2006, to September 30, 2017). Each eligible visit (i.e., rheumatology visits with documented core clinical measures to compute DAS28, CDAI, and RAPID3) with 18 months of enrollment and 2 rheumatology visits with DAS28 during the previous 18 months was classified as having a MTC or no MTC. The study included a baseline measurement period (18 months before the eligible visit) to measure covariates and potential confounders and an exposure period (7 days prior to 30 days after the eligible visit) to assess if a MTC occurred. The 7-day pre-visit exposure period was selected to identify any interventions that may have occurred immediately prior to the visit, (e.g., steroid dose escalation via telephone call or electronic message), and the 30-day post-visit period was designed to capture interventions that started at the visit. An outcome period (2–6 months after the eligible visit) was used to identify patients who achieved an ACR20 response (Supplemental Fig. S1).
Visit eligibility criteria
Eligible visits were identified for patients meeting the following criteria: enrolled in VARA registry, ≥ 18 years of age, rheumatology visit with all components of DAMs (DAS28, CDAI, RAPID3), documented (referenced as an eligible patient visit), ≥ 18 months of enrollment in VA health care system prior to the eligible visit, and 2 rheumatology visits with documented DAS28 scores during the 18-month baseline period ≥ 60 days apart from each other and ≥ 60 days before the eligible patient visit (to measure disease stability) (Supplemental Fig. S1). The key exclusion criteria included active cancer, organ transplant, diagnosis of other autoimmune disorders (e.g., systemic lupus erythematosus), any surgical procedure within 90 days after the eligible visit, or any hospitalization within 30 days of the eligible visit.
Youden Index and empirical decision threshold
The Youden Index is a measure of diagnostic accuracy that is used to identify optimal thresholds that discriminate a dichotomous outcome from a continuous scale [7]. The Youden Index has traditionally been used to identify optimal cut points for diagnostic tests. In this analysis, the Youden Index was used to identify the DAM value that maximized the correct classification of MTC where equal weighting was given to sensitivity and specificity.
The Youden Index (J) was calculated for each cut point/threshold (c), i.e., every value of the DAM.
$$ J(c)=\mathrm{sensitivity}(c)+\mathrm{specificity}(c)-1 $$
The goal of this analysis was to maximize J to identify the optimal cut point where c represents the set of candidate cut points/thresholds:
$$ {c}^{\mathrm{opt}}=\arg\ {\max}_{c\in C}J(c) $$
Measurements
Exposure: MTC
MTC has been previously defined [5, 6]. Briefly, a visit was associated with a MTC if (1) a new disease-modifying antirheumatic drug (DMARD) was initiated (including switching agents within the same drug class) either as a new agent or after a 90-day gap following the last date of prior therapy, (2) DMARD dose was escalated by ≥ 25%, (3) prednisone was initiated, (4) monthly average prednisone dose increased by 25%, and (5) and/or intra-articular injection of ≥ 2 with corticosteroids.
Outcome: clinical improvement measured by ACR20 response criteria
An ACR20 response was defined as improvement of 20% in both tender and swollen joint counts and 20% improvement in 3 of the ACR core disease activity measures (patient assessment of pain, patient global assessment of disease activity, physician global assessment, patient assessment of physical function, and acute-phase reactant laboratory value) [12]. ACR20 response was chosen to measure the treatment effects because it is a validated and common outcome measure in clinical trials, has standardized outcome assessments across the DAMs, and can detect clinical response to treatment in a time frame consistent with routine follow-up care (~ 3 months) [13, 14]. A window of 2–6 months after the index visit to document outcomes was used to account for variability in observed visit intervals and reduce the risk of exposure misclassification due to subsequent treatment modification. If multiple visits with documented core clinical measures were observed during the follow-up period, data from the visit closest to 3 months after the index visit were used.
Covariates: potential confounders between MTC and ACR20 response
Covariate adjustment was used to remove confounding between MTC and ACR20 response. Potential confounders included demographic characteristics, duration of RA, level of disease activity, Rheumatic Disease Comorbidity Index (RDCI) [15], disease stability [5], DMARD use at baseline, and MTC within 90 days of the eligible visit.
The standard criteria have been established to classify disease activity into remission, low, moderate, and high disease activity (Supplemental Table 1) [16,17,18]. For this analysis, we also evaluated disease activity stratification that included a division of moderate disease activity based on Youden thresholds. With this method, categories of disease activity included the following: (1) remission and low disease activity for DAS28 (< 3.2), CDAI (< 10.0), and RAPID3 (< 2.0); (2) low-moderate (lower bound of moderate disease to the Youden-identified threshold) for DAS28 (3.20–4.02), CDAI (10.0–12.9), and RAPID3 (2.00–3.81); (3) high-moderate (greater than Youden-identified threshold to the high bound of moderate disease activity) for DAS28 (4.03–5.10), CDAI (13.0–22.0), and RAPID3 (3.82–4.0); and (4) high disease activity for DAS28 (> 5.1), CDAI (> 22.0), and RAPID3 (> 4.0).
Types of MTC were descriptively analyzed based on category: changes in oral prednisone (initiating medication, restarting medication after a gap, and/or increase in medication dose), intra-articular corticosteroid injections, changes in bDMARD, and changes in csDMARD.
Estimating the impact of MTC on ACR20 response
Crude (bivariate) associations between MTC and ACR20 response were represented by risk difference (RD) and risk ratio (RR) with 95% confidence intervals (CIs). Impacts of MTC on ACR20 response were further evaluated using G-computation [19, 20] for the marginal and disease activity level conditional effects. The population average generalized estimating equation (GEE) model with an exchangeable correlation structure [21] was used with the G-computation approach to account for within-patient correlation, as multiple visits per patient were possible. Since the G-computation approach allowed us to predict potential outcomes for the entire population under both treatment conditions (with MTC and without MTC), we first built a model using a complete case analysis (i.e., in visits during the follow-up window with documented core measures), and then applied this model to the full population, including those with missing data, to estimate potential outcomes for every patient. We computed 95% CIs using a bootstrapping method, in which the random sampling (1000 samples) was done with replacement [19].
G-computation models were fit using patient age at visit, sex, race, ACPA status, RF status, disease duration, RDCI score, DAM stability (worsening or not), csDMARDs, bDMARDs, and prednisone dispensed in the month prior to visit and in the previous year, and the baseline MTC (MTC during previous 90 days). An interaction term was used to evaluate how the effect of MTC on ACR20 response was modified by different levels of disease activity. The marginal effect (overall effect) was produced by averaging the differences between the potential outcomes under MTC and the potential outcomes under no MTC, accounting for the fact that treatment effects vary across disease activity levels.
The probability of ACR20 response was shown to be independent of follow-up month when conditioning on MTC and levels of disease activity [6]; we therefore did not adjust for the follow-up interval in our ACR20 response model and used the G-computation models to estimate the population-level effects under the assumption of no loss to follow-up.
Descriptive statistics included the number of observations and percentages for dichotomous and continuous variables, and the number of observations, means, standard deviations (SDs), and 95% CIs [11] for continuous variables. We used several statistical software packages for these analyses, including Microsoft SQL Server, SAS version 9.4, and Enterprise Guide version 7.1. Data preparation and statistical analyses were conducted using Stata 14.
