Skip to main content

A Canadian evaluation framework for quality improvement in childhood arthritis: key performance indicators of the process of care

Abstract

Background

The evaluation of quality of care in juvenile idiopathic arthritis (JIA) is critical for advancing patient outcomes but is not currently part of routine care across all centers in Canada. The study objective is to review the current landscape of JIA quality measures and use expert panel consensus to define key performance indicators (KPIs) that are important and feasible to collect for routine monitoring in JIA care in Canada.

Methods

Thirty-seven candidate KPIs identified from a systematic review were reviewed for inclusion by a working group including 3 pediatric rheumatologists. A shortlist of 14 KPIs was then assessed using a 3-round modified Delphi panel based on the RAND/UCLA Appropriateness Method. Ten panelists across Canada participated based on their expertise in JIA, quality measurement, or lived experience as a parent of a child with JIA. During rounds 1 and 3, panelists rated each KPI on a 1–9 Likert scale on themes of importance, feasibility, and priority. In round 2, panelists participated in a moderated in-person discussion that resulted in minor modifications to some KPIs. KPIs with median scores of ≥ 7 on all 3 questions without disagreement were included in the framework.

Results

Ten KPIs met the criteria for inclusion after round 3. Five KPIs addressed patient assessments: pain, joint count, functional status, global assessment of disease activity, and the clinical Juvenile Arthritis Disease Activity Score (cJADAS). Three KPIs examined access to care: wait times for consultation, access to pediatric rheumatologists within 1 year of diagnosis, and frequency of clinical follow-up. Safety was addressed through KPIs on tuberculous screening and laboratory monitoring. KPIs examining functional status using the Childhood Health Assessment Questionnaire (CHAQ), quality of life, uveitis, and patient satisfaction were excluded due to concerns about feasibility of measurement.

Conclusions

The proposed KPIs build upon existing KPIs and address important processes of care that should be measured to improve the quality of JIA care. The feasibility of capturing these measures will be tested in various data sources including the Understanding Childhood Arthritis Network (UCAN) studies. Subsequent work should focus on development of meaningful outcome KPIs to drive JIA quality improvement in Canada and beyond.

Introduction

Juvenile idiopathic arthritis (JIA) is an inflammatory arthritis that affects approximately 3 in 1000 Canadian children [1]. Timely diagnosis and treatment in a pediatric rheumatology center is key to improving outcomes of patients with JIA. While contemporary treatments have resulted in excellent outcomes for many patients, there can be variability in outcomes with a severe disease course in about 20% of patients in contemporary cohorts [2]. For some with JIA, disease may persist into adulthood [3,4,5]. In a Norwegian study, after 30 years of follow-up, persistence of active disease or medication use was seen in 41% of patients with JIA, with up to 28% having a high symptom state [6]. Consequences of inadequately treated disease include pain, functional limitations due to joint deformities or damage, growth abnormalities, and psychological impacts [7, 8]. Beyond the burden on patients and families, JIA can also be a costly disease. Patients who fail first-line disease-modifying anti-rheumatic drugs (DMARDs) are often treated with biologic therapies, which are highly effective in improving outcomes, but are costly [9]. Indeed, tumor necrosis factor inhibitors (anti-TNF), the biologic DMARDs most often used in JIA, rheumatoid arthritis (RA), and inflammatory bowel disease, are the drug class that accounted for the highest proportion of public drug spending in Canada in 2018 [9].

Variability in practice and care can contribute to suboptimal patient outcomes. Over the last number of decades, there has been increasing interest in measuring and monitoring processes and outcomes of care to reduce unwarranted variability and improve quality of care, defined as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge” [10]. In the USA, the American College of Rheumatology (ACR) published a White Paper on Quality Measurement in 2011 highlighting the quality landscape for rheumatologists and prioritizing areas for measure development [11]. In that White Paper, JIA ranked 3rd as an area for future ACR quality measure development following RA and osteoporosis. Despite this, there currently exist no ACR-endorsed JIA quality measures [12]. Nevertheless, groups such as the Pediatric Rheumatology Care and Outcomes Improvement Network (PR-COIN) in the USA and the British Society for Pediatric and Adolescent Rheumatology (BSPAR) in the UK have proposed measurement tools for quality improvement in JIA [13, 14]. In Canada, the Arthritis Alliance of Canada (AAC), a non-profit group representing 36 arthritis stakeholders’ groups, developed a set of system-level performance measures for inflammatory arthritis including JIA [15]. The measures focused on early access to care and treatment, and 3 of the measures were applicable to JIA. However, the measures addressed only wait times, yearly follow-up, and pediatric rheumatology access within 1 year [15].

The objective of this study was to review the current landscape of JIA quality measures and through expert panel consensus to define key performance indicators (KPIs) that are important and feasible to collect for routine monitoring of JIA care in Canada. The KPIs will be implemented and tested in future studies and will help assess strategies for care improvement. The KPI framework also represents a first step in the evaluation of individual patient and health economic outcomes.

Methods

The evaluation framework was developed over 3 phases using a modification of the RAND Corporation/University of California Los Angeles/University of California Los Angeles (RAND/UCLA) Appropriateness Method [16]. A summary of this process is shown in Fig. 1.

Fig. 1
figure1

Summary of key performance indicators development process

Phase 1: Systematic literature review and search strategy

A systematic search of KPIs for inflammatory arthritis including JIA that has previously been published was updated to ensure the framework was aligned with existing KPIs, quality improvement efforts, and clinical practice guidelines [17]. To identify any new indicators that were developed since the original search, the search strategy was updated in January 2019. A grey literature search was also repeated of select websites (see Additional file 1) to identify any indicators not available in the peer-reviewed literature. To ensure appropriate scope and relevance, the grey literature search was limited to websites that pertained to pediatric rheumatology organizations and organizations that develop and report on quality indicators from Canada, the USA, and the UK. Experts in the field from the PR-COIN network and Canadian Rheumatology Association Quality of Care Committee and the American College of Rheumatology Quality Measures sub-committees were also contacted about unpublished quality measurement frameworks, and the reference lists of included articles were searched for additional sources.

Existing KPI publications were included in our updated review if (1) they were written in English, (2) identified an indicator of JIA quality of care, and (3) the method of indicator development was available. KPIs were excluded if there was no description of how they were selected and/or developed.

JIA guidelines and standards of care were identified using a separate targeted search strategy of pediatric rheumatology organizations in Canada and internationally (see Additional file 2). The guidelines and standards of care were used to (1) identify gaps in measurement when compared to the existing KPIs identified from the systematic review and (2) to ensure that the identified KPIs were supported by established clinical practice guidelines.

Phase 2: Establishing a JIA expert working group

A working group was assembled to guide and oversee the development of the evaluation framework (CB, DM, MT, LS, and NB). Working group members were recruited based on their clinical expertise with JIA (MT, LS, and NB) and/or performance measurement (CB, DM, and MT). No individuals declined the invitation to participate. Members were asked to participate in a series of conference calls to review the project methodology and draft KPIs before presentation to the modified Delphi panel.

Several steps were taken to develop the candidate list of KPIs for the modified Delphi panel (Fig. 1). Working group members reviewed the 37 KPIs identified from the systematic review separately, and then members convened as a group to discuss the exclusion criteria (Table 1). Two members of the team (CB and TP) consolidated the working group feedback to determine the shortlist of candidate KPIs to be discussed by the Delphi panel.

Table 1 Exclusion criteria for the candidate key performance indicators for the modified Delphi panel

Following this process, the working group members were asked to review measurement gaps that were identified when comparing the KPIs to the established guidelines and standards of care for JIA. At this stage, there were a few noted overlaps between KPIs, and additional measures were proposed by the working group to address this issue. In addition, slight modifications to the wording and/or specification of some KPIs were made to ensure concordance with Canadian practice guidelines and standards of care.

Phase 3: Modified Delphi panel

Panelist recruitment

Fourteen JIA stakeholders including 9 pediatric rheumatologists, 2 allied health professionals, and 3 parents of a child with JIA were invited to participate in the modified Delphi panel to select the KPIs in the evaluation framework. Participants are part of the larger Understanding Childhood Arthritis Network (UCAN) CURE team as either collaborators or as part of the patient engagement committee and were selected based on their clinical background as pediatric rheumatologists caring for patients with JIA, professional expertise as allied health professionals in pediatric rheumatology, or personal experience with JIA. To ensure diversity of representation, panelists were recruited from various centers across Canada. Participants did not receive any honoraria or incentives for their participation in the study. The University of Calgary Conjoint Health Research Ethics Board approved this study (REB19-0098).

Modified Delphi panel protocol

The modified Delphi panel consisted of 3 rounds, including 2 rounds of voting using an online survey with an in-person panel discussion in between. In round 1, the modified Delphi panel rated each KPI on a Likert scale from 1 to 9 using the following criteria: (1) does the measure target an important gap in JIA care, (2) how likely is it that the information required to report this measure will be available to health care providers in a typical pediatric rheumatology clinic, 3) overall priority of including this indicator in the evaluation framework. The round 1 survey also included basic demographic questions such as the number of years in practice for health professionals. Panelists were given a background document of the study rationale, methodology, supporting guideline summary, and measurement specifications for reference during the voting process. In round 2, panelists participated in an in-person meeting to review the results from round 1 voting and to discuss any concerns. In round 3, panelists re-rated the KPIs using the same criterion questions as in round 1.

Analysis of panelist responses

Panel ratings for each criterion were categorized into “high” (median scores of 7–9), “uncertain” (median scores of 4–6), and “low” (median scores of 1–3). Median scores of 3.5 or 6.5 were included in the next higher rating category as recommended for panels comprised of an even number of panelists [16]. To be included in the evaluation framework, KPIs had to have median scores of ≥ 7 on all 3 questions with no disagreement among panelists. Disagreement was calculated using the method outlined by the RAND/UCLA Manual [16] and exists when the interpercentile range (difference between the 30th and 70th percentiles) is larger than the Interpercentile Range Adjusted for Symmetry (IPRAS), which was calculated using the following formula: IPRAS = 2.35 + (Asymmetry Index × 1.5).

Results

Phase 1: Systematic literature review and search strategy

The systematic literature search identified 276 articles, of which 9 were reviewed in full text. For consideration in this phase of framework development, only one [19] was included from the update of the search (see Additional file 1 for flow diagram), in addition to two other publications previously identified [15, 20]. Thirty-seven KPIs were abstracted from the 3 articles and categorized into clinically-relevant themes (e.g., assessment, medication). To understand the breadth and depth of the KPIs, they were further classified as structure, process, or outcome indicators based on the Donabedian’s framework [21] and the dimensions of quality according to the Institute of Medicine (IOM) [10], and the Alberta Health Quality Matrix which is based on the IOM framework [22]. Lastly, the indicators were classified according to the AAC Pan-Canadian Model of Care (MOC) (e.g., specialized access to care, medical management) [23]. Several gaps in measures were found when comparing the abstracted KPIs to JIA guidelines and standards of care. These gaps mainly pertained to the use of non-drug therapies, transitional care, therapy assessment, and vaccinations.

Phase 2: Establishing a JIA expert working group

The working group members reviewed the draft KPIs, and a total of 14 candidate KPIs were included after an iterative review process (see Fig. 2). A brief summary of the recommendations and minor wording and specification changes made by the working group members are outlined in Additional file 3. Lastly, the working group members reviewed the measurement gaps (see Additional file 4). The consensus was that these areas of JIA care were challenging to measure and not feasible to assess given the scope and overall objectives of the evaluation framework.

Fig. 2
figure2

Flow diagram of key performance indicators included in modified Delphi panel

Phase 3: Modified Delphi panel

Of the 14 individuals approached, 10 agreed to participate in the modified Delphi panel. The panel consisted of 7 pediatric rheumatologists, 2 allied health professionals, and 1 parent of a child with JIA. There was representation from various provinces across Canada including Alberta, British Columbia, Nova Scotia, Ontario, and Quebec. The number of years in practice for health care providers ranged from 5 to 30 years. All panelists completed rounds 1 and 3 of the modified Delphi panel. Nine panelists participated in the in-person discussion. The panelist who could not participate in the in-person discussion reviewed detailed notes of the meeting and participated in round 3.

After round 1, several KPIs (KPIs 5, 10, 12, 13, and 14) scored in the uncertain range (median score of 4–6) on the feasibility criterion (without disagreement), while other KPIs were rated high (median score of 7–9) on all 3 rating questions. Following round 2 discussions, minor modifications were made to the wording or specification of some of the KPIs based on the feedback from the panelists. These changes are summarized in Additional file 3. After round 3 voting, 10 KPIs met the threshold for inclusion and 4 KPIs were excluded. The results of the panel voting from round 3 and the final included KPIs are shown in Table 2 and Table 3 respectively. The descriptions and reporting of the KPIs are shown in Additional file 5, and the complete specification of each KPI is available upon request.

Table 2 Results of modified Delphi panel (round 3)
Table 3 Final set of key performance indicators

The final set of KPIs examines important processes of JIA care including access to care and the measurement of patient outcomes (Table 3). Panelists voted in KPIs that assessed pain, joint count assessments, functional status, global assessment of disease activity, and the clinical Juvenile Arthritis Disease Activity Score (cJADAS). KPIs for wait time consultation, access to pediatric rheumatologist within 1 year of diagnosis, and follow-up care were included. Safety was addressed through tuberculous screening and laboratory monitoring KPIs. Although KPIs examining functional status using the Childhood Health Assessment Questionnaire (CHAQ) and quality of life were deemed important, there were concerns about the availability of this information in usual clinical practice, leading to the exclusion of these KPIs. Concerns regarding feasibility of measurement also led to the exclusion of KPIs related to uveitis and patient satisfaction.

Discussion

The present study identified 10 KPIs that examine important processes of care for JIA that will be tested in future studies. This work builds upon previous national efforts to develop a measurement framework for monitoring and improving care for patients with inflammatory arthritis that was developed in collaboration with the AAC [15]. The AAC System-Level Performance Measure set included 6 measures that have been used to evaluate early access to care and treatment for inflammatory arthritis, including JIA [24,25,26,27,28]. All 3 of the measures previously included in the AAC measurement set (wait times for JIA care, percentage of patients seen within 1 year of diagnosis, and percentage of patients seen in yearly follow-up) were retained in the current measurement set. While many of these measures build upon existing published measures, some modifications were made to better reflect Canadian practice and/or current guidelines. Furthermore, a new measure was proposed to ensure measurement of disease activity with the cJADAS to optimize efforts in treating JIA to target [29].

Internationally, this work is complementary to a number of national and international efforts to monitor and improve the quality of care for patients living with JIA. For example, PR-COIN is a quality improvement learning network that uses patient registry data to inform quality improvement strategies to optimize processes of care and patient outcomes and includes 20 centers in the USA and Canada [30]. Beyond measurement of processes and outcomes, PR-COIN uses a number of strategies to improve care including pre-visit planning, strategies and toolkits for self-management, and patient/parent engagement [30]. PR-COIN’s quality measurement framework includes several process and outcome measures adapted from a set of 12 measures assessing processes of arthritis care that were also considered in the present study [20]. They were developed by a working group including representatives from the ACR, American Academy of Pediatrics, American Board of Pediatrics, and the Association of Rheumatology Health Professionals [20]. PR-COIN uses sophisticated strategies for audit and feedback based on their measurement framework and improvement science methods to drive change. Importantly, the network sites have been mapping captured data to electronic records to facilitate data entry and automate reporting.

In 2010, the BSPAR developed standards of care for children and young people with JIA that addressed the empowerment of patients, identification of JIA, referral to pediatric rheumatology care, access to a pediatric rheumatology multidisciplinary team, eye screening, access to treatment including joint injections, regular review to target “tight” control strategies, clinical networks and arrangements for shared care, and the care of adolescent patients [31]. An audit in 10 pediatric rheumatology centers using retrospective chart review demonstrated variable adherence to the standards of care including delays in access to care [32]. The audit highlighted a need for consensus on measurable JIA quality indicators, which prompted the development of the BSPAR National Audit Tool for JIA which was funded by the Health Care Quality Improvement Partnership, an independent organization that aims to promote quality in healthcare [19, 33]. The audit tool contains not only quality measures but also prospectively collected patient and care giver reported outcome measures and experience measures (PROMs and PREMs) in a questionnaire. In parallel, a core dataset for JIA was developed with the vision of collecting standardized items in the audit tool to facilitate quality improvement and research efforts [19]. In the present study, items from the BSPAR audit tool were considered for inclusion in the framework; however, none were included in the final set of KPIs.

While our study represents the first Canadian national effort to define KPIs that are feasible and important to measure to drive improvements in processes of care in JIA, the ultimate goal is to improve patient outcomes. We did not, however, include any outcome KPIs in our framework at the present time for several reasons. Firstly, we relied on published quality measures as a starting point for our framework to ensure measures were comparable with other centers. At the time of our study, outcome measures from PR-COIN were not published and available in the public domain for consideration of inclusion [20]. The outcome measures reported from the BSPAR audit tool relied heavily on patient experience or outcome questionnaires, which were deemed less feasible to collect in our healthcare setting. Secondly, agreeing upon and monitoring of the process measures was considered a first step to the evaluation of outcomes and setting benchmarks. Following our study completion, the American College of Rheumatology/Arthritis Foundation guidelines for treatment of JIA were published [34]. These were unfortunately not available for consideration during our study; however, none of the KPIs currently included in our set addresses treatment at this time and it is unlikely based on the KPIs considered by panelists that the guidelines would have altered the final resulting set. In the future, the updated guidelines should be considered when developing additional JIA treatment KPIs. Lastly, there exist a number of challenges for the development of outcome measures including but not limited to outcome attribution, risk adjustment, and defining the period of risk (reviewed in detail by Suter et al. [35]). We therefore propose that this set represent a starter set for data collection and monitoring as the processes required for documenting important patient outcomes including pain, disease activity, and function are included. Future projects would work to define appropriate outcome measures and benchmarks for care.

It should be noted that while all KPIs considered for inclusion in the framework were rated highly by our panel in terms of importance and priority for measurement, there were significant concerns around the feasibility of measurement of some KPIs, which lead to their exclusion. This should not be interpreted as a lack of importance of these measures necessarily; however, this highlights that improved methods of data collection could result in the addition of some of these measures to the framework at a future date. For example, while critical to high-quality JIA care, uveitis screening documentation was felt to be a challenge in many centers, which could have led to falsely low measure reporting and hence lower feasibility ratings by our panelists. Future work should therefore focus on improving reliable ways to track screening in electronic records. There was also skepticism about how patient experience and quality of life questionnaires could be used to improve patient care and/or health leading to the exclusion of measures relating to these concepts. A lack of standardized questionnaires in these domains was also highlighted, as were concerns about length of existing questionnaires and burden to patient and families receiving care and collection of such questionnaires is not currently routinely completed in practice. Further research in the area of patient experience and quality of life can be used to drive clinical improvements and may help inform future performance measurement in this area.

Conclusions

Our KPI framework was developed building upon existing frameworks in the USA and the UK and highlights important processes of JIA care with measures adapted to the Canadian context. The feasibility of capturing these measures will be examined in available data sources, including the data collected prospectively as part of a national study (UCAN CURE: Precision Decisions in Childhood Arthritis). Ultimately, these measures may also inform routine clinical practice and quality reporting. Importantly, the collection of these process KPIs systematically will allow us to evaluate the impact on patient outcomes and inform the development of meaningful outcome KPIs in the future.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

AAC:

Arthritis Alliance of Canada

ACR:

American College of Rheumatology

Anti-TNF:

Anti-tumor necrosis factor inhibitors

BSPAR:

British Society for Paediatric and Adolescent Rheumatology

CHAQ:

Childhood Health Assessment Questionnaire

cJADAS:

Clinical Juvenile Arthritis Disease Activity Score

DMARDs:

Disease-modifying anti-rheumatic drugs

IOM:

Institute of Medicine

IPRAS:

Interpercentile Range Adjusted for Symmetry

JIA:

Juvenile idiopathic arthritis

KPIs:

Key performance indicators

PR-COIN:

Pediatric Rheumatology Care and Outcomes Improvement Network

RAND/UCLA:

RAND Corporation/University of California Los Angeles

UCAN:

Understanding Childhood Arthritis Network

References

  1. 1.

    The Arthritis Society. Childhood arthritis. https://arthritis.ca/about-arthritis/arthritis-types-(a-z)/types/childhood-arthritis. Accessed 17 Dec 2019.

  2. 2.

    Rypdal V, Guzman J, Henrey A, Loughin T, Glerup M, Arnstad ED, et al. Validation of prediction models of severe disease course and non-achievement of remission in juvenile idiopathic arthritis: part 1-results of the Canadian model in the Nordic cohort. Arthritis Res Ther. 2019;21(1):270.

    CAS  Article  Google Scholar 

  3. 3.

    Guzman J, Oen K, Tucker LB, Huber AM, Shiff N, Boire G, et al. The outcomes of juvenile idiopathic arthritis in children managed with contemporary treatments: results from the ReACCh-Out cohort. Ann Rheum Dis. 2015;74(10):1854–60.

    CAS  Article  Google Scholar 

  4. 4.

    Bertilsson L, Andersson-Gare B, Fasth A, Petersson IF, Forsblad-D'elia H. Disease course, outcome, and predictors of outcome in a population-based juvenile chronic arthritis cohort followed for 17 years. J Rheumatol. 2013;40(5):715–24.

    Article  Google Scholar 

  5. 5.

    Glerup M, Rypdal V, Arnstad ED, Ekelund M, Peltoniemi S, Aalto K, et al. Long-term outcomes in juvenile idiopathic arthritis: 18 years of follow-up in the population-based Nordic juvenile idiopathic arthritis (JIA) cohort. Arthritis Care Res (Hoboken). 2019. https://doi.org/10.1002/acr.23853.

  6. 6.

    Selvaag AM, Aulie HA, Lilleby V, Flatø B. Disease progression into adulthood and predictors of long-term active disease in juvenile idiopathic arthritis. Ann Rheum Dis. 2016;75(1):190–5.

    Article  Google Scholar 

  7. 7.

    Tollisen A, Selvaag AM, Aulie HA, Lilleby V, Aasland A, Lerdal A, et al. Physical functioning, pain, and health-related quality of life in adults with juvenile idiopathic arthritis: a longitudinal 30-year followup study. Arthritis Care Res (Hoboken). 2018;70(5):741–9.

    Article  Google Scholar 

  8. 8.

    McCann LJ, Wedderburn LR, Hasson N. Juvenile idiopathic arthritis. Archives Disease Childhood Educ Practice. 2006;91(2):ep29–36.

    Article  Google Scholar 

  9. 9.

    Canadian Institue for Health Information. Prescribed drug spending in Canada, 2018: a focus on public drug programs. Ottawa: CIHI; 2018.

    Google Scholar 

  10. 10.

    Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington (DC): National Academies Press (US). Copyright 2001 by the National Academy of Sciences. All rights reserved.; 2001.

    Google Scholar 

  11. 11.

    Saag KG, Yazdany J, Alexander C, Caplan L, Coblyn J, Desai SP, et al. Defining quality of care in rheumatology: the American College of Rheumatology white paper on quality measurement. Arthritis Care Res (Hoboken). 2011;63(1):2–9.

    Article  Google Scholar 

  12. 12.

    American College of Rheumatology. ACR Endorsed Measures 2016. https://www.rheumatology.org/Practice-Quality/Clinical-Support/Quality-Measurement/ACR-Endorsed-Measures. Accessed 8 Nov 2019.

  13. 13.

    Pediatric Rheumatology Care and Outcomes Improvement Network. About PR-COIN. https://pr-coin.org/about. Accessed 8 Nov 2019.

  14. 14.

    British Society for Paediatric and Adolescent Rheumatology. About BSPAR. https://www.bspar.org.uk/about-bspar/. Accessed 8 Nov 2019.

  15. 15.

    Barber CEH, Marshall DA, Mosher DP, Akhavan P, Tucker L, Houghton K, et al. Development of system-level performance measures for evaluation of models of care for inflammatory arthritis in Canada. J Rheumatol. 2016;43(3):530–40.

    Article  Google Scholar 

  16. 16.

    Fitch K, Bernstein S, Aguilar MD, Burnand B, LaCalle JR, Lazaro P, et al. The RAND/UCLA appropriateness method user’s manual. RAND; 2001.

  17. 17.

    Cooper M, Rouhi A, Barber CEH. A systematic review of quality measures for inflammatory arthritis. J Rheumatol. 2018;45(2):274–83.

    Article  Google Scholar 

  18. 18.

    Canadian Rheumatology Association. Wait time benchmarks for arthritis care. http://www.waittimealliance.ca/benchmarks/arthritis-care/. Accessed 8 Nov 2019.

  19. 19.

    McErlane F, Foster HE, Armitt G, Bailey K, Cobb J, Davidson JE, et al. Development of a national audit tool for juvenile idiopathic arthritis: a BSPAR project funded by the Health Care Quality Improvement Partnership. Rheumatology (Oxford). 2018;57(1):140–51.

    Article  Google Scholar 

  20. 20.

    Lovell DJ, Passo MH, Beukelman T, Bowyer SL, Gottlieb BS, Henrickson M, et al. Measuring process of arthritis care: a proposed set of quality measures for the process of care in juvenile idiopathic arthritis. Arthritis Care Res. 2011;63(1):10–6.

    Article  Google Scholar 

  21. 21.

    Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260(12):1743–8.

    CAS  Article  Google Scholar 

  22. 22.

    Health Quality Council of Alberta. The Alberta Quality Matrix for Health. https://hqca.ca/about/how-we-work/the-alberta-quality-matrix-for-health-1/. Accessed 8 Nov 2019.

  23. 23.

    Arthritis Alliance of Canada. Pan-Canadian approach to IA models of care. http://www.arthritisalliance.ca/en/pan-canadian-approach-to-ia-models-of-care. Accessed 8 Nov 2019.

  24. 24.

    Barber CEH, Schieir O, Lacaille D, Marshall DA, Barnabe C, Hazlewood G, et al. High adherence to system-level performance measures for rheumatoid arthritis in a national early arthritis cohort over eight years. Arthritis Care Res (Hoboken). 2018;70(6):842–50.

    Article  Google Scholar 

  25. 25.

    Barber CEH, Thorne JC, Ahluwalia V, Burt J, Lacaille D, Marshall DA, et al. Feasibility of measurement and adherence to system performance measures for rheumatoid arthritis in 5 models of care. J Rheumatol. 2018;45(11):1501–8.

    Article  Google Scholar 

  26. 26.

    Barber CEH, Lacaille D, Faris P, Mosher D, Katz S, Homik J, et al. Reporting of Arthritis Alliance of Canada (ACC) system-level performance measures for patients with rheumatoid arthritis (RA) in Alberta [abstract]. CRA Annual Scientific Meeting and the AHPA Annual Meeting 2019. https://rheum.ca/wp-content/uploads/2019/03/2019-Poster-Presentations-for-JRheum.pdf. Accessed 10 Nov 2019.

  27. 27.

    Barber CEH, Lix LM, Lacaille D, Marshall DA, Kroeker K, Benseler S, et al. Testing population-based performance measures identifies gaps in juvenile idiopathic arthritis (JIA) care. BMC Health Serv Res. 2019;19(1):572.

    Article  Google Scholar 

  28. 28.

    Marshall DA, Faris P, Mosher D, Katz S, Homik J, Yee K, et al. Regional variation in system-level performance measures for rheumatoid arthritis in Alberta [abstract]. CRA Annual Scientific Meeting and the AHPA Annual Meeting 2019. https://rheum.ca/wp-content/uploads/2019/03/2019-Poster-Presentations-for-JRheum.pdf. Accessed 10 Nov 2019.

  29. 29.

    Ravelli A, Consolaro A, Horneff G, Laxer RM, Lovell DJ, Wulffraat NM, et al. Treating juvenile idiopathic arthritis to target: recommendations of an international task force. Ann Rheum Dis. 2018;77(6):819–28.

    PubMed  Google Scholar 

  30. 30.

    Harris JG, Bingham CA, Morgan EM. Improving care delivery and outcomes in pediatric rheumatic diseases. Curr Opin Rheumatol. 2016;28(2):110–6.

    Article  Google Scholar 

  31. 31.

    Davies K, Cleary G, Foster H, Hutchinson E, Baildam E. BSPAR Standards of Care for children and young people with juvenile idiopathic arthritis. Rheumatology (Oxford). 2010;49(7):1406–8.

    Article  Google Scholar 

  32. 32.

    Kavirayani A, Foster HE. Paediatric rheumatology practice in the UK benchmarked against the British Society for Paediatric and Adolescent Rheumatology/Arthritis and Musculoskeletal Alliance Standards of Care for juvenile idiopathic arthritis. Rheumatology (Oxford). 2013;52(12):2203–7.

    Article  Google Scholar 

  33. 33.

    Health Quality Improvement Partnership. Measuring and improving our healthcare services. https://www.hqip.org.uk/about-us/#.XcSVkVdKiUk. Accessed 8 Nov 2019.

  34. 34.

    Ringold S, Angeles-Han ST, Beukelman T, Lovell D, Cuello CA, Becker ML, et al. 2019 American College of Rheumatology/Arthritis Foundation Guideline for the treatment of juvenile idiopathic arthritis: therapeutic approaches for non-systemic polyarthritis, sacroiliitis, and enthesitis. Arthritis Care Res. 2019;71(6):717–34.

    Article  Google Scholar 

  35. 35.

    Suter LG, Barber CE, Herrin J, Leong A, Losina E, Miller A, et al. American College of Rheumatology white paper on performance outcome measures in rheumatology. Arthritis Care Res (Hoboken). 2016;68(10):1390–401.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

Genome Canada, Ontario Genomics, Genome Alberta, the Ontario Ministry of Economic Development, Job Creation and Trade; The Arthritis Society, Alberta Children’s Hospital Research Institute, University of Calgary, UCAN Utrecht, and the Hospital for Sick Children. DM was funded through Canada Research Chair (Health Systems and Services Research, 2008-2018) and the Arthur J.E. Child Chair in Rheumatology Research. The funding bodies had no role in the design of the study and collection, analysis, and interpretation of the data, and in writing the manuscript.

Author information

Affiliations

Authors

Contributions

C.E.H.B contributed to the study design and acquisition, analysis, and interpretation of data and wrote the manuscript. M.T contributed to the study design and interpretation of data and reviewed the manuscript. T.P contributed to the acquisition, analysis, and interpretation of data and wrote the manuscript. G.C contributed to the study design and interpretation of data and reviewed the manuscript. S.B contributed to the study design and reviewed the manuscript. R.S.M.Y contributed to the study design and reviewed the manuscript. M.B, N.B, J.G, B.L, C.L, D.L, C.O, H.S, G.S, L.S, and K.W contributed to the interpretation of data and reviewed the manuscript. D.A.M contributed to the study design and acquisition, analysis, and interpretation of data and reviewed the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Deborah A. Marshall.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the University of Calgary Conjoint Health Research Ethics Board (REB19-0098).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Select websites searched for grey literature review. Sources and website links for sites searched for grey literature review.

Additional file 2.

Select guidelines or standards of care or recommendations endorsed by various medical societies. Sources and links/references for guidelines, standards of care and recommendations endorsed by various medical societies.

Additional file 3.

Summary of proposed Key Performance Indicators (KPIs), modifications and rationale. Summary of modifications to KPIs and rationale by the working group or Delphi panel.

Additional file 4.

Gaps in measures according to guidelines, standards of care and recommendations for JIA treatment. Gaps in measure that were identified when comparing the measures found through the systematic review to published JIA guidelines, standards of care and recommendations.

Additional file 5.

Final set of KPIs: descriptions and reporting. Full descriptions and reporting details for the final set of KPIs. Is an extension of Table 4 in manuscript.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Barber, C.E.H., Twilt, M., Pham, T. et al. A Canadian evaluation framework for quality improvement in childhood arthritis: key performance indicators of the process of care. Arthritis Res Ther 22, 53 (2020). https://doi.org/10.1186/s13075-020-02151-w

Download citation

Keywords

  • Juvenile idiopathic arthritis
  • Quality improvement
  • Quality of care
  • Quality indicators