- Open Access
Reply to the comment on “Accuracy and usability of a diagnostic decision support system in the diagnosis of three representative rheumatic diseases: a randomized controlled trial among medical students”
Arthritis Research & Therapy volume 24, Article number: 123 (2022)
With great interest, we read the comment by Gilbert and Wicks on our recent publication  testing the accuracy and usability of Ada’s symptom checker among medical students.
We fully agree with Gilbert et al. that the accuracy and usability of the Ada app is extremely user and use-case dependent. Whereas Ada’s accuracy was extremely high (98%) in the rheumatoid arthritis vignette, it was clearly lower for the other two vignettes (granulomatosis with polyangiitis 43%; systemic lupus erythematosus 51%). In a recent publication  with actual patients, we analyzed in detail the varying perceived usability regarding Ada compared to a similar rheumatology-specific system (Rheport). Usability of both tools was good, although usability of Ada was significantly lower compared to Rheport. Importantly, usability significantly decreased with age.
The authors of the comment state that the app was designed for layperson users to test what underlying disease might be causing their health issues and state that the app explicitly is not developed for health care professionals (HCP). It is noteworthy that the underlying Ada intelligence is identical to Ada’s HCP focused system. Remarkably, the same authors (Gilbert et al., Ada Health GmbH ) published a study in which primary care physicians (GPs) tested 200 clinical vignettes with Ada and other digital symptom assessment apps. Also urgency advice was analyzed. Inspired and analog to that study, we intentionally chose medical students and rheumatology case vignettes (source public online learning center and Rheum2Learn section American college of Rheumatology) with very typical disease symptoms over laypersons to create a “best-case scenario.” Gilbert and Wicks further argue that the used Ada app in our study is not a diagnostic decision support system (DDSS); however, the Ada app provides diagnostic terms upon symptom entry and intentionally recommends urgency advices to support appointments. In our opinion and on the basis of the work by Sutton et al. , apps interpreting or translating symptoms into diagnoses fulfill characteristics of DDSS, but could also be named symptom assessment apps, self-diagnosis tools, technology-supported clinical decision support tools, etc. Nevertheless, arguing over the exact description should not distract from proper utilization of this technology, since we believe that DDSS or similar tools could operate through patients or HCPs as a gatekeeper for optimization of patient flows. Unfortunately, these tools still lack accuracy in certain rheumatologic rare diseases.
Diagnostic decision support system
Health care professionals
Knitza J, Tascilar K, Gruber E, Kaletta H, Hagen M, Liphardt A-M, et al. Accuracy and usability of a diagnostic decision support system in the diagnosis of three representative rheumatic diseases: a randomized controlled trial among medical students. Arthr Res Ther. 2021;23:233.
Knitza J, Muehlensiepen F, Ignatyev Y, Fuchs F, Mohn J, Simon D, et al. Patient’s perception of digital symptom assessment technologies in rheumatology: results from a multicentre study. Front Public Health. 2022;10:844669. https://doi.org/10.3389/fpubh.2022.844669.
Gilbert S, Mehl A, Baluch A, Cawley C, Challiner J, Fraser H, et al. How accurate are digital symptom assessment apps for suggesting conditions and urgency advice? A clinical vignettes comparison to GPs. BMJ Open. 2020;10(12):e040269. https://doi.org/10.1136/bmjopen-2020-040269.
Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020;3:17. https://doi.org/10.1038/s41746-020-0221-y.
Ethics approval and consent to participate
The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Knitza, J., Hueber, A.J. Reply to the comment on “Accuracy and usability of a diagnostic decision support system in the diagnosis of three representative rheumatic diseases: a randomized controlled trial among medical students”. Arthritis Res Ther 24, 123 (2022). https://doi.org/10.1186/s13075-022-02806-w