Can erosions on MRI of the sacroiliac joints be reliably detected in patients with ankylosing spondylitis? - A cross-sectional study
© Weber et al.; licensee BioMed Central Ltd. 2012
Received: 17 December 2011
Accepted: 24 May 2012
Published: 24 May 2012
Erosions of the sacroiliac joints (SIJ) on pelvic radiographs of patients with ankylosing spondylitis (AS) are an important feature of the modified New York classification criteria. However, radiographic SIJ erosions are often difficult to identify. Recent studies have shown that erosions can be detected also on magnetic resonance imaging (MRI) of the SIJ early in the disease course before they can be seen on radiography. The goals of this study were to assess the reproducibility of erosion and related features, namely, extended erosion (EE) and backfill (BF) of excavated erosion, in the SIJ using a standardized MRI methodology.
Four readers independently assessed T1-weighted and short tau inversion recovery sequence (STIR) images of the SIJ from 30 AS patients and 30 controls (15 patients with non-specific back pain and 15 healthy volunteers) ≤45 years old. Erosions, EE, and BF were recorded according to standardized definitions. Reproducibility was assessed by percentage concordance among six possible reader pairs, kappa statistics (erosion as binary variable) and intraclass correlation coefficient (ICC) (erosion as sum score) for all readers jointly.
SIJ erosions were detected in all AS patients and six controls by ≥2 readers. The median number of SIJ quadrants affected by erosion recorded by four readers in 30 AS patients was 8.6 in the iliac and 2.1 in the sacral joint portion (P < 0.0001). For all 60 subjects and for all four readers, the kappa value for erosion was 0.72, 0.73 for EE, and 0.63 for BF. ICC for erosion was 0.79, 0.72 for EE, and 0.55 for BF, respectively. For comparison, the kappa and ICC values for bone marrow edema were 0.61 and 0.93, respectively.
Erosions can be detected on MRI to a comparable degree of reliability as bone marrow edema despite the significant heterogeneity of their appearance on MRI.
Erosions of the sacroiliac joints (SIJ) on pelvic radiographs of patients with ankylosing spondylitis (AS) are an important feature of the modified New York classification criteria . However, SIJ erosions are often difficult to identify on pelvic radiographs and training to recognize radiographic structural changes of the SIJ did not improve the performance of radiologists and rheumatologists in detecting radiographic sacroiliitis . A comparison of SIJ radiographs with computed tomography (CT) scans showed a higher sensitivity of CT scans to detect structural changes indicative of sacroiliitis (86% versus 72%), but the same specificity (84%) . However, assessment of SIJ erosions by CT in clinical practice is limited given two recent reports consistently indicating an association of malignancy with CT of the pelvis [3–5].
Recent studies have shown that erosions can also be detected on magnetic resonance imaging (MRI) of the SIJ early in the disease course before they can be seen on radiography  and that erosions may occur in the absence of bone marrow edema (BME) . In the first report, 59% of non-radiographic spondyloarthritis (SpA) patients showed erosions on MRI in at least two SIJ quadrants . The latter study demonstrated that recognition of erosions on T1-weighted spin echo (T1SE) MRI sequences contributes significantly to diagnostic utility in early SpA and that training to recognize lesions on T1SE MRI improves rheumatologist performance to diagnose SpA on MRI . A recent retrospective analysis confirmed that erosion on SIJ MRI is a highly specific lesion in patients with SpA . However, data on the reliability of detection of SIJ erosion by MRI are scarce [9, 10].
Materials and methods
Characteristics of the study participants and frequency of MRI lesions recorded by ≥2 readers.
AS ≤ 5 years duration
AS > 5/≤10 years duration
Symptom duration, years
HLA-B27 positive, %
CRP level, mg/l
Reading exercise and MRI protocol
Four readers from three rheumatology university hospitals, who were blinded to the diagnosis and patient characteristics, independently assessed semicoronal T1SE and short tau inversion recovery (STIR) sequences of SIJ MRI scans in random order on electronic work stations. These are the sequences used in daily routine for MRI evaluation of SpA patients in the involved institutions. The detailed MRI parameters have been published previously [6, 7]. The reading exercise of SIJ MRI of the 30 AS patients and the 30 non-specific back pain and healthy controls from the original cohort of 187 subjects was conducted two years after the original assessment of these MRI scans.
Standardized assessment of MR images
These newly described features representing the spectrum of abnormalities associated with SIJ erosions on T1SE MRI were added to a revised reference SIJ MR images set developed by consensus among study investigators . This reference image set containing active and structural lesions served to calibrate the reader team. Calibration contained three international video-teleconference sessions using SIJ MRI scans of AS patients not involved in this study population. MR images of the SIJ were assessed according to the standardized methodology outlined in an online training module . This standardizes the assessment of consecutive semicoronal slices through the SIJ from anterior to posterior outlining which are the first anterior and last posterior slices that are scored according to anatomical landmarks. We recorded erosions, bone marrow edema (BME) and fat infiltration (FI) in each SIJ quadrant on each semicoronal slice in a customized online data entry module described previously . EE and BF were assessed per four iliac and sacral joint surfaces irrespective of the number of SIJ slices affected.
At the patient level, we calculated frequencies of AS patients and controls with specific MRI abnormalities reported concordantly by ≥2 readers and also by all four readers. We also calculated the median (interquartile range (IQR)) number of quadrants showing erosion, BME and FI by ≥2 readers and also by all four readers. We repeated this analysis to assess frequencies of MRI lesions according to each of the four joint surfaces (right and left iliac, right and left sacral). We compared the frequencies of lesions observed in the iliac versus the sacral portions of the joint using the Wilcoxon test.
Interobserver reproducibility of specific MRI lesions
We calculated percentage agreement for specific MRI lesions according to positive and negative concordance among the six possible reader pairs for lesions detected at the four iliac and sacral joint surfaces per patient (total = 120 for each group). We also compared concordance separately for the iliac and sacral joint surfaces (total = 60 for each group).
Kappa statistics and intraclass correlation coefficients (ICC) were used to calculate the reproducibility of specific MRI lesions for all four readers jointly and for all 60 study participants (30 AS patients and 30 controls), per iliac and sacral joint portion, and per subject. The inter-reader agreement was defined as slight, fair, moderate, substantial and almost perfect by values of the estimated Cohen's kappa κ < 0.2, 0.2 ≤κ < 0.4, 0.4 ≤κ < 0.6, 0.6 ≤κ < 0.8, and 0.8 ≤κ < 1, respectively . Among the six reported variants of ICC, we report the results of the ICC(3, 1) and ICC(2, 1) model [16–18]. The ICC(3, 1) approach considers the study readers to be a fixed sample and thus not representative of a larger population of raters. In the ICC (2, 1) approach, study readers are considered to be a random sample and, therefore, representative of a larger population of raters. ICC values > 0.4, > 0.6, > 0.8, and > 0.9 were regarded as representing moderate, good, very good, and excellent reproducibility, respectively. For kappa, we provide bootstrap confidence intervals based on 1,000 bootstrap replications. All confidence intervals are computed using a confidence level of 95% and statistical tests are considered significant if the P-value is ≤0.05.
Frequency of MRI lesions in the AS and control groups
Frequency of MRI lesions recorded by four readers jointly and percentage concordance among six possible reader pairs.
P-value Ilium versus Sacrum
AS patients (number = 30)
Frequency: median (IQR)a
% Concordance six RP: total (positive/negative)b
Controls (number = 30)
Frequency: median (IQR)a
% Concordance six RP: total (positive/negative)b
Reproducibility of MRI lesions
Percentage concordance among six possible reader pairs
Among the 30 AS patients, the percentage concordance for the detection of erosions in the iliac joint portion was high (80.0%; positive/negative concordance - 75.0% and 5.0%, respectively) and comparable with the percentage concordance for BME (79.7%; positive/negative concordance - 60.3% and 19.4%, respectively) (Table 2). The percentage concordance in the sacral joint portion, which had a statistically significantly lower frequency of erosions compared with the ilium, showed a lower concordance between the six possible reader pairs for erosion (67.8%) than for BME (90.3%) resulting in a lower percentage concordance also at the patient level (73.9% for erosion versus 85.0% for BME). The lowest percentage concordance was observed for FI, both for the AS and the control group (69.7% and 83.8%, respectively, per patient level). At the patient level, the percentage concordance for EE (75.7%) and BF (80.0%) was similar to erosion (73.9%). The percentage concordance for the control group, where comparatively few lesions were observed, was high both for erosion (92.1%) and BME (89.3%).
Kappa statistics (MRI lesions as binary variables) for four readers jointly
Agreement by kappa statistics and intraclass correlation coefficients by four readers jointly and for 60 study subjects.
Kappa value (95% CI)
0.72 (0.57 to 0.84)
0.67 (0.53 to 0.79)
0.56 (0.42 to 0.70)
0.61 (0.47 to 0.74)
0.64 (0.50 to 0.76)
0.71 (0.57 to 0.82)
0.55 (0.41 to 0.68)
0.45 (0.32 to 0.58)
0.60 (0.46 to 0.72)
0.73 (0.63 to 0.84)
0.68 (0.58 to 0.79)
0.45 (0.35 to 0.56)
0.63 (0.53 to 0.73)
0.60 (0.50 to 0.70)
0.32 (0.22 to 0.42)
ICC(3, 1)/ICC(2, 1)
Intraclass correlation coefficients (MRI lesions as sum scores) for four readers jointly
At the subject level, reader agreement regarding sum scores of the 60 study participants was higher for BME (ICC(3, 1) 0.93 and ICC(2, 1) 0.92, respectively) than for erosion (ICC(3, 1) 0.79 and ICC(2, 1) 0.75, respectively) (Table 3). This difference was also observed for both the iliac and sacral joint portion. Consistent with reader pair percentage concordance and kappa values, assessment of FI also had the lowest reliability according to ICC values (ICC(3, 1) 0.71 and ICC(2, 1) 0.63, respectively). Assessment of EE and BF was less reliable than of erosions overall but assessment for these two lesions was based on evaluation of the four cortical surfaces in the entire joint as compared to the erosion score based on evaluation of eight SIJ quadrants.
This systematic and controlled evaluation of erosion and related features on SIJ MRI in patients with AS had two findings of clinical relevance. First, the reliability of detecting erosion on SIJ MRI was substantial and comparable to BME, provided that readers are trained to recognize abnormalities on T1SE MRI. Familiarity with the variable features of erosions on MRI may also improve reliability. These features include EE and BF. Second, erosions occurred statistically significantly more frequently in the ilium compared with the sacrum. This finding has to be taken into account when comparing various lesions detected on SIJ MRI in AS patients, because BME and FI showed a similar distribution across both joint surfaces.
A debriefing analysis of the reading exercise in the original study population  retrospectively identified EE and BF as major sources of inter-observer disagreement regarding erosion. It is possible that these appearances represent stages in the evolution of an erosive lesion because 15 (78.9%) of 19 AS patients showing BF also showed EE. We, therefore, consider it appropriate to incorporate these features into a comprehensive definition of SIJ erosion. We interpret the signal alteration of BF on T1SE sequences as metaplastic tissue which refills the excavated subchondral bone. For obvious ethical reasons, histopathologic evidence in support of this assumption is not available. Serial MRI assessments of the SIJ over several years may help determine if BF represents an intermediate stage between SIJ erosion and ankylosis. In contrast to rheumatoid arthritis, where regression of erosion has been reported only rarely [19, 20], erosion in many SpA patients may be followed by refilling of the excavated bone and eventually ankylosis.
Two previous reports focused on agreement data for structural MRI lesions of the SIJ [9, 10]. In a cohort of 68 patients with inflammatory back pain according to Calin criteria, the concordance rate for structural lesions defined as a composite index of ankylosis, sclerosis and erosions was 81% and 88%, and kappa values were 0.37 and 0.66, respectively . However, differences in study design preclude a direct comparison with our reliability data. Unlike our study cohort, the study population consisted of patients with inflammatory back pain and only 14 patients (20.6%) met the radiographic modified New York criteria, structural MRI lesions were less frequent (16%), assessment was based on different definitions of MRI lesions, lesions were recorded for the entire right and left SIJ rather than according to joint surface, and there was no control group. Another study which evaluated SIJ MRI in 41 inflammatory back patients meeting the European Spondylarthropathy Study Group (ESSG) criteria , reported an interreader agreement (percentage agreement/kappa value) between two senior radiologists of 77%/0.54 for erosion . The lesion analysis according to eight SIJ quadrants noted a predominance of erosions in the iliac joint portion, but no formal comparison of lesion frequency between the two joint surfaces was performed. Again, there were major differences in study design compared to our work regarding study population, MRI lesion definitions and imaging technique, and lack of a control group.
Our study was conducted with T1SE and STIR sequences representing the routine protocol used for MRI evaluation of SpA patients. Additional so-called 'cartilage MRI sequences', such as T1-weighted fat saturated (T1FS) or T2-weighted gradient echo (T2GE) sequences, may offer advantages to recognize erosion. However, they require further evaluation as to their reliability for detection of erosion compared to T1SE sequence alone and their implementation into routine MRI scanning protocols with regard to examination time and costs . A study using both T1SE and T1FS sequences to detect SIJ erosions in 37 SpA patients meeting the ESSG criteria reported a good inter-observer agreement by kappa statistics between two trained radiologists (0.76 for erosion at joint level and 0.80 at patient level) . Erosion was defined as loss of marrow signal on T1SE and T1FS images together with a defect in the overlaying cortical bone; erosions were scored regarding their extent on the joint surface and the presence of ankylosis was added to the erosion score. However, this study did not directly compare T1SE and T1FS sequences with regard to reliability for detection of SIJ erosions, a separate analysis without the contribution of ankylosis to the erosion score was not performed, and there was no control group. A recent report compared T1FS sequences with two variants of T2GE sequences, three-dimensional -fast low angle shot (3D-FLASH) and three-dimensional-double excitation in the steady state (3D-DESS) sequences, in a retrospective analysis of scans of 30 patients with clinically suspected sacroiliitis and nine healthy controls . There was no difference in the number of erosions detected by all three sequences, but erosion scores based on the extent of joint involvement were significantly higher in both T2GE sequences compared with T1FS sequences. The reliability for detection of erosion by these three 'cartilage MRI sequences' could not be assessed because only one reader evaluated the scans and there was no comparison with the T1SE sequence often used in daily routine.
There are no controlled data on whether CT may have higher sensitivity to detect SIJ erosions compared with MRI. However, two recent studies consistently reported an increased risk of malignancy associated with CT of the pelvis [3–5]. The adjusted lifetime attributable risk of cancer was two cancers per 1,000 exposed women for 20-year-old women who undergo pelvic CT examination . This concern about radiation dose may limit the use of CT to assess SIJ erosions in daily routine and particularly in studies involving healthy controls.
We found a marked difference for erosion kappa values in the iliac versus the sacral joint portion. However, kappa values depend on the prevalence of the finding under observation [24–26]. Kappa values for erosion and BME were virtually identical in the ilium, which may be partly due to a higher occurrence of erosion in this area, comparable to the occurrence of BME. In contrast, erosion kappa values for the sacrum were lower than for BME which occurred more frequently than erosion in the sacrum. Another reason for the difference in detecting erosion between the ilium and sacrum may be different MRI appearances of erosion in the two articular surfaces. Such differences in erosion phenotype may relate to cartilage thickness (which is usually thinner on the iliac compartment), size and depth of erosions, or MRI artifacts, such as chemical shift, which may impair the assessment of subchondral bone [27, 28].
This systematic, standardized, and controlled evaluation of SIJ MRI scans in AS patients demonstrated that the reliability between four readers for detection of erosion on SIJ MRI was substantial and comparable to BME and that, in contrast to BME and FI, erosion occurred significantly more frequently on the iliac side. The spectrum of appearance of erosion on MRI is much more heterogeneous than reported previously and recognition of variants such as 'extended erosion' and 'backfill' may facilitate overall detection of erosion. Moreover, further assessment in prospective studies is required to understand the characteristics of these variants and their role in the evolution of sacroiliitis.
Bath Ankylosing Spondylitis Disease Activity Index
Bath Ankylosing Spondylitis Functional Index 
bone marrow edema
European Spondylarthopathy Study Group
intraclass correlation coefficient
human leucocyte antigen B27
magnetic resonance imaging
numeric rating scale
non-specific back pain
short tau inversion recovery sequence
T1-weighted fat saturated sequence
T1-weighted spin echo sequence
T2-weighted gradient echo sequence.
The authors thank the patients and the healthy volunteers for their participation; Tracey Clare, Clinical Research Manager, and Paul Filipow, Data Manager, Department of Radiology, University of Alberta, Edmonton, Canada, for coordinating the Web-based SPARCC scoring index; Christian Streng, Medical Documentation, Balgrist University Hospital, Zurich, Switzerland, for his technical assistance with Figures 2 to 5. Walter P. Maksymowych is a Scientist of the Alberta Heritage Foundation for Medical Research. This study was supported by the Canadian Arthritis Society, National Research Initiative Award; Alberta Innovates Health Solutions; the Walter L. and Johanna Wolf Foundation, Zurich, Switzerland; and the Foundation for Scientific Research at the University of Zurich, Switzerland. All funding bodies played no role in study design, in the collection and interpretation of data, or in the decision to submit the manuscript for publication.
- Van der Linden S, Valkenburg HA, Cats A: Evaluation of diagnostic criteria for ankylosing spondylitis. A proposal for modification of the New York criteria. Arthritis Rheum. 1984, 27: 361-368. 10.1002/art.1780270401.View ArticlePubMedGoogle Scholar
- Van Tubergen A, Heuft-Dorenbosch L, Schulpen G, Landewé R, Wijers R, van der Heijde D, van Engelshoven J, van der Linden S: Radiographic assessment of sacroiliitis by radiologists and rheumatologists: does training improve quality?. Ann Rheum Dis. 2003, 62: 519-525. 10.1136/ard.62.6.519.PubMed CentralView ArticlePubMedGoogle Scholar
- Berrington de Gonzalez A, Mahesh M, Kim KP, Bhargavan M, Lewis R, Mettler F, Land C: Projected cancer risks from computed tomographic scans performed in the United States in 2007. Arch Intern Med. 2009, 169: 2071-2077. 10.1001/archinternmed.2009.440.View ArticlePubMedGoogle Scholar
- Smith-Bindman R, Lipson J, Marcus R, Kim KP, Mahesh M, Gould R, Berrington de Gonzalez A, Miglioretti DL: Radiation dose associated with common computed tomography examinations and the associated lifetime attributable risk of cancer. Arch Intern Med. 2009, 169: 2078-2086. 10.1001/archinternmed.2009.427.PubMed CentralView ArticlePubMedGoogle Scholar
- Redberg RF: Cancer risks and radiation exposure from computed tomographic scans. How can we be sure that the benefits outweigh the risks? [editorial]. Arch Intern Med. 2009, 169: 2049-2050. 10.1001/archinternmed.2009.453.View ArticlePubMedGoogle Scholar
- Weber U, Lambert RGW, Østergaard M, Hodler J, Pedersen SJ, Maksymowych WP: The diagnostic utility of magnetic resonance imaging in spondylarthritis. An international multicenter evaluation of one hundred eighty-seven subjects. Arthritis Rheum. 2010, 62: 3048-3058. 10.1002/art.27571.View ArticlePubMedGoogle Scholar
- Weber U, Lambert RGW, Pedersen SJ, Hodler J, Østergaard M, Maksymowych WP: Assessment of structural lesions in sacroiliac joints enhances diagnostic utility of magnetic resonance imaging in early spondylarthritis. Arthritis Care Res (Hoboken). 2010, 62: 1763-1771.View ArticleGoogle Scholar
- Wick MC, Weiss RJ, Jaschke W, Klauser AS: Erosions are the most relevant magnetic resonance imaging features in quantification of sacroiliac joints in ankylosing spondylitis. J Rheumatol. 2010, 37: 622-627. 10.3899/jrheum.090602.View ArticlePubMedGoogle Scholar
- Heuft-Dorenbosch L, Weijers R, Landewé R, van der Linden S, van der Heijde D: Magnetic resonance imaging changes of sacroiliac joints in patients with recent-onset inflammatory back pain: inter-reader reliability and prevalence of abnormalities. Arthritis Res Ther. 2006, 8: R11-10.1186/ar1859.PubMed CentralView ArticlePubMedGoogle Scholar
- Puhakka KB, Jurik AG, Egund N, Schiottz-Christensen B, Stengaard-Pedersen K, van Overeem Hansen G, Christiansen JV: Imaging of sacroiliitis in early seronegative spondylarthropathy. Assessment of abnormalities by MRI in comparison with radiography and CT. Acta Radiol. 2003, 44: 218-229. 10.1034/j.1600-0455.2003.00034.x.View ArticlePubMedGoogle Scholar
- Madsen KB, Jurik AG: Magnetic resonance imaging grading system for active and chronic spondyloarthritis changes in the sacroiliac joint. Arthritis Care Res (Hoboken). 2010, 62: 11-18. 10.1002/acr.20008.View ArticleGoogle Scholar
- Garret S, Jenkinson T, Kennedy LG, Whitelock H, Gaisford P, Calin A: A new approach to defining disease status in ankylosing spondylitis: The Bath Ankylosing Spondylitis Disease Activity Index. J Rheumatol. 1994, 21: 2286-2291.Google Scholar
- Maksymowych WP, Dhillon SS, Chiowchanwisawakit P, Pedersen SJ, Martinez B, Østergaard M, Lambert RGW: Development and validation of web-based training modules for systematic evaluation of active inflammatory lesions in the spine and sacroiliac joints in spondyloarthritis. J Rheumatol. 2009, 36: (suppl 84):48-57.Google Scholar
- Arthritis Doctor.ca. [http://www.arthritisdoctor.ca]
- Landis J, Koch G: The measurement of observer agreement for categorical data. Biometrics. 1977, 33: 159-174. 10.2307/2529310.View ArticlePubMedGoogle Scholar
- Shrout PE, Fleiss JL: Intraclass correlation: uses in assessing rater reliability. Psychol Bull. 1979, 86: 420-428.View ArticlePubMedGoogle Scholar
- McGraw KO, Wong SP: Forming inferences about some intraclass correlation coefficients. Psychological Methods. 1996, 1: 30-46.View ArticleGoogle Scholar
- Rousson V, Gasser T, Seifert B: Assessing intrarater, interrater and test-retest reliability of continuous measurements. Stat Med. 2002, 21: 3431-3446. 10.1002/sim.1253.View ArticlePubMedGoogle Scholar
- Sharp JT, van der Heijde D, Boers M, Boonen A, Bruynesteyn K, Emery P, Genant HK, Herborn G, Jurik A, Lassere M, McQueen F, Østergaard M, Peterfy C, Rau R, Strand V, Wassenberg S, Weissman B: Repair of erosions in rheumatoid arthritis does occur. Results from 2 studies by the OMERACT subcommittee on healing of erosions. J Rheum. 2003, 30: 1102-1107.PubMedGoogle Scholar
- Dohn UM, Ejbjerg B, Boonen A, Hetland ML, Hansen MS, Knudsen LS, Hansen A, Madsen OR, Hasselquist M, Moller JM, Østergaard M: No overall progression and occasional repair of erosions despite persistent inflammation in adalimumab-treated rheumatoid arthritis patients: results from a longitudinal comparative MRI, ultrasonography, CT and radiography study. Ann Rheum Dis. 2011, 70: 252-258. 10.1136/ard.2009.123729.View ArticlePubMedGoogle Scholar
- Dougados M, van der Linden S, Juhlin R, Huitfeldt B, Amor B, Calin A, Cats A, Dijkmans B, Olivieri I, Pasero G, Veys E, Zeidler H: The European spondylarthropathy study group preliminary criteria for the classification of spondylarthropathy. Arthritis Rheum. 1991, 34: 1218-1227. 10.1002/art.1780341003.View ArticlePubMedGoogle Scholar
- Maksymowych WP, Weber U: Diagnostic utility of MR in early spondyloarthritis. Curr Rheumatol Rep. 2011, 13: 402-408. 10.1007/s11926-011-0190-x.View ArticlePubMedGoogle Scholar
- Algin O, Gokalp G, Ocakoglu G: Evaluation of bone cortex and cartilage of spondyloarthropathic sacroiliac joint: efficiency of different fat-saturated MR sequences (T1-weighted, 3D-FLASH, and 3D-DESS). Acad Radiol. 2010, 17: 1292-1298. 10.1016/j.acra.2010.05.019.View ArticlePubMedGoogle Scholar
- Hoehler FK: Bias and prevalence effects on kappa viewed in terms of sensitivity and specificity. J Clin Epidemiol. 2000, 53: 499-503. 10.1016/S0895-4356(99)00174-2.View ArticlePubMedGoogle Scholar
- Viera AJ, Garrett JM: Understanding interobserver agreement: the kappa statistic. Fam Med. 2005, 37: 360-363.PubMedGoogle Scholar
- Cunningham M: More than just the kappa coefficient: a program to fully characterize inter-rater reliability between two raters. SAS global forum. 2009, 242-2009.Google Scholar
- McQueen F, Østergaard M, Peterfy C, Lassere M, Ejbjerg B, Bird P, O'Connor P, Genant H, Shnier R, Emery P, Edmonds J, Conaghan P: Pitfalls in scoring MR images in rheumatoid arthritis wrist and metacarpophalangeal joints. Ann Rheum Dis. 2005, 64 (Suppl I): i48-i55.PubMed CentralView ArticlePubMedGoogle Scholar
- Morelli JN, Runge VM, Ai F, Attenberger U, Vu L, Schmeets SH, Nitz WR, Kirsch JE: An image-based approach to understanding the physics of MRI artifacts. RadioGraphics. 2011, 31: 849-866. 10.1148/rg.313105115.View ArticlePubMedGoogle Scholar
- Calin A, Garrett S, Whitelock H, Kennedy LG, O'Hea J, Mallorie P, Jenkinson T: A new approach to defining functional ability in ankylosing spondylitis: the development of the Bath Ankylosing Spondylitis Functional Index. J Rheumatol. 1994, 21: 2281-2285.PubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.