JBJS | Interobserver Reliability and Intraobserver Reproducibility of the System of King et al. for the Classification of Adolescent Idiopathic Scoliosis*

The Journal of Bone & Joint Surgery, Volume 80, Issue 8

Articles | August 01, 1998

Interobserver Reliability and Intraobserver Reproducibility of the System of King et al. for the Classification of Adolescent Idiopathic Scoliosis*

R. JAY CUMMINGS, M.D.†; ERIC A. LOVELESS, M.D.†; JOSEPH CAMPBELL, M.D.‡; STEPHEN SAMELSON, M.D.§; JOHN M. MAZUR, M.D.†, JACKSONVILLE, FLORIDA

View Disclosures and Other Information

Investigation performed at Nemours Children's Clinic, Jacksonville

The Journal of Bone & Joint Surgery. 1998; 80:1107-11

5 Recommendations (Recommend) | 3 Comments | Saved by 3 Users Save Case

text A A A

Abstract

The classification of adolescent idiopathic scoliosis with use of the system of King et al. has become widely accepted since its introduction. The purpose of the present study was to establish the interobserver reliability and intraobserver reproducibility of this classification system.The preoperative radiographs of sixty-three patients who were managed operatively for adolescent idiopathic scoliosis were classified by five observers with the system of King et al. Interobserver reliability was assessed by comparison of the classification of the curves among the observers, and intraobserver reproducibility was evaluated by comparison of the classifications of each set of radiographs by each observer on two occasions three weeks apart.The median interobserver reliability kappa coefficient for the classification system of King et al. was 0.44 (range, 0.28 to 0.50), and the median intraobserver reproducibility kappa coefficient was 0.64 (range, 0.44 to 0.72).According to the definition of Landis and Koch, the classification system of King et al. is substantially reproducible but is only moderately reliable. However, according to the stricter definition of Svanholm et al., its reproducibility is only fair and its reliability is poor.

Figures in this Article

Introduction

Introduction | Materials and Methods | Results | Discussion | References

In 1905, Schulthess described a system for the classification of scoliotic curves, and for some time most authors used that system when discussing idiopathic scoliosis. In 1983, King et al. presented a new classification with the intention of enabling surgeons to determine which curves should be treated with arthrodesis in patients who have multiple curves. According to this system, a type-I curve is an s-shaped curve in which the lumbar curve is larger and less flexible than the thoracic curve, a type-II curve is an s-shaped curve in which the thoracic curve is larger and less flexible than the lumbar curve, a type-III curve is a single thoracic curve in which no compensatory lumbar curve crosses the midline, a type-IV curve is a long thoracic curve in which the fourth lumbar vertebra tilts into the thoracic curve, and a type-V curve is a double thoracic curve. Since its publication, this classification has been used in many articles and scientific presentations to describe idiopathic scoliotic curves. The purpose of the present study was to evaluate the interobserver reliability and intraobserver reproducibility of the classification system of King et al.

*No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article. No funds were received in support of this study.

†Nemours Children's Clinic, 807 Nira Street, Jacksonville, Florida 32207.

‡Department of Orthopaedics, Naval Hospital, Camp Le Jeune, North Carolina 28547.

§42nd Medical Group/SGSO, 3030 Kirkpatrick Avenue, Maxwell Air Force Base, Alabama 36112.

*No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article. No funds were received in support of this study.

†Nemours Children's Clinic, 807 Nira Street, Jacksonville, Florida 32207.

‡Department of Orthopaedics, Naval Hospital, Camp Le Jeune, North Carolina 28547.

§42nd Medical Group/SGSO, 3030 Kirkpatrick Avenue, Maxwell Air Force Base, Alabama 36112.

View Large | Download Slide(.ppt)
Add to My JBJS

Figs. 1-A, 1-B, and 1-C: The curve in this patient was classified as type III by one scoliosis specialist and both orthopaedic residents and as type V by the other scoliosis specialist. The orthopaedist who did not manage patients with scoliosis classified the curve as type II. Fig. 1-A: Posteroanterior radiograph.

View Large | Download Slide(.ppt)
Add to My JBJS

Fig. 1-B: Right-side-bending radiograph.

View Large | Download Slide(.ppt)
Add to My JBJS

Fig. 1-C: Left-side-bending radiograph.

View Larger

TABLE I FREQUENCY OF ASSIGNMENT OF EACH TYPE OF CURVE*

*The values are given as the number of curves, with the percentage in parentheses.

Type of Curve	First Observation (N = 315)	Second Observation (N = 315)
I	39 (12)	46 (15)
II	107 (34)	96 (31)
III	71 (23)	70 (22)
IV	56 (18)	60 (19)
V	36 (11)	33 (11)
VI (unclassifiable)	6 (2)	10 (3)

Add to My JBJS

TABLE I FREQUENCY OF ASSIGNMENT OF EACH TYPE OF CURVE*

*The values are given as the number of curves, with the percentage in parentheses.

Type of Curve	First Observation (N = 315)	Second Observation (N = 315)
I	39 (12)	46 (15)
II	107 (34)	96 (31)
III	71 (23)	70 (22)
IV	56 (18)	60 (19)
V	36 (11)	33 (11)
VI (unclassifiable)	6 (2)	10 (3)

View Larger

TABLE II AGREEMENT AMONG THE OBSERVERS WITH REGARD TO CLASSIFICATION OF THE SIXTY-THREE CURVES*

*The values are given as the number of curves, with the percentage in parentheses.

Agreement	First Observation (N = 63)	Second Observation (N = 63)
All 5 observers	15 (24)	16 (25)
At least 4 observers	32 (51)	35 (56)
At least 3 observers	52 (83)	54 (86)

Add to My JBJS

TABLE II AGREEMENT AMONG THE OBSERVERS WITH REGARD TO CLASSIFICATION OF THE SIXTY-THREE CURVES*

*The values are given as the number of curves, with the percentage in parentheses.

Agreement	First Observation (N = 63)	Second Observation (N = 63)
All 5 observers	15 (24)	16 (25)
At least 4 observers	32 (51)	35 (56)
At least 3 observers	52 (83)	54 (86)

View Larger

TABLE III INTEROBSERVER RELIABILITY

*Type VI is not part of the system of King et al. †The values for the Type-VI curves are not included.

Type of Curve	Percentage of Simple Agreement (N = 63)	Kappa Coefficient	P Value
I	51	0.44	<0.001
II	66	0.50	<0.001
III	53	0.39	<0.001
IV	59	0.50	<0.001
V	36	0.28	<0.001
VI (unclassifiable)*	6	0.04	>0.100
Median†	53	0.44

Add to My JBJS

TABLE III INTEROBSERVER RELIABILITY

*Type VI is not part of the system of King et al. †The values for the Type-VI curves are not included.

Type of Curve	Percentage of Simple Agreement (N = 63)	Kappa Coefficient	P Value
I	51	0.44	<0.001
II	66	0.50	<0.001
III	53	0.39	<0.001
IV	59	0.50	<0.001
V	36	0.28	<0.001
VI (unclassifiable)*	6	0.04	>0.100
Median†	53	0.44

View Larger

TABLE IV INTRAOBSERVER REPRODUCIBILITY

Observer	Percentage of Simple Agreement (N = 63)	Kappa Coefficient
1	56	0.44
2 (specialist)	73	0.64
3 (specialist)	79	0.72
4	73	0.64
5	70	0.60
Median	73	0.64

Add to My JBJS

TABLE IV INTRAOBSERVER REPRODUCIBILITY

Observer	Percentage of Simple Agreement (N = 63)	Kappa Coefficient
1	56	0.44
2 (specialist)	73	0.64
3 (specialist)	79	0.72
4	73	0.64
5	70	0.60
Median	73	0.64

View Larger

TABLE V INTEROBSERVER RELIABILITY FOR THE MODIFIED CLASSIFICATION

Type of Curve	Percentage of Simple Agreement (N = 63)	Kappa Coefficient	P Value
I	61	0.55	<0.00001
II	67	0.52	<0.00001
III & IV	79	0.61	<0.00001
V	36	0.29	<0.00001
Median	64	0.54	<0.00001

Add to My JBJS

TABLE V INTEROBSERVER RELIABILITY FOR THE MODIFIED CLASSIFICATION

Type of Curve	Percentage of Simple Agreement (N = 63)	Kappa Coefficient	P Value
I	61	0.55	<0.00001
II	67	0.52	<0.00001
III & IV	79	0.61	<0.00001
V	36	0.29	<0.00001
Median	64	0.54	<0.00001

View Larger

TABLE VI DEFINITIONS OF ACCEPTABILITY ACCORDING TO KAPPA COEFFICIENT

Kappa Coefficient	Acceptability
Svanholm et al.
<0.50	Poor
0.50—0.75	Fair
>0.75	Good to excellent

Landis and Koch
0.0—0.20	Slight
0.21—0.40	Fair
0.41—0.60	Moderate
0.61—0.80	Substantial
0.81—1.0	Excellent

Add to My JBJS

TABLE VI DEFINITIONS OF ACCEPTABILITY ACCORDING TO KAPPA COEFFICIENT

Kappa Coefficient	Acceptability
Svanholm et al.
<0.50	Poor
0.50—0.75	Fair
>0.75	Good to excellent

Landis and Koch
0.0—0.20	Slight
0.21—0.40	Fair
0.41—0.60	Moderate
0.61—0.80	Substantial
0.81—1.0	Excellent

View Larger

TABLE VII INTEROBSERVER RELIABILITY AND INTRAOBSERVER REPRODUCIBILITY FOR SEVERAL ORTHOPAEDIC CLASSIFICATION SYSTEMS

Study	Classification System	Interobserver Kappa Coefficient	Intraoberver Kappa Coefficient
Ward et al.	Severin	0.16	0.34
Thomsen et al.	Lauge-Hansen	0.49	0.64
	Weber	0.58	0.68
Sidor et al.	Neer	0.48	0.66
Siebenrock and Gerber	Neer	0.40	0.60
	AO/ASIF shoulder fracture	0.53 (types); 0.42 (groups)	0.58 (types); 0.48 (groups)
Roberts et al.	Arthrographic classification of Legg-Perthes disease	0.62	0.57
	Herring	0.36	0.60
	Catterall	0.27	0.39
Lenke et al.⁷	King et al.	0.49	0.61
Present study	King et al.	0.44	0.62

Add to My JBJS

TABLE VII INTEROBSERVER RELIABILITY AND INTRAOBSERVER REPRODUCIBILITY FOR SEVERAL ORTHOPAEDIC CLASSIFICATION SYSTEMS

Study	Classification System	Interobserver Kappa Coefficient	Intraoberver Kappa Coefficient
Ward et al.	Severin	0.16	0.34
Thomsen et al.	Lauge-Hansen	0.49	0.64
	Weber	0.58	0.68
Sidor et al.	Neer	0.48	0.66
Siebenrock and Gerber	Neer	0.40	0.60
	AO/ASIF shoulder fracture	0.53 (types); 0.42 (groups)	0.58 (types); 0.48 (groups)
Roberts et al.	Arthrographic classification of Legg-Perthes disease	0.62	0.57
	Herring	0.36	0.60
	Catterall	0.27	0.39
Lenke et al.⁷	King et al.	0.49	0.61
Present study	King et al.	0.44	0.62

Materials and Methods

Introduction | Materials and Methods | Results | Discussion | References

Two postgraduate fourth-year orthopaedic residents, a pediatric orthopaedist who was not actively involved in the care of patients with scoliosis, and two pediatric orthopaedists who were actively involved in the management of patients with scoliosis and who routinely used the classification system of King et al. participated in this study. All of the observers independently reviewed a description of the classification by King et al. before the study, and a copy of the article was available in the room where the radiographs were viewed. No tutorial on the use of the classification was held before the study because we believed that the provision of such a tutorial would not reflect how most physicians learn the classification system.

The five observers independently classified the preoperative posteroanterior radiographs, made with the patient standing, and the left and right-side-bending radiographs, made with the patient supine (Figs. 1-A, 1-B, and 1-C), of sixty-three randomly selected patients who had adolescent idiopathic scoliosis. The radiographs for each patient were placed in a numbered folder. All of the radiographs for each patient were viewed simultaneously. To minimize fatigue, the observers were given the radiographs of thirty patients first, and once those had been classified they were given the radiographs of the remaining thirty-three patients. Three weeks after the sixty-three sets of radiographs had been classified, each observer repeated the classification exercise. All identifying data on the radiographs were obscured. The radiographs were arranged in random order and were rearranged for the second viewing.

Statistical Analysis

Interobserver reliability was assessed by comparison of the classifications of the curves among the five observers. A chance-corrected, unweighted agreement index ? was calculated for each observer². The kappa coefficient adjusts for the proportion of agreement between or among observers by correcting for the proportion of agreement that could have occurred by chance. Kappa coefficients range from +1.0 (complete agreement) to less than 0.0 (less agreement than that expected by chance). A value of 0.0 indicates that agreement is no better than that expected by chance. Similarly, the intraobserver reproducibility for each type of curve at the two viewings across all observers was evaluated with use of the kappa coefficient.

Results

Introduction | Materials and Methods | Results | Discussion | References

The frequency with which each type was used to classify a curve ranged from thirty-three (11 per cent) of 315 for type V to 107 (34 per cent) of 315 for type II (Table I). Compared with the study by King et al., a slightly lower percentage of curves in the present study were classified as type III (seventy-one [23 per cent] of 315 compared with 133 [33 per cent] of 405) and a slightly higher percentage were classified as type IV (fifty-eight [18 per cent] of 315 compared with thirty-seven [9 per cent] of 405). The percentages of curves classified as type I, II, or V were similar in the two studies.

All five observers in the present study agreed on the classification of fifteen (24 per cent) of the sixty-three curves at the first evaluation and that of sixteen curves (25 per cent) at the second evaluation. At least three of the five observers agreed on the classification of fifty-two curves (83 per cent) at the first evaluation and fifty-four curves (86 per cent) at the second evaluation (Table II).

The overall median interobserver reliability kappa coefficient for the classification system of King et al. was 0.44. The type-II and type-IV curves had the highest kappa coefficient (? = 0.50 for both), and the type-V curves had the lowest kappa coefficient (? = 0.28) (Table III). It should be noted that the p values for the kappa coefficients for each group to some extent depend on the number of curves in that group. The mean interobserver reliability kappa coefficient was 0.28 for the two observers who were actively involved in the management of patients with scoliosis and 0.41 for the two residents and the pediatric orthopaedist who did not see patients with scoliosis. This difference was not found to be significant with the numbers available.

The overall median intraobserver reproducibility kappa coefficient was 0.64. The highest reproducibility kappa coefficient for an observer was 0.72 and the lowest was 0.44 (Table IV). The mean intraobserver reproducibility kappa coefficient was 0.68 for the two observers who were actively involved in the management of patients with scoliosis and 0.56 for the two residents and the pediatric orthopaedist who did not see patients with scoliosis. This difference was not found to be significant with the numbers available.

Discussion

Introduction | Materials and Methods | Results | Discussion | References

The results of this study will probably disappoint individuals who have relied on the classification system of King et al. for the determination of levels of arthrodesis for idiopathic curves and for investigators who have used this classification when reporting the results of studies of idiopathic scoliosis. The median interobserver reliability kappa coefficient of 0.44 is considered poor acceptability and the median intraobserver reproducibility kappa coefficient of 0.64 is considered fair acceptability according to the definitions of Svanholm et al.

We decided to test the effect on reliability of simplifying the classification. The type-III and IV classifications were combined because they both define single curves. As a result, the observers would not have been required to make a decision as to the relative flexibility of two different curves when they evaluated these curves. We found that combining these two types improved the reliability only slightly (? = 0.54) (Table V).

It would be interesting to be able to compare the results of this study with those of a similar study performed when the Harrington rod was in use. The introduction of instrumentation systems that were stronger than the Harrington rod was accompanied by occasional decompensation of apparently flexible lumbar curves caudad to and thoracic curves cephalad to right thoracic curves that had been treated with instrumentation^5,6. It might therefore be postulated that, with our increased ability to correct a curve with instrumentation, concern about the capacity for compensation by the apparently flexible curves cephalad or caudad to it when those curves have not been included in the instrumentation might lead to indecision regarding the rating of type-I, II, and V curves according to the classification system of King et al. Our finding that the type-V curves had the lowest interobserver reliability kappa coefficient tends to support that conclusion. However, our finding that the type-II curves had the highest interobserver kappa coefficient argues against that conclusion.

The number of observers and the number of subjects being evaluated with the classification system are important factors to consider when reliability and reproducibility studies are compared. For example, it was not possible to demonstrate a significant difference with regard to the reliability of the more experienced observers compared with that of the less experienced observers in the present study because the greater number of less experienced observers (ratio, 1.5 to one) made it easier for them to agree more often. As far as we know, Lenke et al.⁷ are the only other investigators to have studied the reliability and reproducibility of the classification system of King et al. There were seven observers in that study. Almost twice as many curves were reclassified by the five observers in the present study as were reclassified by the six observers in the reproducibility part of the study by Lenke et al.

All seven of the observers in the study by Lenke et al.⁷ were spine specialists, while, in general, the observers in our study had less experience. We believe that the inclusion of only observers who are frequent users of the classification system being studied results in a best-case analysis of the problem and that the results might not be valid for individuals who use the system less frequently. Comparison of the median interobserver reliability reported by Lenke et al. (? = 0.49) with that reported in our study (? = 0.44) suggests that scoliosis specialists do not agree on classifications derived with the system of King et al. much more often than non-specialists do. Therefore, it appears that an observer's reliability does not increase with increased use of the system. Our inability to detect a significant difference between the intraobserver reproducibility for spine specialists and that for non-specialists suggests that intraobserver reproducibility also may not improve with experience. A study involving more spine specialists and non-specialists is needed to determine if these findings are significant.

Another difference between the methodology used by Lenke et al.⁷ and that used by us is that the curves in their study were assigned a type by one of the creators of the classification system (King) and agreement was determined on the basis of that assignment. It appears to us that, as a result of that methodology, their study determined how well the observers agreed with the individual who made the assignments rather than how reliable the classification system is. An individual's classifications can be used as the so-called gold standard only if that individual's reproducibility is perfect or at least markedly better than that of all of the other observers. Unfortunately, it is not clear if King participated in the reproducibility portion of the study by Lenke et al. In our study, the curves were not preclassified by one individual. Still, despite the differences in the number of observers, the number of curves classified, the level of experience of the observers, and the methodology, our study and that of Lenke et al. revealed very similar levels of reliability and reproducibility for the classification system of King et al.

In recent years, a number of investigators have examined the reliability and reproducibility of various classification systems used in orthopaedic practice^{7,8,10,11,13,14}. It was frequently found that the classification system being examined was not as precise as had been assumed. In an editorial on fracture classification systems, Burstein expressed a reluctance to publish studies that relied on classification systems that had not been shown to be valid. It is important to realize that statisticians do not agree completely as to what kappa coefficient should be considered acceptable for medical classification systems (Table VI)^4,12 and that agreement that is better than fair (as defined by Svanholm et al.) or moderate (as defined by Landis and Koch) is uncommon for the orthopaedic classification systems studied to date (Table VII). Thus, what constitutes acceptable agreement is a matter of perspective and the perceived need for precision.

In summary, the fair-to-moderate reliability⁴ of the classification system of King et al. does not appear sufficient to allow surgeons to make decisions regarding whether to operate on the basis of the results reported by other surgeons with use of the classification. Clearly, a more reliable system is needed for that purpose. If one accepts the definition of acceptability proposed by Landis and Koch, the classification system of King et al. demonstrates moderate-to-substantial reproducibility with regard to a surgeon's ability to formulate an operative decision on the basis of his or her experience with other patients whom he or she has classified with the system.

We agree that editors should be reluctant to accept manuscripts concerning studies that relied on classification systems that have not been shown to be reliable and reproducible. We think that a uniform editorial definition of acceptable reliability and reproducibility levels is desirable. It appears that if orthopaedists are to continue to use any of the orthopaedic classification systems that have been developed, the more lenient definition of acceptability proposed by Landis and Koch will need to be adopted. We hope that, as investigators devote more attention to the reliability and reproducibility of the classification systems that they develop, the more stringent definition of acceptability proposed by Svanholm et al. can be achieved.

References

Introduction | Materials and Methods | Results | Discussion | References

Burstein, A. H.: Editorial. Fracture classification systems: do they work and are they useful?. J. Bone and Joint Surg.,75-A: 1743-1744, Dec. 1993.75-A1743 1993

Cohen, J. A.: A coefficient of agreement for nominal scales. Educat. and Psychol. Measure.,20: 37-46, 1960.2037 1960

King, H. A.; Moe, J. H.; Bradford, D. S.; and Winter, R. B.: The selection of fusion levels in thoracic idiopathic scoliosis. J. Bone and Joint Surg.,65-A: 1302-1313, Dec. 1983.65-A1302 1983

Landis, J. R., and Koch, G. G.: The measurement of observer agreement for categorical data. Biometrics,33: 159-174, 1977.33159 1977 [PubMed]

Lee, C. K.; Denis, F.; Winter, R. B.; and Lonstein, J. E.: Analysis of the upper thoracic curve in surgically treated idiopathic scoliosis. A new concept of the double thoracic curve pattern. Spine,18: 1599-1608, 1993.181599 1993 [PubMed]

Lenke, L. G.; Bridwell, K. H.; Baldus, C.; and Blanke, K.: Preventing decompensation in King type II curves treated with Cotrel-Dubousset instrumentation. Strict guidelines for selective thoracic fusion. Spine,17: 274-S281, 1992.17274 1992

Lenke, L.; Betz, R.; Bridwell, K.; Clements, D.; Harms, J.; King, H.; Lowe, T.; and Shufflebarger, H.: Classification of adolescent idiopathic scoliosis by the King classification method: is it reliable? Read at the Annual Meeting of the Scoliosis Research Society, Ottawa, Ontario, Canada, Sept. 26, 1996.

Roberts, J. M.; Ain, M.; and Abromowitz, A.: Arthrographic classification of Legg-Perthes disease. Read at the Annual Meeting of the Pediatric Orthopaedic Society of North America, Phoenix, Arizona, May 13, 1996.

Schulthess, W.: Die Pathologie und Therapie der Rückgratsverkrümmungen. In Handbuch der orthopädischen Chirurgie, edited by G. Joachimsthal. Vol. 1, part 2. Jena, Germany, Gustav Fischer, 1905-1907.

Sidor, M.; Zuckerman, J. D.; Lyon, T.; Koval, K.; Cuomo, F.; and Schoenberg, N.: The Neer classification system for proximal humeral fractures. An assessment of interobserver reliability and intraobserver reproducibility. J. Bone and Joint Surg.,75-A: 1745-1750, Dec. 1993.75-A1745 1993

Siebenrock, K. A., and Gerber, C.: The reproducibility of classification of fractures of the proximal end of the humerus. J. Bone and Joint Surg.,75-A: 1751-1755, Dec. 1993.75-A1751 1993

Svanholm, H.; Starklint, H.; Gundersen, H. J.; Fabricius, J.; Barlebo, H.; and Olsen, S.: Reproducibility of histomorphologic diagnoses with special reference to the kappa statistics. APMIS: Acta Pathol., Microbiol., Immunol. Scandinavica,97: 689-698, 1989.97689 1989

Thomsen, N. O. B.; Overgaard, S.; Olsen, L. H.; Hansen, H.; and Nielsen, S. T.: Observer variation in the radiographic classification of ankle fractures. J. Bone and Joint Surg.,73-B(4): 676-678, 1991.73-B(4)676 1991

Ward, W. T.; Vogt, M.; Grudziak, J. S.; Tümer, Y.; Cook, P. C.; and Fitch, R. D.: Severin classification system for evaluation of the results of operative treatment of congenital dislocation of the hip. A study of intraobserver and interobserver reliability. J. Bone and Joint Surg.,79-A: 656-663, May 1997.79-A656 1997

Topics

View Large | Download Slide(.ppt)
Add to My JBJS

Fig. 1-B: Right-side-bending radiograph.

View Large | Download Slide(.ppt)
Add to My JBJS

Fig. 1-C: Left-side-bending radiograph.

View Larger

TABLE I FREQUENCY OF ASSIGNMENT OF EACH TYPE OF CURVE*

*The values are given as the number of curves, with the percentage in parentheses.

Type of Curve	First Observation (N = 315)	Second Observation (N = 315)
I	39 (12)	46 (15)
II	107 (34)	96 (31)
III	71 (23)	70 (22)
IV	56 (18)	60 (19)
V	36 (11)	33 (11)
VI (unclassifiable)	6 (2)	10 (3)

Add to My JBJS

TABLE I FREQUENCY OF ASSIGNMENT OF EACH TYPE OF CURVE*

*The values are given as the number of curves, with the percentage in parentheses.

Type of Curve	First Observation (N = 315)	Second Observation (N = 315)
I	39 (12)	46 (15)
II	107 (34)	96 (31)
III	71 (23)	70 (22)
IV	56 (18)	60 (19)
V	36 (11)	33 (11)
VI (unclassifiable)	6 (2)	10 (3)

View Larger

TABLE II AGREEMENT AMONG THE OBSERVERS WITH REGARD TO CLASSIFICATION OF THE SIXTY-THREE CURVES*

*The values are given as the number of curves, with the percentage in parentheses.

Agreement	First Observation (N = 63)	Second Observation (N = 63)
All 5 observers	15 (24)	16 (25)
At least 4 observers	32 (51)	35 (56)
At least 3 observers	52 (83)	54 (86)

Add to My JBJS

TABLE II AGREEMENT AMONG THE OBSERVERS WITH REGARD TO CLASSIFICATION OF THE SIXTY-THREE CURVES*

*The values are given as the number of curves, with the percentage in parentheses.

Agreement	First Observation (N = 63)	Second Observation (N = 63)
All 5 observers	15 (24)	16 (25)
At least 4 observers	32 (51)	35 (56)
At least 3 observers	52 (83)	54 (86)

View Larger

TABLE III INTEROBSERVER RELIABILITY

*Type VI is not part of the system of King et al. †The values for the Type-VI curves are not included.

Type of Curve	Percentage of Simple Agreement (N = 63)	Kappa Coefficient	P Value
I	51	0.44	<0.001
II	66	0.50	<0.001
III	53	0.39	<0.001
IV	59	0.50	<0.001
V	36	0.28	<0.001
VI (unclassifiable)*	6	0.04	>0.100
Median†	53	0.44

Add to My JBJS

TABLE III INTEROBSERVER RELIABILITY

*Type VI is not part of the system of King et al. †The values for the Type-VI curves are not included.

Type of Curve	Percentage of Simple Agreement (N = 63)	Kappa Coefficient	P Value
I	51	0.44	<0.001
II	66	0.50	<0.001
III	53	0.39	<0.001
IV	59	0.50	<0.001
V	36	0.28	<0.001
VI (unclassifiable)*	6	0.04	>0.100
Median†	53	0.44

View Larger

TABLE IV INTRAOBSERVER REPRODUCIBILITY

Observer	Percentage of Simple Agreement (N = 63)	Kappa Coefficient
1	56	0.44
2 (specialist)	73	0.64
3 (specialist)	79	0.72
4	73	0.64
5	70	0.60
Median	73	0.64

Add to My JBJS

TABLE IV INTRAOBSERVER REPRODUCIBILITY

Observer	Percentage of Simple Agreement (N = 63)	Kappa Coefficient
1	56	0.44
2 (specialist)	73	0.64
3 (specialist)	79	0.72
4	73	0.64
5	70	0.60
Median	73	0.64

View Larger

TABLE V INTEROBSERVER RELIABILITY FOR THE MODIFIED CLASSIFICATION

Type of Curve	Percentage of Simple Agreement (N = 63)	Kappa Coefficient	P Value
I	61	0.55	<0.00001
II	67	0.52	<0.00001
III & IV	79	0.61	<0.00001
V	36	0.29	<0.00001
Median	64	0.54	<0.00001

Add to My JBJS

TABLE V INTEROBSERVER RELIABILITY FOR THE MODIFIED CLASSIFICATION

Type of Curve	Percentage of Simple Agreement (N = 63)	Kappa Coefficient	P Value
I	61	0.55	<0.00001
II	67	0.52	<0.00001
III & IV	79	0.61	<0.00001
V	36	0.29	<0.00001
Median	64	0.54	<0.00001

View Larger

TABLE VI DEFINITIONS OF ACCEPTABILITY ACCORDING TO KAPPA COEFFICIENT

Kappa Coefficient	Acceptability
Svanholm et al.
<0.50	Poor
0.50—0.75	Fair
>0.75	Good to excellent

Landis and Koch
0.0—0.20	Slight
0.21—0.40	Fair
0.41—0.60	Moderate
0.61—0.80	Substantial
0.81—1.0	Excellent

Add to My JBJS

TABLE VI DEFINITIONS OF ACCEPTABILITY ACCORDING TO KAPPA COEFFICIENT

Kappa Coefficient	Acceptability
Svanholm et al.
<0.50	Poor
0.50—0.75	Fair
>0.75	Good to excellent

Landis and Koch
0.0—0.20	Slight
0.21—0.40	Fair
0.41—0.60	Moderate
0.61—0.80	Substantial
0.81—1.0	Excellent

View Larger

TABLE VII INTEROBSERVER RELIABILITY AND INTRAOBSERVER REPRODUCIBILITY FOR SEVERAL ORTHOPAEDIC CLASSIFICATION SYSTEMS

Study	Classification System	Interobserver Kappa Coefficient	Intraoberver Kappa Coefficient
Ward et al.	Severin	0.16	0.34
Thomsen et al.	Lauge-Hansen	0.49	0.64
	Weber	0.58	0.68
Sidor et al.	Neer	0.48	0.66
Siebenrock and Gerber	Neer	0.40	0.60
	AO/ASIF shoulder fracture	0.53 (types); 0.42 (groups)	0.58 (types); 0.48 (groups)
Roberts et al.	Arthrographic classification of Legg-Perthes disease	0.62	0.57
	Herring	0.36	0.60
	Catterall	0.27	0.39
Lenke et al.⁷	King et al.	0.49	0.61
Present study	King et al.	0.44	0.62

Add to My JBJS

TABLE VII INTEROBSERVER RELIABILITY AND INTRAOBSERVER REPRODUCIBILITY FOR SEVERAL ORTHOPAEDIC CLASSIFICATION SYSTEMS

Study	Classification System	Interobserver Kappa Coefficient	Intraoberver Kappa Coefficient
Ward et al.	Severin	0.16	0.34
Thomsen et al.	Lauge-Hansen	0.49	0.64
	Weber	0.58	0.68
Sidor et al.	Neer	0.48	0.66
Siebenrock and Gerber	Neer	0.40	0.60
	AO/ASIF shoulder fracture	0.53 (types); 0.42 (groups)	0.58 (types); 0.48 (groups)
Roberts et al.	Arthrographic classification of Legg-Perthes disease	0.62	0.57
	Herring	0.36	0.60
	Catterall	0.27	0.39
Lenke et al.⁷	King et al.	0.49	0.61
Present study	King et al.	0.44	0.62

Burstein, A. H.: Editorial. Fracture classification systems: do they work and are they useful?. J. Bone and Joint Surg.,75-A: 1743-1744, Dec. 1993.75-A1743 1993

Cohen, J. A.: A coefficient of agreement for nominal scales. Educat. and Psychol. Measure.,20: 37-46, 1960.2037 1960

King, H. A.; Moe, J. H.; Bradford, D. S.; and Winter, R. B.: The selection of fusion levels in thoracic idiopathic scoliosis. J. Bone and Joint Surg.,65-A: 1302-1313, Dec. 1983.65-A1302 1983

Landis, J. R., and Koch, G. G.: The measurement of observer agreement for categorical data. Biometrics,33: 159-174, 1977.33159 1977 [PubMed]

Siebenrock, K. A., and Gerber, C.: The reproducibility of classification of fractures of the proximal end of the humerus. J. Bone and Joint Surg.,75-A: 1751-1755, Dec. 1993.75-A1751 1993

Accreditation Statement

These activities have been planned and implemented in accordance with the Essential Areas and policies of the Accreditation Council for Continuing Medical Education (ACCME) through the joint sponsorship of the American Academy of Orthopaedic Surgeons and The Journal of Bone and Joint Surgery, Inc. The American Academy of Orthopaedic Surgeons is accredited by the ACCME to provide continuing medical education for physicians.

CME Activities Associated with This Article

Submit a Comment

Please read the other comments before you post yours. Contributors must reveal any conflict of interest.
Comments are moderated and will appear on the site at the discretion of JBJS editorial staff.

* = Required Field

Comment Author(s) * (if multiple authors, separate names by comma)

Example: John Doe

Affiliation & Institution*

Comment Title*

Comment*

Cancel

Print
PDF
Email

Recipient(s) will receive an email with a link (good for 5 Days) to 'Interobserver Reliability and Intraobserver Reproducibility of the System of King et al. for the Classification of Adolescent Idiopathic Scoliosis*' and do not need to have The Journal of Bone & Joint Surgery account to access the content.
Your Name:*
Example: John Doe

Email Address:* CC Me:
Enter your valid email address. Example: jdoe@example.com

Recipient's Email Address:*

Separate multiple email address with semi-colons (up to 5).

Subject:* 's The Journal of Bone & Joint Surgery: 'Interobserver Reliability and Intraobserver Reproducibility of the System of King et al. for the Classification of Adolescent Idiopathic Scoliosis*'
Subject for your email.

Message:

(Optional, message will truncate at 1000 characters)

Processing your request... Please Wait...

Copyright © in the material you requested is held by The Journal of Bone & Joint Surgery, Inc. (unless otherwise noted). This email ability is provided as a courtesy, and by using it you agree that that you are requesting the material solely for personal, non-commercial use, and that it is subject to Journal of Bone & Joint Surgery, Inc.'s Terms of Use. The information provided in order to email this topic will not be used to send unsolicited email, nor will it be furnished to third parties. Please refer to The Journal of Bone & Joint Surgery Inc.'s Privacy Policy for further information.

Copyright © The Journal of Bone & Joint Surgery Inc. All rights reserved.
Share
Get Citation

R. JAY CUMMINGS, ERIC A. LOVELESS, JOSEPH CAMPBELL, STEPHEN SAMELSON, JOHN M. MAZUR; Interobserver Reliability and Intraobserver Reproducibility of the System of King et al. for the Classification of Adolescent Idiopathic Scoliosis*. The Journal of Bone & Joint Surgery. 1998 Aug;80(8):1107-11.

Download citation file:

RIS (Zotero)

EndNote

BibTex

Medlars

ProCite

RefWorks

Reference Manager

Copyright © The Journal of Bone and Joint Surgery, Inc.
Rights & Permissions
Sign Up for Alerts on this
Article

Processing your request... Please Wait.
Submit a Comment