Background: The American edition of The Journal of Bone and
Joint Surgery (JBJS-A) has included a level-of-evidence rating for each
of its clinical scientific papers published since January 2003. The purpose of
this study was to assess the type and level of evidence found in nine
different orthopaedic journals by applying this level-of-evidence rating
system.
Methods: We reviewed all clinical articles published from January
through June 2003 in nine orthopaedic journals. Studies of animals, studies of
cadavera, basic-science articles, review articles, case reports, and expert
opinions were excluded. The remaining 382 clinical articles were randomly
assigned to three experienced reviewers and two inexperienced reviewers, who
rated them with the JBJS-A grading system. Each reviewer determined whether
the studies were therapeutic, prognostic, diagnostic, or economic, and each
rated the level of evidence as I, II, III, or IV. Reviewers were blinded to
the grades assigned by the other reviewers.
Results: According to the reviewers' ratings, 70.7% of the articles
were therapeutic, 19.9% were prognostic, 8.9% were diagnostic, and 0.5% were
economic. The reviewers graded 11.3% as Level I, 20.7% as Level II, 9.9% as
Level III, and 58.1% as Level IV. The kappa values for the interobserver
agreement between the experienced reviewers and the inexperienced reviewers
were 0.62 for the level of evidence and 0.76 for the study type. The kappa
values for the interobserver agreement between the experienced reviewers were
0.75 for the level of evidence and 0.85 for the study type. The kappa values
for the agreement between the reviewers' grades and the JBJS-A grades were
0.84 for the level of evidence and 1.00 for the study type. All kappa values
were significantly different from zero (p < 0.0001 for all). The percentage
of articles that were rated Level I or II increased in accordance with the
2003 journal impact factors for the individual journals (p = 0.0061).
Conclusions: Orthopaedic journals with a higher impact factor are
more likely to publish Level-I or II articles. The type and level of
information in orthopaedic journals can be reliably classified, and clinical
investigators should pursue studies with a higher level of evidence whenever
feasible.