To the Editor:

Since 1992, when the term "evidence-based medicine" first appeared in the medical literature (1), there has been an upsurge in the establishment of practice guidelines that adhere to its principles. The JBJS apparently took the lead in the orthopedic field in emphasizing the "evidence" (2,3). Evidence-based medicine ranks clinical evidence with respect to the influence of biases that beset medical research. However, we want to express our concern regarding a key issue in documenting the "evidence". Thus far, the clinical research reports regarding evidence-based medicine have been based solely on study design; whereas information about the degree of proficiency of professionals who conduct these studies is rarely documented.

To varying extents in all medical practices, treatment outcomes are influenced and biased by the skill with which medical personnel perform procedures. In clinical studies, because experience with implemented techniques is not paralleled by job position, simply categorizing caregivers as residents, attending surgeons, or consultants, etc., provides little, if any, scientific information regarding their expertise in specific techniques adopted in clinical studies. Individuals grouped into one such category often differ in their abilities to perform specific techniques. Because of such differences, in reports of treatments that rely heavily on methodology, the use of technique-performance evaluation criteria is critical for clear, objective interpretation of treatment outcomes. This concept holds particular importance in comparable studies conducted at different institutions or in different geographical areas which exhibit divergences in prevalence of, and consequently physicians exposure to, the study condition.

We proposed a method for the documentation of physicians' technique-experience levels as fitting one of the following categories:

Level I. Non-specialist. A physician in training, or a general physician/surgeon.

Level II. Specialist, less experienced. A specialist who has not yet acquired in-depth knowledge or high-volume experience in the use of the study technique(s). His/her lesser degree of experience can be judged by: 1. a shorter duration of practice as a specialist (i.e. < 5 years), and/or 2. a limited exposure to the investigated disorder.

Level III. Specialist, experienced. A physician who has obtained sufficient experience in the use of the treatment techniques pertinent to the study.

Level IV. Specialist, highly-experienced. A specialist who possesses in-depth knowledge of and experience in the use of techniques under study. He/she has been a leading participant in studies relevant to the disorder or techniques.

Level V. Expert. The pioneer of the technique in the study, or a recognized contributor to the advancement of knowledge and or treatments related to the disorder being investigated.

We believe that documentation of levels of experience would benefit a clinical report. First, doing so would facilitate scientific comparison and appraisal of reported treatment outcomes from different reports. Categorizing levels of experience in a report would allow for scientific analysis of the impact of physicians' capability on treatment outcomes. It may even enable statistical analysis of the relationship of physicians' proficiency levels with treatment outcomes. Second, documentation of levels of experience would merit the appraisal of clinical outcomes reported from different regions of the world. Worldwide the levels of physicians' experience with a disease vary enormously because of the divergence in the natural incidence of the disease in different regions in various parts of the world. Third, the use of criteria to scale levels of experience can indicate how capable study participants are at validating the efficacy of a given treatment procedure. Physicians with lower levels of experience would be less qualified.

We suggest that the expertise levels of the physicians who conduct the treatment be reported, perhaps, under "Methods" with further specification of the percentage of subjects treated by each experience-category, and as a supplement of currently used "Level of Evidence".

The approach we have presented is an example of a much-needed, standardized method for reporting a caregiver's technique-proficiency, an essential element currently missing in the documentation of procedure-based clinical studies. By eliminating a large degree of bias from their interpretation, such a system would enhance the utility of clinical studies in evaluating the efficacy of clinical treatments.

We wish that the JBJS and our colleagues in such a specialty that heavily relies on the proficiency of caregivers put emphasis on this issue and present their technique-proficiency in clinical reports.

The authors did not receive any outside funding or grants in support of their research for or preparation of this work. Neither they nor a member of their immediate families received payments or other benefits or a commitment or agreement to provide such benefits from a commercial entity. No commercial entity paid or directed, or agreed to pay or direct, any benefits to any research fund, foundation, division, center, clinical practice, or other charitable or nonprofit organization with which the authors, or a member of their immediate families, are affiliated or associated.

References

1. Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992;268:2420-5.

2. Hanzlik S, Mahabir RC, Baynosa RC, Khiabani KT. Levels of evidence in research published in The Journal of Bone and Joint Surgery (American Volume) over the last thirty years. J Bone Joint Surg Am. 2009;91:425-8.

3. Heckman JD. Further emphasis on evidence. J Bone Joint Surg Am. 2009;91:1557.