We appreciate the interest in our paper expressed by Mr. Nayagam and Ms.Davies. The method used in our paper is only a problem if the procedures used to identify infection differed among the three groups; however, the procedures used to screen for pin infection were exactly the same among the three groups. There may be different ways to classify pin infections, but so long as the procedures are the same across the study arms, then the study maintains its internal validity.

We agree that some readers may mistakenly attach importance to the number of p- values reported in a paper. The reader must be aware that the sheer number of p-values reported in a paper conveys no special meaning about the quality or importance of a study or its conclusions.

The third point is about the use of the word “moderate” with respect to correlations. Although some statisticians have offered guidelines for the use of the word “moderate” we doubt that there is a strong consensus on the meaning and use of the term in the orthopaedic community, nor do we think that much effort should be expended on standardizing the use of this term. Critical consumers of scientific information form opinions by examining data rather than relying on interpretative or descriptive words.

To The Editor:

We thank the authors for contributing to the subject of pin site-related problems in external fixation but we wish to draw attention to several points which may adversely influence their conclusions.

Whilst the authors are to be congratulated for conducting a randomised controlled trial, issues over the sample size and power(1) and the absence of matching during randomisation need to be raised. The latter has resulted in covariate imbalance as exemplified by a greater number of patients needing open surgery in the Biopatch and hydrogen peroxide groups. The need for open surgery may have been for a variety of reasons but suggest the cases in the samples were not truly comparable.

The definition of what constitutes a clinically relevant pin site infection has also been raised(1). The use of prevalence in the data analysis by the authors asks the question of whether the categories of erythema, cellulitis, drainage, pin or radiographic loosening were mutually exclusive. If not, and pin sites displayed several of these characteristics (as they often do), which criterion defined placement in the various categories of pin complications? Furthermore, if patients experienced more than one episode of infection in the six weeks of external fixation, either in the same group or separate group of pins, were the data recorded as discrete events or grouped? The absence of standardisation in counting episodes of pin site complications may underlie the widely different reported rates. This may occur between different published studies and even between different observers in the same study. We have found the method of data collection traditionally carried out in this field of research to be limiting.

In a study we published in 2005, we offered a different approach which we feel reduces some of the problems of ‘counting’ the event of a pin site infection(2). It avoids classifying degrees of pin site problems until a valid and reliable system has been produced. Whilst it may be argued that this can be artificial, the same has been said of randomized controlled trials(3). We believe it may be a reasonable way of answering questions around pin site sepsis and accommodates the variable of time, which is often ‘hidden’ in more standard analyses.

There are many p values cited in this study. This may lead the casual reader to attach much importance to statistical significance from hypothesis testing. An example is the Spearman rho value quoted for association between co-morbid factors and the categories of pin site complications. These values ranged from r=0.21 to r=0.17, with statistical significance cited for cellulitis and the ASA classification as well as the number of co-morbidities and cellulitis. Both were described as ‘moderate’ correlations. It may be of interest that in each case only 4.4% and 2.9% of the variability in the data respectively could be explained by this association. This belies the description of moderate correlation.

Many professionals (both medical and allied) scan abstracts when searching for evidence in online databases. The firm conclusions expressed in the abstract of this article may lead many to adopt the view reached by the authors, and to hold this as Level 2 evidence. Sadly this misses the methodological problems raised here and elsewhere(1) about this study; it is a case of one step forward but two backwards in the field of pin site related research.

References:

1. Commentary & Perspective by Alexander Y. Shin, MD, and Dirk R. Larson eJBJS http://www.ejbjs.org/cgi/content/full/88/2/349/DC1

2. Davies R, Holt N, Nayagam S. The care of pin sites with external fixation. J Bone Joint Surg Br. 2005;87:716-9.

3. A. Maynard Evidence-based medicine: an incomplete method for informing treatment choices. The Lancet 1997; Volume 349 (Issue 9045): 126-128