June 11, 2014- By Steven E. Greer, MD
When the recent Canadian studied published in the NEJM failed to show any benefit from the WHO surgical checklist championed by Atul Gawande, The Healthcare Channel pointed out that the original Gawande paper was possibly the problem. It was designed poorly, and collected data from non-U.S. countries with little oversight.
Now, in the current online NEJM, letters to the editor are coming in. One writer echoes our concerns about the original Gawande paper.
To the Editor:
The lack of benefit with implementation of the WHO surgical safety checklist in Ontario warrants sober reflection. Many highly publicized efficacy studies have shown improved outcomes with simple interventions, and standard-of-care procedures are often changed on the basis of rapid implementation of their findings. However, attempts to replicate studies often fail or find opposite results.1
This is unsurprising considering the impetus to publish positive results, publication bias in peer-reviewed journals, and statistical explanations for overoptimistic and false findings.2 The revision of evidence that has been embraced has the potential to promote cynicism among clinicians and within society.
Pointing out limitations of initial findings should be the responsibility of unbiased editorials. Although Leape’s editorial mentions the limitations of the Ontario study, it does not address whether the trial that popularized the WHO checklist3 might have shown inflated attributable benefit.
When an intervention has already been implemented and its effectiveness is subsequently challenged, policy reversal and de-implementation might be indicated.
In view of the potential harm of premature implementation, independent replication should be required before new evidence is implemented.
Michael S. Avidan, M.B., B.Ch.
Alex S. Evers, M.D.
Washington University School of Medicine, St. Louis, MO
avidanm@anest.wustl.edu