It is considered one of the insidious problems in clinical research: Researchers tend to publish the results of trials that show an intervention works, while not even submitting those that don't. As a result, systematic reviews of the literature to determine the efficacy of a particular treatment or preventive measure are likely to be biased, and doctors may end up prescribing useless or even harmful medications to their patients. Now, the editors of 100 journals around the world have proposed a novel way to deal with this problem.
Editorials in this week's issues of the British Medical Journal (BMJ) and The Lancet call for "an amnesty" for unpublished trials. The idea, explains Richard Smith, editor of the BMJ, is to get researchers to register the existence of trials they have completed but never published, and post the registry on the World Wide Web. Then, when other researchers perform a systematic review of the literature, they can track down the unpublished results to see if they should be included in the review.
The amnesty idea was initiated by Ian Roberts, director of the Child Health Monitoring Unit at The Institute of Child Health in England. Roberts has been reviewing the effectiveness of interventions in the treatment of brain and spinal cord injuries for the Cochrane Collaboration (Science, 5 April 1996, p. 22), an international network of medical researchers who prepare, maintain, and disseminate systematic reviews of the effectiveness of treatments and preventive measures. He realized that unpublished results could skew his analysis. "It's quite clear from the work we've been doing," says Roberts, "that unpublished trials pose a major threat to the validity of systematic reviews."
Other epidemiologists have come to similar conclusions. Over the past decade, says University of Maryland epidemiologist Kay Dickersin, researchers have done five studies assessing the percentage of clinical trials that are completed but never published. "The final publication rate varies from 50% to 90%," says Dickersin, "and on average it is probably closer to 50." David Naylor, a clinical epidemiologist at Ontario's Institute of Clinical Evaluative Sciences, says the bulk of unpublished studies will simply be inconclusive. But he agrees that researchers are more likely to publish results that suggest a treatment works than that it doesn't, resulting in "an unduly inflated and excessively precise estimate of the treatment effectiveness in meta-analyses."
The amnesty idea will be discussed at the International Conference on Biomedical Peer Review, a gathering of journal editors from around the world being held this week in Prague, Czech Republic. Although the editors of 100 journals have already endorsed the proposal, Smith says he hopes others will sign on and help disseminate the call for unpublished work throughout the world. "We want to make it as easy as possible for people to be able to say 'Well yes, this study did happen.' And anybody can report one of these trials. It doesn't have to be the person who did it. It can be any trial you know about that's unreported. We're quite prepared to have the same trial reported more than once."
The idea has not won universal approval in the biomedical publishing world, however. The editor of one journal that is not involved suggested it would "encourage a vast gemische of junk." He questions the logic of inviting studies that have never been peer reviewed to be included in systematic reviews. Even Naylor, who wrote the editorial in the BMJ introducing the amnesty, wonders whether there is any real incentive to register unpublished trials, as they won't count toward a researcher's publication record. Moreover, he says, a disproportionate number of unpublished inconclusive or negative trials may have been sponsored by pharmaceutical companies that may not want the results publicized.
Naylor suggests that editors should increase the incentives by offering to review trials as submissions and publishing them annually in an electronic supplement. "The amnesty is a nice altruistic idea," he says, "and it's a sign of increasing activism of medical journals to improve the standards of reportage of clinical research. But some kind of more tangible benefit may be needed for this to have a real impact."
Material presented on this home page constitutes opinion of the author.
Copyright © 1997 Steven J. Milloy. All rights reserved. Site developed and hosted by WestLake Solutions, Inc.