On 27 July 2018 an article was published in the journal BMJ Evidence-Based Medicine relating to the recently published Cochrane Review on prophylactic human papillomavirus (HPV) vaccines. The article is based on analyses undertaken at the Nordic Cochrane Centre, and two of the authors are experienced Cochrane researchers: Professors Peter Gøtzsche and Tom Jefferson. It made several criticisms of the Cochrane Review, most notable of which was that the Cochrane Review was incomplete due to missing "nearly half of the eligible trials".
Cochrane takes all criticisms and feedback seriously, seeing this as one mechanism among many to improve the quality of Cochrane Reviews. The organisation has 10 long-standing principles that we hold dear, and they include a commitment to quality and the minimisation of bias, transparency, and a recognition of the need for our work to be relevant to the needs of evidence users and decision makers. Cochrane aims to create the best current evidence to guide health decisions.
We initiated an investigation in response to the criticisms, working with the review authors and editors and with independent researchers who had not been involved in the original publication.
The key findings of our investigation are that:
- The Cochrane Review did not miss "nearly half of the eligible trials". A small number of studies were missed due to the primary focus on peer-reviewed reports in scientific journals, but addition of these data makes little or no difference to the results of the review for the main outcome;
- The trials comparators were unambiguously, transparently, and accurately described;
- The selection of outcomes for benefits was appropriate and was consistent with World Health Organization guidance;
- The review included published and unpublished data on serious harms, and the findings on mortality were reported transparently and responsibly;
- The review was compliant with Cochrane’s current conflict of interest policy;
- Cochrane’s media coverage was cautious and balanced, but we recognize that there could be improvements in relation to transparency where external experts are quoted;
- The BMJ Evidence-Based Medicine article substantially overstated its criticisms.
David Tovey, Editor in Chief, Cochrane
Karla Soares-Weiser, Deputy Editor in Chief, Cochrane