In 1999, Chalmers and Altman stated that medical journals should consider publishing protocols to aid with the primary prevention of poor medical research . In the intervening period, transparency has become a major part of the healthcare research discourse. The Declaration of Helsinki (DoH) is in support of explicit research protocols: “The design and performance of each research study involving human subjects must be clearly described and justified in a research protocol.” Indeed, in 2013, the registration part of the declaration was updated to state: “Every research study involving human subjects must be registered in a publicly accessible database before recruitment of the first subject”, a clear broadening of the language from “every trial” must be registered in 2008 version . We have gone some way to addressing this already with the launch of the Research Registry® www.researchregistry.com where we already described the benefits of protocol registration . The launch of this registry ensured that every research study submitted to the International Journal of Surgery (IJS, www.journal-surgery.net) and Annals of Medicine and Surgery (AMS, www.annalsjournal.com) could be registered, as mandated in the DoH.
Publication bias is a major problem within healthcare research. Dwan et al. conducted a systematic review of publication and outcome reporting bias . They found direct empirical evidence for the existence of study publication and outcome reporting bias. There was strong evidence that positive or statistically significant results were more likely to be published. In surgery, Chapman et al. found that one in five surgical RCTs are discontinued early and one in three remains unpublished two years after their conclusion . They stated that negative findings were a barrier to publication and appealed for journals to publish them. They concluded by stating that public trust in research will be badly damaged if we continue to ‘mothball’ negative studies and recruitment will become an even greater challenge .
Ross et al. showed that only 42% of trials that stated they concluded in 2005 were actually published over two years later . They concluded by stating that: “The scientific community should be prioritizing the timely and accurate publication and dissemination of clinical trial results, regardless of the strength and direction of the trial results.” This is a challenge to the human behaviour clinicians and researchers exhibit when editors express their bias through their decisions – favoring positive studies. Indeed, whilst Negative studies don’t get published and positive studies sometimes get published twice .
The other aspect we must acknowledge are the lessons lost and the learning that wasn’t disseminated to the wider research community. Kasenda et al. found that 24.9% of 1017 RCTs were discontinued with the most frequent reason being poor recruitment . These ‘failures’ are often not reported. But learning from them and disseminating problems encountered together with possible solutions could help prevent others repeating the mistake. Was it poor recruitment, costs, a flawed hypothesis or lack of equipoise that led to the studies demise. We can learn from these valuable experiences and they should be added to the ever expanding corpora of knowledge in our field.
We need to know what works and what doesn’t work in order to drive research in the correct direction, aid collaboration, prevent duplication and wasted resources. Other issues our community grapples with, is underpowered studies, poor statistical methods, poor reproducibility and external validity, poor methodology and reporting of studies , , , , , . Publishing Protocols and negative results brings the focus away from the results themselves to the research questions, the hypothesis and the robustness of the methodology used to investigate it.
Peer-review of protocols allows for independent external feedback at an important stage and allows for important course corrections that could save significant time, money and resources spent on a study that was asking the wrong research question or which deployed methodology that could never reasonably be expected to answer it. Institutions and funding bodies would certainly appreciate it, they often organize it themselves anyway for institutional review board approval and grant funding respectively. Once published, it may facilitate collaboration leading to a multicenter study with greater power and external validity. It also gives the research team greater assurance on the likelihood of publication of the final paper, if they stick to the protocol. Of course this does not preclude exploratory or additional analyses, as long as the final paper mentions these in a transparent manner. For many years now the life sciences have had protocols journals. Medicine has not been far behind, although Protocol review and publication at the Lancet just recently ceased . Both fields even have dedicated negative results journals , , , . It about time that surgeons had dedicated journals for their protocols and negative studies. Such journals should provide rapid submission to decision times whilst maintaining an effective and robust peer-review process, allowing surgeons and researchers to get on with the next phase of their work whilst maintaining high scientific standards. A swift process has certainly been lacking in my own experience of journals publishing protocols and this view is supported by colleagues I have asked.
In view of these issues, the IJS Publishing Group is launching a suite of online-only, international, peer-reviewed, open access journals which aim to publish protocols and negative results, starting with IJS Protocols. We aim to have a rapid submission to decision time whilst maintaining a rigorous peer-review process (we will not publish bad science or unethical research). We will publish basic science, clinical and translational research. These launches are unique in that IJS Protocols and IJS Short Reports will be the first dedicated surgical journals to publish protocols and negative results respectively. We hope that you will support us as we develop these journals and aim to improve and rebalance the scholarly literature in our field for the better.