Accepted on 15 Aug 2017
Submitted on 3 Aug 2017
Cohort studies have a major investigative role in medical and surgical literature. A cohort study is a comparative study typically involving two or more groups of patients that are sampled only on the basis of a specific exposure or intervention [1]. Cohort studies essentially compare outcomes between people who received the intervention or exposure and those who did not and provide clues to aetiological epidemiology.
Well-designed cohort studies can provide powerful results and valuable information unavailable from randomised trials that haven’t been done yet. Randomised controlled trials (RCTs) represent the criterion standard for primary research evidence, and although the focus of current medical research, they are often not possible for logistical or ethical reasons. RCTs are expensive, time-consuming and recruitment may be difficult. Therefore RCTs are often conducted over a limited period of time and determine efficacy of a given intervention under strict conditions in a highly controlled, carefully selected patient population [2]. Randomisation of patients to a potentially harmful intervention is often not possible. Cohort studies are observational, a more ‘natural experiment’ representing real word conditions, hence they inherently have a pragmatic design. This means cohort studies can be used to determine if the efficacy observed in randomised trials translates into effectiveness in broader populations and more realistic settings and to provide information on adverse events and risks [3]. Likewise, cohort studies can provide ethical grounds for performing RCTs by providing evidence towards a given association. Cohort studies are able to assess causality by reviewing a large databases of patients, and can provide an opportunity for long follow-up [2]. Cohort studies in the surgical literature are popular. Level 2 or 3 evidence accounted for 20–55% of published research in the top three journals by impact factor in 2013 across six surgical specialties, a significant rise compared with the 2003 figures of 17–50% [4]. However, work by our group has shown them to be poorly reported in Plastic Surgery [5].
Currently there is little guidance to authors with regards to the minimum necessary criteria to report in cohort studies. Journals and authors may adopt the guidelines for other study types, including guidelines for case series (PROCESS) or case reports (Surgical case report ‘SCARE’ guidelines) developed by our group. However, these are not tailored for use in cohort studies where multiple groups exposed to different interventions exist. Not all the points in these criteria are applicable to cohort studies. For example, mentioning the drug history and past medical history of each participant in cohort studies is not feasible, but is essential when reporting on a single case. However, comorbidities can be summarised in order to assess and compare the fitness of different groups. Additionally, cohort studies are often comparative and additional items required for high-quality cohort studies are not included in PROCESS or SCARE. The STROBE guideline provides guidance on reporting cohort studies, but these are not tailored to Surgery [6]. The development of the SCARE guidelines, when compared with the CARE guidelines, illustrates just how important a surgery specific guideline is.
Cohort studies must report on the comparison being made, as well as the clinical relevance of a given comparison [2]. The main form of bias in cohort studies is selection bias. Patients are not randomly allocated to treatment or control groups. Thus, such studies are vulnerable to the possibility that pre-existing baseline differences in prognosis and hidden confounders between two groups may bias the assessment of the effect of an intervention. To minimise selection bias, comparison groups should be carefully selected, for example, by excluding patients with a defined diagnosis or specific characteristics that is known to confound the outcome of interest [7]. All cohort studies should recognise that unknown confounders could affect the results. Effort should be made to identify and measure potential confounders, followed by a control for any pre-existing confounding factors in statistical analysis, often multivariate analyses [2], [8]. Authors must then also provide information on the distribution of potential confounders in the intervention and comparison [8].
There is therefore a need to develop guidelines specifically for use in cohort studies. The objective of this research is to conduct a Delphi consensus exercise amongst experienced cohort study reviewers and editors to develop STROCSS (Strengthening the Reporting of Cohort Studies in Surgery) guidelines.
A similar methodological approach will be taken to that used for developing the SCARE and PROCESS guidelines including Moher et al’s guidance on developing reporting guidelines [1], [9]. Relevant points from the PROCESS [1], STROBE [10] and CONSORT guidelines [11], will be used to create a template table to use as baseline material in the Delphi rounds. Guidance from articles appraising cohort studies will also be considered. [2], [8], [12].
The Delphi questionnaire will be administered by Google Forms. The questionnaire will be conducted using standard Delphi Methodology [13]. The same questionnaire will be completed by all stakeholders throughout the process. Participants will be invited to recommend adaptations to the current STROBE items and suggest new ones, with a relevance to surgical cohort studies.
In each success round participants will rate the importance of reporting each item according to a nine-point Likert scale, where 1 indicates not important, and 9 indicates critical. This methodology follows the recommendations outlined by the Grading Recommendations, Assessment, Development, and Evaluations (GRADE) working group [13]. Items scoring: 1 to 3 – indicates the item is of little important; 4 to 6 – indicate an item is important but not critical; 7 to 9 – indicates the items is critically important.
Items will proceed to the reporting guideline if they are scored between 7 and 9 by 70% of respondents and between 1 to 3 by fewer than 30% of respondents. Likewise, items will not proceed to reporting guidelines if they score between 1 and 3 by 70% of respondents, and between 7 and 9 by 30% or more of respondents.
The questionnaire will be administered through sequential rounds until a final set of items is agreed. The entire process will be conducted electronically. There is no predetermined number of Delphi rounds, but judging on previous work by our group [1], [14] it is expected that two or three rounds will be required.
The panel who helped develop the SCARE and PROCESS guidelines will be invited back. Further participants will be drawn from the Editorial Board of the International Journal of Surgery.
The research will be published in a peer-reviewed journal and will be disseminated electronically and presented at national and international conferences. Journals publishing surgical cohort studies will be encouraged to endorse the statement.
No ethical approval is required as patient data is not being directly collected.
None received.
RAA – idea conception, revision and final manuscript approval.
MRB – drafting, revision, final manuscript approval.
MVB – drafting, revision, final manuscript approval.
RT – drafting, revision, final manuscript approval.
None declared. None of the authors have a financial interest in any of the products, devices or drugs mentioned in the manuscripts.
No conflicts of interest are declared. The authors have no financial, consultative, institutional and other relationships that might lead to bias or conflict of interest.
Riaz A. Agha.
This is not a human study.
Agha, R.A., Fowler, A.J. and Rajmohan, S. (2016). Preferred reporting of case series in surgery; the PROCESS guidelines. Int. J. Surg. 36: 319–323. [PubMed]
Rochon, P.A., Gurwitz, J.H. and Sykora, K. (2005). Reader's guide to critical appraisal of cohort studies: 1. Role and design. BMJ 330(7496): 895. [PubMed]
Black, N. (1996). Why we need observational studies to evaluate the effectiveness of health care. BMJ 312(7040): 1215. [PubMed]
Agha, R.A., Devesa, M. and Whitehurs, K. (2016). Levels of evidence in plastic surgery – bibliometric trends and comparison with five other surgical specialties. Eur J Plast Sur 39(5): 365–370.
Agha, R.A., Lee, S. and Jeong, K.J.L. (2016). Reporting quality of observational studies in plastic surgery needs improvement: a systematic review. Ann Plast Surg 76(5): 585–589. PMID: 25643190. [PubMed]
Von Elm, E., Altman, D.G. and Egger, M. (2014). The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Int J Surg 12(12): 1495–1499. [PubMed]
McKee, M., Britton, A. and Black, N. (1999). Methods in health services research: interpreting the evidence: choosing between randomised and non-randomised studies. BMJ 319(7205): 312. [PubMed]
Mamdani, M., Sykora, K. and Li, P. (2005). Reader's guide to critical appraisal of cohort studies: 2. Assessing potential for confounding. BMJ 330(7497): 960. [PubMed]
Moher, D., Schulz, K.F. and Simera, I. (2010). Guidance for developers of health research reporting guidelines. PLoS Med 7(2)e1000217
STROBE Statement. Strobe Checklist for cohort studies. https://www.strobe-statement.org/index.php?id=available-checklists, 2007 (accessed 13.08.17).
Boutron, I., Altman, D.G. and Moher, D. (2017). CONSORT statement for randomized trials of nonpharmacologic treatments: a 2017 update and a CONSORT extension for nonpharmacologic trial abstracts. Ann. Intern. Med. 167: 40–47. [PubMed]
Normand, S.L., Sykora, K. and Li, P. (2005). Readers guide to critical appraisal of cohort studies: 3. Analytical strategies to reduce confounding. BMJ 330(7498): 1021–1023. [PubMed]
Sinha, I.P., Smyth, R.L. and Williamson, P.R. (2011). Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med 8(1): e1000393. [PubMed]
Agha, R.A., Fowler, A.J. and Saeta, A. (2016). The SCARE statement: consensus-based surgical case report guidelines. Int. J. Surg. 34: 180–186. [PubMed]
Supplementary data associated with this article can be found, in the online version, at https://s3-eu-west-1.amazonaws.com/ubiquity-partner-network/ijs/journal/ijsp/IJSP11.zip.