Surgical innovation has undoubtedly led to dramatic improvements in patient care. Unstandardised introduction of surgical procedures and devices, however, has the potential to cause significant patient harm [1, 2]. This is particularly true in the early stages of surgical innovation when new techniques or technologies are still being modified and refined before they are optimised . Current ways of reporting and sharing important incremental learning arising from each case, however, are informal and insufficient . Consequently, individual surgeons may simultaneously refine the same technical steps, or even repeat ineffective or harmful modifications . This can lead to delays in uptake of promising innovation and may critically increase the risk of avoidable patient harm .
Systematic monitoring of modifications is crucial to evaluating new procedures/devices during their introduction into clinical practice and before definitive evaluation in randomised controlled trials . The importance of modifications has been recognised in a recent core outcome set developed for early phase surgical studies (the COHESIVE Study), with a ‘modifications’ domain agreed as one of the core outcome domains to be measured and reported in all studies . Measurement and reporting of modifications in current empirical studies of surgical innovation is, however, heterogenous and lacks detail [9, 10, 11]. Standardisation of reporting and sharing of modifications is therefore integral to the safe and efficient introduction of new procedures/devices into clinical practice.
There is now an urgent need to define measurement and reporting of modifications. No agreed definition of what constitutes a modification to an evolving surgical procedure currently exists. Likewise, guidance on how modifications should be measured or classified to aid systematic reporting and sharing of modifications is lacking [9, 12]. A framework is needed to facilitate transparent reporting and sharing of modifications in future studies of innovative invasive procedures/devices. It can provide surgeon innovators, device manufacturers and trialists with a standardised tool to be used throughout the innovation lifecycle to promote their efficient evaluation. Consequently, a necessary step to informing such a reporting framework is to examine and summarise existing relevant literature on modifications.
This systematic scoping review aims to examine how modifications to innovative surgical procedures/devices might be defined, perceived and classified, including views on how they should be measured and reported. Specific objectives of this scoping review are to identify literature in the context of innovative invasive procedures and devices that provide any existing:
An invasive procedure is defined as “one where purposeful/deliberate access to the body is gained via an incision, percutaneous puncture, where instrumentation is used in addition to the puncture needle, or instrumentation via a natural orifice. It begins when entry to the body is gained and ends when the instrument is removed, and/or the skin is closed. Invasive procedures are performed by trained healthcare professionals using instruments, which include, but are not limited to, endoscopes, catheters, scalpels, scissors, devices and tubes” .
There is no agreed definition of innovative invasive procedures [14, 15, 16], and no validated methods to identify phase of evaluation retrospectively in published literature. Innovative invasive procedures were therefore defined as those where authors self-report an invasive procedure as “new” or “modified”, corresponding to IDEAL phases 1 and 2a [17, 18].
A scoping review was identified as a suitable method for mapping a complex topic where no prior comprehensive investigation has been performed [19, 20, 21]. This study will be conducted in accordance with the PRISMA guidelines for scoping reviews (PRISMA-ScR) and will adhere to established scoping review frameworks [20, 21, 22]. A study flow chart is presented in Figure 1.
Ethical approval is not required for this scoping review.
Two specific, distinct publication types of interest have been identified a priori by the research team. It is hypothesised that i) review articles and ii) opinion pieces will represent the most useful data sources for this review. This is because modifications are more likely to be discussed in greater detail outside of the existing traditional reporting parameters of surgical research studies and case reports. In addition, review articles and opinion pieces are thought to hold valuable information about desired reporting practices in view of deficient reporting practices in empirical studies [9, 10, 11].
A wide range of review articles will be considered and may include, but are not limited to, systematic literature reviews (including Cochrane reviews), narrative, critical and scoping reviews. Opinion pieces will be described as published reports or expressions of “original and personal views” and are typically, but not limited to, letters, debates, editorials, previews, comments, responses or position papers .
Tailored, specific searches are required to identify a manageable number of potentially relevant articles. Therefore, two systematic database searches will be undertaken to identify review articles and opinion pieces, respectively. These searches will be supplemented with snowball searches to ensure a comprehensive approach to identifying relevant literature. Details for each search are described below.
Previous reviews undertaken by the study team have found that the use of search terms related to “modification” and “invasive procedures” retrieve a large number of irrelevant records. Development of a sufficiently focused search strategy is needed to avoid retrieving an unmanageable number of records for full text screening. Therefore, targeted internet keyword searches will be used to inform a comprehensive search strategy to apply to the MEDLINE library catalogue database. Targeted internet searches will be performed in Google Scholar using keywords related to “modifications”, “definition”, “classification” and “invasive procedures” to identify relevant literature that reports definitions, perceptions and/or classifications of modifications. Identified articles will be read in full to derive further relevant keywords to be added to a long list of search terms. The long list will inform development of a comprehensive database search strategy which will be refined through discussions with the with the study team (consisting of surgeons, methodologists, trialists) and in collaboration with an expert subject librarian. Two separate searches will then be performed in MEDLINE (Ovid version) to identify records relevant to the two distinct publication types of interest (described below). Unique aspects inherent to each publication type will be considered when applying the search strategy. Searches will be limited to publications in English language and to studies in humans only.
A comprehensive search will be developed to identify all potentially relevant review articles, which can subsequently be narrowed down by screening titles and abstracts. This search will include a modified, previously validated search filter for reviews in order to identify relevant publications classified as reviews . Specific changes to the search strategy will include the addition of the word “review*” and “narrative” as well as the omission of keywords related to meta-analyses, as these are presumed unlikely to yield relevant results. To reduce the number of potential records, publication dates will be limited to articles published between 2010 and 2021 to ensure the search focuses on contemporary literature that is more likely to provide information about current descriptions of modifications.
A targeted search for published opinion pieces is necessary due a lack of abstracts available and/or detail provided in any abstracts for these types of publications, meaning screening for eligibility will be resource intensive. As a result, full-text articles will likely need to be obtained for the majority of opinion pieces. A more detailed search will be developed (e.g. using narrower proximity operators and/or fewer keywords), to obtain fewer results and limit the number of records necessary to be screened. Search filters for editorials and commentaries will be applied. No limits to publication date will be applied because this search is expected to produce a smaller number of potential records.
Backwards and forwards searching of reference lists (snowball searches) will be applied to all eligible records until no new relevant publications will be identified. This will detect records which may have been missed due to lack of indexing terms .
Included will be any relevant review article or opinion piece published in a peer-reviewed journal that discusses modifications in the context of innovative invasive procedures/devices. Specifically, publications will be included that:
Studies discussing modifications to accompanying (concomitant) interventions (e.g. drains, dressings or analgesia) or changes to patient selection will also be included as these have been identified as relevant aspects of modifications in previous studies [4, 8].
The following publications are outside the scope of this work and will be excluded:
Other exclusion criteria are:
Studies identified through the search process will undergo a two-stage study selection process.
First, titles, abstracts and keywords will be screened independently by two reviewers. Keyword screening is included in the first stage because it is anticipated that abstracts will hold less detail about modifications and/or may not use a common structure (in particular opinion pieces). Full texts will be retrieved where eligibility is confirmed, uncertain or where no abstract is presented.
Decisions on inclusion/exclusion during the first stage of study selection are expected to require a degree of subjective interpretation because titles, abstracts and keywords are expected to hold limited information about modifications with which eligibility can be assessed. Therefore, initial additional steps are planned to ensure consistency of study selection approach between the two independent reviewers. A random selection of a small number of studies will be used to (i) independently screen titles, abstracts and keywords to decide whether records are included, excluded or uncertain; (ii) discuss and investigate reasons for discrepancies of decisions, (iii) jointly screen full texts of remaining uncertain records to assess final eligibility, which will lead to refinements of study selection approach. This process will be repeated until reviewer agreement of independent inclusion/exclusion decisions has reached sufficient levels (80%).
Second, full texts will be screened by the same reviewers to assess eligibility of identified records. Each reviewer will screen half (50%) of all retrieved publications to decide their inclusion or exclusion in this review. Articles deemed eligible for inclusion and those where inclusion is uncertain will be discussed between the two reviewers to reach a decision. In addition, a third independent reviewer, will subsequently assess eligibility of all articles discussed between the two reviewers.
Reasons for exclusion will be documented at each screening stage and will follow a predefined order of priority (1; not related to surgery, 2; not a review article or opinion piece, 3; not related to modifications, 4; no information on definition/perception/classification/reporting of modifications). Double-screening will be carried out for at least 20% of title and abstracts and 10% of full text articles. Any discrepancies will be discussed between the three reviewers and any further disagreements will be arbitrated in discussions with the wider study team. The process of study selection will be reported in line with PRISMA-ScR guidelines .
A study-specific data extraction form in line with the research objectives will be developed by the research team. An initial proforma will be piloted with a small number of articles and iteratively refined to ensure all emerging relevant detail is comprehensively captured. The final data extraction form will be applied to all included articles. Information related to publication characteristics, details about author(s) and affiliations and conflict of interest statements will be extracted. Detail as to whether the article discusses surgical procedures, devices or both will also be captured to enable potential sub-group analysis. Any details related to modifications and their definition, perception, classification or views on their measurement and reporting will be extracted verbatim. At least 10% of included articles will undergo initial double data extraction. These will be compared and discussed to ensure consistency of the approach to data extraction. Data extraction will be conducted independently by two reviewers.
Study characteristics will be summarised using descriptive statistics and presented in a table. Data relating to study characteristics will be grouped into categories where appropriate. Categorisation might be based on existing guidelines when available (e.g. national statistics country classifications will be used to categorise geographical origin of included articles) or will be finalised in discussions with the wider team wherever necessary.
Verbatim text will be analysed using inductive thematic analysis and will be guided by the study objectives . The extent of sub-group analyses will be led by the data. Analyses may be conducted, for example, separately for information categories (e.g. definitions, perceptions, classifications of modifications, views on how to measure and report them) and for article context (e.g. procedures, devices). Two reviewers (SH, CH) will independently perform analyses and will discuss interim findings during regular meetings. Data familiarisation will be achieved through reading and re-reading extracted data. Initial codes will be assigned to the textual data and discussed between the reviewers. Thematic structures for analyses will be developed jointly between the two reviewers and iteratively refined as new themes emerge from the data. Consensus discussions within the wider team will agree on a final thematic structure. Themes will be presented in a narrative summary. Where thematic analysis is not appropriate (e.g. any identified classification systems, taxonomies or typologies), data will be descriptively summarised and presented in tabulated format.
No quantitative analyses are planned as is it not expected that opinion pieces and review articles will yield data appropriate for this type of synthesis.
This study was supported by the National Institute for Health Research (NIHR) Biomedical Research Centre (BRC) at University Hospitals Bristol and Weston NHS Foundation Trust and the University of Bristol (BRC-1215-20011). This work was supported by the Royal College of Surgeons of England Bristol Surgical Trials Centre. The views expressed in this publication are those of the authors and not necessarily those of the NHS, the NIHR, the Royal College of Surgeons of England or the Department of Health and Social Care. SP and AM are NIHR Clinician Scientists (NIHR CS-2016-16-019, NIHR CS-2017-17-010). NB is an MRC Clinician Scientist. JB is an NIHR Senior Investigator.
JB is a member of the Core Outcome Measures for Effectiveness Trials (COMET) Initiative Management Group. All other authors declare no conflict of interests.
Christin Hoffmann: Conceptualisation; Data curation; Investigation; Methodology; Project administration; Validation; Roles/Writing – original draft. Sina Hossaini: Data curation; Investigation; Methodology; Project administration; Validation; Roles/Writing – original draft. Sian Cousins: Conceptualisation; Methodology; Project administration; Validation; Writing – review & editing. Natalie Blencowe: Conceptualisation; Validation; Writing – review & editing. Angus McNair: Conceptualisation; Validation; Writing – review & editing. Jane Blazeby: Conceptualisation; Funding acquisition; Validation; Writing – review & editing. Kerry Avery: Conceptualisation; Methodology; Supervision; Validation; Writing – review & editing. Shelley Potter: Conceptualisation; Methodology; Supervision; Validation; Writing – review & editing. Rhiannon Macefield: Conceptualisation; Investigation; Methodology; Supervision; Validation; Writing – review & editing.
Christin Hoffmann and Sina Hossaini are joint first authors.
Kerry N. L. Avery, Shelley Potter and Rhiannon Macefield are joint senior authors.
Whealon M, Vinci A, Pigazzi A. Future of Minimally Invasive Colorectal Surgery. Clin. Colon Rectal Surg. 2016; 29: 221–231. DOI: https://doi.org/10.1055/s-0036-1584499
Miller ME, Siegler M, Angelos P. Ethical issues in surgical innovation. World J. Surg. 2014; 38: 1638–1643. DOI: https://doi.org/10.1007/s00268-014-2568-1
Barkun JS, Aronson JK, Feldman LS, Maddern GJ, Strasberg SM. Evaluation and stages of surgical innovations. Lancet. 2009; 374: 1089–1096. DOI: https://doi.org/10.1016/S0140-6736(09)61083-7
Dimick JB, Sedrakyan A, McCulloch P. The IDEAL Framework for Evaluating Surgical Innovation: How It Can Be Used to Improve the Quality of Evidence. JAMA Surg. 2019; 154: 685–686. DOI: https://doi.org/10.1001/jamasurg.2019.0903
Angelos P, Kohler L. A. Professor, Ethics and surgical innovation: challenges to the professionalism of surgeons. Int. J. Surg. 2013; 11: S2–S5. DOI: https://doi.org/10.1016/S1743-9191(13)60003-5
Angelos P. Double-edged sword of surgical innovation: ethical issues of early and late adoption of novel techniques. Br. J. Surg. 2021; 108: 107–108. DOI: https://doi.org/10.1093/bjs/znaa162
McCulloch P, Altman DG, Campbell WB, Flum DR, Glasziou P, Marshall JC, Nicholl J. No surgical innovation without evaluation: the IDEAL recommendations. Lancet. 2009; 374: 1105–1112. DOI: https://doi.org/10.1016/S0140-6736(09)61116-8
Avery KNL, Wilson N, Macefield R, McNair A, Hoffmann C, Blazeby JM, Potter S. A core Outcome Set for Seamless, Standardized Evaluation of Innovative Surgical Procedures and Devices (COHESIVE). Ann. Surg; Publish Ah 2021. DOI: https://doi.org/10.1097/SLA.0000000000004975
Macefield RC, Wilson N, Hoffmann C, Blazeby JM, McNair AGK, Avery KNL, Potter S. Outcome selection, measurement and reporting for new surgical procedures and devices: a systematic review of IDEAL/IDEAL-D studies to inform development of a core outcome set. BJS Open. 2020; 4: 1072–1083. DOI: https://doi.org/10.1002/bjs5.50358
Hoffmann C, Macefield RC, Wilson N, Blazeby JM, Avery KNL, Potter S, McNair AGK. A systematic review and in-depth analysis of outcome reporting in early phase studies of colorectal cancer surgical innovation. Color. Dis. 2020; 22: 1862–1873. DOI: https://doi.org/10.1111/codi.15347
Khachane A, Philippou Y, Hirst A, McCulloch P. Appraising the uptake and use of the IDEAL Framework and Recommendations: A review of the literature. Int. J. Surg. 2018; 57: 84–90. DOI: https://doi.org/10.1016/j.ijsu.2018.07.008
Avery K, Blazeby J, Wilson N, Macefield R, Cousins S, Main B, Blencowe NS, Zahra J, Elliott D, Hinchliffe R, Potter S. Development of reporting guidance and core outcome sets for seamless, standardised evaluation of innovative surgical procedures and devices: A study protocol for content generation and a Delphi consensus process (COHESIVE study). BMJ Open. 2019; 9: e029574. DOI: https://doi.org/10.1136/bmjopen-2019-029574
Cousins S, Blencowe NS, Blazeby JM. What is an invasive procedure? A definition to inform study design, evidence synthesis and research tracking. BMJ Open. 2019; 9: 28576. DOI: https://doi.org/10.1136/bmjopen-2018-028576
Hutchison K, Rogers W, Eyers A, Lotz M. Getting Clearer About Surgical Innovation: A New Definition and a New Tool to Support Responsible Practice. Ann. Surg. 2015; 262: 949–954. DOI: https://doi.org/10.1097/SLA.0000000000001174
Rogers WA, Lotz M, Hutchison K, Pourmoslemi A, Eyers A. Identifying surgical innovation: A qualitative study of surgeons’ views. Ann. Surg. 2014; 259: 273–278. DOI: https://doi.org/10.1097/SLA.0b013e31829ccc5f
Birchley G, Ives J, Huxtable R, Blazeby J. Conceptualising Surgical Innovation: An Eliminativist Proposal. Heal. Care Anal. 2019; 28: 73–97. DOI: https://doi.org/10.1007/s10728-019-00380-y
McCulloch P, Cook JA, Altman DG, Heneghan C, Diener MK, IDEAL Group. IDEAL framework for surgical innovation 1: the idea and development stages. BMJ. 2013; 346. DOI: https://doi.org/10.1136/bmj.f3012
Ergina PL, Barkun JS, McCulloch P, Cook JA, Altman DG, IDEAL Group. IDEAL framework for surgical innovation 2: observational studies in the exploration and assessment stages. BMJ. 2013; 346. DOI: https://doi.org/10.1136/bmj.f3011
Fischer F, Lange K, Klose K, Greiner W, Kraemer A. Healthcare Barriers and Strategies in Guideline Implementation-A Scoping Review; 2016. DOI: https://doi.org/10.3390/healthcare4030036
Arksey H, O’malley L. Scoping studies: towards a methodological framework. Int. J. Soc. Res. Methodol. 2005; 8: 19–32. DOI: https://doi.org/10.1080/1364557032000119616
Tricco AC, Lillie E, Zarin W, O’Brien K, Colquhoun H, Kastner M, Levac D, Ng C, Sharpe JP, Wilson K, Kenny M, Warren R, Wilson C, Stelfox HT, Straus SE. A scoping review on the conduct and reporting of scoping reviews. BMC Med. Res. Methodol. 2016; 16: 1–10. DOI: https://doi.org/10.1186/s12874-016-0116-4
Levac D, Colquhoun H, O’Brien KK. Scoping studies: Advancing the methodology. Implement. Sci. 2010; 5. DOI: https://doi.org/10.1186/1748-5908-5-69
Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Heal. Inf. Libr. J. 2009; 26: 91–108. DOI: https://doi.org/10.1111/j.1471-1842.2009.00848.x
BMJ Best Practice, Study design search filters. (n.d.). https://bestpractice.bmj.com/info/toolkit/learn-ebm/study-design-search-filters/ (accessed August 17, 2021).
Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ. 2005; 331: 1064–1065. DOI: https://doi.org/10.1136/bmj.38636.593461.68
Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, Moher D, Peters MDJ, Horsley T, Weeks L, Hempel S, Akl EA, Chang C, McGowan J, Stewart L, Hartling L, Aldcroft A, Wilson MG, Garritty C, Lewin S, Godfrey CM, MacDonald MT, Langlois EV, Soares-Weiser K, Moriarty J, Clifford T, Tunçalp Ö, Straus SE. PRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann. Intern. Med. 2018; 169: 467–473. DOI: https://doi.org/10.7326/M18-0850
Braun V, Clarke V. Using thematic analysis in psychology. Qual. Res. Psychol. 2006; 3: 77–101. DOI: https://doi.org/10.1191/1478088706qp063oa