improving outcomes in pediatric endoscopic third ... · arthur de jager & wico breimer info:...

UITNODIGING

voor het bijwonen van de verdediging van mijn proefschrift

Improving outcomes in pediatric endoscopic third ventriculostomy

through outcome analysis and surgeon training

door

Gerben Breimer

op woensdag 11 januari 2017 om 11:00 uur

in de Aula van het Academiegebouw, Broerstraat 5 te Groningen.

Aansluitend wordt een lunch georganiseerd.

Paranimfen:

Arthur de Jager

&

Wico Breimer

info:[email protected]

Improving outcomes in pediatric endoscopic third ventriculostomy through

outcome analysis and surgeon training

Gerben Breimer

Improving outcom

es in pediatric endoscopic third ventriculostomy through outcom

e analysis and surgeon training G

erben Breim

er

Breimer cover fin.indd 1 29-11-16 13:39

Improving outcomes in pediatric endoscopic third ventriculostomy through outcome analysis

and surgeon training

Gerben Eise Breimer

Improving outcomes in pediatric endoscopic third ventriculostomy through outcome analysis and surgeon trainingAcademic thesis, University of Groningen, Groningen, The Netherlands

ISBN 978-90-367-9434-3 (gedrukt) 978-90-367-9433-6 (ebook)

Author Gerben Breimer

Cover design Gerben Breimer and Eelco Roos

Layout Eelco Roos

Print Ridderprint BV

Copyright © 2017, Gerben Breimer, Amsterdam, the Netherlands.

All rights reserved. No part of this thesis may be reproduced or transmitted in any

form or by any means, without express written permission from the author.

Improving outcomes in pediatric endoscopic third ventriculostomy through outcome analysis

and surgeon training

PhD thesis

to obtain the degree of PhD at the

University of Groningen

on the authority of the

Rector Magnificus Prof. E. Sterken

and in accordance with

the decision by the College of Deans.

This thesis will be defended in public on

Wednesday, 11 January 2017 at 11:00 hours

by

Gerben Eise Breimer

born on 10 February 1988

in Wageningen

SupervisorsProf. Dr. E.W. Hoving Prof. Dr. J.M. Drake

Assessment CommitteeProf. Dr. O.F. Brouwer Prof. Dr. J.A. Grotenhuis Prof. Dr. S.M. Peerdeman

“I cannot say whether things will get better if we change; what I can say is that they

must change if they are to get better.” – Georg Christoph Lichtenberg

7

Contents

Acknowledgements 9 List of abbreviations 13

Chapter 1 General introduction 15

Chapter 2 An external validation of the ETVSS for both short-term and long-term predictive adequacy in 104 pediatric patients 31

Chapter 3 Endoscopic third ventriculostomy (ETV) and repeat ETV in pediatric patients: the Dutch experience 47

Chapter 4 Development and content validation of performance assessments for endoscopic third ventriculostomy 71

Chapter 5 Validity Evidence for the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT) 95

Chapter 6 Design and evaluation of a new synthetic brain simulator for endoscopic third ventriculostomy 113

Chapter 7 Simulation-based Education for Endoscopic Third Ventriculostomy: A Comparison Between Virtual and Physical Training Models 129

Chapter 8 Summary and General Discussion 147

Chapter 9 Future Directions 163

Chapter 10 Conclusions 171

Chapter 11 Appendix 1. Nederlandse samenvatting 177

Appendix 2. Lay summary & samenvatting voor leken 179

Appendix 3. Three initial lists for online survey 181

Appendix 4. Feedback on brain simulator 184

Appendix 5. Simulator assessment - VR v physical 185

Appendix 6. Curriculum vitae 189

9

Acknowlegements

Acknowledgements

Writing this thesis, I realized over and over how lucky I have been in my life on a whole variety of levels. On small scale I have been lucky with the inspiring people I have met, my friends, colleagues and mentors. Without the understanding, patience and kindness of those around me I would not have able to get things done. Eelco Hoving has supported me since the very first time we worked together in Groningen and his positive attitude, work ethos, knowledge, technical skills and integrity are a great example for me and I am very grateful that Dr. Hoving has introduced me to James Drake who is someone I admire and consider to be my mentor. Dr. Drake has helped me in many ways and I am incredibly grateful for his supportive attitude towards me, and for opening doors which would probably have stayed closed if not for him. In Toronto I met many people, one of which is Faizal Haji, he has made serious efforts to explain and guide me through the process of conducting research and writing, always being constructive and offering a helpful hand. All members of the lab of the Centre for Image Guided Innovation and Therapeutic Intervention at the Hospital for Sick Children were tremendously helpful, always available to give advice or suggestions, especially Vivek Bodani and Thomas Looi, Brian Carrillo, Nikoo Saber, Fouzia Khan and Matthew Kang. Most grateful of all I am of meeting Fouzia in the lab in Toronto. On research design and statistics I have had much help from Marjolein Brusse-Keizer (Medisch Spectrum Twente, Enschede), Els Reeuwijk (Universiteit Twente), Riekie de Vet (Vrije Universiteit Amsterdam) and Floor van Leeuwen (Nederlands Kanker Instituut / Antoni van Leeuwenhoek). Margaret Darling helped me with the PubMed search strategies. Fouzia Khan, Jorik Booij, and Roxanne Leung gave much needed advice and suggestions on my written material. The enthusiasm of the international pediatric neurosurgery community has been very stimulating, be it in personal communications, at conferences or at workshops. All the participating residents, fellows and neurosurgeons in Toronto cannot be thanked enough for offering their precious time and for the relevant feedback on the simulator. All the experts contributing to the Delphi study made that project succeed: G. Cinalli, Italy; S. Constantini, Israel; P. Decq, France; S. deRibaupierre, Canada; C. DiRocco, Italy; Y. Ersahin, Turkey; J. Grotenhuis, the Netherlands; N. Gupta, United States of America; E. Hoving, the Netherlands;

10

I. Pollack, United States of America; C. Sainte-Rose, France; S. Santoreneos, Australia; H. Schroeder, Germany; D. Thompson, United Kingdom; B. Warf, United States of America; J. Wellons, United States of America; S. Zymberg, Brazil. For the validation study in Naples I am much grateful for Piero Spennato, Guiseppe Cinalli and Spyros Sgouros for rating all participants, and without consent of the staff and trainees of the workshop this study would have been impossible. In the Netherlands, for the repeat ventriculostomy study, I received enthusiastic help from Ruben Dammers, Dennis Buis, Peter Woerdeman, Hans Delye, and Sebastian Arts. And for the survey study there were Benjamin Warf, Abhaya Kulkarni, Guiseppe Cinalli, Henry Schroeder, Shlomo Constantini, Jonathan Roth and of course Eelco Hoving and James Drake. It was great that Trudell Medical Marketing Limited provided us the neuroendoscopic equipment required for testing the simulator. Lisa Satterthwaite and the technicians at the University of Toronto surgical skills centre at Mount Sinai Hospital provided much support and also the endoscopic tower and other necessary resources for testing the brain simulator. Eelco Roos helped with the layout and cover of the final printed thesis. On a more general level, there are the obvious lucky breaks such as being born in family with open-minded high educated parents with good jobs and unconditional love and support for my brothers and me, and being brought up in relative quiet times in a rich western world country with a good educational system. A country in which getting accepted into medical school is – although it may be better to say was – not as competitive as it is in other first world countries like Canada or the United States of America. I realize that I would never have become a medical doctor in these countries (among other things due to mediocre grades in high school). Since I started collecting data for my first paper in 2011 I have found myself in various stages of an education. First I was a student without a clue about what I wanted out of medical school (and life) and having been on the verge of voluntarily dropping out of university many times. In a later phase of medical school I was a student with a plan, a goal and a dream of becoming a neurosurgeon with a strong research background. Then, as a medical doctor, I was a resident with shifting ideas of what I expected from my life and career, culminating in the decision to leave the field of neurosurgery in December 2014,

11

Acknowlegements

in order to pursue a career in clinical pathology. During these years I had to make many choices: between medical school and another path, between a job as a neurosurgical resident in Groningen and a research position in Toronto, and between neurosurgery and clinical pathology. Writing this all down, knowing that I went with my gut feelings, I feel good about my choices.

13

Abbreviations

List of abbreviations

3D Three-Dimensional ANOVA Analysis of Variance CI Confidence Interval CIGITI Centre for Image Guided Innovation and Therapeutic InterventionCL ChecklistCSF Cerebro-Spinal FluidCT Computed TomographyDICOM Digital Imaging and Communications in MedicineELD External lumbar puncture ETV Endoscopic Third VentriculostomyETVSS Endoscopic Third Ventriculostomy Success ScoreEVD External ventricular drainGRS Global Rating ScaleICC Intra-class Correlation CoefficientIQR Interquartile Range LP Lumbar PunctureMRI Magnetic Resonance ImagingNEVAT Neuro-Endoscopic Ventriculostomy Assessment ToolOR Operating RoomOR Odds ratioOSAT Objective Structured Assessment Tool PGY Postgraduate YearRe-ETV Repeat ETVSD Standard Deviation SIMONT Sinus Model Oto-Rhino Neuro TrainerSPSS Statistical Package for the Social Sciences STL StereolithographyVP Ventriculo-PeritonealVR Virtual Reality

Chapter 1

General introduction

16

CHAPTER 1

17



The focus of this thesis is on endoscopic treatment of hydrocephalic children. Classically two types of hydrocephalus can be distinguished, communicating and obstructive hydrocephalus. Hydrocephalus is generally treated by diversion of cerebrospinal fluid (CSF) outside the central nervous system into another body cavity (abdominal or pleural) or into the circulation (venous system) by means of a shunt. Standard general therapy for all indications of hydrocephalus is ventriculo-peritoneal shunt (VP-shunt) implantation.1 In case of obstructive hydrocephalus treatment by endoscopic fenestration might be indicated. The most common type of fenestration is endoscopic third ventriculostomy (ETV). An ETV restores the CSF circulation via a perforation in the third ventricle floor and bypassing the anatomical obstruction by creating access to the normal cisternal CSF pathways. Subsequently regular absorption of CSF can take place into the venous system. The main advantage of this endoscopic treatment modality for obstructive hydrocephalus is restoration of normal CSF flow without having to use a shunt device with a higher chance on adverse effects.2 The main question remains whether hydrocephalus is being treated sufficiently by means of endoscopy. In order to address this question, we focused our study on certain aspects of ETV treatment. First, we analyzed retrospectively the efficacy ETV and repeat ETV (re-ETV) in hydrocephalic children by using the ETV success score (ETVSS). This might add to delineate further the proper indications for ETV and re-ETV. Second, we focused on the acquisition and the evaluation of endoscopic skills using a newly developed training model. Both aspects might eventually contribute to a better outcome. In rightly selected hydrocephalic patients, ETV might be a preferred option. Kulkarni et al developed a predictive model for adequately selecting patients, the ETV success score (ETVSS).3 In chapter 2 we provide validity evidence for the short and long-term predictive adequacy of the ETVSS. It remains unclear what the right course of action is in patients with failed initial ETV. In chapter 3 we retrospectively analysed children who underwent ETV and in some cases re-ETV between 1998 and 2015 in six major pediatric neurosurgery centers in the Netherlands.4

18

CHAPTER 1

The second part of our thesis covers acquiring and evaluating technical skills of the ETV procedure. As neurosurgery residents or other neuroendoscopic trainees intend to learn a new procedure there are multiple ways to proceed. The classic option is to observe and learn, apprenticeship-based training (Halstedian method). This is how most neurosurgeons are trained. Sometimes the training program is supplemented by simulation-based training in which the procedure is performed on a substitute for a patient. A patient can be simulated by using an animal, a human cadaver, a physical simulator (either high-fidelity or low-fidelity box trainer) or a virtual reality simulator.5–8 The interest in simulation-based education for neurosurgery training has increased recently, and many program directors in Europe and the United States of America think it will become part of the twenty-first century neurosurgical education.9–11 In chapter 4 we describe the development of a series of assessment instruments for evaluating technical skills of trainees performing an ETV procedure, the instruments together are called the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). In chapter 5, we provide validity evidence for the NEVAT by reporting on the internal structure, relations of score on NEVAT and level of expertise and interrater reliability.In chapter 6 we describe how we develop a patient specific synthetic brain simulator for neuroendoscopy – the SickKids brain simulator. The simulator can be used to improve neuroendoscopy skills, specifically for the ETV procedure. In chapter 7 the SickKids brain simulator is compared head-to-head with a virtual reality (VR) simulator for ETV.

Hydrocephalus

The incidence of hydrocephalus is roughly one out of 1000 living births.12 The natural history of untreated hydrocephalus evolves with devastating effects on the brain and high mortality rates.13,14 Hydrocephalus is more than just an abnormal accumulation of CSF within the intracranial compartment, it should be considered as a complex disorder of the central nervous system as the result of abnormal CSF hydrodynamics. The pathogenesis is related to a mismatch of CSF production over absorption. Increased production of CSF is of minor importance since this is a very rare condition related to choroid plexus tumors.

19


Disturbed absorption of CSF explains most types of hydrocephalus, and a subsequent distinction can be made on the site of obstruction.15 In this respect communicating hydrocephalus can be considered obstructive hydrocephalus at the site of absorption.In newborns – with cranial sutures not yet solidified – the accumulation of CSF results in increased head circumference; in older patients it leads to increase of intracranial pressure (the now solid skull does not give way). Hydrocephalic children can be classified using different systems.1,15 A distinction can be made between communicating and non-communicating hydrocephalus. In non-communicating hydrocephalus (also known as obstructive hydrocephalus), an intraventricular obstruction prevents the CSF from reaching the arachnoid villi where CSF normally is reabsorbed into the venous system. In communicating hydrocephalus, the arachnoid villi fail to reabsorb the CSF appropriately or there is obstruction in the subarachnoid space. There are newer classifications and explanations for the mechanisms that are involved in cerebrospinal fluid dynamics.1 Rekate considers all types of hydrocephalus being the result of an obstruction; in his model communicating hydrocephalus is due to an obstruction at the absorption capacity.15 There have been advantages in the care of hydrocephalus over the course of the last decades but success rates have not improved much despite better imaging such as magnetic resonance imaging, different shunt designs, endoscopic equipment and techniques.16

History

Implantation of a CSF shunt has for many years been the main treatment for hydrocephalus and remains standard of care for many hydrocephalic patients.16,17 The CSF is diverted from the intraventricular cavities to another body cavity, often to the peritoneal cavity (ventriculo-peritoneal shunt) but also to other cavities such as the pleura of the lungs and the atrium of the heart (respectively ventriculo-pleural shunt and ventriculo-atrial shunt). Despite extensive research with the goal of finding the ultimate shunt design, no such design was found.18–21 The development of effective treatment techniques has begun with the Torkildsen ventriculo-cisternal shunt around 1920.22 Walter Dandy explored experimental treatment with endoscopy and choroid plexus coagulation.23 Around 1950

20

CHAPTER 1

Holter and Pudenz designed their first unidirectional valve type shunts. These shunts have become more and more sophisticated, and VP-shunts have evolved to the standard treatment of hydrocephalus.18,19

In the 1990’s ventriculoscopy has shown an enormous revival partly due to technical improvements. The ETV has become a preferred alternative treatment option in case of triventricular hydrocephalus. Although enormous improvements in treatment of hydrocephalus has been made, the risk factors of each treatment modality have also become clearer.Shunt implantation creates the risks of infections, haemorrhage and shunt dysfunction.24 Endoscopic treatment carries its own small risks of procedural morbidity and the long-term risk of recurrent obstruction. In addition, the fundamental question of optimal treatment for the brain has become more apparent. The type of treatment, the timing and the prevention of adverse effects appear to be of major importance in this respect. Fundamental knowledge about the CSF hydrodynamics during development of the central nervous system and as integral part of the hemodynamics of the blood circulation remains crucial for further improvements of neurosurgical handling.Our study has focused on two major aspects of the endoscopic treatment in hydrocephalus patients.First the definition of proper indications for ETV was further analyzed using the ETVSS in a retrospective series. The same predictive score has been evaluated in re-ETV cases in a nationwide multicentre study.Second the acquisition and evaluation of technical endoscopic skills have been explored using a newly designed model in combination with an assessment instrument.Optimal performance of a surgical technique may affect the outcome. This in combination with variance of patient exposure in number and morphology, illustrates the potential benefits of acquisition of skills in a controlled standard fashion. Simulation-based education may be ideal for this, as it may help: acquiring new skills for residentsmaintaining proper performance of standardized procedures for more experienced neurosurgeons (life-long learning and maintenance of skills). This safety issue is becoming more and more part of standard practice of care.

21


Endoscopic third ventriculostomy

We focus on ETV as treatment for hydrocephalus. After the first introduction of neuroendoscopic treatment for hydrocephalus in the 1910’s it soon fell out of grace due to high morbidity and mortality, however, with major technological advances neuroendoscopy made a comeback in the 1990’s.25,26 Slowly but surely intraventricular endoscopy became a cornerstone in the treatment of hydrocephalus without any randomized trials to assess superiority of ETV over shunt in selected cases.1,27 There has also been an expansion of patients selected for ETV, whereas in the beginning only obstructive hydrocephalus qualified for the procedure, the indications for ETV now also include communicating hydrocephalus (e.g. following infection or intraventricular hemorrhage).1 The International Infant Hydrocephalus Study is the first comparing ETV and shunt implantation, it is unique in its design as it compares long-term cognitive outcome albeit in small subgroup of patients with hydrocephalus due to aquaductal stenosis.28–30 The most important complications of ETV include intracranial hemorrhage (e.g. due to basilar artery rupture), damage to neural structures such as thalamus or fornix, central nervous system infection, permanent and transient neural complications such as memory disorders or gaze palsy, and hormonal disorders such as diabetes insipidus or weight gain.31–33 Most complications occur in the first period after introducing the procedure to the clinic (indicating a steep mention learning curve).32,34–36 Shunt implantation is a technique all neurosurgery residents will learn and master during residency, but neuroendoscopic surgery is technically more demanding and is performed by more specialized neurosurgeons.37 Still, many residents will encounter neuroendoscopic surgery and will perform these procedures during residency. For this purpose, simulation-based education could prove worthwhile in order to familiarize the resident with the neuroendoscopic equipment and procedures.

Simulation-based education

The medical history is full of examples of simulation, primarily human cadavers have been a prominent educational tool in acquiring anatomical knowledge and training new skills. Cadaver models with dynamic filling of the cerebral vasculature with colored liquid and clear fluid filling of the arachnoid cisterns

22

CHAPTER 1

have been developed,38 and animals were important for training until this day.39,40 However, for a multitude of reasons, there has been a movement towards physical and virtual reality simulators mimicking pathology in the human body (or parts of human body). This started mainly in general surgery but neurosurgery has caught up over the last decades and over the course of last few years there has been a surge in developments, there is a widespread academic and commercial interest in simulation-based education.41 For neuroendoscopy there are specific problems with human or animal models for training procedures. It is necessary to be able to access the ventricular system. In a normal situation, most people would have a slender non-dilated ventricle system and patients who are selected for neuroendoscopic procedures will have a dilated ventricle system allowing an endoscope to penetrate it without doing too much damage to the surrounding anatomical structures (e.g. thalamus, fornix).

Different simulation platforms

For neuroendoscopy there are few reports on physical simulators and virtual reality simulators. First we will give an overview of the physical simulators, followed with an overview of virtual reality platforms for neuroendoscopy. Until recently, most physical models would be rigid and as such lacking realistic haptics.42 There came a change with physical simulators such as the Sinus Model Oto-Rhino Neuro Trainer (SIMONT) simulator, which has been one of the earlier high fidelity synthetic simulators.43,44 Later, the same group presented a pediatric model called Anatomical Simulator for Pediatric Neurosurgery (ASPEN).45 In chapter 6 we present the SickKids brain simulator.46 In the process of writing this introduction there have been more synthetic brain simulators introduced to the field, and with the increased accessibility and ease of use of three-dimensional modelling and printing technology it is probable that more will follow.47 Newer examples are the Realistic Operative Workstation for Educating Neurosurgical Apprentices (ROWENA),48,49 a simulator presented by Waran et al.,50 and the BrightMatter™ Simulate.51 The development of virtual reality simulators for neuroendoscopy has been reviewed extensively before.6,52 The physical and virtual reality simulators that have been developed are very sophisticated but laboratory dissection training remains to be a cornerstone in neurosurgerical training.53

23


Developing simulation-based educational curriculum

The new modes of training are being incorporated into the neurosurgerical residency curriculum, for example as boot camp courses. The model-based simulation has been developed in the United States of America,54 but also in Canada in the form of a rookie camp to ease the transition for medical students into the neurosurgery residency program.55 During these courses or specific modules with focus on cranial, spine or vascular surgery there was also a need for standardized assessment methods.56 Assessment instruments are an important part of an educational curriculum, for tracking the learning curve and providing standardized feedback.

Assessment of surgical skill

There is extensive literature on methods of assessment tool development and the validation process.57–59 A contemporary approach to validity arguments, currently, Kane’s validation framework which is an argument-based approach to validation.59,60 Kane’s validity framework sees validity as a series of arguments,61 but there are some authors who disagree with parts of the framework. For example, should implications of the assessment instrument be regarded as part of a validity argument? An example of an implication of assessment instruments is to use the instrument to evaluate whether or not a neurosurgery resident is ready to operate on a real patient in the operating room. Cook et al. think that the implications are part of the validity argument.60 But others disagree because even though implications do depend on the validity of the test, the validity does not depend on the implications.62 To explain: if strong validity evidence has been published on an assessment instrument, then the instrument may be used for high stakes assessment but on the other hand, the validity of the instrument does not depend on its use. In general surgery, the need for standardized testing prompted developing objective structured assessment of technical skill (OSATS) in the end of the 1990s.63 In subsequent years, many other examples followed, validity evidence was gathered for various assessment instruments but not for neurosurgical procedures in spite of calls for development of such tools.64 Parallel to the increased interested in simulation, there has been a boost in enthusiasm for standardisation of evaluation of technical competency in neurosurgery.

24

CHAPTER 1

We report our experience with developing and gathering validity evidence for the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT).65,66 Sarkiss et al. present an adaptation of OSATS and show high interrater reliability.67 Rooney et al present a series of assessment instruments for evaluation of skills in placing an external ventricle drain, a procedure specific checklist called Ventriculostomy-Procedural Assessment Tool (V-PAT) and a modified OSATS.68 Other examples of neurosurgical assessment tools – all reported in 2015 – are the Northwestern Objective Microanastomosis Assessment Tool (NOMAT),69 and preliminary validity evidence of OSATS for five procedures: pediatric neurosurgery,70 specific spine simulator,71 anterior cervical discectomy and fusion,72 posterior cervical fusion,72 and durotomy repair,72 but thus far, these assessment instruments lack validity evidence.In virtual reality simulation, there are some metrics that are used to assess competence, motion tracking is an example of such metric.73,74 We compared a VR simulator head-to-head with a physical simulator and found that each have their own advantages and disadvantages and selection of a simulation platform – either physical or VR – must be based on educational objectives.5 For learning anatomy the VR simulator was the preferred choice, and developing technical skills and manual dexterity using neuroendoscopic tools may be aided with the physical simulator.

25


References

1. Kahle KT, Kulkarni A V, Limbrick DD, Warf BC. Hydrocephalus in children. Lancet. 2015;55(11):502-507. doi:10.1016/S0140-6736(15)60694-8.

2. Di Rocco C, Massimi L, Tamburrini G. Shunts vs endoscopic third ventriculostomy in infants: are there different types and/or rates of complications? A review. Childs Nerv Syst. 2006;22(12):1573-1589. doi:10.1007/s00381-006-0194-4.

3. Kulkarni A, Drake JM, Mallucci CL, Sgouros S, Roth J, Constantini S. Endoscopic third ventriculostomy in the treatment of childhood hydrocephalus. J Pediatr. 2009;155(2):254-9.e1. doi:10.1016/j.jpeds.2009.02.048.

4. Breimer GE, Dammers R, Woerdeman PA, et al. Endoscopic third ventriculostomy (ETV) and repeat ETV in 624 pediatric patients: the Dutch experience. Neurosurgery. 2016.

5. Breimer GE, Haji FA, Bodani V, et al. Simulation-based Education for Endoscopic Third Ventriculostomy: A Comparison Between Virtual and Physical Training Models. Oper Neurosurg. June 2016:1. doi:10.1227/NEU.0000000000001317.

6. Cohen AR, Lohani S, Manjila S, Natsupakpong S, Brown N, Cavusoglu MC. Virtual reality simulation: basic concepts and use in endoscopic neurosurgery training. Childs Nerv Syst. 2013;29(8):1235-1244. doi:10.1007/s00381-013-2139-z.

7. Coelho G, Zanon N, Warf BC. The role of simulation in neurosurgery. Child’s Nerv Syst. 2014;30:1997-2000. doi:10.1016/j.suc.2010.02.013.

8. Alaraj A, Lemole MG, Finkle JH, et al. Virtual reality training in neurosurgery: Review of current status and future applications. Surg Neurol Int. 2011;2:52. doi:10.4103/2152-7806.80117.

9. Cobb MIPH, Taekman JM, Zomorodi AR, Gonzalez LF, Turner DA. Simulation in Neurosurgery-A Brief Review and Commentary. World Neurosurg. 2015;89:583-586. doi:10.1016/j.wneu.2015.11.068.

10. Ganju A, Aoun SG, Daou MR, et al. The Role of Simulation in Neurosurgical Education: A Survey of 99 United States Neurosurgery Program Directors. World Neurosurg. 2013;80(5):e1-e8. doi:10.1016/j.wneu.2012.11.066.

11. Zanello M, Zerah M, Sainte-Rose C, Di Rocco F. Virtual simulation in neurosurgery: a comparison between pediatric and general neurosurgeons. Acta Neurochir (Wien). 2014;156(11):2215-2216. doi:10.1007/s00701-014-2214-6.

12. Tully HM, Dobyns WB. Infantile hydrocephalus: A review of epidemiology, classification and causes. Eur J Med Genet. 2014;57(8):359-368. doi:10.1016/j.ejmg.2014.06.002.

13. Laurence KM, Coates S. The Natural History of Hydrocephalus: Detailed Analysis of 182 Unoperated Cases. Arch Dis Child. 1962;37(194):345-362. doi:10.1136/adc.37.194.345.

14. Yashon D, Jane JA, Sugar O. The course of severe untreated infantile hydrocephalus. Prognostic significance of the cerebral mantle. J Neurosurg. 1965;23(5):509-516. doi:10.3171/jns.1965.23.5.0509.

15. Rekate HL. A Contemporary Definition and Classification of Hydrocephalus. Semin Pediatr Neurol. 2009;16(1):9-15. doi:10.1016/j.spen.2009.01.002.

16. Drake JM. The surgical management of pediatric hydrocephalus. Neurosurgery. 2008;62 Suppl 2(Supplement 2):633-40-2. doi:10.1227/01.neu.0000316268.05338.5b.

17. Breimer GE, Sival DA, Hoving EW. Low-pressure valves in hydrocephalic children: a retrospective analysis. Childs Nerv Syst. 2012;28(3):469-473. doi:10.1007/s00381-011-1664-x.

26

CHAPTER 1

18. Drake JM, Kestle JRW, Milner R, et al. Randomized trial of cerebrospinal fluid shunt valve design in pediatric hydrocephalus. Neurosurgery. 1998;43:294-305. doi:10.1097/00006123-199808000-00068.

19. Kestle J, Drake J, Milner R, et al. Long-term follow-up data from the shunt design trial. Pediatr Neurosurg. 2000;33(5):230-236. doi:10.1159/000055960.

20. Pollack IF, Albright AL, Adelson PD. A randomized, controlled study of a programmable shunt valve versus a conventional valve for patients with hydrocephalus. Hakim-Medos Investigator Group. Neurosurgery. 1999;45(6):1399-408-11. http://www.ncbi.nlm.nih.gov/pubmed/10598708.

21. Kestle JRW, Walker ML. A multicenter prospective cohort study of the Strata valve for the management of hydrocephalus in pediatric patients. J Neurosurg. 2005;102(2 Suppl):141-145. doi:10.3171/ped.2005.102.2.0141.

22. Eide PK, Lundar T. Arne Torkildsen and the ventriculocisternal shunt: the first clinically successful shunt for hydrocephalus. J Neurosurg. 2016;124(5):1421-1428. doi:10.3171/2015.1.JNS142659.

23. Pudenz RH. The surgical treatment of hydrocephalus—An historical review. Surg Neurol. 1981;15(1):15-26. doi:10.1016/S0090-3019(81)80084-5.

24. Baird LC. First Treatment in Infants With Hydrocephalus: The Case for Endoscopic Third Ventriculostomy/Choroid Plexus Cauterization. Neurosurgery. 2016;63 Suppl 1(1):78-82. doi:10.1227/NEU.0000000000001299.

25. Decq P, Schroeder HWS, Fritsch M, Cappabianca P. A History of Ventricular Neuroendoscopy. World Neurosurg. 2013;79(2):S14.e1-S14.e6. doi:10.1016/j.wneu.2012.02.034.

26. Schmitt PJ, Jane JA. A lesson in history: the evolution of endoscopic third ventriculostomy. Neurosurg Focus. 2012;33(2):E11. doi:10.3171/2012.6.FOCUS12136.

27. Nishiyama K, Yoshimura J, Fujii Y. Limitations of Neuroendoscopic Treatment for Pediatric Hydrocephalus and Considerations from Future Perspectives. Neurol Med Chir (Tokyo). 2015;55(8):611-616. doi:10.2176/nmc.ra.2014-0433.

28. Sgouros S, Kulkharni A V, Constantini S. The International Infant Hydrocephalus Study: concept and rational. Childs Nerv Syst. 2006;22(4):338-345. doi:10.1007/s00381-005-1253-y.

29. Constantini S, Sgouros S, Kulkarni A. Neuroendoscopy in the youngest age group. World Neurosurg. 2013;79(2 Suppl):S23.e1-11. doi:10.1016/j.wneu.2012.02.003.

30. Kulkarni A V., Sgouros S, Constantini S. International Infant Hydrocephalus Study: initial results of a prospective, multicenter comparison of endoscopic third ventriculostomy (ETV) and shunt for infant hydrocephalus. Child’s Nerv Syst. 2016;32(6):1039-1048. doi:10.1007/s00381-016-3095-1.

31.Bouras T, Sgouros S. Complications of endoscopic third ventriculostomy. J Neurosurg Pediatr. 2011;7(6):643-649. doi:10.3171/2011.4.PEDS10503.

32. Schroeder HWS, Niendorf W-R, Gaab MR. Complications of endoscopic third ventriculostomy. J Neurosurg. 2002;96(6):1032-1040. doi:10.3171/jns.2002.96.6.1032.

33. Kawsar KA, Haque MR, Chowdhury FH. Avoidance and management of perioperative complications of endoscopic third ventriculostomy: the Dhaka experience. J Neurosurg. May 2015:1-6. doi:10.3171/2014.11.JNS14395.

34. Navarro R, Gil-Parra R, Reitman AJ, Olavarria G, Grant JA, Tomita T. Endoscopic third ventriculostomy in children: early and late complications and their avoidance. Childs Nerv Syst. 2006;22(5):506-513. doi:10.1007/s00381-005-0031-1.

35. Feng H, Huang G, Liao X, et al. Endoscopic third ventriculostomy in the management of obstructive hydrocephalus: an outcome analysis. J Neurosurg. 2004;100(4):626-633. doi:10.3171/jns.2004.100.4.0626.

27


36. Furlanetti LL, Santos MV, de Oliveira RS. The Success of Endoscopic Third Ventriculostomy in Children: Analysis of Prognostic Factors. Pediatr Neurosurg. July 2013:4-11. doi:10.1159/000353619.

37. van Lindert EJ, Beems T, Grotenhuis JA. The role of different imaging modalities: is MRI a conditio sine qua non for ETV? Child’s Nerv Syst. 2006;22(12):1529-1536. doi:10.1007/s00381-006-0189-1.

38. Aboud E, Al-Mefty O, Yaşargil MG. New laboratory model for neurosurgical training that simulates live surgery. J Neurosurg. 2002;97(6):1367-1372. doi:10.3171/jns.2002.97.6.1367.

39. Becerra Romero ADC, Zicarelli CA, Pinto FCG, Pasqualucci CA, Aguiar PHP. Simulation of endoscopic third ventriculostomy in fresh cadaveric specimens. Minim Invasive Neurosurg. 2009;52(3):103-106. doi:10.1055/s-0029-1231080.

40. Liu JKC, Kshettry VR, Recinos PF, Kamian K, Schlenk RP, Benzel EC. Establishing a surgical skills laboratory and dissection curriculum for neurosurgical residency training. J Neurosurg. May 2015:1-8. doi:10.3171/2014.11.JNS14902.

41. Rehder R, Abd-El-Barr M, Hooten K, Weinstock P, Madsen JR, Cohen AR. The role of simulation in neurosurgery. Child’s Nerv Syst. October 2015. doi:10.1007/s00381-015-2923-z.

42. Sierra R, Dimaio SP, Wada J, et al. Patient specific simulation and navigation of ventriculoscopic interventions. Stud Health Technol Inform. 2007;125:433-435.

43. Zymberg S, Vaz-Guimarães Filho F, Lyra M. Neuroendoscopic training: presentation of a new real simulator. Minim Invasive Neurosurg. 2010;53(1):44-46. doi:10.1055/s-0029-1246169.

44. Filho FVG, Coelho G, Cavalheiro S, Lyra M, Zymberg ST. Quality assessment of a new surgical simulator for neuroendoscopic training. Neurosurg Focus. 2011;30(4):E17. doi:10.3171/2011.2.FOCUS10321.

45. Coelho G, Zymberg S, Lyra M, Zanon N, Warf B. New anatomical simulator for pediatric neuroendoscopic practice. Child’s Nerv Syst. 2015;31(2):213-219. doi:10.1007/s00381-014-2538-9.

46. Breimer GE, Bodani V, Looi T, Drake JM. Design and evaluation of a new synthetic brain simulator for endoscopic third ventriculostomy. J Neurosurg Pediatr. 2015;15(1):82-88. doi:10.3171/2014.9.PEDS1447.

47. Marro A, Bandukwala T, Mak W. Three-Dimensional Printing and Medical Imaging: A Review of the Methods and Applications. Curr Probl Diagn Radiol. 2015:1-8. doi:10.1067/j.cpradiol.2015.07.009.

48. Ashpole RD. Introducing Rowena: a simulator for neurosurgical training. Bull R Coll Surg Engl. 2015;97(7):299-301. doi:10.1308/rcsbull.2015.299.

49. Javed S, Berhanu M. Rowena: the trainee’s perspective on simulation. Bull R Coll Surg Engl. 2015;97(7):303-304. doi:10.1308/rcsbull.2015.303.

50. Waran V, Narayanan V, Karuppiah R, et al. Neurosurgical Endoscopic Training via a Realistic 3-Dimensional Model. Simul Heal. 2015;10(1):43-48. doi:10.1097/SIH.0000000000000060.

51. BrightMatterTM Simulate. http://synaptivemedical.com/products/simulate/. Accessed October 5, 2015.

52. Neubauer A, Wolfsberger S. Virtual endoscopy in neurosurgery: a review. Neurosurgery. 2013;72 Suppl 1(January):97-106. doi:10.1227/NEU.0b013e31827393c9.

53. Kshettry VR, Mullin JP, Schlenk R, Recinos PF, Benzel EC. The Role of Laboratory Dissection Training in Neurosurgical Residency: Results of a National Survey. World Neurosurg. 2014;82(5):554-559. doi:10.1016/j.wneu.2014.05.028.

54. Selden NR, Origitano TC, Hadjipanayis C, Byrne R. Model-based simulation for early neurosurgical learners. Neurosurgery. 2013;73(SUPPL. 4):15-24. doi:10.1227/NEU.0000000000000058.

55. Haji FA, Clarke DB, Matte MC, et al. Teaching for the Transition: the Canadian PGY-1 Neurosurgery “Rookie Camp.” Can J Neurol Sci / J Can des Sci Neurol. 2015;42(1):25-33. doi:10.1017/cjn.2014.124.

28

CHAPTER 1

56. Harrop J, Lobel DA, Bendok B, Sharan A, Rezai AR. Developing a neurosurgical simulation-based educational curriculum: An overview. Neurosurgery. 2013;73(SUPPL. 4):25-29. doi:10.1227/NEU.0000000000000101.

57. Pugh DM, Wood TJ, Boulet JR. Assessing Procedural Competence. Simul Healthc J Soc Simul Healthc. 2015;10(5):288-294. doi:10.1097/SIH.0000000000000101.

58. Cook DA, Zendejas B, Hamstra SJ, Hatala R, Brydges R. What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment. Adv Heal Sci Educ. 2013:1-18. doi:10.1007/s10459-013-9458-4.

59. Downing SM. Validity: on the meaningful interpretation of assessment data. Med Educ. 2003;37(9):830-837. doi:10.1046/j.1365-2923.2003.01594.x.

60. Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane’s framework. Med Educ. 2015;49(6):560-575. doi:10.1111/medu.12678.

61. Schuwirth LWT, van der Vleuten CPM. Programmatic assessment and Kane’s validity perspective. Med Educ. 2012;46:38-48. doi:10.1111/j.1365-2923.2011.04098.x.

62. Knorr M, Klusmann D. The trouble with validity: what is part of it and what is not? Med Educ. 2015;49(6):550-552. doi:10.1111/medu.12738.

63. Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84(2):273-278. doi:10.1046/j.1365-2168.1997.02502.x.

64. Rothstein BD, Selman WR. Evaluating Simulation as a Teaching Tool in Neurosurgery. Virtual Mentor. 2015;17(1):33-36. doi:10.1001/virtualmentor.2015.17.01.medu1-1501.

65. Breimer GE, Haji FA, Cinalli G, Hoving EW, Drake JM. Validity Evidence for the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). Oper Neurosurg. November 2015:1. doi:10.1227/NEU.0000000000001158.

66. Breimer GE, Haji FA, Hoving EW, Drake JM. Development and content validation of performance assessments for endoscopic third ventriculostomy. Child’s Nerv Syst. 2015;31(8):1247-1259. doi:10.1007/s00381-015-2716-4.

67. Sarkiss CA, Philemond S, Lee J, et al. Neurosurgical Skills Assessment: Measuring Technical Proficiency in Neurosurgery Residents Through Intraoperative Video Evaluations. World Neurosurg. 2016;89:1-8. doi:10.1016/j.wneu.2015.12.052.

68. Rooney DM, Tai BL, Sagher O, Shih AJ, Wilkinson DA, Savastano LE. Simulator and 2 tools: Validation of performance measures from a novel neurosurgery simulation model using the current Standards framework. Surgery. 2016:1-9. doi:10.1016/j.surg.2016.03.035.

69. Aoun SG, Ahmadieh TY El, Tecle NE El, et al. A pilot study to assess the construct and face validity of the Northwestern Objective Microanastomosis Assessment Tool. J Neurosurg. 2015. doi:10.3171/2014.12.JNS131814.Disclosure.

70. Hadley C, Lam SK, Briceño V, Luerssen TG, Jea A. Use of a formal assessment instrument for evaluation of resident operative skills in pediatric neurosurgery. J Neurosurg Pediatr. August 2015:1-8. doi:10.3171/2015.1.PEDS14511.

71. Harrop J, Rezai AR, Hoh DJ, Ghobrial GM, Sharan A. Neurosurgical training with a novel cervical spine simulator: posterior foraminotomy and laminectomy. Neurosurgery. 2013;73 Suppl 1(4):94-99. doi:10.1227/NEU.0000000000000103.

72. Zammar SG, Hamade YJ, Aoun RJN, et al. The Cognitive and Technical Skills Impact of the Congress of Neurological Surgeons Simulation Curriculum on Neurosurgical Trainees at the 2013 Neurological Society of India Meeting. World Neurosurg. 2015;83(4):419-423. doi:10.1016/j.wneu.2014.12.006.

29


73. Gélinas-Phaneuf N, Del Maestro RF. Surgical expertise in neurosurgery: Integrating theory into practice. Neurosurgery. 2013;73 Suppl 1(4):30-38. doi:10.1227/NEU.0000000000000115.

74. Alotaibi FE, AlZhrani GA, Mullah MAS, et al. Assessing Bimanual Performance in Brain Tumor Resection With NeuroTouch, a Virtual Reality Simulator. Neurosurgery. 2015;11(1):1. doi:10.1227/NEU.0000000000000631.

Chapter 2

An external validation of the ETVSS for both short-term and

long-term predictive adequacy in 104 pediatric patients

G. E. BreimerD. A. Sival

M. G. J. Brusse-KeizerE. W. Hoving

Child’s Nerv Syst 2013; 29: 1305-1311doi: 10.1007/s00381-013-2122-8

32

CHAPTER 2

Abstract

Purpose

This study aims to provide external validation of the “Endoscopic Third Ventriculostomy Success Score (ETVSS)” for both short-term and long-term predictive adequacy.

Methods

Between 1998 and 2007, we collected clinical follow-up data (after 6 and 36 months) of all 104 hydrocephalic children (<18 years of age) treated by endoscopic third ventriculostomy (ETV) in our hospital. Predictive adequacy of ETVSS for 6- and 36-month periods was tested by means of an unpaired t test, Hosmer–Lemeshow “goodness-of- fit” test, and area under the receiver operating characteristic curve.

Results

Mean follow-up was 73.4 months. For both the short-term (6 months) and the long-term (36 months) periods, the mean predicted probability of ETV for the patients with successful ETV treatment was significantly higher than in the patients with failed ETV treatment. The areas under the curve for the short- and long-term periods were, respectively, 0.82 (95 % CI 0.71–0.92) and 0.73 (95 % CI 0.62–0.84). For patients with moderate ETVSS (50–70), the median age at first ETV was significantly higher for patients with successful ETV for both short- and long-term periods.

Conclusion

In hydrocephalic children, the ETVSS is a useful tool for prediction of outcome after ETV treatment. The ETVSS is more adequate in predicting short-term than long-term success. In our population, it is suggested that success rate for patients with moderate ETVSS could be improved if more weight is attributed to age at first ETV.

33

External validation of the ETVSS

Introduction

Endoscopic third ventriculostomy (ETV) is a commonly used and accepted method for treatment of obstructive hydrocephalus.1–3, 5, 11, 21, 22, 24, 27, 28 A major advantage of ETV in relation with ventriculoperitoneal shunting (VP shunt) is that the first procedure does not leave a foreign body. There is no consensus regarding the role of ETV in young pediatric patients.3, 4, 7, 15, 29 Hopefully, the International Infant Hydrocephalus Study will provide a more definite answer in the future.25 Meanwhile, the ETV Success Score (ETVSS) may be applied to predict whether an ETV is likely to be successful.17 The ETVSS is derived from three variables: patient age, etiology of hydrocephalus, and the presence of previous treatment with VP shunt. After internal and external validation, the ETVSS has been applied in many studies,8, 10, 12, 16–18, 20 revealing a good short-term prediction. This study provides external validation of the ETVSS at the 6-month follow-up. Ultimately, it is essential that the treatment continues to be successful in the long-term and therefore we perform an additional test of the long-term (36 months) adequacy of the ETVSS. A retrospective cohort study is presented of a series of children (n=104) under 18 years of age who all underwent ETV as treatment for hydrocephalus at the University Medical Centre Groningen (UMCG).

Methods

Study design

Between 1998 and 2007, we retrospectively collected all clinical data of children who received an ETV at the UMCG (Fig. 1). Ten patients died before the 6-month follow-up criterion was met; seven patients died of tumor-related causes, three patients died of other causes (Walker–Warburg syndrome, congenital heart disease, and respiratory failure due to bacterial pneumonia), and none died of causes related to hydrocephalus. All ETV procedures were performed in accordance with national guidelines, i.e., either by a neurosurgeon or a resident under direct supervision of a neurosurgeon. ETV failure was diagnosed by a clinician, with additional radiologic information.

34

CHAPTER 2

Data collection

From patient records, we collected data regarding basic patient characteristics, etiology of hydrocephalus, age at first ETV treatment, primary ETV, or secondary ETV (after previous VP shunt treatment). ETV failure was indirectly determined by additional HC requirement or death. Late ETV failure is defined as failure of ETV between 6 and 36 months. Re-ETV is defined as a repeated ETV procedure after failure of the initial ETV. According to Naftel et al., we apply the term “salvage” ETV when the ETV decision is based upon multiple shunt failures and/or poor CSF absorption, instead of straight ETV criteria.20 We report all ETV treatments fitting “salvage” category of both initial ETV and re-ETV treatments. No additional analysis was performed on the “salvage” category. In all patients, MRI scans were made pre-operatively in order to contribute to the definition of the etiology of the hydrocephalus. The following categories were identified in concordance with the original paper on ETVSS: aqueductal stenosis, post-infectious, myelomeningocele, post-intraventricular hemorrhage, tectal tumor, other brain tumor (non-tectal), and other causes.17

For each patient, the ETVSS is subsequently calculated by combining age, etiology, and previous shunt score (Table 1). The calculated scores show a range

Fig. 1 Exclusion

35


from 0 to 90 related to predicted success rates after 6 months. For example, for a 2-month-old child with an aqueductal stenosis in absence of previous CSF shunt treatment, a total score of 50 would be applicable. This score is composed of 10 points for age, 30 points for etiology of hydrocephalus, and 10 points for “previous shunt.” A patient with ETVSS of 50 is postulated to have a 50 % chance of successful ETV treatment for a period of 6 months.

Statistical analysis

We performed data analysis with the statistical package SPSS for Windows (version 20). Baseline characteristics are reported as mean with SD or as numbers with corresponding percentages. If variables were not normally distributed, values are reported as median with interquartile range (IQR). The duration of follow-up after surgery was calculated till the date of subsequent hydrocephalus treatment or date of death, or was ceased at the last date at which the patient was seen at the treating center and known to be well. The cohort was stratified based on ETVSS in three different categories: low ETVSS (≤40), moderate ETVSS (50–70), and high ETVSS (≥80). Kaplan–Meier curves indicated the shunt failure-free survival rate for the different strata. We calculated the hazard ratios for ETV failure for the different ETVSS strata by Cox regression analysis. Differences in mean ETVSS and median age in months at ETV treatment between patients with early ETV failure and late ETV failure were analyzed by, respectively, the unpaired t test and Mann–Whitney U test. We compared differences in median age at ETV treatment between patients with successful and failed ETV in the moderate ETVSS group by Mann–Whitney U test. Between-group comparisons of nominal or ordinal variables were performed

Table 1. Calculation of the ETVSS (after Kulkarni et al.17)

36

CHAPTER 2

for the moderate ETVSS group by Chisquare and/or by Fisher’s exact test. We considered p values <0.05 as statistically significant. We determined external validity of the ETVSS model by Kulkarni et al.17 over a short-term (6 months) and a long-term (36 months) follow-up period. We compared the predicted probability of ETV success between children with successful and failed ETV treatment by an unpaired t test. Performance of the predictive model was subsequently tested by the Hosmer–Lemeshow “goodness-of-fit” statistic; a significant p value rejects the null hypothesis that the model fits the data well. Model discrimination was quantified by the area under the receiver operating characteristic (ROC) curve.

Results

Patient characteristics

Patient characteristics of 104 children are presented in Table 2. The mean follow-up period was 73.4 months (SD, 38.7). The first ETV was performed at a median age of 67.7 months (IQR, 3.1–117.5). Of all ETV treatments performed in the inclusion period, seven patients qualified for the “salvage” category; in three of these patients, ETV appeared successful.

Short-term predictive adequacy (6 months)

Mean ETVSS was 66.2 (SD, 17.0), and actual overall ETV success rate after 6 months was 70.2 %. The remaining 31 (29.8 %) patients received subsequent treatment, consisting of VP shunt implantation or revision (29 children; 93.5 %), or a repeated ETV (2 children; 6.5 %). The mean predicted probability of ETV success was significantly higher in children with successful ETV than ETV failure [0.72 (SD, 0.13) vs. 0.52 (SD, 0.1), with a mean difference of 0.21 (95 % CI 0.15–0.27; p<.001)]. The Hosmer–Lemeshow statistic revealed an adequate model fit (p=.10). The area under the curve is 0.82 (95 % CI 0.71–0.92).

Long-term predictive adequacy (36 months)

In 84 of 104 (80.8 %) patients, a follow-up period of at least 36 months was

37


available. The mean ETVSS for this group was 66.0 (SD, 16.6). The actual overall ETV success rate after 36 months in this group was 48.8 % (n=41). Within the extended follow-up period of 36 months, a total of 43 (51.2 %) patients underwent subsequent treatment, of which 27 (62.8 %) patients received a VP shunt (or revision of a present VP shunt system) and 16 (37.2 %) received a re-ETV. Twenty-seven of 43 (62.8 %) patients had early failure. The mean predicted probability of ETV success was significantly higher in children with successful than failed ETVs [0.73 (SD, 0.14) vs. 0.59 (SD, 0.17), mean difference of 0.14 (95 % CI 0.071–0.20; p<.001)]. The Hosmer–Lemeshow revealed an adequate model fit (p=.77), and the area under the curve is 0.73 (95 % CI 0.62–0.84).

Abbreviations: ETV, endoscopic third ventriculostomy; IVH, intraventricular hemorrhage

Table 2. Characteristics of 104 patients

38

CHAPTER 2

ETVSS strata

According to Kulkarni et al.22, we characterized ETVSS outcomes as low, moderate, and high in 17, 47, and 40 children, respectively. In Tables 3 and 4, mean ETVSS and the actual success rate within these ETVSS strata are displayed for, respectively, the 6- and 36-month follow-up period. The ETVSS and the actual success rate for the different strata for the children under the age of 2 years are shown in Table 5. Patient characteristics of the moderate ETVSS group are indicated in Table 6. Comparing low with high ETVSS reveals a 5.5-fold chance of ETV failure within the next 36 months (95 % CI 2.5–12.3; p<.001) for the low ETVSS group. A patient with moderate ETVSS reveals a 2.2-fold chance of ETV failure than a patient with high ETVSS (95 % CI 1.1–4.6; p=.03).Secondary ETV

Mean ETVSS for the group with secondary ETV (n=20) was 70 (SD, 9.7), and actual overall ETV success rate after 6 months was 75.0 %. After 36 months (n=16), mean ETVSS was 68.8 (SD, 10.3), and the success rate dropped to 43.8 %.

Repeated ETV

Eventually, 20 patients required repeated ETV; 18 of 20 patients with minimal follow-up of 6 months were eligible for short-term analysis. The interval between

Table 3: Stratified success in 6 months’ period

Table 4: Stratified success in 36 months’ period

39


the first and second surgery was for each patient at least 3 months with a median time between first and repeated ETVof 15.8 months (IQR, 9.2–32.0). In 14 of 18 (77.8 %) patients, the re-ETV was successful for the 6-month follow-up period. The mean ETVSS for these patients was 68.9 (SD, 11.8). Sixteen patients with minimal follow-up of 36 months were eligible for long-term analysis; the mean ETVSS was 67.5 (SD, 11.8). In 10 of 16 (62.5 %) patients, the re-ETV treatment was successful in the long-term.

Table 5: Stratified success in 6 months’ period for patients < 2 years

Table 6. characteristics of moderate ETVSS (50 – 70)

Abbreviations: ETV, endoscopic third ventriculostomy; IVH, intra-ventricular hemorrhage

40

CHAPTER 2

Late ETV failure

Late ETV failure is defined as failure of ETV between 6 and 36 months. Sixteen of 43 (37.2 %) had late ETV failure after a median period of 15.8 months (IQR, 9.6–26.8). Comparing patients with “late ETV failures” (n=16) vs. “early ETV failures” within 6 months (n=31) revealed a significantly higher mean ETVSS in the first group [late vs. early ETV failure, 68.8 (SD, 12.6) vs. 51.6 (SD, 16.6), respectively; mean difference of 17.1 (95 % CI 7.6–26.7); p=.001]. Median age in months at first ETV, 79.1 (IQR, 9.7–132.5) and 2.4 (IQR, 0.4–19.1), p=.002, differed significantly between patients with late ETV failure and patients with early ETV failure, respectively.

Mortality

As previously stated, ten patients died before the 6-month follow-up criterion was met. During follow-up, mortality rate was 11.5 %. Twelve of 104 children died, with a mean age 6.7 years (SD, 4.3; min, 1.7; max, 15.2). The underlying etiology for hydrocephalus involved: aqueductal stenosis (n=2), non-tectal brain tumor (n=7), and other (n=3). Six patients received subsequent hydrocephalus treatment prior to death. One patient died of shunt dysfunction. The remaining 11 patients died independent of hydrocephalus treatment, i.e., by: malignancy (n=7), epilepsy (n=1), hemolytic–uremic syndrome after meningitis (n=1), acute respiratory distress syndrome after sepsis (n=1), and unknown cause (n=1).

Discussion

In the present study, we provide data regarding the short-term and long-term predictive adequacy of the ETVSS. We found significantly higher mean predicted probability of ETV success in children with successful than failed ETVs for both follow-up periods. Furthermore, the model discrimination, quantified by the area under the ROC curve, was found to be adequate. Therefore, short-term and long-term analyses reveal that ETVSS is a suitable predictive model.Kulkarni et al. developed the ETVSS model.17 In their internal validation set, the C-statistic (equivalent of the area under the ROC curve) was 0.68. Naftel et al. provided the first external validation of the ETVSS for the 6-month period,

41


with a C-statistic of 0.74 (95 % CI 0.65–0.83).20 For the 6- and 36-month periods, we found a C-statistic of 0.82 (95 % CI 0.71–0.92) and 0.73 (95 % CI 0.62–0.84), respectively. Durnford et al. performed a retrospective analysis for both 6- and 36-month follow-up periods.8 They found an inverse relationship between short- and long-term followup. At the 6-month follow-up period, there was no significant difference between the mean ETVSS for patients with successful vs. patients with failed ETV. In comparison, for the 36-month follow-up period, the mean ETVSS differed significantly in favor of the patients with successful ETV. These findings suggest a positive selection of ETV patients in the long-term group. In contrast to these findings, we report a decreased long-term predictive adequacy. In our study, late failures occurred both in patients with high (14 %) and moderate (21 %) ETVSS (Kaplan–Meier survival curve, Fig. 2), whereas Durnford et al. reported that late ETV failures are predominantly confined to the moderate ETVSS category.In accordance with literature, we observed that a large proportion of patients, 62.8 %, revealed an early ETV failure.17, 24 However, we also observed a relatively large proportion of ETV failures after the first 6-month period (about 40 vs. 20 % by Durnford et al.). This relatively large group of patients with late failure in spite of high or moderate ETVSS in our series might be explained by selection bias in a historical series. It may also be postulated that other factors than the elements of the ETVSS may be involved in order to explain late failure after ETV.Comparison of patient characteristics between the groups with early and late ETV failure revealed a significantly younger age and lower ETVSS in the first group. In literature, late ETV failure is described with potentially fatal consequences.6,

9, 19 Kadrian et al. reported that age is the only significant factor to predict long-term ETV success.14

In 78 %, re-ETV appeared successful after an evaluation period of 6 months. This relatively high re-ETV success rate has also been reported in literature11, 13,

14, 23, 26, 28 and may implicate that re-ETV could be considered before proceeding to CSF diversion by means of a VP shunt. Especially, analysis of the moderate ETVSS category could elucidate whether a patient should be treated by means of ETV or VP shunt. This decision will

42

CHAPTER 2

be based on the personal judgment and preference of the treating surgeon in consultation with patient and parents. More in-depth analysis of the moderate ETVSS group was performed in order to determine which variables may explain the difference in performance of the ETVSS model. The main characteristic differing between failure and success of ETV treatment in this category is patient age at ETV (Table 6).The median age at first ETV is significantly higher in successful ETV for both the short- and long-term follow-up periods. This could suggest that the influence by age at which the ETV is performed might be of pivotal importance. The success rate for the moderate ETVSS group might be improved by attributing more weight to the age factor.We recognize some limitations to our study. The present study concerns a retrospective design with a limited sample size. However, a randomized controlled prospective design would probably not apply to ethical standards, since individual choices between ETV and VPD treatment options should still depend upon the best predicted benefit (ETV success scores) in the first place.We cannot exclude that multiple considerations before ETV treatment have caused a selection bias. Some hydrocephalus patients have a problematic treatment course with multiple shunt revisions and difficulty finding correct shunting profile. In such patients in whom ETV placement could be regarded

Fig. 2 Kaplan-Meier survival curves for ETVs in 104 patients stratified by ETVSS

43


as a last resort, the major advantage of shunt independence could outweigh the limited chance of success. In this specific “salvage” category, risk assessment should therefore be interpreted differently.20

For neurosurgeons considering treating hydrocephalus in a pediatric patient by means of ETV, the ETVSS provides a useful tool in everyday practice. The total score is easy to implement, and the calculated chances of success during the first 6 months are sufficiently validated in previously mentioned series. With the application of the score, the treatment of hydrocephalus could become more uniform and transparent. In addition to predicting chances of successful ETV treatment for individual patients, the ETVSS (when consistently used by future authors) may contribute to a more reliable comparison of pediatric cohorts of hydrocephalic children treated with ETV described in literature.

44

CHAPTER 2

References

1. Baldauf J, Oertel J, Gaab MR, Schroeder HW (2007) Endoscopic third ventriculostomy in children younger than 2 years of age. Childs Nerv Syst 23:623–626

2. Bouras T, Sgouros S (2011) Complications of endoscopic third ventriculostomy. J Neurosurg Pediatr 7(Bouras T, Sgouros S):643–649

3. de Ribaupierre S, Rilliet B, Vernet O, Regli L, Villemure JG (2007) Third ventriculostomy vs ventriculoperitoneal shunt in pediatric obstructive hydrocephalus: results from a Swiss series and literature review. Childs Nerv Syst 23:527–533

4. Di Rocco C, Massimi L, Tamburrini G (2006) Shunts vs endoscopic third ventriculostomy in infants: are there different types and/or rates of complications? A review. Childs Nerv Syst 22:1573–1589

5. Drake JM (2007) Endoscopic third ventriculostomy in pediatric patients: the Canadian experience. Neurosurgery 60:816–881

6. Drake J, Chumas P, Kestle J, Pierre-Kahn A, Vinchon M, Brown J, Pollack IF, Arai H (2006) Late rapid deterioration after endoscopic third ventriculostomy: additional cases and review of the literature. J Neurosurg 105:118–126

7. Drake JM, Kulkarni AV, Kestle J (2009) Endoscopic third ventriculostomy versus ventriculoperitoneal shunt in pediatric patients: a decision analysis. Childs Nerv Syst 25:467–472

8. Durnford AJ, Kirkham FJ, Mathad N, Sparrow OC (2011) Endoscopic third ventriculostomy in the treatment of childhood hydrocephalus: validation of a success score that predicts longterm outcome. J Neurosurg Pediatr 8:489–493

9. Fabiano AJ, Doyle K, Grand W (2010) Delayed stoma failure in adult communicating hydrocephalus after initial successful treatment by endoscopic third ventriculostomy: case report. Neurosurgery 66:E1210–E1211

10. Fani L, de Jong TH, Dammers R, van Veelen ML (2013) Endoscopic third ventriculocisternostomy in hydrocephalic children under 2 years of age: appropriate or not? A single-center retrospective cohort study. Childs Nerv Syst 29:419–423

11. Feng H, Huang G, Liao X, Fu K, Tan H, Pu H, Cheng Y, Liu W, Zhao D (2004) Endoscopic third ventriculostomy in the management of obstructive hydrocephalus: an outcome analysis. J Neurosurg 100:626–633

12. García LG, López BR, Botella GI, Páez MD, da Rosa SP, Rius F, Sánchez MA (2012) Endoscopic Third Ventriculostomy Success Score (ETVSS) predicting success in a series of 50 pediatric patients. Are the outcomes of our patients predictable? Childs Nerv Syst 28:1157–1162

13. Hellwig D, Giordano M, Kappus C (2012) Redo third ventriculostomy. World Neurosurg. doi:10.1016/j.wneu. 2012.02.006

14. Kadrian D, van Gelder J, Florida D, Jones R, Vonau M, Teo C, Stening W, Kwok B (2005) Long-term reliability of endoscopic third ventriculostomy. Neurosurgery 56:1271–1278

15. Kulkarni AV, Drake JM, Kestle JR, Mallucci CL, Sgouros S, Constantini S (2010) Endoscopic third ventriculostomy vs cerebrospinal fluid shunt in the treatment of hydrocephalus in children: a propensity score-adjusted analysis. Neurosurgery 67:588–593

16. Kulkarni AV, Drake JM, Kestle JR, Mallucci CL, Sgouros S, Constantini S (2010) Predicting who will benefit from endoscopic third ventriculostomy compared with shunt insertion in childhood hydrocephalus using the ETV success score. J Neurosurg Pediatr 6:310–315

17. Kulkarni AV, Drake JM, Mallucci CL, Sgouros S, Roth J, Constantini S (2009) Endoscopic third ventriculostomy in the treatment of childhood hydrocephalus. J Pediatr 155:254–259

45


18. Kulkarni AV, Riva-Cambrin J, Browd SR (2011) Use of the ETV success score to explain the variation in reported endoscopic third ventriculostomy success rates among published case series of childhood hydrocephalus. J Neurosurg Pediatr 7:143–146

19. Lipina R, Palecek T, Reguli S, Kovarova M (2007) Death in consequence of late failure of endoscopic third ventriculostomy. Childs Nerv Syst 23:815–819

20. Naftel RP, Reed GT, Kulkarni AV,Wellons JC (2011) Evaluating the Children’s Hospital of Alabama endoscopic third ventriculostomy experience using the Endoscopic Third Ventriculostomy Success Score: an external validation study. J Neurosurg Pediatr 8:494–501

21. O’Brien DF, Javadpour M, Collins DR, Spennato P, Mallucci CL (2005) Endoscopic third ventriculostomy: an outcome analysis of primary cases and procedures performed after ventriculoperitoneal shunt malfunction. J Neurosurg 103:393–400

22. O’Brien DF, Seghedoni A, Collins DR, Hayhurst C, Mallucci CL (2006) Is there an indication for ETV in young infants in aetiologies other than isolated aqueduct stenosis? Childs Nerv Syst 22:1565–1572

23. Peretta P, Cinalli G, Spennato P, Ragazzi P, Ruggiero C, Aliberti F, Carlino C, Cianciulli E (2009) Long-term results of a second endoscopic third ventriculostomy in children: retrospective analysis of 40 cases. Neurosurgery 65:539–547

24. Sacko O, Boetto S, Lauwers-Cances V, Dupuy M, Roux FE (2010) Endoscopic third ventriculostomy: outcome analysis in 368 procedures. J Neurosurg Pediatr 5:68–74

25. Sgouros S, Kulkharni AV, Constantini S (2006) The International Infant Hydrocephalus Study: concept and rational. Childs Nerv Syst 22:338–345

26. Siomin V, Weiner H, Wisoff J, Cinalli G, Pierre-Kahn A, Saint-Rose C, Abbott R, Elran H, Beni-Adani L, Ouaknine G, Constantini S (2001) Repeat endoscopic third ventriculostomy: is it worth trying? Childs Nerv Syst 17:551–555

27. Sufianov AA, Sufianova GZ, Iakimov IA (2010) Endoscopic third ventriculostomy in patients younger than 2 years: outcome analysis of 41 hydrocephalus cases. J Neurosurg Pediatr 5:392–401

28. Surash S, Chumas P, Bhargava D, Crimmins D, Straiton J, Tyagi A (2010) A retrospective analysis of revision endoscopic third ventriculostomy. Childs Nerv Syst 26:1693–1698

29. Tuli S, Alshail E, Drake J (1999) Third ventriculostomy versus cerebrospinal fluid shunt as a first procedure in pediatric hydrocephalus. Pediatr Neurosurg 30:11–15

Chapter 3

Endoscopic third ventriculostomy (ETV) and repeat ETV in

pediatric patients: the Dutch experience

Gerben E. BreimerRuben Dammers

Peter A. WoerdemanDennis R. Buis

Hans DelyeMarjolein Brusse-Keizer

Eelco W. HovingDutch Pediatric Neurosurgery Study Group

Submitted

48

CHAPTER 3

Abstract

Background

After an endoscopic third ventriculostomy (ETV) procedure some patients will develop recurrent symptoms of hydrocephalus. The optimal treatment for these patients is not clear; either repeat ETV (re-ETV) or cerebrospinal fluid shunt.

Objective

To assess the effectiveness of re-ETV relative to initial ETV in pediatric patients and validate the ETV success score (ETVSS) for re-ETV.

Methods

Retrospective data of 624 ETV and 93 re-ETV procedures were collected in six neurosurgical centers in the Netherlands (between 1998 and 2015). We used a multivariable Cox proportional hazards model to provide an adjusted estimate of the hazard-ratio (HR) for re-ETV failure relative to ETV failure. The correlation coefficient between ETVSS and chances of re-ETV success was calculated by Kendall’s tau. Model discrimination was quantified by the c-statistic. We analyzed presence of arachnoid membranes and usage of extra-ventricular drain (EVD) postoperatively on success rates of re-ETV.

Results

The HR for re-ETV failure relative to ETV failure was 1.14 (95% CI: 0.83-1.57), P= .44. At six months success rates for both ETV and re-ETV were 68%. The ETVSS was significantly related to the chances of re-ETV success, τ= 0.37 (95% BCa CI: 0.21-0.52), P< .001. The c-statistic was 0.74 (95% CI: 0.64-0.85). Presence of prepontine arachnoid membranes and usage of an EVD were negatively associated with chances of success, respectively odds ratios of 4.0 (95% CI: 1.5-10.5) and 9.7 (95% CI: 3.4-27.8).

Conclusions

Re-ETV seems to be as safe and effective as initial ETV. The ETVSS adequately predicts chances of success of re-ETV. Presence of prepontine arachnoid membranes and use of EVD negatively influence chance of success.

49

ETV and re-ETV in pediatric patients

Introduction

Endoscopic third ventriculostomy (ETV) has become the standard treatment for obstructive hydrocephalus.1 Its range of indications has extended to communicative hydrocephalus as well.1,2 Unfortunately, to this day no treatment modality guaranties a permanent solution for hydrocephalus.3 Therefore, when primarily treating hydrocephalus, the apparent benefits and disadvantages of ETV and shunting have to be weighted and considered.4–11 To facilitate neurosurgeons in the process of patient selection for ETV, a predictive model was developed which estimates chances of ETV success over a six-month period; the so-called ETV success score (ETVSS).12–17 When a patient is referred for suspicion of recurrent hydrocephalus, the neurosurgeon has two treatment options: either perform a repeat-ETV (re-ETV) or implant a cerebrospinal fluid shunt. In a Dutch external validation study of the ETVSS a 78% success rate was found for the group that underwent a re-ETV after an evaluation period of six months.13 However, the sample size was relatively small with eighteen patients receiving re-ETV. This prompted the set-up of the current retrospective multicenter study in which data of six university neurosurgical centers in the Netherlands were clustered to investigate these findings using a larger sample size. In this paper, we aim to report on the observed effectiveness of a re-ETV relative to the observed effectiveness of initial ETV and to determine the predictive accuracy of the ETVSS in patients undergoing a re-ETV. An additional aim is providing an in-depth analysis of re-ETV patients and which factors influence chances of success.

Methods

Study design

Retrospective data was gathered from electronic patient records in six university medical centers in the Netherlands, in Amsterdam the Academic Medical Center (July 1999 – February 2015) and VU University Medical Center (most between August 2010 – October 2014), in Groningen the University Medical Center Groningen (April 1998 – November 2014), in Nijmegen the Radboud University Medical Center (most between January 2004 – August 2014), in Rotterdam the

50

CHAPTER 3

Erasmus Medical Center, Sophia Children’s Hospital (May 1997 – December 2014), and in Utrecht the University Medical Center Utrecht (most between April 2004 – July 2013), the cooperating centers are united in the DPNSG (Dutch Pediatric Neurosurgical Study Group). All ETV procedures were performed in accordance with national guidelines, i.e. either by a neurosurgeon or a resident under direct supervision of a neurosurgeon. ETV failure was defined as any subsequent surgical procedure for CSF diversion, as determined by a neurosurgeon, or death related to hydrocephalus management.18

The research protocol was submitted to and approved by an ethics committee. Patient consent was not required as we worked with an anonymized retrospective dataset and did not use an intervention.

Data collection

All patients who were younger than 18 years at ETV were eligible for inclusion. The following patient characteristics were collected: etiology (classified into aqueductal stenosis, post-infectious, myelomeningocele, post-intraventricular hemorrhage, tectal tumor, non-tectal brain tumor, and other causes), date of birth, sex, date initial ETV, date ETV failure (defined as any subsequent surgical procedure for cerebrospinal fluid diversion or hydrocephalus related death), date re-ETV, date failure re-ETV (using previously mentioned definition), use of external ventricular drain (EVD), lumbar punctures (LPs) and/or external lumbar drain (ELD) postoperatively, shunt history and last contact at neurosurgery or neurology department for follow-up. Patient age was classified as is used for the ETVSS score into younger than one month, one month to younger than six months, six months to younger than one year, one year to younger than ten years and equal to or older than ten years.The ETVSS for each patient’s re-ETV was calculated using the age, etiology and previous shunt score (Table 1).12 The ETVSS works as follows: if a 9 months old patient with hydrocephalus due to aqueductal stenosis who was previously treated by means of a CSF shunt is selected for an ETV, the chance of ETV success over a period of six months is 60% (30 for patient age, 30 for etiology and 0 for previous shunt = 60).The operating reports were investigated for intraoperative findings at the re-ETV, these findings were reported in different categories; complete occluded

51


stoma without underlying arachnoid membranes, complete occluded stoma with underlying arachnoid membranes, pinpoint stoma without underlying arachnoid membranes, pinpoint stoma with underlying arachnoid membranes, obstruction due to underlying prepontine arachnoid membranes alone, no apparent occlusion.

Statistical analysis

We reported categorical patient characteristics using the number and percentage observed, continuous normally distributed characteristics using means and standard deviations (SD) and continuous non-normally distributed characteristics using medians and interquartile ranges (IQR). We examined differences in groups of patients using Chi-Square test, Fisher’s exact test for categorical variables, odds ratios (OR) were calculated with a 95% confidence interval (CI). We used independent two-tailed t-tests and Mann-Whitney U tests for continuous normally and not normally distributed variables, respectively. The follow-up was calculated from point of entry to the date at which the patient was seen at the clinic and known to be well, the date of subsequent treatment or date of death, whichever came first. Survival curves were calculated using the Kaplan-Meier method. Survival analysis covering the time to ETV failure was performed using a Cox proportional hazards model. The proportional hazards assumption was checked visually with the Kaplan Meier-curves. The variables used for the ETVSS (age of patient, etiology and prior shunt) and “ETV vs re-ETV” were entered into a multivariable Cox proportional hazards model, because literature supports these

Table 1. Calculation of the ETVSS

Used with permission from Elsevier; Kulkarni et al. J Pediatr 2009;155:254-9

52

CHAPTER 3

as the most important confounders.12 Subsequently the potential confounders with the highest P-value were eliminated from the model step by step until the fit of the model decreased significantly (based on the -2 log likelihood). The final multivariate model was used to provide an adjusted estimate and the 95% CI of the hazard ratio (HR) for re-ETV failure relative to ETV failure. Successful cases with less than six months follow-up were excluded from testing predictive adequacy of the ETVSS as this score predicts chances of ETV success over a six-month period. We determined the predictive adequacy of the ETVSS by comparing the predicted probability of re-ETV success between patients with successful and failed re-ETV treatment by an independent t-test. Model discrimination was quantified by the area under the receiver operating characteristic curve (hereafter called the c-statistic) with 95% CI and the correlation coefficient between ETVSS and chances of re-ETV success was calculated by Kendall’s tau with 95% bootstrapped CI based on 1000 replications. We performed the same analysis for initial ETV procedure of all patients operated between the previously mentioned dates in the cooperating centers. We performed all data analysis using IBM SPSS Statistics for Windows, Version 23.0 (Armonk, NY: IBM Corp) and considered P-values less than .05 as statistically significant.

Results

Demographics

A total of 624 ETV cases and 93 re-ETV cases were included for analysis, see Table 2 for baseline demographics and Figure 1 for flow chart. In five patients a third ETV was performed, in three of these no subsequent treatment was needed, two received a shunt. Seven patients were excluded from analysis, these had a patent ventriculostomy site and no other obstruction was found during neuro-endoscopic surgery. Three patients died from hydrocephalus related death: two had earlier failure of the ETV after which they received a shunt which subsequently failed, they were referred too late and died; the last ex-premature patient with neonatal cerebral hemorrhage (resulting in hydrocephalus treated with a shunt) was a severely mentally challenged, spastic child with infantile

53


Figure 1. Flow chart for ETV and re-ETV

Table 2. Demographics ETV and re-ETV cases

Abbreviations: ETV, endoscopic third ventriculostomy

54

CHAPTER 3

encephalopathy, at the age of 17 he was admitted to the intensive care unit with suspected aspiration of blood and obstructed airways, gastroscopy and multiple other diagnostics were to no avail, eventually the patient developed bradycardia after which a MRI cerebrum showed a trapped fourth ventricle and compression of the brainstem, ETV with aquaductstenting proved insufficient and eventually the patient died. Several techniques were used for the ETV procedure, most used a dumbbell shaped balloon for dilating the stoma, others used Thulium laser ablation of the third ventricle floor, and some used “black” fiber tip/diode laser combination for controlled tissue ablation.19 The number of ETV and re-ETV patients per institute are listed in the following tabulation:

Institute No. (%) of ETV patients re-ETV patientsErasmus 189 (30.3) 37 (39.8)UMCG 202 (32.4) 36 (38.7)Radboud-umc 97 (15.5) 8 (8.6)UMCU 42 (6.7) 7 (7.5)AMC 62 (9.9) 3 (3.2)VUmc 32 (5.1) 2 (2.2)

Median duration in months of follow-up of ETV procedure was 31.4 (IQR = 1.6 to 49.7) and the median follow-up of the re-ETV procedure was 14.8 (IQR = 1.2 to 49.9). The median age in years at initial ETV was 4.8 (IQR = 0.5 to 11.6) and the mean ETVSS for the initial ETV was 67 (SD = 17). The median age in years at re-ETV was 6.1 (IQR = 1.0 to 12.4) and the mean ETVSS for the re-ETV was 71 (SD = 16). The median time in months between ETV and failure was 1.4 (IQR = 0.6 to 9.2), the last failure occurred after 11.8 years. The median time in months between re-ETV and failure was 1.7 (IQR = 0.5 to 14.7), the last failure occurred after 8.0 years.

55


See Table 3 for success rates for ETV and re-ETV over time and Figure 2 for Kaplan-Meier survival curves. The hazard ratios are shown in Table 4, etiology was dropped from this model.

Validity evidence for predictive model

For the initial ETV procedure: The mean ETVSS for ETV was significantly higher for successful cases than for failed cases at six months 72 (SD = 15) versus

Figure 2. Kaplan-Meier survival curve for ETV and re-ETV Log Rank test: P = .58

Table 3. Survival tables for ETV and re-ETV

Abbreviations: CI, confidence interval; ETV, endoscopic third ventriculostomy

56

CHAPTER 3

57 (SD = 18), P < .001. The c-statistic for the six-month follow-up was 0.74 (95% CI: 0.70-0.79). The ETVSS of the initial ETV was significantly related to the chances of ETV success, τ = 0.36 (95% BCa CI: 0.29-0.42), P < .001

For the re-ETV procedure: The mean ETVSS for re-ETV was significantly higher for successful cases than for failed cases at six months 76 (SD = 13) versus 63 (SD = 16), P < .001. The c-statistic for the six-month follow-up was 0.74 (95% CI: 0.64-0.85). The ETVSS of the re-ETV was significantly related to changes of re-ETV success, τ = 0.37 (95% BCa CI: 0.21-0.52), P < .001.

Succesfully versus unsuccesfully treated re-ETV patients

For the 93 re-ETV patients, an in-depth analysis was conducted, see Table 5 for demographics of failed and successful cases. Seven cases had less than six months follow-up or died before six months (not hydrocephalus related) and were excluded from analysis in this section. Only one patient received additional choroid plexus cauterization at re-ETV, two months after the procedure the patient received a CSF shunt.In successfully treated patients, the median age in years at re-ETV was significantly higher than in unsuccessfully treated patients, at six months follow-up: 8.2 (IQR = 1.8 to 13.5) versus 1.6 (IQR = 0.4 to 8.6), P = .002. The median time in months between ETV and re-ETV did not differ significantly between successful and unsuccessful cases: 12.1 (IQR = 5.6 to 29.4) versus 7.6 (IQR = 1.6 to 29.7), P = .17.

Table 4. Hazard ratios from multivariable Cox proportional hazards model

Abbreviations: CI, confidence interval; ETV, endoscopic third ventriculostomy; HR, hazard ratio

57


Table 5. Characteristics of re-ETV success and failure at six months

Abbreviations: ELD, external lumbar drain; ETV, endoscopic third ventriculostomy;IQR, interquartile range; LP, lumbar drainSeven cases lost to follow-up or died within six months.a For etiologies in category ‘other’, see appendix 1.b Four missing values.c Four missing values.d Four missing values.e Three missing values.

58

CHAPTER 3

Effect of intraoperative findings

If the ventriculostomy site was closed and this was reopened, the patient had a significantly higher chance of success than when no closed ventriculostomy site was found during surgery, based on the odds ratio, the odds of success were 3.4 times higher (95% CI: 1.3-9.1, P = .01). When additional subarachnoid membranes are observed during re-ETV surgery, the patient had a significantly higher chance to need subsequent hydrocephalus treatment within six months, based on the odds ratio, the odds of failure were 4.0 times higher (95% CI: 1.5-10.5, P = .004).

Effect of postoperative management

Two patients received an EVD after intra-operative bleeding, one patient had multiple thick prepontine arachnoid membranes and the CSF of one patient was a bit cloudy (suspected infection). In the remaining cases no specific reasons for leaving an EVD was mentioned and as such we considered the EVD as being for escape reasons. When an EVD was placed, the patient had a significantly higher chance to need subsequent hydrocephalus treatment within six months, based on the odds ratio, the odds of failure were 9.7 times higher (95% CI: 3.4-27.8, P = .001). We did not discriminate between an open EVD or an escape EVD (which remains closed and would only be used if the patient develops clinical signs of raised intracranial pressure). Administering LPs or ELDs after the re-ETV did not influence chances of success at six months, P = .97.

Discussion

The present study observed a seven-year success rate for re-ETV of 44.3% (95% CI: 32.5-56.0). One large study from the CURE Children’s Hospital of Uganda including 215 pediatric patients reported corresponding findings with a seven-year success for re-ETV of 51%.20 The population of this group is not necessarily equal to the patient-population found in western world (e.g. more post-infectious hydrocephalus). The few published studies with 20 to 40 patients have reported reasonable good success rates between 63 and 81%.13,21–24 In other reports with fewer patients, the success rates range from 0 to 100%.25–35

59


The various authors come with various recommendations. Whereas some conclude that re-ETV should be considered in all patients that are referred with recurrent symptoms, others mention that re-ETV remains controversial. We think it is worthwhile to give ETV another chance in selected patients. Our primary motivation for this suggestion is to avoid shunt dependency. Also, the chances of ETV success and re-ETV success are similar: the adjusted estimate of the hazard ratio for re-ETV failure relative to ETV failure is 1.14 (95% CI: 0.83-1.57).

Effect of time

There is a great heterogeneity of patient populations between the abovementioned studies (e.g. not all are pediatric population) and it is rare to find a notion of the method used to calculate duration of follow-up. In literature on re-ETV, there seems to be a trend towards lower success rates in younger patients.12 This does not come as a surprise as similar findings have been reported for initial ETV procedures.30,36 In the ETVSS proposed by Kulkarni et al., 50% of the total score is estimated by age alone.12 We also found that patients in which re-ETV was successful at six months after treatment were significantly older than unsuccessfully treated patients but they did not have more time between their initial and re-ETV than unsuccessful cases. Other authors suggested that for patients with late failure of their ETV, re-ETV seems worthwhile.20,24 Furthermore, as has been extensively reported in literature on initial ETV we found that 64% of failures occurred within the first six months after re-ETV.29,33,34,37–40 Another reason for choosing ETV over shunt treatment is the apparent difference in concern among parents of hydrocephalic children.41 Although the comfort of parents could be a reason to prefer ETV over shunting, it can also make parents less cautious (false sense of security) as there is no certainty of cure for hydrocephalus and late failure of ETV does occur with potential devastating results.28,42–44 In our study, the latest failure occurred twelve years after ETV and eight years after re-ETV.

60

CHAPTER 3

Intraoperative findings

Some intraoperative and preoperative factors have been postulated to have predictive effects on ETV outcome (e.g. third ventricle floor thickness, prepontine scarring).21,44–49 A thicker third ventricle floor has been negatively associated with ETV success,50 unfortunately we could not retrospectively analyze this aspect. The presence of cisternal scarring has been negatively associated with success of the first ETV.47 We found that presence of subarachnoid membranes and younger patient age had a negative effect on chances of success. Similar findings were reported before.21,44 We found a positive effect of observed closure of the ventriculostomy site at re-ETV on chances of short-term success. Other authors suggested that if closure of the ventriculostomy site by gliosis and scarring is found during the second neuroendoscopic procedure a re-ETV should be considered.51–55

Choroid plexus cauterization

Combination with choroid plexus cauterization might give better results than ETV alone as was shown in a cohort from the previously mentioned Uganda based authors.56,57 In another study it was shown that even when an ETV could not be performed, and the procedure had to be converted into a cerebrospinal fluid shunt, the implanted shunt had better chances of success if choroid plexus cauterization was performed.58 It may be that ETV with choroid plexus cauterization has the most effect in a subset of patients (e.g. myelomeningocele patients).59,60 However, the possible favorable influence of choroid plexus cauterization in combination with ETV in developed countries is still being researched and debated and is beyond the scope of this study.61–67

Postoperative management

Usage of EVD (we did not discriminate between open and closed/emergency EVD) had a negative effect on chance of re-ETV success. It seems likely that a perioperative event made the surgeon to proceed with caution but deterred him from converting to an internal shunt (e.g. bleeding during endoscopy or suspect infection). Postoperative administering LPs or ELDs did not influence the chances of re-ETV success. The retrospective nature of our study is an obstacle

61


in this aspect. It is not clear whether usage of LPs or ELDs postoperatively are always well documented.

Predictive model

We performed an additional analysis of the ETVSS and found that the score adequately predicts chances of success for patients undergoing a re-ETV. The ETVSS of successful cases was significantly higher than the ETVSS of unsuccessful cases. The c-statistic has a range from 0.5 (no discrimination) to 1.0 (perfect discrimination).68 As rough guideline, values between 0.7 and 0.9 indicate moderate accuracy.69,70 The c-statistic for ETVSS for the re-ETV of 0.74 (95% CI: 0.64-0.85) was comparable to findings in other validation studies for initial ETV procedures including one by our group in which we found 0.82 (95% CI: 0.71-0.92).12,13,15 Kendall’s tau showed a significant correlation between ETVSS and chances of both ETV and re-ETV success. That the ETVSS predicts chances of re-ETV success as accurately as chances of initial ETV success is another argument for the comparability of the two procedures. The ETVSS takes few variables into account that have been shown over and over again to have an effect on the chances of (short-term) success of ETV, and now we show that these same factors can be used to predict chances of re-ETV success.

Limitations and future

The results of our study should be interpreted with caution due to limitations of the retrospective design. Due to this retrospective nature, we could not focus on some aspects that might be of importance. The thickness of the third ventricle floor has already been mentioned above. Another is the motive of choosing re-ETV over shunt implantation (this is rarely clearly stated in documentation) with the possibility of selection bias. The decision-making for treating surgeons in selecting cases for re-ETV (or ETV) was beyond our control, surgeons will have their own algorithm and will only select patients that they think will benefit from an ETV or re-ETV. The relatively small sample of re-ETV cases is also a limitation and finally, this study used a Dutch study population and the extent to which our results are generalizable to world-wide practice is not certain. The description of flow-void on post-operative MRI imaging is missing in our analysis. Again, the retrospective design is a limiting factor. It has been postulated

62

CHAPTER 3

that if a patient returns with symptoms of raised intracranial pressure and disappearance of the flow-void artefact (whereas it was shown before on MRI) a re-ETV should be considered.51 In other cases, shunt implantation may be a better option. Within our group the opinions differed on follow-up MRI’s, on what time-period patients should remain under routine surveillance and under supervision of who this surveillance should take place: neurologist, pediatrician, neurosurgeon or general practitioner. One of the more notable findings was the diverse pattern of practice when the initial ETV failed. We intend to reveal these patterns with a survey study in which we target pediatric neurosurgeons who face the difficulties of treating pediatric hydrocephalus on a regular basis.

Conclusion

Re-ETV seems to be as safe and effective as initial ETV.23,71 The factors related to success of re-ETV are similar to those related to success of initial ETV (e.g. patient age and etiology of hydrocephalus). Other elements that negatively influence chances of success are presence of prepontine arachnoid membranes, use of EVD after the endoscopic procedure and a short interval between initial and re-ETV. The avoidance of lifelong shunt dependency in successful re-ETV cases is important, and therefore, we recommend to perform re-ETV in selected patients. The ETVSS adequately predicts chances of success and can be used for patient selection for re-ETV.

Disclosure

The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

Acknowledgments

Sebastian Arts helped with the gathering of data in Nijmegen and Mariam Slot helped in the VU University Medical Center. Futhermore, we want to thank the Dutch Pediatric Neurosurgery Study Group for sharing of their data: Peter Vandertop MD PhD, Emma Children’s Hospital, Neurosurgical Center

63


Amsterdam; Dennis Buis MD PhD, Emma Children’s Hospital, Neurosurgical Center Amsterdam; Hedy Folkersma MD PhD, Emma Children’s Hospital, Neurosurgical Center Amsterdam; Pim van Ouwerkerk MD PhD, VU University Medical Center; Eelco Hoving MD PhD, Beatrix Children’s Hospital, University Medical Center Groningen; Radboud Koot MD PhD, Willem-Alexander Children’s Hospital, Leiden University Medical Center; Erwin Cornips MD, Maastricht University Medical Center; Erik J. van Lindert MD PhD, Amalia Children’s Hospital, Radboud University Nijmegen Medical Center; Hans Delye MD PhD, Amalia Children’s Hospital, Radboud University Nijmegen Medical Center; Marie Lise van Veelen MD, Sophia Children’s Hospital, Erasmus University Medical Center; Ruben Dammers MD PhD, Sophia Children’s Hospital, Erasmus University Medical Center; Rob de Jong MD, Sophia Children’s Hospital, Erasmus University Medical Center; Sen Han MD PhD, Wilhelmina Children’s Hospital, University Medical Center Utrecht; Peter Woerdeman MD PhD, Wilhelmina Children’s Hospital, University Medical Center Utrecht

64

CHAPTER 3

References

1. Hellwig D, Grotenhuis JA, Tirakotai W, et al. Endoscopic third ventriculostomy for obstructive hydrocephalus. Neurosurg Focus. 2005;28:1-34-8.

2. Nishiyama K, Yoshimura J, Fujii Y. Limitations of Neuroendoscopic Treatment for Pediatric Hydrocephalus and Considerations from Future Perspectives. Neurol Med Chir (Tokyo). 2015;55:611-616.

3. Kahle KT, Kulkarni A V, Limbrick DD, Warf BC. Hydrocephalus in children. Lancet. 2015;55:502-507.

4. Drake JM, Kulkarni A V, Kestle J. Endoscopic third ventriculostomy versus ventriculoperitoneal shunt in pediatric patients: a decision analysis. Childs Nerv Syst. 2009;25:467-472.

5. Kulkarni A V., Drake JM, Kestle JRW, Mallucci CL, Sgouros S, Constantini S. Endoscopic third ventriculostomy vs cerebrospinal fluid shunt in the treatment of hydrocephalus in children: a propensity score-adjusted analysis. Neurosurgery. 2010;67:588-593.

6. El-Ghandour NMF. Endoscopic third ventriculostomy versus ventriculoperitoneal shunt in the treatment of obstructive hydrocephalus due to posterior fossa tumors in children. Childs Nerv Syst. 2011;27:117-126.

7. Tuli S, Alshail E, Drake J. Third ventriculostomy versus cerebrospinal fluid shunt as a first procedure in pediatric hydrocephalus. Pediatr Neurosurg. 1999;30:11-15.

8. Di Rocco C, Massimi L, Tamburrini G. Shunts vs endoscopic third ventriculostomy in infants: are there different types and/or rates of complications? A review. Childs Nerv Syst. 2006;22:1573-1589.

9. Jernigan SC, Berry JG, Graham DA, Goumnerova L. The comparative effectiveness of ventricular shunt placement versus endoscopic third ventriculostomy for initial treatment of hydrocephalus in infants. J Neurosurg Pediatr. 2014;13:295-300.

10. Kulkarni A V., Hui S, Shams I, Donnelly R. Quality of life in obstructive hydrocephalus: Endoscopic third ventriculostomy compared to cerebrospinal fluid shunt. Child’s Nerv Syst. 2010;26:75-79.

11. Kulkarni A V., Shams I, Cochrane DD, McNeely PD. Quality of life after endoscopic third ventriculostomy and cerebrospinal fluid shunting: an adjusted multivariable analysis in a large cohort. J Neurosurg Pediatr. 2010;6:11-16.

12. Kulkarni A, Drake JM, Mallucci CL, Sgouros S, Roth J, Constantini S. Endoscopic third ventriculostomy in the treatment of childhood hydrocephalus. J Pediatr. 2009;155:254-9.e1.

13. Breimer GE, Sival DA, Brusse-Keizer MGJ, Hoving EW. An external validation of the ETVSS for both short-term and long-term predictive adequacy in 104 pediatric patients. Childs Nerv Syst. 2013;29:1305-1311.

14. Durnford AJA, Kirkham FFJ, Mathad N, Sparrow OCEO. Endoscopic third ventriculostomy in the treatment of childhood hydrocephalus: validation of a success score that predicts long-term outcome. J Neurosurg Pediatr. 2011;8:489-493.

15. Naftel RP, Reed GT, Kulkarni A V., Wellons JC. Evaluating the Children’s Hospital of Alabama endoscopic third ventriculostomy experience using the Endoscopic Third Ventriculostomy Success Score: an external validation study. J Neurosurg Pediatr. 2011;8:494-501.

16. Fani L, de Jong THR, Dammers R, van Veelen MLC. Endoscopic third ventriculocisternostomy in hydrocephalic children under 2 years of age: appropriate or not? A single-center retrospective cohort study. Childs Nerv Syst. 2013;29:419-423.

17. García LG, López BR, Botella GI, et al. Endoscopic Third Ventriculostomy Success Score (ETVSS) predicting success in a series of 50 pediatric patients. Are the outcomes of our patients predictable? Childs Nerv Syst. 2012;28:1157-1162.

65


18. Drake JM. Endoscopic third ventriculostomy in pediatric patients: the Canadian experience. Neurosurgery. 2007;60:881-6-6.

19. van Beijnum J, Hanlo PW, Fischer K, et al. Laser-assisted endoscopic third ventriculostomy: long-term results in a series of 202 patients. Neurosurgery. 2008;62:437-43-4.

20. Marano PJ, Stone SSD, Mugamba J, Ssenyonga P, Warf EB, Warf BC. Reopening of an obstructed third ventriculostomy: long-term success and factors affecting outcome in 215 infants. J Neurosurg Pediatr. 2015;15:399-405.

21. Peretta P, Cinalli G, Spennato P, et al. Long-term results of a second endoscopic third ventriculostomy in children: retrospective analysis of 40 cases. Neurosurgery. 2009;65:539-47; discussion 547.

22. Lam S, Harris D, Rocque BG, Ham SA. Pediatric endoscopic third ventriculostomy: a population-based study. J Neurosurg Pediatr. 2014;14:455-464.

23. Siomin V, Weiner H, Wisoff J, et al. Repeat endoscopic third ventriculostomy: is it worth trying? Child’s Nerv Syst. 2001;17:551-555.

24. Mahapatra A, Mehr S, Singh D, Tandon M, Ganjoo P, Singh H. Ostomy closure and the role of repeat endoscopic third ventriculostomy (re-ETV) in failed ETV procedures. Neurol India. 2011;59:867.

25. Kadrian D, van Gelder J, Florida D, et al. Long-term Reliability of Endoscopic Third Ventriculostomy. Neurosurgery. 2005;56:1271-1278.

26. Navarro R, Gil-Parra R, Reitman AJ, Olavarria G, Grant JA, Tomita T. Endoscopic third ventriculostomy in children: early and late complications and their avoidance. Childs Nerv Syst. 2006;22:506-513.

27. Koch D, Grunert P, Filippi R, Hopf N. Re-ventriculostomy for treatment of obstructive hydrocephalus in cases of stoma dysfunction. Minim Invasive Neurosurg. 2002;45:158-163.

28. Surash S, Chumas P, Bhargava D, Crimmins D, Straiton J, Tyagi A. A retrospective analysis of revision endoscopic third ventriculostomy. Childs Nerv Syst. 2010;26:1693-1698.

29. Feng H, Huang G, Liao X, et al. Endoscopic third ventriculostomy in the management of obstructive hydrocephalus: an outcome analysis. J Neurosurg. 2004;100:626-633.

30. Koch-Wiewrodt D, Wagner W. Success and failure of endoscopic third ventriculostomy in young infants: Are there different age distributions? Child’s Nerv Syst. 2006;22:1537-1541.

31. Hellwig D, Giordano M, Kappus C. Redo third ventriculostomy. World Neurosurg. 2013;79:S22.e13-20.

32. Wagner W, Koch D. Mechanisms of failure after endoscopic third ventriculostomy in young infants. J Neurosurg. 2005;103:43-49.

33. Sacko O, Boetto S, Lauwers-Cances V, Dupuy M, Roux F-E. Endoscopic third ventriculostomy: outcome analysis in 368 procedures. J Neurosurg Pediatr. 2010;5:68-74.

34. Grand W, Leonardo J, Chamczuk AJ, Korus AJ. Endoscopic Third Ventriculostomy in 250 Adults With Hydrocephalus. Neurosurgery. 2016;78:109-119.

35. Fukuhara T, Luciano MG, Kowalski RJ. Clinical features of third ventriculostomy failures classified by fenestration patency. Surg Neurol. 2002;58:102-110.

36. Baldauf J, Oertel J, Gaab MR, Schroeder HWS. Endoscopic third ventriculostomy in children younger than 2 years of age. Childs Nerv Syst. 2007;23:623-626.

37. Vulcu S, Eickele L, Cinalli G, Wagner W, Oertel J. Long-term results of endoscopic third ventriculostomy: an outcome analysis. J Neurosurg. July 2015:Epub.

38. Siomin V, Cinalli G, Grotenhuis A, et al. Endoscopic third ventriculostomy in patients with cerebrospinal fluid infection and/or hemorrhage. J Neurosurg. 2002;97:519-524.

66

CHAPTER 3

39. Kulkarni A V., Drake JM, Kestle JRW, Mallucci CL, Sgouros S, Constantini S. Predicting who will benefit from endoscopic third ventriculostomy compared with shunt insertion in childhood hydrocephalus using the ETV Success Score. J Neurosurg Pediatr. 2010;6:310-315.

40. Labidi M, Lavoie P, Lapointe G, et al. Predicting success of endoscopic third ventriculostomy: validation of the ETV Success Score in a mixed population of adult and pediatric patients. J Neurosurg. July 2015:1-9.

41. Kulkarni A V., Shams I, Cochrane DD, McNeely PD. Does treatment with endoscopic third ventriculostomy result in less concern among parents of children with hydrocephalus? Child’s Nerv Syst. 2010;26:1529-1534.

42. Drake J, Chumas P, Kestle J, et al. Late rapid deterioration after endoscopic third ventriculostomy: additional cases and review of the literature. J Neurosurg. 2006;105:118-126.

43. Hader WJ, Drake J, Cochrane D, Sparrow O, Johnson ES, Kestle J. Death after late failure of third ventriculostomy in children. J Neurosurg. 2002;97:211-215.

44. Faggin R, Calderone M, Denaro L, Meneghini L, d’Avella D. Long-term operative failure of endoscopic third ventriculostomy in pediatric patients: the role of cine phase-contrast MR imaging. Neurosurg Focus. 2011;30:E1.

45. Romero L, Ros B, Ibáñez G, Ríus F, González L, Arráez M. Endoscopic third ventriculostomy: Can we predict success during surgery? Neurosurg Rev. 2014;37:89-97.

46. Dlouhy BJ, Capuano AW, Madhavan K, Torner JC, Greenlee JDW. Preoperative third ventricular bowing as a predictor of endoscopic third ventriculostomy success. J Neurosurg Pediatr. 2012;9:182-190.

47. Warf BC, Kulkarni A V. Intraoperative assessment of cerebral aqueduct patency and cisternal scarring: impact on success of endoscopic third ventriculostomy in 403 African children. J Neurosurg Pediatr. 2010;5:204-209.

48. Algin O. Prediction of endoscopic third ventriculostomy (ETV) success with 3D-SPACE technique. Neurosurg Rev. 2014;epub:3-5.

49. Woodworth GF, See A, Bettegowda C, Batra S, Jallo GI, Rigamonti D. Predictors of surgery-free outcome in adult endoscopic third ventriculostomy. World Neurosurg. 2012;78:312-317.

50. Sufianov AA, Sufianova GZ, Iakimov IA. Endoscopic third ventriculostomy in patients younger than 2 years: outcome analysis of 41 hydrocephalus cases. J Neurosurg Pediatr. 2010;5:392-401.

51. Cinalli G, Sainte-Rose C, Chumas P, et al. Failure of third ventriculostomy in the treatment of aqueductal stenosis in children. Neurosurg Focus. 1999;6:e3.

52. Mohanty A, Vasudev MK, Sampath S, Radhesh S, Sastry Kolluri VR. Failed endoscopic third ventriculostomy in children: Management options. Pediatr Neurosurg. 2002;37:304-309.

53. Di Rocco F, Jucá CE, Zerah M, Sainte-Rose C. Endoscopic third ventriculostomy and posterior fossa tumors. World Neurosurg. 2013;79:S18.e15-S18.e19.

54. Sgouros S, Status C, Trends F. Neuroendoscopy. (Sgouros S, ed.). Berlin, Heidelberg: Springer Berlin Heidelberg; 2014.

55. Buxton N, Ho KJ, Macarthur D, Vloeberghs M, Punt J, Robertson I. Neuroendoscopic third ventriculostomy for hydrocephalus in adults: Report of a single unit’s experience with 63 cases. Surg Neurol. 2001;55:74-78.

56. Warf BC, Tracy S, Mugamba J. Long-term outcome for endoscopic third ventriculostomy alone or in combination with choroid plexus cauterization for congenital aqueductal stenosis in African infants. J Neurosurg Pediatr. 2012;10:108-111.

67


57. Warf BC. Comparison of endoscopic third ventriculostomy alone and combined with choroid plexus cauterization in infants younger than 1 year of age: a prospective study in 550 African children. J Neurosurg. 2005;103:475-481.

58. Warf BC, Bhai S, Kulkarni A V., Mugamba J. Shunt survival after failed endoscopic treatment of hydrocephalus. J Neurosurg Pediatr. 2012;10:1-8.

59. Zandian A, Haffner M, Johnson J, Rozzelle CJ, Tubbs RS, Loukas M. Endoscopic third ventriculostomy with/without choroid plexus cauterization for hydrocephalus due to hemorrhage, infection, Dandy-Walker malformation, and neural tube defect: A meta-analysis. Child’s Nerv Syst. 2014;30:571-578.

60. Beuriat P-A, Szathmari A, Grassiot B, Plaisant F, Rousselle C, Mottolese C. Role of Endoscopic Third Ventriculostomy in the management of myelomeningocele related hydrocephalus : a retrospective study in a single French institution. World Neurosurg. August 2015.

61. Kulkarni A V., Riva-Cambrin J, Browd SR, et al. Endoscopic third ventriculostomy and choroid plexus cauterization in infants with hydrocephalus: a retrospective Hydrocephalus Clinical Research Network study. J Neurosurg Pediatr. 2014;14:224-229.

62. Stone SSD, Warf BC. Combined endoscopic third ventriculostomy and choroid plexus cauterization as primary treatment for infant hydrocephalus: a prospective North American series. J Neurosurg Pediatr. 2014;14:439-446.

63.Souweidane MS. Editorial: Combined choroid plexus coagulation and endoscopic third ventriculostomy: is North America ready? J Neurosurg Pediatr. 2014;14:221-223.

64. Chamiraju P, Bhatia S, Sandberg DI, Ragheb J. Endoscopic third ventriculostomy and choroid plexus cauterization in posthemorrhagic hydrocephalus of prematurity. J Neurosurg Pediatr. 2014;13:433-439.

65. Zhu X, Di Rocco C. Choroid plexus coagulation for hydrocephalus not due to CSF overproduction: A review. Child’s Nerv Syst. 2013;29:35-42.

66. Drake J. Editorial: Endoscopic third ventriculostomy. J Neurosurg Pediatr. 2012;10:461-462.

67. Riva-Cambrin J, Spader H, Kulkarni A, Warf B, Mugamba J, Ssenyonga P. Attitudes regarding endoscopic third ventriculostomy and choroid plexus coagulation (ETV+CPC) and the effect of training at CURE Children’s Hospital, Uganda among North American pediatric neurosurgeons. Fluids Barriers CNS. 2015;12:O10.

68. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115:928-935.

69. Fischer JE, Bachmann LM, Jaeschke R. A readers’ guide to the interpretation of diagnostic test properties: Clinical example of sepsis. Intensive Care Med. 2003;29:1043-1051.

70. Akobeng AK. Understanding diagnostic tests 3: Receiver operating characteristic curves. Acta Paediatr. 2007;96:644-647.

71. Gallo P, Sala F. The role of repeat endoscopic third ventriculostomy after failure of the initial procedure. Neurol India. 2011;59:844.

68

CHAPTER 3

Appendix 1

Table 6. Other category (n = 27)

69


Chapter 4

Development and content validation of performance

assessments for endoscopic third ventriculostomy

Gerben E. BreimerFaizal A. Haji

Eelco W. HovingJames M. Drake

Child’s Nerv Syst 2015; 31: 1247-1259doi:10.1007/s00381-015-2716-4

72

CHAPTER 4

Abstract

Purpose

This study aims to develop and establish the content validity of multiple expert rating instruments to assess performance in endoscopic third ventriculostomy (ETV), collectively called the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT).

Methods

The important aspects of ETV were identified through a review of current literature, ETV videos, and discussion with neurosurgeons, fellows, and residents. Three assessment measures were subsequently developed: a procedure-specific checklist (CL), a CL of surgical errors, and a global rating scale (GRS). Neurosurgeons from various countries, all identified as experts in ETV, were then invited to participate in a modified Delphi survey to establish the content validity of these instruments. In each Delphi round, experts rated their agreement including each procedural step, error, and GRS item in the respective instruments on a 5-point Likert scale.

Results

Seventeen experts agreed to participate in the study and completed all Delphi rounds. After item generation, a total of 27 procedural CL items, 26 error CL items, and 9 GRS items were posed to Delphi panelists for rating. An additional 17 procedural CL items, 12 error CL items, and 1 GRS item were added by panelists. After three rounds, strong consensus (>80 % agreement) was achieved on 35 procedural CL items, 29 error CL items, and 10 GRS items. Moderate consensus (50–80 % agreement) was achieved on an additional 7 procedural CL items and 1 error CL item. The final procedural and error checklist contained 42 and 30 items, respectively (divided into setup, exposure, navigation, ventriculostomy, and closure). The final GRS contained 10 items.

Conclusions

We have established the content validity of three ETV assessment measures by iterative consensus of an international expert panel. Each measure provides unique

73

Assessment tool development and content validation

assessment information and thus can be used individually or in combination, depending on the characteristics of the learner and the purpose of the assessment. These instruments must now be evaluated in both the simulated and operative settings, to determine their construct validity and reliability. Ultimately, the measures contained in the NEVAT may prove suitable for formative assessment during ETV training and potentially as summative assessment measures during certification.

Introduction

The climate of neurosurgical education has changed in recent decades. The increased emphasis on patient safety, technological advances (with the associated increase in the complexity of neurosurgical care), and reductions in resident duty hours have increased scrutiny on how residents are trained and how their competence is assessed.1,2 New educational paradigms have emerged in response to these challenges, including a shift toward competency-based training,3 the articulation of educational “milestones”,4 and the emergence of simulation as an adjunct to clinical neurosurgical training.5,6 Alongside these initiatives, there have been calls for objective, valid, and reliable methods to assess competence that can be used both in clinical and simulated environments.1,2,6

Traditionally, workplace-based assessments in neurosurgery involve reviews of resident procedure logs and unstructured global assessments of competence by expert surgeons.7 Although an adequate volume of clinical experience is necessary to achieve competency, the number of procedures completed by a resident alone does not necessarily reflect their level of skill. Similarly, unstructured observations may be unreliable and prone to bias, as they lack standardized procedure-specific assessment criteria.8–10 Direct observation of residents’ performance by an expert using structured measurement instruments, such as procedure-specific checklists (CL) and global rating scales (GRS), may reduce bias and generate more objective, valid, and reliable assessments.8,10–14 Such procedure-specific instruments also enable educators to highlight specific elements in a resident’s performance that require improvement through deliberate practice,15–19 allowing the tool to be used both in a formative manner (to accelerate the learning process) and as a

74

CHAPTER 4

summative assessment. As a result, these methods may be more appropriate than traditional approaches for assessing neurosurgical skills in the operating room (OR) and during simulation training.In order to adequately address patient safety, procedural assessments must not only address psychomotor skills but also surgical errors. This is particularly important in neuroendoscopy, given multiple reports suggest many complications in endoscopic third ventriculostomy (ETV) are attributable to intraoperative errors committed by surgeons who have recently begun performing the technique.20–23 As there are currently no standardized measures for assessing competence in neuroendoscopic surgery,24 the goals of this study were to (1) develop a battery of instruments to assess technical skills and intraoperative errors of neurosurgical trainees performing ETV, known as the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT) and (2) establish the content validity of these instruments, to ensure that assessments generated using these instruments reflect the skills required to perform an ETV and potential errors that may be committed during the procedure.25

Methods

The development and content validation of the NEVAT was achieved using the modified Delphi method, through which consensus on the content and structure of three assessment instruments was established among an international expert panel of neurosurgeons.17, 26, 27 The Delphi method is an iterative, multi-step process28,29 that has been extensively applied to the validation of assessment instruments for many medical and surgical procedures, including gastrointestinal endoscopy,30 ultrasound-guided regional anesthesia,31 lumbar puncture,32 endoscopic sinus surgery,33 and various general surgery procedures.34,35

Panelist selection

An international panel of neurosurgical educators was convened to participate in the Delphi survey. In keeping with current guidelines for sampling in Delphi surveys, a purposive, criterion-based sampling technique (rather than a random sample) was used to ensure the expertise of the group.28,29,36,37 Specifically, the authors generated a list of neurosurgeons who had published on the ETV

75


technique were experienced in performing the procedure or who were involved in teaching ETV to residents and fellows. To ensure the representativeness of the group, experts from multiple countries practicing in both pediatric and adult settings were included. These individuals were identified during the literature review conducted by the primary author and based on the recommendations of the two senior authors (EWH and JMD), who provided the names and contact information (if available) for surgeons that (i) had completed fellowships in which ETV was explicitly taught and had subsequently taken on positions as neurosurgical consultants in which ETV was routinely part of their clinical and teaching responsibilities or (ii) who were part of neurosurgical societies or attended neurosurgical conferences in which ETV techniques and research were often discussed.As 15–30 panelists is considered adequate for a Delphi study,38 a total of 35 surgeons were invited to participate to increase the likelihood that this target sample size would be achieved. Invited surgeons received information about the study objectives, methods, and anticipated time commitment and explicitly asked to only agree to participate if they were able to complete all rounds of the Delphi. Those who did not respond received weekly reminder e-mails for 2 weeks. Surgeons who agreed to participate received an e-mail with a link to the first online questionnaire. The identity of panelists was kept confidential throughout the study.

Item generation and initial tool development

The primary author conducted a review of the ETV literature to inform the development of the initial version of the assessment instruments. Two searches of the PubMed database were completed: the first to identify the procedural steps and skills required to perform an ETV (search terms: ETV, endoscopic third ventriculostomy, ventriculostomy, methods and standards) and the second to identify potential errors and complications associated with the procedure (search terms: ETV, endoscopic third ventriculostomy, ventriculostomy, postoperative complications, adverse effects, complication*). The resulting citations and abstracts were screened and full texts of relevant publications (along with their reference lists) were reviewed to extract data on ETV procedural steps, surgical errors, and technical skills. A selection of assessment measures from other

76

CHAPTER 4

surgical fields were also reviewed.34 From this, a preliminary version of three ETV-specific assessment instruments was developed: a CL of procedural steps, a CL of surgical errors, and a GRS of technical skills. In keeping with previous expert-rating assessments, checklist items were graded on a binary scale (1 = done correctly or 0 = done incorrectly) and global rating items were graded on a 5-point Likert scale (with explicit verbal anchors associated with ratings of 1, 3, and 5).39 These preliminary instruments were edited for comprehensiveness, clarity, and redundancy by the co-authors, who are experienced in performing and teaching ETV and/or have experience in surgical education research and assessment.

Survey design and administration

An online questionnaire outlining the preliminary version of the three assessment instruments was created using Survey Monkey (SurveyMonkey Inc., Palo Alto, CA, USA). Panelists rated their agreement on including each item the final assessment instruments on a 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree). Free-text boxes were provided after each item and at the end of each assessment instrument, so panelists could provide additional comments.An a priori decision to conduct a maximum of three survey rounds was made to balance the benefit of achieving consensus on all survey items with the risk of panelists not responding to all survey rounds. In the first round, panelists were asked to rate each item, add items they felt were missing, and suggest revisions to the wording or content of individual items as necessary. Panelists were also invited to suggest changes to the wording of GRS verbal anchors and the rating scales for each instrument. A reminder was sent to those who did not respond within 2 weeks of receiving the online link, with weekly reminders thereafter until all panelists had responded. Prior to round 2, participants received feedback on the results of round 1, including their rating, the median group rating, and interquartile range (IQR), whether consensus on the item had been reached and an anonymized summary of all comments provided by panelists [40]. In round 2, participants were asked to consider this feedback and re-rate any items for which “strong” consensus had not been obtained (see below for consensus

77


criteria), as well as any items whose wording had been changed or which had been added by panelists in the previous round. In addition, panelists were explicitly asked to provide a reason if they selected a rating more than one point away from the median response.40 Reminders were sent in similar fashion to round 1. Once all panelists had completed the questionnaire, the results of round 2 were summarized and the process was repeated for a third and final round.

Data analysis

All data were analyzed using the Survey Monkey analysis tool and IBM SPSS Statistics for Windows, Version 20.0 (Armonk, NY: IBM Corp). Demographic characteristics were summarized descriptively and reported as medians and IQRs or frequency counts and percentages. After each survey round, the proportion of responses in each rating category were compared to against pre-set criteria to determine if consensus had been achieved on each item. In keeping with prior Delphi studies,31,36 a “strong” consensus threshold was set at >80 % of panelists providing a Likert rating >4 (indicating strong positive agreement) or <2 (indicating strong negative agreement). A “moderate” consensus threshold was also set, defined as 50–80% of panelists providing a Likert rating of >4 or <2 (indicating moderate positive and negative agreement, respectively). All items reaching moderate or strong positive consensus were included in the NEVAT.

Results

Panelist demographics

Of 35 experts invited to participate in the survey, 19 (54 %) agreed to participate and 17 (49 %) completed all survey rounds. Of those who did not participate, one expert declined while the remaining 15 did not respond to the invitation. The final panel included neurosurgeons from multiple geographic regions with extensive experience in performing and teaching ETV in both adult and pediatric settings (Table 1).

Initial instrument development

The first and second literature searches resulted in 876 and 848 citations,

78

CHAPTER 4

respectively, from which a selection of relevant articles was reviewed.20,21,39,41–53 The items in the global rating scale were particularly informed by citations from two review papers on observational tools for assessment of procedural skills9,17,39,54 and by the objective-structured assessment of technical skill (“OSATS”) GRS.39 The structure of both the CL and GRS instruments (including wording of grading scales and verbal anchors) was also based on the instruments described by Martin et al.39 After removal of redundant and duplicate items, a total of 27 procedural steps and 26 surgical errors (each sub-divided into set-up, exposure, navigation, ventriculostomy, confirmation of adequate ventriculostomy, and closure), as well as 9 GRS domains were included in the preliminary version of the instruments (Appendix 1).

Table 1. Panelist demographics

Abbreviations: ETV, endoscopic third ventriculostomy; IQR, interquartile range

79


Delphi survey rounds

Over the 3 Delphi rounds, panelists added 15 procedural steps, 4 surgical errors, and 1 GRS item to the initial version of the instruments. The authors also revised a number of items based on panelists’ comments (see Fig. 1 for details of the number of items added, revised, and removed in each round of the survey). Panelists’ also suggested that some procedural steps were “optional” and dependent on individual surgeon preference (e.g., use of an articulating arm to hold the endoscope) or specific patient populations (e.g., dural closure in young children). Similarly, some surgical errors were viewed as applicable only when

Figure 1. Details of three iterative rounds

80

CHAPTER 4

specific techniques were used (e.g., failure to establish a track with a smaller instrument before advancing the trocar or sheath). By the third round, strong consensus (>80 % agreement among panelists) was achieved on 35 procedural steps, 29 surgical errors, and all GRS items. Moderate consensus was achieved on the remaining 7 procedural steps and 1 surgical error. The final NEVAT procedural steps CL, surgical errors CL, and GRS are provided in Tables 2, 3, and 4, respectively.

Discussion

The decreased time available for clinical training resulting from duty hour restrictions, coupled with the steep learning curve associated with ETV,20,21,49 emphasizes the need for objective measures to assess changes in trainees’ ETV skills over time. The NEVAT was developed specifically for this purpose. To our knowledge, it is the first tool of its kind in neuroendoscopy.Expert-based assessment methods like the NEVAT have many advantages over other methods to assess trainees’ performance. For instance, while alternative measures (e.g., pressure exerted on neural elements or motion efficiency)55,56 have been developed for various simulation platforms, these assessments are not easily transferred to the clinical setting. Similarly, outcome measures such as time to completion,55,57 performance accuracy,58,59 and complications59 are important but may be influenced by external factors (such as individual patient characteristics).60,61 Furthermore, neither of these approaches provide direct assessment of the knowledge and skills of the trainee and are thus less useful as formative assessments to guide training. Conversely, the instruments contained within the NEVAT provide trainees with feedback on various aspects of the ETV procedure. This allows for personalized feedback on elements of the procedure that have been mastered and aspects that require further training. As the NEVAT instruments are comprehensive, flexible, and relative unintrusive, they may be used to assess performances in multiple contexts (including during simulation-based education and in the OR). In this way, it can also be used to evaluate the nature and quality of skills transfer from the simulation environment to the operative setting.17

81


Table 2. Final version of NEVAT – procedural steps CL

82

CHAPTER 4

Continued table 2. Final version of NEVAT – procedural steps CL

83


Table 3. Final version of NEVAT – surgical errors CL

84

CHAPTER 4

Each of the three instruments included in the NEVAT provide specific and unique assessment information; thus, each scale can be used on its own or in combination with the other measures, depending on the nature of the assessment and the available time and human resources. However, given their specific strengths and limitations, each measure may be more or less useful depending on the nature of the case or characteristics of the trainee being assessed. Specifically, the checklist of procedural steps may be particularly helpful as a formative assessment for learners in the early stages of training (e.g., junior residents), given its purpose is to determine if the trainee knows each procedural step (in sequence) and how to perform it correctly. However, this instrument may be less useful among more experienced learners who have mastered the steps of ETV, because a ceiling effect is likely to be observed.62–64 In this setting, global ratings may be preferable, as checklists and GRS scores are highly correlated and global ratings tend to be more sensitive to smaller changes in expertise.13,62–65 As the GRS is also the shortest of the three instruments, it is ideally suited to situations where there is limited time to complete the assessment. A limitation of this tool is that given the general nature of the items in the instrument, less detailed feedback will be generated for the learner, reducing its utility as a formative assessment for novices or intermediate learners. Where additional granularity (i.e., level of detail) is important, the checklist of errors may be a useful adjunct, as it can be used to screen for common mistakes among participants at all levels. Furthermore, this

Continued table 3. Final version of NEVAT – surgical errors CL

85


instrument may be a useful research tool to identify whether training leads not only to improved technical performance but also patient safety.The use of expert-rating-based assessments is not unique to this study. A number of recent reports used similar assessment instruments modified from the OSATS tool39 for simulation-based training in microsurgery,66 cranial trauma,67 anterior cervical discectomy and fusion,68 and posterior laminectomy and foraminotomy.69 While the modification of pre-existing instruments is reasonable, it is important to evaluate whether the resulting tool can generate valid and reliable assessments

Table 4. Final version of NEVAT – GRS

86

CHAPTER 4

in the context to which it will be applied. Engaging in a comprehensive and systematic approach to instrument development that combined modification of existing tools with a literature review and expert consensus ensures that the resulting instrument adequately represents the domain of interest.30,70

By using the modified Delphi method, the content validity of the three instruments in the NEVAT has been established based on consensus among an international panel of expert neurosurgical educators. The limited dropout rate between each consecutive round may indicate the perceived need and value of a validated assessment tool for ETV.26 The geographic diversity of panelists and inclusion of both adult and pediatric neurosurgeons improves the generalizability of the tool, which reflects clinical practice patterns in most settings where ETV is performed globally. Similarly, the anonymous nature of the survey allowed panelists to present and modify their opinions without undue social pressure and prevented dominant individuals from influencing group opinion.28

However, these factors may also have contributed to the difficulty in obtaining strong consensus on all items (especially in the checklist instruments). The comments made by panelists suggest that some procedural steps or operative errors may not be applicable in all settings, given variations in procedural technique of surgeons practicing in different geographic areas or patient populations. While a moderate consensus was achieved on all items included in the tool, to ensure the flexibility of the instruments without losing potentially important assessment information, we modified the grading scale of the checklist instruments to include a “not applicable” option.Additional limitations of the Delphi method is that no universal guidelines exist on an “ideal” sample size, method of expert selection, number of survey rounds, or threshold for consensus. While our panelist selection and sample size were determined based on existing Delphi literature, our approach limited the inclusion of panelists not well known to the authors or for whom current contact information was not available. In addition, while we chose to limit the survey to three rounds and set strong and moderate consensus criteria to balance the likelihood of achieving a strong response rate through all three survey rounds, this approach may have affected the number and wording of items included in the final assessment tools. In particular, our choice to include all items that achieved “moderate” consensus, as well as those which may not be applicable

87


to every ETV case lengthened both the procedural steps and procedural errors checklist. Future work that compares the psychometric properties’ different variations of the NEVAT instruments (e.g., all items vs. only those with strong consensus, etc.) may be useful and may lead to instruments that are simpler but with similar reliability and sensitivity.It is important to note that while the current study has generated “content” validity evidence for the NEVAT instruments, this is necessary but not sufficient to justify their use in evaluating trainees technical skills.71 Other psychometric properties must also be evaluated, particularly if it may be used for high-stakes assessment or credentialing.71 A number of frameworks exist to guide investigators in this regard.72–74 For instance, at minimum, the reliability or internal consistency of these assessment tools must also be established.71 Where a “gold standard” is available, it is also helpful to evaluate the degree to which the scores of the NEVAT instruments align with this gold standard.25 Additional evidence on the “construct validity” (the degree to which the assessment measures the underlying construct) of the assessment should be determined.25,75 In the surgical education literature, a common approach is to use the instrument to assess the performance of individuals of different skill level (e.g., residents vs. practicing surgeons) or in the same individuals over time,76 to ensure that the tool can distinguish between performers of varying expertise.Ultimately, as contemporary views of validity contend that validation is an “argument” based on evidence that supports the use of an assessment tool in a given context,71 validity evidence supporting the NEVAT instruments may need to be generated among different learner populations (e.g., junior, intermediate, and senior residents) and in different educational settings (e.g., simulation vs. clinical), depending on how each instrument is ultimately be used. In addition to these psychometric properties, the acceptability and feasibility of the tool should also be established (e.g., through a survey of experts using the tool), to ensure it will be implemented by neurosurgical educators in the long term.75,77 Our research group is conducting additional studies to generate such evidence to support the NEVAT, which may ultimately support its use both as a formative tool to guide neuroendoscopic training in the simulation lab and the OR and as a summative assessment instrument to be used for the purposes of credentialing.

88

CHAPTER 4

Conclusion

We applied a comprehensive, systematic approach to develop and establish the content validity of the NEVAT for assessment of procedural skills and operative errors in ETV. The three assessment instruments in the tool represent a consensus of opinion among an international panel of expert neurosurgical educators generated through the Delphi method. These tools must now be evaluated in both the simulated and operative settings, to generate additional evidence supporting their construct validity, reliability, acceptability, and feasability. Ultimately, these tools prove suitable for formative assessment during ETV training and potentially as summative assessment measures during certification.

Acknowledgments

Thanks to all the experts for their valuable contribution to this Delphi study: Dr. G. Cinalli, Italy; Dr. S. Constantini, Israel; Dr. P. Decq, France; Dr. S. deRibaupierre, Canada; Dr. C. DiRocco, Italy; Dr. Y. Ersahin, Turkey; Dr. J. Grotenhuis, the Netherlands; Dr. N. Gupta, United States of America; Dr. E. Hoving, the Netherlands; Dr. I. Pollack, United States of America; Dr. C. Sainte-Rose, France; Dr. S. Santoreneos, Australia; Dr. H. Schroeder, Germany; Dr. D. Thompson, United Kingdom; Dr. B. Warf, United States of America; Dr. J. Wellons, United States of America; and Dr. S. Zymberg, Brazil.

89


References

1. Reznick RK, MacRae H (2006) Teaching surgical skills — changes in the wind. N Engl J Med 355:2664–2669

2. Epstein RM (2007) Assessment in medical education. N Engl J Med 356:387–396

3. Frank JR, Snell LS, Ten Cate O et al (2010) Competency-based medical education: theory to practice. Med Teach 32:638–645

4. American Board of Neurological Surgery (2013) Primary Certification Process: Training in Neurological Surgery. http://www.abns.org/content/primary_certification_process_pr.asp

5. Burchiel K (2013) Commentary: simulation training in neurological surgery. Neurosurgery 73:6–7

6. Dagi TF (2013) Commentary: the roles and future of simulation in neurosurgery. Neurosurgery 73(Suppl 1):4–5

7. Fallah A, Ebrahim S, Haji F et al (2010) Surgical activity of first-year Canadian neurosurgical residents. Can J Neurol Sci 37:855–860

8. Reznick RK (1993) Teaching and testing technical skills. Am J Surg 165:358–361

9. Memon MA, Brigden D, Subramanya MS, Memon B (2010) Assessing the surgeon’s technical skills: analysis of the available tools. Acad Med 85:869–880

10. Beard JD (2007) Assessment of surgical competence. Br J Surg 94:1315–1316

11. Cristancho S, Moussa F, Dubrowski A (2012) Simulation-augmented training program for off-pump coronary artery bypass surgery: developing and validating performance assessments. Surgery 151:785–795

12. Scavone BM, Sproviero MT, McCarthy RJ et al (2006) Development of an objective scoring system for measurement of resident performance on the human patient simulator. Anesthesiology 105:260–266

13. Moorthy K, Munz Y, Sarker SK, Darzi A (2003) Objective assessment of technical skills in surgery. BMJ 327:1032–1037

14. Epstein RM, Hundert EM (2002) Defining and assessing professional competence. JAMA 287:226–235

15. Sugden C, Aggarwal R (2010) Assessment and feedback in the skills laboratory and operating room. Surg Clin N Am 90:519–533

16. Sanfey H, Dunnington G (2010) Verification of proficiency: a prerequisite for clinical experience. Surg Clin N Am 90:559–567

17. Ahmed K, Miskovic D, Darzi A et al (2011) Observational tools for assessment of procedural skills: a systematic review. Am J Surg. doi: 10.1016/j.amjsurg.2010.10.020

18. Eubanks TR, Clements RH, Pohl D et al (1999) An objective scoring system for laparoscopic cholecystectomy. J Am Coll Surg 189:566–574

19. Haase J, Boisen E (2009) Neurosurgical training: more hours needed or a new learning culture? Surg Neurol 72:89–95, discussion 95–7

20. Navarro R, Gil-Parra R, Reitman AJ et al (2006) Endoscopic third ventriculostomy in children: early and late complications and their avoidance. Childs Nerv Syst 22:506–513

21. Schroeder HWS, Niendorf W-R, Gaab MR (2002) Complications of endoscopic third ventriculostomy. J Neurosurg 96:1032–1040

90

CHAPTER 4

22. Furlanetti LL, Santos MV, de Oliveira RS (2012) The success of endoscopic third ventriculostomy in children: analysis of prognostic factors. Pediatr Neurosurg. doi: 10.1159/000353619

23. Jones RF, Kwok BC, Stening WA, Vonau M (1994) The current status of endoscopic third ventriculostomy in the management of non-communicating hydrocephalus. Minim Invasive Neurosurg 37:28–36

24. Cohen AR, Lohani S, Manjila S et al (2013) Virtual reality simulation: basic concepts and use in endoscopic neurosurgery training. Childs Nerv Syst 29:1235–1244

25. Mokkink LB, Terwee CB, Patrick DL et al (2010) The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 63:737–745

26. Boogaard S, Heymans MW, Patijn J et al (2011) Predictors for persistent neuropathic pain—a Delphi survey. Pain Physician 14:559–568

27. Hasson F, Keeney S, McKenna H (2000) Research guidelines for the Delphi survey technique. J Adv Nurs 32:1008–1015

28. Hsu C, Sandford BA (2007) The Delphi technique: making sense of consensus. Pract Assess Res Eval 12:1–8

29. Clayton MJ (1997) Delphi: a technique to harness expert opinion for critical decision-making tasks in education. Educ Psychol 17:373–386

30. Walsh CM, Ling SC, Khanna N et al (2013) Gastrointestinal endoscopy competency assessment tool: development of a procedure-specific assessment tool for colonoscopy. Gastrointest Endosc. doi: 10.1016/j.gie.2013.10.035

31. Cheung JJH, Chen EW, Darani R et al (2012) The creation of an objective assessment tool for ultrasound-guided regional anesthesia using the Delphi method. Reg Anesth Pain Med 37:329–333

32. Kessler DO, Auerbach M, Pusic M et al (2011) A randomized trial of simulation-based deliberate practice for infant lumbar puncture skills. Simul Healthc 6:197–203

33. Lin SY, Laeeq K, Ishii M et al (2009) Development and pilot-testing of a feasible, reliable, and valid operative competency assessment tool for endoscopic sinus surgery. Am J Rhinol Allergy 23:354–359

34. Peyre SE, Peyre CG, Hagen JA et al (2009) Laparoscopic Nissen fundoplication assessment: task analysis as a model for the development of a procedural checklist. Surg Endosc 23:1227–1232

35. Palter VN, MacRae HM, Grantcharov TP (2011) Development of an objective evaluation tool to assess technical skill in laparoscopic colorectal surgery: a Delphi methodology. Am J Surg 201:251–259

36. Morgan PJ, Lam-McCulloch J, Herold-McIlroy J, Tarshis J (2007) Simulation performance checklist generation using the Delphi technique. Can J Anaesth 54:992–997

37. De Villiers MR, de Villiers PJT, Kent AP (2005) The Delphi technique in health sciences education research. Med Teach 27:639–643

38. Linstone HA, Turoff M (1975) The Delphi method—techniques and applications. Addison-Wesley, Reading

39. Martin J, Regehr G, Reznick R et al (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278

40. Powell C (2003) The Delphi technique: myths and realities. J Adv Nurs 41:376–382

41. Beems T, Grotenhuis JA (2004) Long-term complications and definition of failure of neuroendoscopic procedures. Childs Nerv Syst 20:868–877

42. Bouras T, Sgouros S (2011) Complications of endoscopic third ventriculostomy. J Neurosurg Pediatr 7:643–649

91


43. Brockmeyer D, Abtin K, Carey L, Walker ML (1998) Endoscopic third ventriculostomy: an outcome analysis. Pediatr Neurosurg 28:236–240

44. Chowdhry SA, Cohen AR (2013) Intraventricular neuroendoscopy: complication avoidance and management. World Neurosurg. doi: 10.1016/j.wneu.2012.02.030

45. Peretta P, Ragazzi P, Galarza M et al (2006) Complications and pitfalls of neuroendoscopic surgery in children. J Neurosurg 105:187–193

46. Teo C (2005) Complications of endoscopic third ventriculostomy. In: Cinalli G, Sainte-Rose C, Maixner WJ (eds) Pediatric hydrocephalus. Springer Milan, Milano, pp 411–420

47. Dusick JR, McArthur DL, Bergsneider M (2008) Success and complication rates of endoscopic third ventriculostomy for adult hydrocephalus: a series of 108 patients. Surg Neurol 69:5–15

48. Ersahin Y, Arslan D (2008) Complications of endoscopic third ventriculostomy. Childs Nerv Syst 24:943–948

49. Haji FA, Dubrowski A, Drake J, de Ribaupierre S (2013) Needs assessment for simulation training in neuroendoscopy: a Canadian national survey. J Neurosurg 118:250–257

50. Jones RF, Stening WA, Brydon M (1990) Endoscopic third ventriculostomy. Neurosurgery 26:86–91, discussion 91–2

51. García LG, López BR, Botella GI et al (2012) Endoscopic third ventriculostomy success score (ETVSS) predicting success in a series of 50 pediatric patients. Are the outcomes of our patients predictable? Childs Nerv Syst 28:1157–1162

52. Hopf NJ, Grunert P, Fries G et al (1999) Endoscopic third ventriculostomy: outcome analysis of 100 consecutive procedures. Neurosurgery 44:795–804, discussion 804–6

53. Bouramas D, Paidakakos N, Sotiriou F et al (2012) Endoscopic third ventriculostomy in obstructive hydrocephalus: surgical technique and pitfalls. Acta Neurochir Suppl 113:135–139

54. Van Hove PD, Tuijthof GJM, Verdaasdonk EGG et al (2010) Objective assessment of technical surgical skills. Br J Surg 97:972–987

55. Mattei TA, Frank C, Bailey J et al (2013) Design of a synthetic simulator for pediatric lumbar spine pathologies. J Neurosurg Pediatr 12:192–201

56. Choudhury N, Gélinas-Phaneuf N, Delorme S, Del Maestro R (2012) Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg. doi: 10.1016/j.wneu.2012.08.022

57. Miranpuri AS, Nickele CM, Akture E et al (2013) Neuroangiography simulation using a silicone model in the angiography suite improves trainee skills. J Neurointerv Surg. doi: 10.1136/neurintsurg-2013-010826

58. Banerjee PP, Luciano CJ, Lemole GM et al (2007) Accuracy of ventriculostomy catheter placement using a head- and hand-tracked high-resolution virtual reality simulator with haptic feedback. J Neurosurg 107:515–521

59. Luciano CJ, Banerjee PP, Bellotte B et al (2011) Learning retention of thoracic pedicle screw placement using a high-resolution augmented reality simulator with haptic feedback. Neurosurgery 69:14–19, discussion 19

60. Downing SM (2006) Face validity of assessments: faith-based interpretations or evidence-based science? Med Educ 40:7–8

61. Bjorkman D, Popp JJ (2006) Measuring the quality of endoscopy. Am J Gastroenterol 101:864–865

62. Ma IWY, Zalunardo N, Pachev G et al (2012) Comparing the use of global rating scale with checklists for the assessment of central venous catheterization skills using simulation. Adv Health Sci Educ Theory Pract 17:457–470

92

CHAPTER 4

63. Regehr G, MacRae H, Reznick RK, Szalay D (1998) Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med 73:993–997

64. Hodges B, Regehr G, McNaughton N et al (1999) OSCE checklists do not capture increasing levels of expertise. Acad Med 74:1129–1134

65. Gélinas-Phaneuf N, Del Maestro RF (2013) Surgical expertise in neurosurgery: integrating theory into practice. Neurosurgery 73(Suppl 1):30–38

66. El Ahmadieh TY, Aoun SG, El Tecle NE et al (2013) A didactic and hands-on module enhances resident microsurgical knowledge and technical skill. Neurosurgery 73(Suppl 1):51–56

67. Lobel D, Elder J, Schirmer C et al (2013) A novel craniotomy simulator provides a validated method to enhance education in the management of traumatic brain injury. Neurosurgery 73:57–65

68. Ray WZ, Ganju A, Harrop JS, Hoh DJ (2013) Developing an anterior cervical diskectomy and fusion simulator for neurosurgical resident training. Neurosurgery 73(Suppl 1):100–106

69. Harrop J, Rezai AR, Hoh DJ et al (2013) Neurosurgical training with a novel cervical spine simulator: posterior foraminotomy and laminectomy. Neurosurgery 73(Suppl 1):94–99

70. Guyatt G, Bombardier C, Tugwell P (1986) Measuring disease-specific quality of life in clinical trials. CMAJ 134:889–895

71. Downing SM (2003) Validity: on meaningful interpretation of assessment data. Med Educ 37:830–837

72. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1994) Standards for educational and psychological testing. American Educational Research Association, Washington, DC

73. American Psychologoy Association (1966) Standards for educational and psychological tests and manuals. American Psychological Association, Washington, DC

74. Kane MT (2006) Validation. In: Brennan RL (ed) Educational measurement, 4th edn. Praeger, Westport

75. De Vet HCW, Terwee CB, Mokkink LB, Knol DL (2011) Measurement in medicine, 1st edn. Cambridge University Press, New York

76. Hogle NJ, Liu Y, Ogden RT, Fowler DL (2014) Evaluation of surgical fellows laparoscopic performance using Global Operative Assessment of Laparoscopic Skills (GOALS). Surg Endosc. doi: 10.1007/s00464-013-3324-6

77. Van der Vleuten C (1996) The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ 1:41–67

93


Chapter 5

Validity Evidence for the Neuro-Endoscopic Ventriculostomy

Assessment Tool (NEVAT)

Gerben E. BreimerFaizal A. Haji

Giuseppe CinalliEelco W. HovingJames M. Drake

Operative Neurosurgery 2016DOI: 10.1227/NEU.0000000000001158

96

CHAPTER 5

Abstract

Background

Growing demand for transparent and standardized methods for evaluating surgical competence prompted the construction of the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT).

Objective

To provide validity evidence of the NEVAT by reporting on the tool’s internal structure and its relationship with surgical expertise during simulation-based training.

Methods

The NEVAT was used to assess performance of trainees and faculty at an international neuroendoscopy workshop. All participants performed an endoscopic third ventriculostomy (ETV) on a synthetic simulator. Participants were simultaneously scored by 2 raters using the NEVAT procedural checklist and global rating scale (GRS). Evidence of internal structure was collected by calculating interrater reliability and internal consistency of raters’ scores. Evidence of relationships with other variables was collected by comparing the ETV performance of experts, experienced trainees, and novices using Jonckheere’s test (evidence of construct validity).

Results

Thirteen experts, 11 experienced trainees, and 10 novices participated. The interrater reliability by the intraclass correlation coefficient for the checklist and GRS was 0.82 and 0.94, respectively. Internal consistency (Cronbach’s a) for the checklist and the GRS was 0.74 and 0.97, respectively. Median scores with interquartile range on the checklist and GRS for novices, experienced trainees, and experts were 0.69 (0.58-0.86), 0.85 (0.63-0.89), and 0.85 (0.81-0.91) and 3.1 (2.5-3.8), 3.7 (2.2-4.3) and 4.6 (4.4-4.9), respectively. Jonckheere’s test showed that the median checklist and GRS score increased with performer expertise (P = .04 and .002, respectively).

97

Validity evidence for NEVAT

Conclusion

This study provides validity evidence for the NEVAT to support its use as a standardized method of evaluating neuroendoscopic competence during simulationbased training.

Introduction

Significant changes in the landscape of surgical education, including resident duty hour restrictions, concerns about patient safety, mounting costs of health care, and the shift toward competency-based education, are putting significant strain on traditional methods of training and assessing residents’ surgical skills. These factors, along with the increase in competencybased education, have led to growing interest in the use of simulation in postgraduate surgical education.1-5

Whether simulation is used for training or to evaluate the skills of trainees at various points during residency, the need for assessment tools that are valid, reliable, and easily implemented has never been greater. Contemporary approaches to the development and evaluation of assessment tools in health professions education view validity as an argument, whereby evidence is collected to support the use of an assessment instrument in a particular setting.6 A widely cited framework describes 5 different types of evidence that can be used to support the validity of scores generated from an assessment tool: content, response process, internal structure, relationships with other variables, and consequences.7 Depending on the intended use of the instrument (eg, formative assessment vs credentialing decision), the level of evidence needed to support the tool will change. As such, the process of validation is not considered to be a one-time event.6,8,9 For example, if we demonstrate that 1 or more assessment scales are “valid” for assessing junior resident performance during simulation-based training of a given procedure, this does not necessarily translate into evidence supporting the use of these tools in the assessment of senior residents in the operating room (OR).10

In a previous study by our research group, a series of assessment instruments for the endoscopic third ventriculostomy (ETV) procedure were constructed to fulfill the need for standardized assessment methods for ventricular neuroendoscopy.11 Together these are called the Neuro-Endoscopic Ventriculostomy Assessment Tool

98

CHAPTER 5

(NEVAT). We previously generated content validity evidence for the NEVAT through a Delphi study of an international panel of expert neuroendoscopists.11 However, before these tools can be readily implemented in the assessment of ETV performance, further validity evidence related to internal structure and to relationships with other variables is required. The phrase “relationship to other variables” is a form of validity evidence in the framework that we used. It measures whether scores from the assessment instrument generate similar results as would be predicted from other measures of the underlying construct. In this study, it refers to whether the NEVAT scores can differentiate between the expertise of performers that would be predicted based on their previous experience with ETV (also known as construct validity). The purpose of this study is twofold. First, we sought to provide validity evidence related to internal structure by examining the interrater reliability and internal consistency of scores generated using the NEVAT assessment tools. Second, we sought to examine NEVAT’s relationships with other variables by comparing scores observed among novices, experienced trainees, and expert surgeons performing ETV on a physical simulator. Both of these aims were realized by collecting data on the NEVAT during an international neuroendoscopy workshop.

Methods

Workshop and Participants

The study took place during the annual international hands-on workshop on cerebral and ventricular neuroendoscopy at the Centre of Biotechnologies, Cardarelli Hospital, Naples, Italy (January 2014). All faculty were expert neuroendoscopists. The participants were a mix of neurosurgical residents, fellows, and consultant neurosurgeons. Preceding the course, participants and faculty alike received an invitation letter with information on the goals and format of the study. Thirty-four participants were included in the study, 13 of whom were faculty and 21 of whom were trainees. At the outset of the workshop, participants completed a demographic survey in which data regarding their status (resident [plus postgraduate year], fellow, or consulting neurosurgeons), previous experience

99


with ETV (observed, performed with assistance, and performed independently), and previous exposure to simulation-based training. The participants were subsequently divided into 3 groups: novices, “experienced trainees,” and experts. Novices were defined as assisting in fewer than 5 ETVs and performing none independently. The “experienced trainees” were defined as assisting in 5 or more ETVs and/or performing at least 1 independently. All faculty were regarded as experts.

Study Protocol

After providing informed consent, all participants were given the opportunity to perform an ETV procedure on a synthetic phantom, the S.I.M.O.N.T. Neurosurgical Endotrainer (Sinus Model Oto-Rhino Neuro Trainer, Pro Delphus Co.). Participants were instructed to complete a full ETV procedure as they would in the OR; all steps that could not be carried out on the simulator had to be mentioned as if they also were physically executed. Standard neuroendoscopic instruments were used during the simulation, including a Gaab universal rigid 0° endoscope (Karl-Storz, Tuttlingen, Germany), trocar, and grasping and dissecting forceps. Irrigation with normal saline solution was also provided. Participants were videotaped while performing the ETV in an anonymous manner (ie, by only videotaping hands and screen plus operating site, without including the face of the participant in the video).

Assessment Tool and Rating Procedure

The NEVAT contains 3 assessment measures: a procedure-specific checklist, a checklist of surgical errors, and a global rating scale (GRS).11 Checklist items were graded on a binary scale (done correctly vs done incorrectly) and the GRS items were graded on a 5-point Likert scale (with explicit verbal anchors associated with ratings of 1, 3, and 5). The GRS and procedure-specific checklist of the NEVAT were used for scoring all participants consisting of, respectively, 42 and 10 items (see Appendix, Supplemental Digital Content, http://links.lww.com/NEU/A820). A number of checklist items could not be completed on the simulator (corrected checklist with nonscorable items removed: n = 21, items 1-7, 9, 10, 20-22, 24-26, 29, 31-33, 36, 39). The participants were requested to

100

CHAPTER 5

talk the rater through the procedure, saying what they did or would do in a real OR setting on every consecutive step, thereby potentially allowing the raters to score all items, including those that could not be completed on the simulator (see Figure 1 for trainees at work).All 3 faculty raters were practicing pediatric neurosurgeons, each of whom had extensive experience with neuroendoscopic surgery and were familiar with the steps involved in an ETV procedure. For each trainee, 2 of the 3 faculty raters graded each participant’s ETV performance live during the workshop. They received oral and written instructions on how to use the NEVAT procedural checklist and the GRS. Overall, they were encouraged to score the checklist as the participant was performing the procedure and score the GRS directly afterward. The final score was expressed as a percentage of items completed correctly on the checklist (total points/scored items) and as the mean score for the GRS (average of scores across all GRS items).

Statistical Analysis

Differences in scores between novices, experienced trainees, and experts were analyzed by Jonckheere’s trend test. If the outcome of Jonckheere’s test was

Figure 1. Trainees at the workshop performing an endoscopic third ventriculostomy.

101


significant, post hoc testing was performed with pairwise Mann-Whitney U tests with Dunn-Bonferroni correction. Internal consistency of the checklist and GRS were evaluated using Cronbach’s a with the 95% confidence interval (CI) plus the mean interitem correlation. The intraclass correlation coefficient (ICC) plus the 95% CI was reported as a measure of interrater reliability, based on data from the 2 raters who scored performances live during the workshop. All data were analyzed using SPSS Statistics for Windows, Version 22.0 (IBM Corp, Armonk, New York) and R for Windows, Version 3.1.2 (R Foundation, Vienna, Austria) with statistical package “psychometric.”12,13 We considered P values ,.05 as statistically significant. Missing data were taken care of by having calculations based on the mean total scores of the checklist and GRS. Descriptive demographic characteristics were reported as medians and interquartile ranges or frequency counts and percentages.

Results

Demographic Data

All faculty were neurosurgical consultants for whom ETV was a routine part of their clinical and teaching responsibilities (Table 1). For details on trainees, see Table 2. A total of 67 evaluations were completed (34 participants rated by 2 raters) of 13 faculty, 11 experienced trainees, and 10 novices. One missing GRS form (ie, all items of 1 GRS) belonged to a novice.

Internal Structure

High interrater reliability was observed between the 2 live raters, with the ICC for the checklist scores being 0.82 (95% CI: 0.63-0.91) and the ICC for the GRS

Table 1. Demographics of experts (N = 13)

Abbreviation: IQR, interquartile range; ETV, endoscopic third ventriculostomy

102

CHAPTER 5

being 0.94 (95% CI: 0.89-0.97). A moderate level of internal consistencywas observed for the checklist, with Cronbach’s a = 0.74 (95% CI: 0.60-0.85) and mean interitem correlation = 0.066. The GRS had excellent internal consistency, with Cronbach’s a = 0.97 (95% CI: 0.95-0.98), with mean interitem correlation of 0.78.

Relationships With Other Variables

The Jonckheere’s test revealed a significant trend in the data (J =256, z = 2.04, P = .04, r = 0.35), with improving checklist scores observed as participant’s level of expertise increased; the trend did not remain when the corrected checklist was used with nonscorable items removed (J = 236, z = 1.41, P = .16, r = 0.24). Pairwise comparisons with adjusted P values showed no significant differences. Jonckheere’s test revealed a significant trend in the data, with the median GRS score improving with increased level of expertise (J = 291, z = 3.15, P = .002, r = 0.54) (Table 3). Pairwise comparisons with adjusted P values showed significant differences between novices and experts (P = .01). There was also a trend toward significance between experienced trainees and experts (P = .05); however, there was no significant difference between novices and experienced trainees (P = 1.0).

Table 2. Demographics of trainees (N = 21)

Abbreviations: IQR, interquartile range; PGY, post-graduate yeara One trainee did not disclose statusb Definition experienced trainee: assisting 5 or more ETVs and/or performing at least one independently

103


Discussion

Simulation and Competency-Based Training

With current resident work-hour restrictions, mounting concerns regarding patient safety, and the shift toward competency-based methods of surgical training, the role of simulation-based training and assessment in postgraduate neurosurgical education is growing.14-17 Simulation provides a low-risk environment where trainees can practice and their performance can be assessed before they begin performing procedures on real patients.18,19 Although simulation has recently been applied in various aspects of neurosurgical training (eg, cerebral aneurysm clipping, intracranial tumor resection, pedicle screw placement, cervical laminoforaminotomy, and spinal durotomy),1-5,20-30 applications to neuroendoscopy may be particularly beneficial for improving patient safety. In the literature, several authors report a steep learning curve for ETV, with most errors occurring in the early stages of a surgeon’s experience with the procedure.31-36 However, simulation-based training has been noted to reduce surgical errors for other procedures and may demonstrate similar results when applied to ETV.37-39 These characteristics make simulation-based neuroendoscopic training an important target for simulation-based training assessment within the postgraduate neurosurgical curriculum.To realize the potential of simulation and support the paradigm shift toward competency-based neurosurgical education, there is urgent need for standardized, procedure-specific methods for assessing surgical competence.17,40 The NEVAT was developed specifically for this purpose. In accordance with existing expert-based methods of assessment, the NEVAT relies on checklists and global ratings to evaluate a trainees’ technical skills.41-44 In addition to the content validity evidence previously generated by our research group on this tool,11 the current

Table 3. Median scores on checklists and GRS per group

Abbreviations: GRS, global rating scale; IQR, interquartile-range

104

CHAPTER 5

study generates evidence regarding the NEVAT’s relationships with other variables (ie, association with level of surgical expertise, in the classic validity framework known as construct validity) and internal structure (ie, interrater reliability and internal consistency).

Relationships With Other Variables

The validity evidence of the NEVAT with respect to its ability to distinguish between individuals at different levels of surgical expertise is demonstrated by the differences in scores observed among novices, experienced trainees, and expert surgeons performing ETV on a physical simulator. Even with our relatively small sample size, the GRS of the NEVAT was able to discriminate between novices and experts, and a trend toward significant differences between experienced trainees and experts was observed. There was also an overall significant trend in the median scores of the 3 groups that suggests that with increased level of training, GRS scores of onNEVAT improve, corroborating the relationship between NEVAT scores and surgical expertise. Furthermore, the experienced trainee cohort demonstrated the largest interquartile range, and their median score was closer to the novices than to the experts. As has been reported previously with other global rating instruments,45 these results suggest that the GRS of the NEVAT captures nuanced elements of surgical performance and as such may be particularly useful for assessment of more experienced trainees.Conversely, on the procedure-specific checklist of the NEVAT, the median score of experienced trainees was closer to that of experts than that of novices. This finding is not surprising, as such checklists are designed to evaluate a trainee’s knowledge of the steps required to complete a procedure rather than nuances regarding surgical technique. It is possible that a plateau or ceiling effect among experienced trainees and experts was observed, as these individuals had likely already mastered the basic steps of ETV. Furthermore, the fact that the median checklist score was not close to perfect also highlights that, with experience, a surgeon may not always complete a procedure in a standard fashion, but in a more fluid manner that reflects variations based on their expert judgment of the case at hand. In turn, this supports the argument that procedural checklists are most useful for tracking learning among novices in the initial stages of training

105


on a given procedure and is thus particularly useful for low-stakes formative assessment in which feedback is the goal.11 To facilitate this, the NEVAT checklist is divided into subsections, reflecting different parts of the ETV procedure. This allows educators to provide specific feedback and for trainees to focus on specific elements of the procedure in which they require further improvement (ie, deliberate practice).46

Internal Structure

Our study also provides validity evidence regarding the internal structure of the NEVAT, given that the interrater reliability observed for both the checklist (0.82) and GRS (0.94) is comparable to published values for other checklist and GRS tools.45 In general, ICC values of ≥0.7 are considered acceptable, with values ≥0.9 desired for high-stakes assessment.47 As further validity evidence regarding internal structure, a high level of internal consistency of the GRS was demonstrated in our study. The observed Cronbach’s a value (0.97) and interitem correlation (0.78) exceeded published guidelines for minimum acceptable values, which typically rangebetween 0.7 and 0.9 for Cronbach’s a and 0.15 to 0.5 for interitem correlations.47,48 However, as a values .0.9 may indicate that there is redundancy among items within the GRS,it is possible that some items of the GRS could be removed without losing much information from the assessment.47,49,50 As such, future work that tests shorter (and thus potentially more efficient) versions of the NEVAT GRS would be of value. Whereas Cronbach’s a of the checklist (0.74) is within acceptable values, the poor interitem correlation (0.066) suggests that this value may have been inflated by the large number of checklist items.48 One explanation for the poor interitem correlation observed in the checklist scores is that raters were given the option to mark an item with “not applicable.” In the synthetic simulator that is used for this study, it was not possible to perform certain aspects of the ETV procedure (eg, making an incision, placing the burr hole, closing the incision). To account for this, participants were asked to talk the raters through the procedure while they were performing an ETV. If a step of the procedure could not be physically performed, the participant would say what action normally would be taken. This

106

CHAPTER 5

may have led to a situation in which some raters scored “not applicable,” whereas others scored the item based on a candidate’s description of the step in question. Further work to clarify the consistency of checklist ratings during operative cases (in which the full checklist can be assessed) may address this issue.

Limitations and Future Work

There are some limitations to this study. First, the small sample size decreased the power in our study to differentiate between performances of novice, experienced, and expert participants. An additional limitation is the number of items in the checklists and GRS. Although reducing the number of items may improve usability and feasibility of these tools (thereby improving their uptake in both the clinical and simulated setting), such item reduction may come at a cost because the ability to discriminate between individuals at different levels of experience might decrease.47 Finally, as contemporary validity frameworks view validation as an argument supporting the use of an assessment instrument in a specific population for a specific purpose,6,10,51 based on the current data, the validity of the NEVAT cannot be assumed to extend beyond the simulated setting (ie, to assessment of performance in the OR) or to other neuroendoscopic procedures other than ETV. Future work to generate validity evidence related to response process (eg, think-aloud protocols of expert raters as they complete the assessment to determine what information they are using to generate their ratings)7,52,53 will also be useful to ensure that scores generated using the NEVAT accurately reflect surgical competence in ETV.The NEVAT could eventually be used as a standard evaluation method especially in the context of competency-based education. There is a need for ongoing assessment of trainees’ performance at various stages along the educational continuum. Based on the validity evidence generated in this study, the NEVAT can now be used for simulation-based assessment of ETV for both novice and experienced learners. These tools could also be used to assess the effectiveness of different simulation platforms or curricular designed for ETV training. As was previously proposed by Hooten et al,28 residents could also be required to demonstrate a certain level of competency (ie, achieve certain scores on the NEVAT) in a simulation-based setting before performing an ETV on a patient. Finally, the NEVAT also has the potential to be useful as an assessment tool for

107


postgraduate credentialing; however, stronger validity evidence (eg, related to response process and consequences) will need to be generated to justify its use in such high-stakes assessments.

Conclusion

We generated validity evidence for a series of assessment instruments designed to evaluate neuroendoscopic competence in the ETV procedure by reporting on the internal structure and its relationships with other variables during simulation-based training. Additional validation studies need to be conducted because the process of validation is not a one-time event. The development and evaluation of assessment tools such as theNEVATcan support the paradigm shift toward competency-based education by providing a method for standardized and transparent evaluation of surgical competence for neuroendoscopy training.

Acknowledgments

The authors thank all faculty and trainees of the neuroendoscopy workshop for their participation, and specifically Dr Spyros Sgouros and Dr Piero Spennato for their help with the live rating of the participants. They also thank Dr Santolo Cozzoline, Director of the Center of Biotechnologies of the Cardarelli Hospital, for logistic support of this study.

108

CHAPTER 5

References

1. Breimer G, Bodani V, Drake J. Design and evaluation of a new synthetic brain simulator for endoscopic third ventriculostomy. J. Neurosurg Pediatr. 2015;15(1):82-88.

2. Coelho G, Zymberg S, Lyra M. New anatomical simulator for pediatric neuroendoscopic practice. Childs Nerv Syst. 2015;31(2):213-219.

3. Waran V, Narayanan V, Karuppiah R, et al. Neurosurgical endoscopic training via a realistic 3-dimensional model. Simul Heal. 2015;10(1):43-48.

4. Lemole GM, Banerjee PP, Luciano C, Neckrysh S, Charbel FT. Virtual reality in neurosurgical education: part-task ventriculostomy simulation with dynamic visual and haptic feedback. Neurosurgery. 2007;61(1):142-148; discussion 148-149.

5. Cohen AR, Lohani S, Manjila S, Natsupakpong S, Brown N, Cavusoglu MC. Virtual reality simulation: basic concepts and use in endoscopic neurosurgery training. Childs Nerv Syst. 2013;29(8):1235-1244.

6. Downing SM. Validity: on meaningful interpretation of assessment data. Med Educ. 2003;37(9):830-837.

7. Cook DA, Zendejas B, Hamstra SJ, Hatala R, Brydges R. What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment. Adv Health Sci Educ Theory Pract. 2013;19(2):233-250.

8. Kane M, Crooks T, Cohen A. Validating measures of performance. Educ Meas Issues Pract. 1999;18:5-17.

9. Schuwirth LW, van der Vleuten CP. Programmatic assessment and Kane’s validity perspective. Med Educ. 2012;46(1):38-48.

10. Cook DA. When I say . . . validity. Med Educ. 2014;48(10):948-949.

11. Breimer GE, Haji FA, Hoving EW, Drake JM. Development and content validation of performance assessments for endoscopic third ventriculostomy. Childs Nerv Syst. 2015;31(8):1247-1259.

12. Fletcher TD. Psychometric: Applied Psychometric Theory. 2010.

13. R Core Team. R: A Language and Environment for Statistical Computing. 2014.

14. Coelho G, Zanon N, Warf BC. The role of simulation in neurosurgery. Childs Nerv Syst. 2014;30:1997-2000.

15. Rothstein BD, Selman WR. Evaluating simulation as a teaching tool in neurosurgery. Virtual Mentor. 2015;17(1):33-36.

16. Zanello M, Zerah M, Sainte-Rose C, Di Rocco F. Virtual simulation in neurosurgery: a comparison between pediatric and general neurosurgeons. Acta Neurochir (Wien). 2014;156(11):2215-2216.

17. Kavic MS. Simulators: a new use for an old paradigm. JSLS. 2006;10(3):281-283.

18. Haji FA, Dubrowski A, Drake J, de Ribaupierre S. Needs assessment for simulation training in neuroendoscopy: a Canadian national survey. J Neurosurg. 2013;118(2):250-257.

19. Kockro RA. Neurosurgery simulators—beyond the experiment. World Neurosurg. 2013;80(5):e101-e102.

20. Zymberg S, Vaz-Guimarães Filho F, Lyra M. Neuroendoscopic training: presentation of a new real simulator. Minim Invasive Neurosurg. 2010;53(1):44-46.

109


21. Choudhury N, Gélinas-Phaneuf N, Delorme S, Del Maestro R. Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg. 2013;80(5):e9-e19.

22. Delorme S, Laroche D, DiRaddo R, Del Maestro RF. NeuroTouch: a physics based virtual simulator for cranial microneurosurgery training. Neurosurgery. 2012;71(1 suppl operative):32-42.

23. Yudkowsky R, Luciano C, Banerjee P, et al. Practice on an augmented reality/haptic simulator and library of virtual brains improves residents’ ability to perform a ventriculostomy. Simul Healthc. 2013;8(1):25-31.

24. Müns A, Meixensberger J, Lindner D. Assessment of a novel phantom-based neurosurgical training system. Surg Neurol Int. 2014;5:173.

25. Harrop J, Rezai AR, Hoh DJ, Ghobrial GM, Sharan A. Neurosurgical training with a novel cervical spine simulator: posterior foraminotomy and laminectomy. Neurosurgery. 2013;73(suppl 1):94-99.

26. Ghobrial GM, Balsara K, Maulucci CM, et al. Simulation training curricula for neurosurgical residents: cervical foraminotomy and durotomy repair modules. World Neurosurg. 2015;84(3):751-755.e7.

27. Alaraj A, Luciano CJ, Bailey DP, et al. Virtual reality cerebral aneurysm clipping simulation with real-time haptic feedback. Neurosurgery. 2015;11(suppl 2):52-58.

28. Hooten KG, Lister JR, Lombard G, et al. Mixed reality ventriculostomy simulation: experience in neurosurgical residency. Neurosurgery. 2014;10(suppl 4):576-581.

29. Zammar SG, El Tecle NE, El Ahmadieh TY, et al. Impact of a vascular neurosurgery simulation based course on cognitive knowledge and technical skills in European neurosurgical trainees. World Neurosurg. 2015;84(2):197-201.

30. Alotaibi FE, AlZhrani GA, Mullah MA, et al. Assessing Bimanual performance in brain tumor resection with NeuroTouch, a virtual reality simulator. Neurosurgery. 2015;11(suppl 2):89-98.

31. Navarro R, Gil-Parra R, Reitman AJ, Olavarria G, Grant JA, Tomita T. Endoscopic third ventriculostomy in children: early and late complications and their avoidance. Childs Nerv Syst. 2006;22(5):506-513.

32. Schroeder HW, Niendorf WR, Gaab MR. Complications of endoscopic third ventriculostomy. J Neurosurg. 2002;96(6):1032-1040.

33. Furlanetti LL, Santos MV, de Oliveira RS. The success of endoscopic third ventriculostomy in children: analysis of prognostic factors. Pediatr Neurosurg. 2013;48(6):4-11.

34. Jones RF, Kwok BC, Stening WA, Vonau M. The current status of endoscopic third ventriculostomy in the management of non-communicating hydrocephalus. Minim Invasive Neurosurg. 1994;37(1):28-36.

35. Marton E, Feletti A, Basaldella L, Longatti P. Endoscopic third ventriculostomy in previously shunted children: a retrospective study. Childs Nerv Syst. 2010;26(7):937-943.

36. Teo C, Rahman S, Boop FA, Cherny B. Complications of endoscopic neurosurgery. Childs Nerv Syst. 1996;12(5):248-253; discussion 253.

37. Filho FVG, Coelho G, Cavalheiro S, Lyra M, Zymberg ST. Quality assessment of a new surgical simulator for neuroendoscopic training. Neurosurg Focus. 2011;30(4):E17.

38. Ahlberg G, Enochsson L, Gallagher AG, et al. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg. 2007;193(6):797-804.

39. Seymour NE, Gallagher AG, Roman SA, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002;236(4):458-463; discussion 463-464.

110

CHAPTER 5

40. Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638-645.

41. Eubanks TR, Clements RH, Pohl D, et al. An objective scoring system for laparoscopic cholecystectomy. J Am Coll Surg. 1999;189(6):566-574.

42. Taylor JB, Binenbaum G, Tapino P, Volpe NJ. Microsurgical lab testing is a reliable method for assessing ophthalmology residents’ surgical skills. Br J Ophthalmol. 2007;91(12):1691-1694.

43. Beard JD, Choksy S, Khan S. Assessment of operative competence during carotid endarterectomy. Br J Surg. 2007;94(6):726-730.

44. Moorthy K, Munz Y, Dosis A, Bello F, Chang A, Darzi A. Bimodal assessment of laparoscopic suturing skills: construct and concurrent validity. Surg Endosc. 2004; 18(11):1608-1612.

45. Ilgen JS, Ma IW, Hatala R, Cook DA. A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Med Educ. 2015;49(2):161-173.

46. Ericsson KA, Krampe RT, Tesch-Römer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev. 1993;100:363-406.

47. De Vet HC, Terwee CB, Mokkink LB, Knol DL, eds. Measurement in Medicine. New York: Cambridge University Press; 2011.

48. Clark LA, Watson D. Constructing validity: basic issues in objective scale development. Psychol Assess. 1995;7:309-319.

49. DeVellis RF. Scale Development—Theory and Applications. Thousand Oaks, CA:Sage Publications, Inc; 2012.

50. Streiner DL, Norman GR, eds. Health Measurement Scales: A Practical Guide to Their Development and Use. Kettering, United Kingdom: Oxford University Press; 2008.

51. Streiner DL, Kottner J. Recommendations for reporting the results of studies of instrument and scale development and testing. J Adv Nurs. 2014;70(9):1970-1979.

52. Van Hove PD, Tuijthof GJM, Verdaasdonk EGG, Stassen LPS, Dankelman J. Objective assessment of technical surgical skills. Br J Surg. 2010;97(7):972-987.

53. Ahmed K, Miskovic D, Darzi A, Athanasiou T, Hanna GB. Observational tools for assessment of procedural skills: a systematic review. Am J Surg. 2011;202(4):469-480.e6.

111


Chapter 6

Design and evaluation of a new synthetic brain simulator for

endoscopic third ventriculostomy

Gerben E. BreimerVivek BodaniThomas Looi

James M. Drake

J Neurosurg Pediatr 2015; 15: 82-88DOI: 10.3171/2014.9.PEDS1447

114

CHAPTER 6

Abstract

Object

Endoscopic third ventriculostomy (ETV) is an effective but technically demanding procedure with significant risk. Current simulators, including human cadavers, animal models, and virtual reality systems, are expensive, relatively inaccessible, and can lack realistic sensory feedback. The purpose of this study was to construct a realistic, low-cost, reusable brain simulator for ETV and evaluate its fidelity.

Methods

A brain silicone replica mimicking normal mechanical properties of a 4-month-old child with hydrocephalus was constructed, encased in the replicated skull, and immersed in water. Realistic intraventricular landmarks included the choroid plexus, veins, mammillary bodies, infundibular recess, and basilar artery. The thinned-out third ventricle floor, which dissects appropriately, is quickly replaceable. Standard neuroendoscopic equipment including irrigation is used. Bleeding scenarios are also incorporated. A total of 16 neurosurgical trainees (Postgraduate Years 1–6) and 9 pediatric and adult neurosurgeons tested the simulator. All participants filled out pediatric (5-point Likert-type items) to rate the simulator for face and content validity.

Results

The simulator is portable, robust, and sets up in minutes. More than 95% of participants agreed or strongly agreed that the simulator’s anatomical features, tissue properties, and bleeding scenarios were a realistic representation of that seen during an ETV. Participants stated that the simulator helped develop the required hand-eye coordination and camera skills, and the training exercise was valuable.

Conclusions

A low-cost, reusable, silicone-based ETV simulator realistically represents the surgical procedure to trainees and neurosurgeons. It can help them develop the technical and cognitive skills for ETV including dealing with complications.

115

Synthetic brain simulator for ETV

Introduction

The development of high levels of technical competence and excellent decision-making skills is a key goal of all neurosurgical residency training programs. Integral to this development is the exposure of residents to a large and diverse number of operative cases.13 However, over the past 10 years, there has been a significant decline in operative case volumes during residency.19 Many factors have contributed to this trend, including resident work hour restrictions, changes in disease management, rapid technological advancement, increased subspecialization, demand for more time- and technological practices, and increased demand for patient safety.11,19–21 Because of these challenges, there has been an increased interest in using models and simulators for neurosurgical education.21,22,26 An ideal procedure for simulation training is endoscopic third ventriculostomy (ETV). Although ETV is an effective and widely accepted treatment for obstructive hydrocephalus,9,23,29 it is technically challenging and carries the risk for serious errors, such as getting lost or disoriented and being unfamiliar with or unable to use the equipment, and technical complications, such as making an inadequate ventriculostomy or causing a basilar artery, hypothalamic, cranial nerve, or forniceal injury.4

Simulators create a risk-free environment in which residents can learn the basic neurosurgical skills that better prepare them for the actual operating room experience.21,22,26

Various training models have been used for neurosurgical training, including human cadavers, live animals, virtual reality, and synthetic phantoms. These models have been used to simulate both open transcranial and minimally invasive neurosurgical procedures.1–3,5–8, 10,12,14,15,17,18,24,25,30 Human cadaver and animal models provide high fidelity and allow residents to practice entire operations. However, they are expensive, difficult to obtain, raise ethical concerns, require special facilities and personnel, lack tissue properties that closely resemble living human brain tissue, and cannot simulate ventriculo-megaly.1,5,8,10,30 Virtual reality platforms are reusable and allow for accurate measurement of resident performance but are expensive, are difficult to maintain, lack convincing haptic feedback and visualization, and do not allow the use of actual convincing instruments.2,3,6,14,15,17,24,25 Synthetic phantom models have shown increasing

116

CHAPTER 6

promise as effective simulators for neurosurgical training.7,12,18 Such models are inexpensive, portable, and reusable and allow for the use of actual surgical instruments. However, they lack realistic tissue properties, are not patient specific, and cannot simulate complex and dynamic operations that require critical thinking and decision-making skills and measures to avoid and mitigate complications. To address these needs, we have developed and evaluated a low-cost, reusable, and patient-specific synthetic simulator that can be used to increase familiarity with the camera skills, endoscopic instruments, and hand-eye coordination required to successfully perform an ETV.

Methods

This study was carried out in 2 parts. Part I involved the design and construction of a synthetic brain simulator for ETV. Part II involved the evaluation of the simulator’s face and content validity based on feedback from neurosurgery residents, fellows, and attending staff at the University of Toronto.

Part I: Design and Construction of the ETV Simulator

Overview of the Simulator’s Features

Haji et al.17 conducted a needs assessment for simulation training in neuroendoscopy and identified the following features as the most important to simulate: 1) instrument setup, 2) following of cortical entry/trajectory, 3) insertion of the instrument into the ventricles, 4) navigation within the ventricles, 5) selection of the ventriculostomy site, 6) performing the fenestration, 7) confirming adequacy of the ventriculostomy, and 8) removing the endoscope and inspecting adequacy fornix. All of these features have been incorporated into our simulator.

Construction of the ETV Simulator

The preoperative CT and MRI studies obtained in a 4-month-old child with hydrocephalus secondary to a pineal tumor were obtained with permission from the parents. All images were de-identified and were stored on the hospital’s secure server to maintain patient confidentiality.

117


The preoperative CT and MRI images (DICOM format) were segmented using Mimics (Materialise NV), a commercial image-processing software, to produce 3D surface models of the brain, ventricles, brainstem, prepontine cistern, and the skull (Fig. 1). Using Magics (Ma- terialise NV), a commercial modeling software, the 3D surface models were edited to eliminate imperfections from the segmentation process and to generate brain and brainstem molds using the principles of negative space. The generated models were split in half along the midsagittal plane and were then saved in the stereolithography file format. The models were then printed using a Spectrum Z510 3D printer (Z Corp.). The printed ventricular system model (lateral and third ventricles) was used to create a “negative” silicone mold of the ventricles. This mold was then used to produce a clay model of the ventricles that exactly replicated the printed model. This was done to ensure easy extraction of the ventricle model once the final brain model was completed. The prepontine cistern was created in a similar way To create the brain and brainstem, Dragon Skin liquid silicone (Smooth-On, Inc.) was used to cast the negative molds. The mechanical properties of the silicone were adjusted by adding Slacker (Smooth-On, Inc.), a softener that allows the silicone to closely resemble brain tissue. Based on volume, we used a 2:1 mix ratio of silicone to Slacker. The clay ventricular and printed prepontine cistern models were appropriately positioned within one- half of the brain mold. The silicone/Slacker mixture was then poured into the mold and cured for 16 hours. The second half of the mold was then filled with the silicone/ Slacker mixture, and the first hemisphere was positioned on top so that the 2 hemispheres would merge together once the silicone cured. The “brain” was removed from both

Figure 1. Digital 3D surface molds of the ventricles, brain, and skull generated from the preoperative CT and MRI of a 4-month patient with hydrocephalus using image processing and modeling software.

118

CHAPTER 6

halves of the mold as a single piece. The clay ventricular model and printed prepontine cistern model were then extracted leaving the brain with an open ventricular system and prepontine cistern within which the brainstem and basilar artery could be placed.The anterior septal and thalamostriate veins, mammillary bodies, and infundibular recess were then added by hand using appropriately colored silicone. The basilar artery was created by layering red silicone over a 3D printed model of the vessel lumen. The basilar artery was then attached to the anterior surface of the brainstem. The choroid plexus was created using red foam and it was positioned within the choroid fissure of the lateral ventricles. The layer of silicone within the third ventricle floor located between the infundibular recess anteriorly and mammillary bodies posteriorly was removed. The floor was then reconstructed using wax paper supported on a plastic ring to simulate the thinned-out floor of the third ventricle. The dimensions of the floor were based on previously reported cadaveric studies.27 This type of floor was easily replaced after each ventriculostomy. Bleeding scenarios were incorporated within the simulator to train surgeons how to manage them appropriately. Silicone tubing was inserted into the choroid plexus and at the tip of the basilar artery. Simulated blood (milk with red food coloring) was injected from a 10-ml syringe via hidden tubing by the simulation monitor to simulate bleeding from the choroid hidden or a basilar artery injury. The final brain model was placed within the 3D printed skull model with a predrilled right frontal bur hole located 1 cm anterior to the printed coronal suture along the mid- pupillary line. The brain and skull were stabilized in a custom-made skull holder and placed into a water-filled container.

Part II: Evaluation of the Face and Content Validity of the ETV Simulator

Participants and Data Collection

Residents (Postgraduate Year [PGY] 1–6), fellows, and staff surgeons were recruited from the Division of Neurosurgery at the University of Toronto. Each subject was asked to independently perform an ETV on the simulator. Standard neuroendoscopic instruments were used, including a MINOP 0° rigid endoscope, trocar, and grasping and dissecting forceps (Aesculap, Inc.). Irrigation

119


with normal saline was also provided. Video was captured from the endoscope during the procedures and was stored on a personal computer for editing and analysis. After the procedure, the surgeons answered a questionnaire regarding the realism (face validity) and teaching effectiveness (content validity) of the simulator. Nine items were included in the questionnaire (Table 1). Items 1–4 evaluated the simulator’s face validity, while Items 5–9 evaluated its content validity. Participants were asked to rate their level of agreement with each item by means of a 5-point Likert scale, according to the following de- scriptions: 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree. Participants were also asked to provide open-ended comments regarding the simulator’s realism and effectiveness as a training tool.

Table 1. Summary of the questionnaire results (n = 25)a

a For item 1, n = 23 and for item 5, n = 24.

120

CHAPTER 6

Data Analysis

Questionnaire data were analyzed using SPSS Statistics for Windows (version 22, IBM Corp.). The distribution of responses for each questionnaire item was calculated. Questionnaire items that were partially or incorrectly completed were excluded from the analysis. The average overall rating (Items 1–9), average rating of realism (Items 1–4), and average rating of efficacy (Items 5–9) were calculated for each participant. Mean values for each rating were calculated for 3 evaluator groups differentiated by level of experience (PGY 1–2 = novices, PGY 3–6 = senior residents, and fellows and staff = neurosurgeons). A 1-way ANOVA was conducted to determine if differences in ratings between evaluator groups were statistically significant (p < 0.05).

Results

We successfully built a low-cost, reusable (and reproducible) silicone-based ETV simulator that realistically mimicked the normal mechanical properties of a 4-month-old child with hydrocephalus (Video: https://www.youtube.com/watch?v=C57wr0b9k-E ) .Realistic intraventricular landmarks including the choroid plexus, veins, mammillary bodies, infundibular recess, and basilar artery were successfully created (Figs. 2 and 3). The thinned-out third ventricle floor dissected appropriately and was quickly replaceable. Bleeding scenarios were successfully incorporated. Standard neuroendoscopic equipment including irrigation was used to perform the ETV. The simulator could be set up in minutes, was easily cleaned, and did not require maintenance or special storage. The production costs were $452 for the molds (fixed costs) and $160.50 for each simulator. The simulator required 55 hours to build, with 10–15 hours of labor. A limited number of models for academic courses can be fabricated on a cost recovery basis (contact corresponding author for details).Twenty-five neurosurgeons in various stages of their career participated in the second part of the study: 16 residents (PGY 1 [n = 4], PGY 3 [n = 3], PGY 4 [n = 4], PGY 5 [n = 4], and PGY 6 [n = 1]), 5 fellows, and 4 adult and pediatric staff neurosurgeons. The median number of ETVs previously performed by this group was 10 (interquartile range 0–20).

121


Figure 2. Photographs of the ETV simulator. From top left clockwise: a) Brain, brainstem, ventricle, and prepontine cistern molds cast with silicone, b) Anatomical structures of the right lateral ventricle including septum pellucidum, foramen of Munro, choroid plexus, anterior septal and thalamostriate veins, d) and e) Replaceable third ventricular floor, f) Installed third ventricular floor, g) Brainstem and basilar artery.

Figure 3. Intraventricular views. Left: Right lateral ventricle. AS = anterior septal vein; CP = choroid plexus; F = fornix; FM = foramen of Monro; TS = thalamostriate vein. Right: Third ventricle. IR = infundibular recess; MB = mammillary bodies; TC = tuber cinereum.

122

CHAPTER 6

Results of the questionnaire are reported in Table 1. Three participants had incompletely or incorrectly answered a single item on the questionnaire and therefore their responses to these items were omitted from the analysis. More than 95% of participants agreed or strongly agreed that the camera view was comparable to that seen in a real patient and that performing the ventriculostomy in the model felt like it does in reality. Most participants agreed or strongly agreed that the mechanical properties of the simulator closely matched live tissue properties, and all agreed that the bleeding looked realistic. Furthermore, all participants agreed or strongly agreed that the ETV simulator helped develop the camera skills and hand-eye coordination needed for ETV and that it was a valuable training exercise and would increase resident competency prior to performing their first actual ETV. All participants agreed or strongly agreed that they would be interested in using the model to train residents. There were no statistically significant differences between average overall ratings from the 3 evaluator groups (1-way ANOVA, p = 0.19): novices (mean 4.3 ± 0.3 [± SD]), senior residents (mean 4.6 ± 0.3), and neurosurgeons (mean 4.6 ± 0.2). Additionally, no statistically significant differences were found between the average rating on realism (Items 1–4) and efficacy (Items 5–9) for each of the 3 groups (p = 0.46 and p = 0.16, respectively).The provided open-ended comments were consistent with the results of the questionnaire; the participants appreciated the ability to use real instruments, the realistic tactile feedback especially when passing through the third ventricular floor, the realistic appearance of bleeding, and the ability to build teamwork. Several suggestions were provided to improve future iterations of the simulator: designing complex third ventricular floors (i.e., thick, opaque, bulging, and with presence of Liliequist’s membrane) and modifying the intraventricular anatomy (reducing the size of the foramen of Monro and the size of the mammillary bodies, including the infundibulum, and adjusting the orientation of the veins).

Discussion

Endoscopic third ventriculostomy is an effective and widely accepted treatment for obstructive hydrocephalus but it is technically demanding due to limited

123


depth perception, narrow ranges of motion, limited tool dexterity, and the presence of critical surrounding anatomy.4,9,23,29 As a result, it is a challenging but essential procedure to learn during residency. To learn, practice, and perfect the skills required for ETV, exposure to a large volume of cases is needed. Several factors have contributed toward this trend of declining case volumes, including work hour limits, changes in disease management, rapidly advancing technology, subspecialization, and increased demand for patient safety and cost-efficient health care practices.11,20,21 As a result, there has been an increased interest in the use of simulation to train residents in a risk-free environment without time or resource constraints.22,26 Simulators have been shown to improve real-life surgical performance and accelerate learning curves, resulting in improved patient outcomes, fewer complications, and decreased operative times.16,28,31

Currently available ETV simulators include virtual reality systems (NeuroTouch)3,6,25 and synthetic phantoms (S.I.M.O.N.T. Neurosurgical Endotrainer [Sinus Model Oto-Rhino Neuro Trainer, Pro Delphus Co.]).7,12,18 A key advantage of NeuroTouch is its ability to provide real-time feedback on important performance metrics such as tool trajectories, contact forces with critical structures, procedure time, and ventriculostomy location.3 However, high cost, inaccessibility, the lack of a sufficiently accurate haptic interface, and unrealistic dissection of the third ventricular floor are its biggest obstacles.25 The S.I.M.O.N.T. synthetic simulator has been shown to be highly realistic and improve surgical performance, with a 42% reduction in trainee errors after use of the simulator.7 However, a key drawback is that the model is not patient specific and therefore it does not allow preoperative rehearsal of actual patient cases. In addition, the brain and ventricular system needs to be replaced after each use, leading to increased costs. To our knowledge, the presented simulator is the first patient-specific synthetic simulator developed for neurosurgery. Several key advantages were demonstrated during the evaluation process, including its low cost, robustness, quick setup time, portability, reusability, and lack of special maintenance or storage requirements. Printing the brain and ventricular system of other patients would follow the same process, and, with the advancement of 3D printing technology, could potentially be automated. Results of the questionnaire established the simulator’s face and

124

CHAPTER 6

content validity. More than 95% of participants agreed that the simulator’s anatomical features and tissue properties were realistic representations of those experienced during an ETV. All participants agreed that the simulator helped develop the hand-eye coordination, instrument handling, and camera skills required for ETV; that it was a valuable training exercise; and that it provided realistic bleeding scenarios. During the evaluation process, some limitations of the simulator became clear. First, it was not possible for the participants to plan the ideal location of the skin incision and bur hole based on preoperative MRI. Second, participants were unable to practice making the skin incision, drill the bur hole, or perform the corticotomy. Third, the model was not integrated with image guidance (either ultrasound or neuronavigation) to help guide trocar insertion into the ventricle. Fourth, participants were not able to use bipolar cautery to achieve hemostasis, relying only on irrigation. Fifth, the third ventricle floor was not patient specific. For example, certain features such as floor thickness, (downward) bulging of the floor, distance between the mammillary bodies and infundibular recess, presence of Liliequist’s membrane, and the exact relationship of the basilar artery to the floor could not be specifically designed. Sixth, it was not possible to observe pulsations of the edges of the ventriculostomy site to ensure the stoma was adequate. Seventh, it was not possible to practice closure of the incision. Lastly, we were unable to track instrument motions or applied tool-tip forces to evaluate surgeon performance. These limitations will be addressed in future iterations of the simulator design.There were some limitations in the design of the evaluation study. First, some participants, in particular the PGY 1 residents, had little or no previous experience performing an ETV, thereby limiting their ability to accurately evaluate the simulator’s realism and teaching effectiveness. Second, the subjective and retrospective nature of comparisons between the simulator and real-life cases may have introduced recall bias. Lastly, there were a limited number of participants, all from the University of Toronto, affecting the generalizability of the results.

125


Conclusions

We have successfully created a low-cost, reusable, patient-specific silicone-based ETV simulator that has high face and content validity. This simulator may help residents to increase their familiarity with the camera skills, instrument handling, and hand-eye coordination required to successfully perform an ETV. The development of future iterations will focus on integration with image guidance systems, incorporation of presurgical planning, design of patient-specific intraventricular landmarks, and providing real-time performance feedback. In addition, models for other age groups and intraventricular pathologies such as tumors or colloid cysts will be developed. Once these models have been fully developed, large prospective randomized trials involving novice and expert surgeons will be required to evaluate the simulator’s face, content, and construct validity. Furthermore, comparisons against other training models such as virtual reality simulation will be required.

126

CHAPTER 6

References

1. Aboud E, Al-Mefty O, Yasargil MG: New laboratory model for neurosurgical training that simulates live surgery. J Neu- rosurg 97:1367–1372, 2002

2. Alaraj A, Charbel FT, Birk D, Tobin M, Luciano C, Banerjee PP, et al: Role of cranial and spinal virtual and augmented reality simulation using immersive touch modules in neurosurgical training. Neurosurgery 72 Suppl 1:115–123, 2013 (Erratum in Neurosurgery 73:E913, 2013)

3. Becerra Romero ADC, Zicarelli CA, Pinto FCG, Pasqualucci CA, Aguiar PHP: Simulation of endoscopic third ventriculostomy in fresh cadaveric specimens. Minim Invasive Neurosurg 52:103–106, 2009

4. Bouras T, Sgouros S: Complications of endoscopic third ventriculostomy. A review. J Neurosurg Pediatr 7:643–649, 2011

5. Chan S, Conti F, Salisbury K, Blevins NH: Virtual reality simulation in neurosurgery: technologies and evolution. Neurosurgery 72 (Suppl 1):154–164, 2013

6. Choudhury N, Gélinas-Phaneuf N, Delorme S, Del Maestro R: Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg 80:e9–e19, 2013

7. Coelho G, Kondageski C, Vaz-Guimarães Filho F, Ramina R, Hunhevicz SC, Daga F, et al: Frameless image-guided neuroendoscopy training in real simulators. Minim Invasive Neurosurg 54:115–118, 2011

8. Cohen AR, Lohani S, Manjila S, Natsupakpong S, Brown N, Cavusoglu MC: Virtual reality simulation: basic concepts and use in endoscopic neurosurgery training. Childs Nerv Syst 29:1235–1244, 2013

9. Cook DA, Hatala R, Brydges R, Zendejas B, Szostek JH, Wang AT, et al: Technology-enhanced simulation for health professions education: a systematic review and meta-analysis. JAMA 306:978–988, 2011

10. Delorme S, Laroche D, DiRaddo R, Del Maestro RF: Neuro-Touch: a physics-based virtual simulator for cranial micro-neurosurgery training. Neurosurgery 71 (1 Suppl Operative):32–42, 2012

11. Drake FT, Horvath KD, Goldin AB, Gow KW: The general surgery chief resident operative experience: 23 years of national ACGME case logs. JAMA Surg 148:841–847, 2013

12. Drake JM: Endoscopic third ventriculostomy in pediatric patients: the Canadian experience. Neurosurgery 60:881–886, 2007

13. Feng H, Huang G, Liao X, Fu K, Tan H, Pu H, et al: Endoscopic third ventriculostomy in the management of obstructive hydrocephalus: an outcome analysis. J Neurosurg 100:626–633, 2004

14. Filho FV, Coelho G, Cavalheiro S, Lyra M, Zymberg ST: Quality assessment of a new surgical simulator for neuroendoscopic training. Neurosurg Focus 30(4):E17, 2011

15. Gurusamy K, Aggarwal R, Palanivelu L, Davidson BR: Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. Br J Surg 95:1088–1097, 2008

16. Haase J, Boisen E: Neurosurgical training: more hours needed or a new learning culture? Surg Neurol 72:89–97, 2009

17. Haji FA, Dubrowski A, Drake J, de Ribaupierre S: Needs assessment for simulation training in neuroendoscopy: a Canadian national survey. Clinical article. J Neurosurg 118:250–257, 2013

18. Halm EA, Lee C, Chassin MR: Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med 137:511–520, 2002

127


19. Hayashi N, Kurimoto M, Hamada H, Kurosaki K, Endo S, Cohen AR: Preparation of a simple and efficient laboratory model for training in neuroendoscopic procedures. Childs Nerv Syst 24:749–751, 2008

20. Jiang D, Hovdebo J, Cabral A, Mora V, Delorme S: Endoscopic third ventriculostomy on a microneurosurgery simulator. Simulation 89:1442–1449, 2013

21. McCall T, Rao G, Kestle J: Work hour restrictions: impact on neurosurgical resident training at the University of Utah. AANS Bulletin 14:17–22, 2005

22. Neubauer A, Wolfsberger S: Virtual endoscopy in neurosurgery: a review. Neurosurgery 72 (Suppl 1):97–106, 2013

23. Reznick RK, MacRae H: Teaching surgical skills—changes in the wind. N Engl J Med 355:2664–2669, 2006

24. Robison RA, Liu CY, Apuzzo MLJ: Man, mind, and machine: the past and future of virtual reality simulation in neu- rologic surgery. World Neurosurg 76:419–430, 2011

25. Romero Adel C, Aguiar PH, Borchartt TB, Conci A: Quantitative ventricular neuroendoscopy performed on the third ventriculostomy: anatomic study. Neurosurgery 68 (2 Suppl Operative):347–354, 2011

26. Sacko O, Boetto S, Lauwers-Cances V, Dupuy M, Roux FE: Endoscopic third ventriculostomy: outcome analysis in 368 procedures. Clinical article. J Neurosurg Pediatr 5:68–74, 2010

27. Scott DJ, Cendan JC, Pugh CM, Minter RM, Dunnington GL, Kozar RA: The changing face of surgical education: simulation as the new paradigm. J Surg Res 147:189–193, 2008

28. Tubbs RS, Loukas M, Shoja MM, Wellons JC, Cohen-Gadol AA: Feasibility of ventricular expansion postmortem: a novel laboratory model for neurosurgical training that simulates intraventricular endoscopic surgery. Laboratory investigation. J Neurosurg 111:1165–1167, 2009

29. Walker RL: Human and animal subjects of research: the moral significance of respect versus welfare. Theor Med Bioeth 27:305–331, 2006

30. Zendejas B, Brydges R, Hamstra SJ, Cook DA: State of the evidence on simulation-based training for laparoscopic surgery: a systematic review. Ann Surg 257:586–593, 2013

31. Zymberg S, Vaz-Guimarães Filho F, Lyra M: Neuroendoscopic training: presentation of a new real simulator. Minim Invasive Neurosurg 53:44–46, 2010

Chapter 7

Simulation-based Education for Endoscopic Third

Ventriculostomy: A Comparison Between Virtual and Physical

Training Models

Gerben E. BreimerFaizal A. HajiVivek Bodani

Melissa S. CunninghamAdriana-Lucia Lopez-Rios

Allan OkrainecJames M. Drake

Oper Neurosurg. 2016DOI: 10.1227/NEU.0000000000001317

130

CHAPTER 7

Abstract

Background

The relative educational benefits of virtual reality (VR) and physical simulation models for endoscopic third ventriculostomy (ETV) have not been evaluated “head to head.”

Objective

To compare and identify the relative utility of a physical and VR ETV simulation model for use in neurosurgical training.

Methods

Twenty-three neurosurgical residents and 3 fellows performed an ETV on both a physical and VR simulation model. Trainees rated the models using 5-point Likert scales evaluating the domains of anatomy, instrument handling, procedural content, and the overall fidelity of the simulation. Paired t tests were performed for each domain’s mean overall score and individual items.

Results

The VR model has relative benefits compared with the physical model with respect to realistic representation of intraventricular anatomy at the foramen of Monro (4.5, standard deviation [SD] = 0.7 vs 4.1, SD = 0.6; P = .04) and the third ventricle floor (4.4, SD = 0.6 vs 4.0, SD = 0.9; P = .03), although the overall anatomy score was similar (4.2, SD = 0.6 vs 4.0, SD = 0.6; P = .11). For overall instrument handling and procedural content, the physical simulator outperformed the VR model (3.7, SD = 0.8 vs 4.5; SD = 0.5, P,.001 and 3.9; SD = 0.8 vs 4.2, SD = 0.6; P = .02, respectively). Overall task fidelity across the 2 simulators was not perceived as significantly different.

Conclusion

Simulation model selection should be based on educational objectives. Training focused on learning anatomy or decision-making for anatomic cues may be aided with the VR simulation model. A focus on developing manual dexterity and technical skills using endoscopic equipment in the operating room may be better learned on the physical simulation model.

131

Comparing VR and physical training model

Introduction

Multiple factors have prompted a recent surge in the development of neurosurgical simulators, including restricted resident work hours, increased focus on patient safety, and rapid technological advancements.1,2 Both physical and virtual reality (VR) simulators have been used to teach a variety of neuro-endoscopic procedures, in particular, endoscopic third ventriculostomy (ETV).3-7 Similarly, tools recently have been developed to assess the surgical performance of trainees on these simulators, including standardized objective assessment measures specific for ETV.8,9 With these advances there has been a call to incorporate neuroendoscopic simulation into the neurosurgical training curriculum.4,10-12 While this is an important goal, presently there is limited evidence to guide neurosurgical educators on how best to incorporate existing simulation modalities within postgraduate training to optimize learning outcomes.13-15 For instance, the relative educational benefits of virtual and physical simulation models for ETV have not been evaluated “head to head.” This is an important line of inquiry as the relative cost, durability, and acceptability of these different platforms may vary. Further insight on these issues may help to inform neurosurgical educators about which platforms are worth investing in, and for what purpose. This study compares the perceived utility of physical and VR ETV simulators for neurosurgical training during a neuroendoscopy workshop organized by the Division of Neurosurgery at the University of Toronto. We set out to address the properties of 2 different platforms, a physical synthetic simulator (SickKids brain simulator3) and an established VR simulator (NeuroTouch Simulator6,16), and provide suggestions on how each could be used within a broader neuroendoscopic training curriculum. The evaluation focused on 4 essential domains of the simulators: modeling of anatomy, instrument handling, content of the simulated task, and overall fidelity.

Methods

Study approval was obtained from the University Health Network Research Ethics Board (ID: 10-0195-BE). Participants were recruited and consented to participate on a voluntary basis during 2 consecutive hands-on neuroendoscopy training courses organized yearly by the Division of Neurosurgery of the

132

CHAPTER 7

University of Toronto at the Surgical Skills Centre in Mount Sinai Hospital (May 2, 2014 and May 23, 2015).

Description of Simulators

For both simulators, there was no simulated skin for incision, and the trainee used a pre-drilled burr hole for introducing the endoscope. As such, on both platforms the procedure started with the introduction of the endoscope.

Physical ETV Simulation Model

A patient-specific, silicone-based, reusable ETV simulator based on a 4-month-old child with hydrocephalus was used for this study.3 The SickKids brain simulator includes intraventricular landmarks such as the choroid plexus, anterior septal and thalamostriate veins, mammillary bodies, bodies, infundibular recess, and basilar artery (Figure 1). The thinned- out third ventricle floor is replaceable quickly. The third ventricle floor does not pulsate. Bleeding scenarios are incorporated to simulate injury to the basilar artery and/or choroid plexus. Standard neuroendoscopic equipment is used to perform the ETV. The simulator can be set up in minutes and does not require maintenance or special storage.

Virtual Reality Simulation Model

The virtual reality ETV simulator used for this study was developed by the National Research Council of Canada, as a part of the NeuroTouch system.6,16

Figure 1. SickKids brain simulator. The left side shows the view of foramen of Monro, and the right side shows the view of third ventricle. Used with permission from Breimer et al (J. Neurosurg Pediatr. 2015;15:82-88).

133


NeuroTouch is a general neurosurgical simulator that contains modules for ETV, endoscopic endonasal surgery, and microscopic cranial procedures. The ETV module includes intraventricular landmarks such as the choroid plexus, veins, mammillary bodies, and infundibular recess (Figure 2). Trainees can select the location of the burr hole and the insertion trajectory to cannulate the lateral ventricle, followed by intraventricular navigation and placement of the ventriculostomy in the floor of the third ventricle using modified endoscopic instruments designed for the simulator. The third ventricle floor moves according to the combined effects of breathing and blood pulsations. A haptic system (PHANTOM Omni [Sensable, Wilmington, Massachusetts]) attached to the neuro-endoscope provides tactile feedback when contact is made with the deformable ventricle walls, using a collision detection algorithm.17 The endoscopic views are displayed on a monitor in front of the user, similar to a real operating room setting.

Study Design

The virtual reality and physical ETV simulation models were set up for use, and participants were asked to perform the ETV procedure as they would in a clinical setting. Although we did not randomize the order of presenting the physical vs virtual model, the order of presentation was balanced (with half of the participants using the virtual model first, while half used the physical model first). Thus, any systematic differences related to the order of presentation of the model would have been reduced. Randomization was not practically possible

Figure 2. NeuroTouch. The left side shows the view of foramen of Monro, and the right side shows the view of third ventricle.

134

CHAPTER 7

due to logistical constraints associated with the number of models available during the course. Participants were then given a paper-based survey consisting of 3 sections: one pertaining to demographic data, the second pertaining to the physical simulator, and the final pertaining to the virtual simulator (see Survey, Supplemental Digital Content, http://links.lww.com/NEU/A879). They were asked to assess 4 domains for each simulation platform: anatomy, instrument handling, content of the simulated procedure, and overall task fidelity (realism). Within each domain, a series of 3 or 4 questions were presented, and participants provided a rating using a 5-point Likert scale (ranging from 1 = strongly disagree to 5 = strongly agree). Participants also were asked to provide open-ended comments regarding their opinion of each simulator, including which they perceived as being superior and why.

Statistical Analysis

Demographic characteristics were summarized descriptively and reported as frequency counts and percentages. Where the distribution of the differences between the mean scores on the VR and physical simulators were normally distributed, the paired t test was used to evaluate for any significant differences in participants’ ratings of the 2 simulators. Mean values for each rating were calculated for 3 evaluator groups differentiated by level of experience: junior (postgraduate year [PGY] 1-2; n = 7), intermediate (PGY 3-4; n = 12), and senior (PGY 5-6 or fellows; n = 7). A one-way analysis of variance was used to determine differences in ratings between evaluator groups. The difference in overall mean score between the 2 simulators was calculated for each participant (physical simulator score minus VR simulator score). We compared the means with an unpaired t test to analyze whether prior experience with simulation and prior experience with assisting ETV procedures had an effect on the perception of the subject. P values. 05 were considered statistically significant. Incomplete items on the survey forms were excluded from the data analysis in list-wise fashion. Statistical analysis was performed using SPSS Statistics version 18.0 (IBM Corp., Armonk, New York).

135


Results

Demographics

Twenty-three neurosurgical residents (distributed across PGY1 through 6) and 3 neurosurgical fellows completed the study. The participants were from neurosurgical programs across Ontario. Demographic information is summarized in Table 1.

Table 1. Study participant demographics

Abbreviations: ETV, endoscopic third ventriculostomy;PGY, postgraduate year

136

CHAPTER 7

Likert Response Data

Table 2 shows the mean (SD) Likert-scale scores assigned to each domain overall and for individual items. Five participants (4 residents and 1 consulting neurosurgeon) were excluded from the analysis due to missing data. Overall, participants rated the representation of anatomy higher on the VR simulator when compared with the physical simulator; however, the difference was not statistically significant (4.2, SD = 0.6 vs 4.0, SD = 0.6; P = .11). However, the VR simulator showed a statistically significant advantage compared with the physical simulator in terms of the realism of the third ventricle floor (4.4, SD = 0.6 vs 4.0, SD = 0.9; P = .03) and the intraventricular anatomy (4.5, SD = 0.7 vs 4.1, SD = 0.6; P = .04). Conversely, participants rated the instrument handling (3.7, SD = 0.8 vs 4.5, SD = 0.5; P = .001) and procedure content (3.9, SD = 0.8 vs 4.2, SD = 0.6; P = .02) of the VR simulator lower when compared with the physical simulator. These differences were statistically significant. Lastly, mean ratings of overall task fidelity for the VR simulator were marginally higher than the physical simulator; however, the difference was not statistically significant (3.8, SD = 0.6 vs 4.0, SD = 0.5; P = .23). There were no statistically significant differences between the 3 evaluator groups in the average ratings for each domain (both composite scores and individual items, P = .27). Prior experience with simulation and prior experience with assisting ETV procedures did not have a significant effect on the participants’ perception of the simulators (P = .44 and P = .60, respectively).

Open-Ended Data

All participants indicated that they would personally use both simulators for training. Thirteen participants (50%) stated that the physical model was superior, especially for its overall realism, tactile feedback, and use of real instruments. Three participants (12%) thought the VR model was superior, especially due to its simulation of the anatomy. Five participants (19%) stated that the VR simulator was superior for anatomy while the physical simulator was superior for instrument handling. Five participants (19%) were undecided about which simulator was better.

137


Table 2. Comparing a virtual reality and physical model

Abbreviations: VR, virtual reality

138

CHAPTER 7

Discussion

Although training in the OR is effective, it also may be inefficient and costly.18,19 These problems can be addressed by using simulation to improve a resident’s surgical skills before they proceed to a real surgical environment in order to maximize training safety and minimize risk.20-23 With increasing recognition of the limitations of clinical surgical training, there is growing interest in simulation-based education in neurosurgery, with several publications in recent years describing a variety of simulators, both physical and VR platforms, their use in “boot camp” courses, and their proposed implementation in neurosurgical curricula.24-29 However, simulation-based education is an expensive undertaking, and due to budget constraints, educators are often faced with the dilemma of having to choose between different training platforms (eg, physical or VR simulators). To guide decision- making, we compared the perceived educational benefit of a physical and VR simulator for ETV. The findings demonstrate that both simulators have their own benefits and drawbacks, and that the choice of simulator should be tailored to specific learning objectives. In the sections to follow, the advantages, disadvantages, and specific characteristics of both platforms are described.

Accessibility

Both physical and VR simulators for neuroendoscopy are available commercially; however, they differ significantly in terms of cost. The NeuroTouch simulator can be purchased for approximately $80 000 while the SickKids brain simulator costs approximately $6000. In addition to its high upfront cost, VR simulators have high maintenance costs. However, VR simulators can be reused many times without additional cost of materials. In contrast, the physical simulator has minimal maintenance and storage requirements, a long shelf life (2-3 years), and a low replacement cost relative to the cost of the VR simulator. The third ventricle floor of the physical simulator is replaced after each use, which entails an additional albeit minimal cost. Modifications to the VR simulator can be made as required; however, this requires the development of new programs, which can be expensive and time-consuming. Modifications to the physical simulator requires the production of new molds and would therefore necessitate the purchase of a new model.

139


Feasibility

While the VR simulator provided a superior third ventricle floor and intraventricular anatomy, the physical simulator provided superior instrument handling and procedural content. Several factors may have contributed to the latter finding. First, the use of actual tools with the physical simulator encourages the use of bimanual instrumentation in which the trainee uses one hand to control the position of the camera/trocar and the other hand the manipulate tissue using a tool such as a blunt probe, forceps, or irrigation. Trainees can compare the use of different instruments and gain exposure to new equipment and techniques. The relative benefits of using actual instruments also was found in a study comparing an animal model and VR system for percutaneous renal access.30 Furthermore, Madan et al31 found that more than 80% of trainees preferred using an inanimate box trainer compared with a VR simulator for laparoscopy training. The physical models also may enhance teamwork and communication skills, as 2 residents can work together (one positioning the camera, the other controlling the tools). Second, the physical model incorporates bleeding scenarios from the choroid plexus and basilar artery that train the surgeon on hemostasis techniques including irrigation and cautery. The NeuroTouch simulator has bleeding scenarios in the other modules (eg, tumor resection) which means it should be possible to incorporate this in the ETV module (being less messy than in physical model, this could be an advantage). Third, the physical models are visible on CT and MR and can therefore be used with neuronavigation to plan an operation. Different materials can be used to construct the brain, which would allow for ultrasound visibility.32 In other VR platforms it is a possibility to upload CT data to perform case rehearsal, and for ETV, preliminary work on patient-specific simulation scenarios based on MRI data has been reported.33,34 Lastly, the physical models can be adapted easily such that the trainee plans the procedure with surface landmarks and navigation, positions and drapes the model, makes a scalp incision, drills a burr hole, opens the dura, and closes the incision, as has been shown in other physical brain simulators.35,36 As mentioned, the above scenarios could be programmed into the VR simulator, but are currently not available for the ETV simulation.

140

CHAPTER 7

Training and Assessment

For both simulators, a trained preceptor is required to familiarize trainees with the instruments and system setup prior to their first use. Both simulators facilitate deliberate practice; however, the physical simulator requires a longer setup time between each attempt in order to replace the third ventricle floor. For real-time evaluation of the trainee, the VR system has the advantage of an objective scoring system, which the computer can generate automatically to assess the trainee. For the physical simulator, an objective assessment tool such as the Neuro- Endoscopic Ventriculostomy Assessment Tool (NEVAT) can be used to assess surgical skills in real-time; however, an educator with knowledge of the ETV procedure is required. To obviate this limitation, trainees can be assessed using a post-procedure video review. However, video-based assessments can be cumbersome (difficult to acquire video, trainee less likely to verbalize their thought process, etc.) and may limit the amount, type, and timeliness of feedback from the instructor. With either simulator, the lack of real-time expert feedback may lead to the development of poor techniques if there is limited understanding of why certain techniques lead to better or worse performance. This is especially problematic for junior trainees who have less experience and for the assessment of nontechnical skills such as situation awareness, teamwork, and communication.

Limitations and Future Research

The main limitation of this study is the limited generalizability of the results to other physical and VR simulators, including those currently available and in development, which may vary in terms of anatomic detail, tactile properties, the use of instruments, and procedural content. A second limitation is the use of subjective outcome measures rather than an objective evaluation of the simulators’ face, content, and construct validity. This limitation may be addressed through an assessment tool like NEVAT, which could be used to evaluate trainee performance before and after a fixed number of practice sessions on each simulator. Differences in the amount and rate of improvement may indicate which platform is superior. Similar studies have been conducted for laparoscopic suturing,37,38 endoscopic surgery,39 and intra-venous cannulation.40

141


Other limitations include the small sample size, the inclusion of only residents and fellows in the study population, the risk of self-reporting bias, and the use of a static measurement (opinions of the simulator may change as experience with the simulators increase). It would be interesting to see if differences across geographical regions influences perceptions about educational utility of the models.

Conclusion

Two neurosurgical simulators for ETV were compared: a physical and a VR simulator. Each simulator has relative advantages and disadvantages. Therefore, educators may consider using either simulator based on training goals. In their present state, the physical simulator is superior for familiarizing trainees with neuroendoscopic instruments and techniques, while the VR simulator is superior for learning the three-dimensional anatomy of the intraventricular workspace. Future studies should compare the impact of each simulator on surgical skill acquisition, maintenance, and transfer to the actual surgical environment.

Acknowledgment

Aesculap, Inc., Center Valley, PA, and KARL STORZ GmbH & Co. KG, Tuttlingen, loaned the endoscopy equipment free of charge.

142

CHAPTER 7

References

1. Kirkman MA, Ahmed M, Albert AF, Wilson MH, Nandi D, Sevdalis N. The use of simulation in neurosurgical education and training. J Neurosurg. 2014;121:228-246. doi:10.3171/2014.5.JNS131766.

2. Haase J, Boisen E. Neurosurgical training: more hours needed or a new learning culture? Surg Neurol. 2009;72:89-95; discussion 95-97. doi:10.1016/j.surneu.2009.02.005.

3. Breimer GE, Bodani V, Looi T, Drake JM. Design and evaluation of a new synthetic brain simulator for endoscopic third ventriculostomy. J Neurosurg Pediatr. 2015;15:82-88. doi:10.3171/2014.9.PEDS1447.

4. Hooten KG, Lister JR, Lombard G, et al. Mixed Reality Ventriculostomy Simulation: Experience in Neurosurgical Residency. Neurosurgery. 2014;10:576-581. doi:10.1227/NEU.0000000000000503.

5. Cohen AR, Lohani S, Manjila S, Natsupakpong S, Brown N, Cavusoglu MC. Virtual reality simulation: basic concepts and use in endoscopic neurosurgery training. Childs Nerv Syst. 2013;29:1235-1244. doi:10.1007/s00381-013-2139-z.

6. Delorme S, Laroche D, DiRaddo R, Del Maestro RF. NeuroTouch: a physics-based virtual simulator for cranial microneurosurgery training. Neurosurgery. 2012;71:32-42. doi:10.1227/NEU.0b013e318249c744.

7. Filho FVG, Coelho G, Cavalheiro S, Lyra M, Zymberg ST. Quality assessment of a new surgical simulator for neuroendoscopic training. Neurosurg Focus. 2011;30:E17. doi:10.3171/2011.2.FOCUS10321.

8. Breimer GE, Haji FA, Hoving EW, Drake JM. Development and content validation of performance assessments for endoscopic third ventriculostomy. Child’s Nerv Syst. 2015;31:1247-1259. doi:10.1007/s00381-015-2716-4.

9. Breimer GE, Haji FA, Cinalli G, Hoving EW, Drake JM. Validity evidence for the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). Neurosurgery. In press.

10. Agrawal A, Kato Y, Sano H, Kanno T. The incorporation of neuroendoscopy in neurosurgical training programs. World Neurosurg. 2013;79:S15.e11-e13. doi:10.1016/j.wneu.2012.02.028.

11. Ganju A, Aoun SG, Daou MR, et al. The Role of Simulation in Neurosurgical Education: A Survey of 99 United States Neurosurgery Program Directors. World Neurosurg. 2013;80:e1-e8. doi:10.1016/j.wneu.2012.11.066.

12. Kockro RA. Neurosurgery Simulators—Beyond the Experiment. World Neurosurg. 2013;80:e101-e102. doi:10.1016/j.wneu.2013.02.017.

13. Haji FA, Hoppe DJ, Morin M-P, et al. What we call what we do affects how we do it: a new nomenclature for simulation research in medical education. Adv Heal Sci Educ. 2014;19:273-280. doi:10.1007/s10459-013-9452-x.

14. Haji FA, Da Silva C, Daigle DT, Dubrowski A. From bricks to buildings: adapting the Medical Research Council framework to develop programs of research in simulation education and training for the health professions. Simul Healthc. 2014;9:249-259. doi:10.1097/SIH.0000000000000039.

15. Haji FA, Dubrowski A, Drake J, de Ribaupierre S. Needs assessment for simulation training in neuroendoscopy: a Canadian national survey. J Neurosurg. 2013;118:250-257. doi:10.3171/2012.10.JNS12767.

16. Scott DJ, Cendan JC, Pugh CM, Minter RM, Dunnington GL, Kozar RA. The changing face of surgical education: simulation as the new paradigm. J Surg Res. 2008;147:189-193. doi:10.1016/j.jss.2008.02.014.

17. Neubauer A, Brooks R, Brouwer I, Debergue P, Laroche D. Haptic collision handling for simulation of transnasal surgery. Comput Animat Virtual Worlds. 2013;24:127-141. doi:10.1002/cav.1489.

143


18. Scott DJ, Bergen PC, Rege R V, et al. Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg. 2000;191:272-283. doi:10.1016/S1072-7515(00)00339-2.

19. Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg. 1999;177:28-32. doi:10.1016/S0002-9610(98)00289-X.

20. Yudkowsky R, Luciano C, Banerjee P, et al. Practice on an augmented reality/haptic simulator and library of virtual brains improves residents’ ability to perform a ventriculostomy. Simul Healthc. 2013;8:25-31. doi:10.1097/SIH.0b013e3182662c69.

21. Van Sickle KR, Ritter EM, Smith CD. The pretrained novice: using simulation-based training to improve learning in the operating room. Surg Innov. 2006;13:198-204. doi:10.1177/1553350606293370.

22. Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: an ethical imperative. Acad Med. 2003;78:783-788. doi:10.1097/01.SIH.0000242724.08501.63.

23. Gaab MR. Neuroendoscopic training by low-priced universal serial bus endoscopes. World Neurosurg. 2013;79:255-257. doi:10.1016/j.wneu.2012.10.027.

24. Selden NR, Origitano TC, Hadjipanayis C, Byrne R. Model-based simulation for early neurosurgical learners. Neurosurgery. 2013;73:15-24. doi:10.1227/NEU.0000000000000058.

25. Lobel D, Elder J, Schirmer C, Bowyer M, Rezai A. A novel craniotomy simulator provides a validated method to enhance education in the management of traumatic brain injury. Neurosurgery. 2013;73:57-65. doi:10.1227/NEU.0000000000000116.

26. Mattei TA, Frank C, Bailey J, et al. Design of a synthetic simulator for pediatric lumbar spine pathologies. J Neurosurg Pediatr. 2013;12:192-201. doi:10.3171/2013.4.PEDS12540.

27. Harrop J, Lobel DA, Bendok B, Sharan A, Rezai AR. Developing a neurosurgical simulation-based educational curriculum: An overview. Neurosurgery. 2013;73:25-29. doi:10.1227/NEU.0000000000000101.

28. Alaraj A, Luciano CJ, Bailey DP, et al. Virtual reality cerebral aneurysm clipping simulation with real-time haptic feedback. Neurosurgery. 2015;11:52-58. doi:10.1227/NEU.0000000000000583.

29. Haji FA, Clarke DB, Matte MC, et al. Teaching for the Transition: the Canadian PGY-1 Neurosurgery “Rookie Camp.” Can J Neurol Sci / J Can des Sci Neurol. 2015;42:25-33. doi:10.1017/cjn.2014.124.

30. Mishra S, Kurien A, Ganpule A, Muthu V, Sabnis R, Desai M. Percutaneous renal access training: Content validation comparison between a live porcine and a virtual reality (VR) simulation model. BJU Int. 2010;106:1753-1756. doi:10.1111/j.1464-410X.2010.09753.x.

31. Madan AK, Frantzides CT, Tebbit C, Quiros RM. Participants’ opinions of laparoscopic training devices after a basic laparoscopic training course. Am J Surg. 2005;189:758-761. doi:10.1016/j.amjsurg.2005.03.022.

32. Chen SJ-S, Hellier P, Marchal M, et al. An anthropomorphic polyvinyl alcohol brain phantom based on Colin27 for use in multimodal imaging. Med Phys. 2012;39:554. doi:10.1118/1.3673069.

33. Arora A, Swords C, Khemani S, et al. Virtual reality case-specific rehearsal in temporal bone surgery: A preliminary evaluation. Int J Surg. 2014;12:141-145. doi:10.1016/j.ijsu.2013.11.019.

34. Armstrong R, Eagleson R, De Ribaupierre S. Patient-specific pipeline to create virtual endoscopic third ventriculostomy scenarios. Stud Health Technol Inform. 2014;196:14-16. doi:10.3233/978-1-61499-375-9-1.

35. Ryan JR, Chen T, Nakaji P, Frakes DH, Gonzalez LF. Ventriculostomy Simulation Using Patient-specific ventricular anatomy, 3D Printing, and Hydrogel Casting. World Neurosurg. 2015. doi:10.1016/j.wneu.2015.06.016.

144

CHAPTER 7

36. Tai BL, Rooney D, Stephenson F, et al. Development of a 3D-printed external ventricular drain placement simulator: technical note. J Neurosurg. 2015. doi:10.3171/2014.12.JNS141867.

37. Kothari SN, Kaplan BJ, DeMaria EJ, Broderick TJ, Merrell RC. Training in Laparoscopic Suturing Skills Using a New Computer-Based Virtual Reality Simulator (MIST-VR) Provides Results Comparable to Those with an Established Pelvic Trainer System. J Laparoendosc Adv Surg Tech. 2002;12:167-173. doi:10.1089/10926420260188056.

38. Munz Y, Kumar BD, Moorthy K, Bann S, Darzi a. Laparoscopic virtual reality and box trainers: Is one superior to the other? Surg Endosc Other Interv Tech. 2004;18:485-494. doi:10.1007/s00464-003-9043-7.

39. Lehmann KS, Ritz JP, Maass H, et al. A Prospective Randomized Study to Test the Transfer of Basic Psychomotor Skills From Virtual Reality to Physical Reality in a Comparable Training Setting. Ann Surg. 2005;241:442-449. doi:10.1097/01.sla.0000154552.89886.91.

40. Chang KKP, Chung JWY, Wong TKS. Learning intravenous cannulation: A comparison of the conventional method and the CathSim Intravenous Training System. J Clin Nurs. 2002;11:73-78. doi:10.1046/j.1365-2702.2002.00561.x.

145


Chapter 8

Summary and General Discussion

148

Chapter 8

149

Summary and General discussion

Summary

We have presented a series of studies on various aspects of ETV. We started with patient selection for ETV in chapter 2 where we saw that the ETVSS is a valid tool for this purpose.1 In chapter 3 we concluded that for patients in which the initial ETV fails, it might be worthwhile to perform a repeat ETV and for patient selection the ETVSS could once again be used with additional consideration of the findings during the first endoscopic operation.2

We developed a series of assessment instruments for the assessment of technical competence of a surgeon performing an ETV. Together, the series of instruments are called the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). In chapter 4 we describe the development and content validation of the NEVAT.6 A group of international experts on ETV achieved consensus on the content of the assessment instrument over the course of three consecutive rounds using the modified Delphi method. We followed this up with a study providing validity evidence for the NEVAT as was described in chapter 5, we compared the scores of three groups with different experience levels and evaluated inter-rater reliability.7 In chapter 6 we describe the process of developing a synthetic brain simulator for tackling the problem of learning the technically demanding ETV procedure, the simulator which was well received by a group of neurosurgical resident and fellows and neurosurgeons.4 In chapter 7 we compare the synthetic simulator with a virtual reality simulator and saw that both platforms have their advantages and disadvantages; whereas the physical simulator may be more appropriate for developing familiarity with the steps of the procedure and the neuroendoscopic equipment, the virtual reality simulator may be more appropriate for learning the ventricular anatomy and landmarks for the procedure.5 Together, the physical simulator and the NEVAT may prove useful in developing and evaluating technical competency before operating on real patients. Patient selection for ETV

The patient selection for ETV and for repeat ETV remains an issue. Kulkarni et al. developed the ETV success score (ETVSS) for initial ETV,8 and external validity evidence was provided in chapter 2.1 However, the ETVSS consists of three scores which have to be added for a total score, one of the pillars is the

150

Chapter 8

etiology and in etiology there is an ‘other’ category; there may be a problem with this, as more and more patients get selected for ETV. The ETVSS was developed with rather strict indications for ETV, whereas in the present, a broader spectrum of etiologies may be considered as candidates for ETV (e.g. Arnold–Chiari malformation).9 This may compromise the reliability of the ETVSS when used for the ‘other’ category. We did not encounter this, an explanation for this may be that we presented a retrospective cohort study for which most procedures overlapped the period used for development of ETVSS (majority of cases between 1995 and 2006 for Kulkarni et al., and between 1998 and 2007 for our study). Patient selection for re-ETV

For selection of patient for re-ETV, it may be beneficial to also take the operative findings of the first operation into account.10 For example, during the development phase of an international survey study on practice patterns after initial ETV fails (unpublished work) prof. Schroeder mentioned: “If there were no prepontine membranes in prior surgery you may expect a different outcome than when there are massive arachnoid adhesions after hemorrhage or infection. In some cases, prepontine membranes are the cause of ETV failure, after opening these the re-ETV will probably be successful.” The neurosurgeon can gather extra information from MRI, a re-ETV may be favorable if the MRI shows occlusion of the ETV and signs of obstructive hydrocephalus (i.e. CSF pathway obstruction within the ventricles or prepontine subarachnoid space).11 Siomin et al. identified two groups of patient who will likely benefit from re-ETV:12

1. patients with successful initial ETV, the absorption system is likely to be functional and as such restoring circulation of CSF might alleviate the symptoms. 2. patients with unsuccessful initial ETV, but only in case of technically suboptimal procedure or if the stomy site might be blocked (e.g. by debris). Choroid plexus cauterization with ETV

The first experience with choroid plexus cauterization (CPC) was described by Dandy, but the initial results were poor.13 In the following decades there

151


was sparse interest in the procedure. Some surgeons reported on their series of patients treated with choroid plexus cauterization but it did not become the standard of care. The comeback of the procedure was achieved through Benjamin Warfs’ efforts in Africa.14,15 He treated hydrocephalic infants under 1 year of age with either ETV alone or combined ETV with CPC and found that ETV-CPC was more successful than ETV alone. More recently, results of ETV-CPC treatment of hydrocephalic children in United States of America were published which show comparable results as were reported in the African series by Warf.16 Furthermore, the results of the Hydrocephalus Research Group show that ETV-CPC is a safe procedure in selected patients.17 However, not everybody is as enthusiastic about the introduction of ETV-CPC, even though some positive results have been reported in Africa, they have not sufficiently been validated in a Western world patient population.18 Another issue is that the CPC part of the procedure is executed using a flexible endoscope which is a technically demanding tool and is presently not being taught in most neurosurgery programs although this is an issue relatively easily overcome.19

Even though a subset of patients fare well with ETV-CPC, still approximately half of the youngest children (younger than 6 months) will need subsequent treatment whereas series using a shunt show that about 65% of these patients are successful.20

Follow-up after surgery

There is no consensus on follow-up after ETV, for example on the duration of the monitoring period after treatment, or on the need to use medical imaging modalities after treatment (and if so, which modalities?).21 There are differing opinions on this topic, some propose that dedicated MRI sequences should be performed after every ETV before discharge to verify patency of the stoma and to have adequate imaging of the outline of the ventricles,22 others write that follow-up MRI’s should be performed over the course of years in a select group of patients because it is shown that even after 8 years an ETV can occlude and late rapid deterioration could occur.23 Presence of a flow-void on MRI imaging seems to correlate with clinical success.24 During writing process of the Dutch

152

Chapter 8

experience with ETV and repeat ETV it became clear that some surgeons believe that if a flow-void is shown on MRI, follow-up MRI’s are not necessary and further follow-up appointments can be scheduled at the neurologist or a pediatrician. Essential information concerning follow-up should be provided to the patients and caregivers, they should know that symptoms of increased intracranial pressure may recur, e.g. due to occlusion of stomy opening. In that case they should be advised to contact their physician.Unfortunately, definite outcome parameters to differentiate successful from unsuccessful cases are lacking. In comparison with patients with VP-shunts, the change in size of the ventricles appears to be much subtler in ETV patient.24,25 The decision for intracranial pressure measurement by means of a lumbar puncture, might be influenced by the uncertainty about the patency of the ventriculostomy (may be due to inadequate fenestration, or additional arachnoid membranes). Finally, it remains very difficult to evaluate the sufficiency of improvement of CSF circulation in ETV patients; are these patient optimally treated or are they being undertreated by ETV? This matter can only be judged in long-term outcome measurements which are currently lacking, although the International Infant Hydrocephalus Study does focus on this domain but the definite results are still pending.26 Furthermore, some patients may be better left without subsequent treatment, differentiation between (slowly) progressive and compensated hydrocephalus seems possible.27 This is a major problem in the treatment of hydrocephalus, since arrested hydrocephalus might be a suboptimal result of long lasting slowly progressive hydrocephalus. If these patients were treated earlier, they might have ended with a better performance. The question remains what would have been the optimal outcome of treated hydrocephalus in an individual case?Additional cost may be another side-effect of diagnostics: financial cost due to use of facilities (e.g. MRI machine, hospital bed) and medical manpower (e.g. neurosurgeon, radiologist, anesthesiologist, nurses), but also psychosocial cost as there may be stress about outcome, the child may have to miss school, parents may have to take time off work, also, the patient and/or parents are kept in their patient role, which might affect their lives.

153


Pathophysiology

Understanding of pathophysiology of hydrocephalus is still marginal and more basic research is needed on this subject in order to find clarity and possibly other treatment options for this disease. Thus far there are multiple models in existence for explaining the mechanisms of hydrocephalus but none seem to explain the phenomenon completely.28 For example, it has been postulated that hydrocephalus is a disorder of macromolecular clearance, rather than a disorder of circulation.29 This theory suggests that excess of macromolecules in the CSF result in an osmotic gradient which attracts more fluid into the ventricles and results in hydrocephalus. The macromolecules may normally be cleared from the CSF via paravascular or lymphatic pathways. Recently, a lymphatic system has been discovered in the central nervous system,30 but the details of lymphatic drainage in the central nervous system remain unclear.31 Also, the role of cilia and of arachnoid granulations in hydrocephalus is obscure. During fetal development, the ventricular lining differentiates into ependymal cells which are ciliated.32 It may be postulated that in an immature brain the cilia are not fully developed, which in turn may negatively affect CSF flow and thus may increase changes of ETV failure in young infants. The involvement of ciliary dysfunction (e.g. due to primary ciliary dyskinesia) in some forms of hydrocephalus has recently been picked up,33–35 although the idea of an association between ciliary motility dysfunction and hydrocephalus is not new.36,37

In adults, the arachnoid granulations play an important role in the absorption of CSF from subarachnoidal space into the circulation but in the youngest category of patients these arachnoid granulations are not fully formed and alternative pathways of CSF absorptions may play an important role.38,39 Examples of alternative routes of CSF reuptake routes are transependymal intraparenchymal CSF pathway and absorption via interstitial and perivascular space, and perineural lymphatic channels.40 The lack of development of arachnoid granulations in infant may contribute to high ETV failure rate in young children, although in non-hydrocephalic children the alternative routes are clearly sufficient to prevent accumulation of CSF which makes unripe arachnoid granulations a less plausible candidate for causing ETV failure.

154

Chapter 8

One hypothesis is that the combination of unripe arachnoid granulations and incomplete intraventricular cilia-system in an already fragile CSF equilibrium in a vulnerable infant makes the alternative pathways insufficient in some patients and thus contribute to the failure rates. As Drake said it: “Perhaps the fundamental lesson from the last 30 years is that hydrocephalus, which appears to be an alluringly simple problem of CSF accumulation, is anything but.”28

Hydrocephalus may be a result of a variety of causes at a certain moment of time in development. This requires a thorough analysis of every single patient, and the definition of an appropriate treatment algorithm. Every endoscopic procedure provides a better understanding of the nature of the hydrocephalus in this particular patient. Evaluation of the procedure is crucial to judge whether the treatment was sufficient and to better understand the nature of the hydrocephalus itself. Simulation-based education

If one is able to define a clear definition of a standardized technique together with a list of consecutive required steps of performance concerning the ETV, one might be more certain that the ETV has been executed appropriately. This is an essential aspect in outcome analysis and it contributes to clear definition of indications for ETV treatment. Simulation-based education provides unique tools for acquiring skills in a standardized fashion which makes it very suitable for ETV. The essential elements of the procedure can be defined by a group of experienced neurosurgeons. This adds complete values to this learning tool. A second aspect is a suitable training model for optimizing eye-hand coordination required for any endoscopic procedure.Before deciding to develop a new simulator (same applies to developing new assessment instruments),41 it is necessary to first conduct a systematic literature review of existing simulators (especially simulators that simulate the intended procedure). One can get ideas for what is missing in existing simulators, how to manufacture the simulator, etc. This may prevent much additional work. Also before setting out to develop a new simulator, it is necessary to first conduct a needs assessment study to evaluate what educators and trainees think is most crucial to incorporate into a simulator.42

The development of the different brain simulators, virtual reality and physical

155


models alike, is good and helpful. In chapter 7 we compared the Neurotouch simulator (virtual reality) with the SickKids brain simulator (physical). It did not come as a surprise to find that both platforms have their own advantages and disadvantages. For mastering the equipment, the physical model was more suitable (as real neuroendoscopic instruments were used) and for learning intraventricular anatomy, the Neurotouch system was more appropriate. Although we only compared one option of the virtual reality group and one of the physical simulator group, the results and conclusions may still be applicable to others as the general features over the various virtual reality and physical simulators will not differ all too much. The key function of physical simulator is to familiarize oneself with the equipment and the steps of the procedure. Virtual reality simulation

There is much work to be done to improve virtual reality simulators for neurosurgical procedures. Significant improvements are necessary on haptic feedback and also on graphics and use of instruments that look realistic (to make the experience as close as possible to the OR experience). The prime objective of simulation-based education is to get trainees further on their learning curve, i.e. to achieve reasonable technical competence before operating on real patients. There are other goals and purposes, such as training co-operation in an operating room setting between the members of the operating team (e.g. surgeon, resident, anesthesiologist, nurses and assistants). Patient-specific simulation

There is some interest in case rehearsal by means of patient specific simulation modules. This seems like a great idea, and might even be helpful for surgeons in choosing approach and technique. The rehearsal of specific cases can also give a false sense of security because it is impossible to predict and mimic all features of the patient exactly. An example on intracranial aneurysm simulation by Rothstein and Selman illustrates this:43 It is not realistic to expect a one-to-one copy of the patient even if it was possible to upload patient specific data into a simulation platform (which will be possible eventually). On the most basic level, imaging of aneurysms, and actual aneurysm anatomy can differ and to predict what would happen to the aneurysm (e.g. deformations or bursting) if one applies pressure

156

Chapter 8

is very hard. At best, you could design an algorithm to create a virtual aneurysm with physical properties of the aneurysm wall strength based on characteristics of averages, if that data is available, or would be gathered. In the words of Rothstein and Selman: “Aneurysms of the same size, configuration, and location can have different susceptibilities of rupture during a procedure.” Nevertheless, simulation, even the patient-specific aneurysm simulators that have been developed are still useful for things like planning trajectory and choosing the optimal clip.44

Fidelity of simulator

In simulators, there are basically two things that seem to matter: physical resemblance (how the simulator appears) and functional task alignment (what the simulator does).45 Although intuitively physical resemblance would appear to be of great importance, research shows that simulators with low physical resemblance still can be highly effective training tools.45,46 Simulators which are not as realistic (i.e. low physical resemblance) can often be produced for less money (easier to create and replace). As long as the functional task alignment is adequate (i.e. the simulator does what it is supposed to do). For each task, different properties may be needed. For example, to train stitching, a simple rubber band may be sufficient. For team training in complex trauma situations, a dynamic simulator (with different scenarios) with high physical resemblance may be more appropriate. Evaluation of technical competency

A trainee could train on a VR simulator and get real-time monitoring via the simulator but ideally there is a proctor present when a resident is training a new skill on a simulator. Giving a resident opportunity to practice on his/her own advantages and disadvantages: it would save time for the consulting surgeons and would reduce fear of failure and thus reluctance to try but on the other hand there is a danger of developing wrong techniques in the absence of an expert teacher when using these real-time feedback or self-scoring systems. Dr. G.M. Fried observed that in absence of a proctor sometimes his residents learn bad habits that give them good scores: “It is a little like a video game; they learn shortcuts. These shortcuts can reinforce bad habits.”47 The ETV procedure is well suited for standardization, in chapter 4 we constructed

157


the NEVAT with the help of a group of international experts on ETV. The expert derived list of consecutive steps of the procedure can be used to acquire skills of this procedure in a very standardized and complete fashion. In the analysis of treatment failure, the cause of failure may be a technically procedural failure. It may be hypothesized that some of the technical failures could be preventable, if all steps that the experts deemed essential to the procedure are taught. This has not been proved. A standardized approach as can be created in a simulation-based environment cannot be offered to the same extent in patient-related acquisition of skills. Patients require individualized treatment and there are frequent exceptions to rules. We realize that all possible scenarios cannot be captured in standardized lists or procedure-specific assessment instruments. In addition to this, standardized forms of evaluation cannot replace constructive comments during an operation or one-to-one feedback on operating skills.48

The way medical education for postgraduate residency rotations is structured presently (by time units such as weeks or months) may be an outdated way of achieving acquisition of skill. In a health care environment in which the education of residents is often secondary priority after patient care and financial incentives it may be appropriate to make some changes to how things are done.49 In all disciplines of medicine we should strive to achieve expert-performance and neurosurgery is no exemption. One factor that may help with achieving this goal is focusing on mastery learning during residency. Simulation-based curriculum

It may be beneficial to incorporate this kind of simulation-based education into the neurosurgery curriculum. Mastery learning can be applied to accomplish this. There are at least seven complementary features to mastery learning: 1 baseline testing; 2 clear learning objectives, sequenced as units ordered by increasing difficulty;3 engagement in educational activities (e.g. deliberate skills practice, data interpretation, reading) that are focused on reaching the objectives;4 the establishment of a minimum passing standard (e.g. test score, checklist score) for each educational unit;5 formative testing to gauge unit completion at a preset minimum passing

158

Chapter 8

mastery standard;6 advancement to the next educational unit given measured achievement at or above the mastery standard, or7 continued practice or study on an educational unit until the mastery standard is reached.50

If simulation-based education is incorporated into the curriculum it would be important to evaluate its impact on the program.51,52 The role of simulation-based education is not clear yet, potentially it will be beneficial but cost-benefit analysis must be used as guidance before broadening the scope of its use. Furthermore, even though simulation-based education provides an excellent means of acquiring and training technical skills, it cannot replace the actual operating room as the core of the neurosurgical training program.

159


References

1. Breimer GE, Sival DA, Brusse-Keizer MGJ, Hoving EW. An external validation of the ETVSS for both short-term and long-term predictive adequacy in 104 pediatric patients. Childs Nerv Syst. 2013;29(8):1305–11. doi:10.1007/s00381-013-2122-8.

2. Breimer GE, Dammers R, Buis DR, Delye H, Woerdeman PA, Hoving EW. Repeat endoscopic third ventriculostomy in pediatric patients: the Dutch experience. Child’s Nerv Syst. 2015.

3. Breimer GE, Warf BC, Cinalli G, et al. Current Practice of Treatment After Failure of Initial Endoscopic Third Ventriculostomy. World Neurosurg. 2016.

4. Breimer GE, Bodani V, Looi T, Drake JM. Design and evaluation of a new synthetic brain simulator for endoscopic third ventriculostomy. J Neurosurg Pediatr. 2015;15(1):82–88. doi:10.3171/2014.9.PEDS1447.

5. Breimer GE, Haji FA, Bodani V, et al. Simulation-based Education for Endoscopic Third Ventriculostomy: A Comparison Between Virtual and Physical Training Models. Oper Neurosurg. 2016:1. doi:10.1227/NEU.0000000000001317.

6. Breimer GE, Haji FA, Hoving EW, Drake JM. Development and content validation of performance assessments for endoscopic third ventriculostomy. Child’s Nerv Syst. 2015;31(8):1247–1259. doi:10.1007/s00381-015-2716-4.

7. Breimer GE, Haji FA, Cinalli G, Hoving EW, Drake JM. Validity Evidence for the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). Oper Neurosurg. 2015:1. doi:10.1227/NEU.0000000000001158.

8. Kulkarni A, Drake JM, Mallucci CL, Sgouros S, Roth J, Constantini S. Endoscopic third ventriculostomy in the treatment of childhood hydrocephalus. J Pediatr. 2009;155(2):254–9.e1. doi:10.1016/j.jpeds.2009.02.048.

9. Nishiyama K, Yoshimura J, Fujii Y. Limitations of Neuroendoscopic Treatment for Pediatric Hydrocephalus and Considerations from Future Perspectives. Neurol Med Chir (Tokyo). 2015;55(8):611–616. doi:10.2176/nmc.ra.2014-0433.

10. Warf BC, Kulkarni A V. Intraoperative assessment of cerebral aqueduct patency and cisternal scarring: impact on success of endoscopic third ventriculostomy in 403 African children. J Neurosurg Pediatr. 2010;5(2):204–209. doi:10.3171/2009.9.PEDS09304.

11. Koch D, Grunert P, Filippi R, Hopf N. Re-ventriculostomy for treatment of obstructive hydrocephalus in cases of stoma dysfunction. Minim Invasive Neurosurg. 2002;45(3):158–63. doi:10.1055/s-2002-34350.

12. Siomin V, Weiner H, Wisoff J, et al. Repeat endoscopic third ventriculostomy: is it worth trying? Child’s Nerv Syst. 2001;17(9):551–555. doi:10.1007/s003810100475.

13. Scarff JE. The treatment of nonobstructive (communicating) hydrocephalus by endoscopic cauterization of the choroid plexuses. J Neurosurg. 1970;33:1–18. doi:10.3171/jns.1970.33.1.0001.

14. Warf BC, Tracy S, Mugamba J. Long-term outcome for endoscopic third ventriculostomy alone or in combination with choroid plexus cauterization for congenital aqueductal stenosis in African infants. J Neurosurg Pediatr. 2012;10(2):108–111. doi:10.3171/2012.4.PEDS1253.

15. Warf BC. Comparison of endoscopic third ventriculostomy alone and combined with choroid plexus cauterization in infants younger than 1 year of age: a prospective study in 550 African children. J Neurosurg. 2005;103(6 Suppl):475–481. doi:10.3171/ped.2005.103.6.0475.

16. Stone SSD, Warf BC. Combined endoscopic third ventriculostomy and choroid plexus cauterization as primary treatment for infant hydrocephalus: a prospective North American series. J Neurosurg Pediatr. 2014;14(5):439–446. doi:10.3171/2014.7.PEDS14152.

160

Chapter 8

17. Kulkarni A V., Riva-Cambrin J, Browd SR, et al. Endoscopic third ventriculostomy and choroid plexus cauterization in infants with hydrocephalus: a retrospective Hydrocephalus Clinical Research Network study. J Neurosurg Pediatr. 2014;14(3):224–229. doi:10.3171/2014.6.PEDS13492.

18. Souweidane MS. Editorial: Combined choroid plexus coagulation and endoscopic third ventriculostomy: is North America ready? J Neurosurg Pediatr. 2014;14(3):221–223. doi:10.3171/2014.4.PEDS1450.

19. Baird LC. First Treatment in Infants With Hydrocephalus: The Case for Endoscopic Third Ventriculostomy/Choroid Plexus Cauterization. Neurosurgery. 2016;63 Suppl 1(1):78–82. doi:10.1227/NEU.0000000000001299.

20. Kulkarni A V. First Treatment in Infants With Hydrocephalus: The Case for Shunt. Neurosurgery. 2016;63 Suppl 1(1):73–7. doi:10.1227/NEU.0000000000001287.

21. Vinchon M, Rekate H, Kulkarni A V. Pediatric hydrocephalus outcomes: a review. Fluids Barriers CNS. 2012;9(1):18. doi:10.1186/2045-8118-9-18.

22. Peretta P, Cinalli G, Spennato P, et al. Long-term results of a second endoscopic third ventriculostomy in children: retrospective analysis of 40 cases. Neurosurgery. 2009;65(3):539–47; discussion 547. doi:10.1227/01.NEU.0000350228.08523.D1.

23. Faggin R, Calderone M, Denaro L, Meneghini L, d’Avella D. Long-term operative failure of endoscopic third ventriculostomy in pediatric patients: the role of cine phase-contrast MR imaging. Neurosurg Focus. 2011;30(4):E1. doi:10.3171/2011.1.FOCUS10303.

24. Kulkarni A V, Drake JM, Armstrong DC, Dirks PB. Imaging correlates of successful endoscopic third ventriculostomy. J Neurosurg. 2000;92(6):915–919. doi:10.3171/jns.2000.92.6.0915.

25. van Lindert EJ, Beems T, Grotenhuis JA. The role of different imaging modalities: is MRI a conditio sine qua non for ETV? Child’s Nerv Syst. 2006;22(12):1529–1536. doi:10.1007/s00381-006-0189-1.

26. Kulkarni A V., Sgouros S, Constantini S. International Infant Hydrocephalus Study: initial results of a prospective, multicenter comparison of endoscopic third ventriculostomy (ETV) and shunt for infant hydrocephalus. Child’s Nerv Syst. 2016;32(6):1039–1048. doi:10.1007/s00381-016-3095-1.

27. Leliefeld PH, Gooskens RHJM, Tulleken CAF, et al. Noninvasive detection of the distinction between progressive and compensated hydrocephalus in infants: is it possible? J Neurosurg Pediatr. 2010;5(6):562–568. doi:10.3171/2010.2.PEDS09309.

28. Drake JM. The surgical management of pediatric hydrocephalus. Neurosurgery. 2008;62 Suppl 2(Supplement 2):633–40; discussion 640–2. doi:10.1227/01.neu.0000316268.05338.5b.

29. Krishnamurthy S, Li J. New concepts in the pathogenesis of hydrocephalus. Transl Pediatr. 2014;3(3):185–194. doi:10.3978/j.issn.2224-4336.2014.07.02.

30. Louveau A, Smirnov I, Keyes TJ, et al. Structural and functional features of central nervous system lymphatic vessels. Nature. 2015;523(7560):337–41. doi:10.1038/nature14432.

31. Raper D, Louveau A, Kipnis J. How Do Meningeal Lymphatic Vessels Drain the CNS? Trends Neurosci. 2016;xx:1–6. doi:10.1016/j.tins.2016.07.001.

32. Gould SJ, Howard S, Papadaki L. The development of ependyma in the human fetal brain: an immunohistological and electron microscopic study. Dev Brain Res. 1990;55(2):255–267. doi:10.1016/0165-3806(90)90207-F.

33. Kahle KT, Kulkarni A V, Limbrick DD, Warf BC. Hydrocephalus in children. Lancet. 2015;55(11):502–507. doi:10.1016/S0140-6736(15)60694-8.

34. Lee L. Riding the wave of ependymal cilia: Genetic susceptibility to hydrocephalus in primary ciliary dyskinesia. J Neurosci Res. 2013;91(9):1117–1132. doi:10.1002/jnr.23238.

161


35. Banizs B, Pike MM, Millican CL, et al. Dysfunctional cilia lead to altered ependyma and choroid plexus function, and result in the formation of hydrocephalus. Development. 2005;132(23):5329–39. doi:10.1242/dev.02153.

36. Greenstone MA, Jones RW, Dewar A, Neville BG, Cole PJ. Hydrocephalus and primary ciliary dyskinesia. Arch Dis Child. 1984;59(5):481–2. Available at: http://www.ncbi.nlm.nih.gov/pubmed/23686703.

37. De Santi MM, Magni A, Valletta EA, Gardi C, Lungarella G. Hydrocephalus, bronchiectasis, and ciliary aplasia. Arch Dis Child. 1990;65(5):543–4. Available at: http://www.ncbi.nlm.nih.gov/pubmed/2357097.

38. Squier W, Lindberg E, MacK J, Darby S. Demonstration of fluid channels in human dura and their relationship to age and intradural bleeding. Child’s Nerv Syst. 2009;25(8):925–931. doi:10.1007/s00381-009-0888-5.

39. Mack J, Squier W, Eastman JT. Anatomy and development of the meninges: implications for subdural collections and CSF circulation. Pediatr Radiol. 2009;39(3):200–210. doi:10.1007/s00247-008-1084-6.

40. Oi S, Di Rocco C. Proposal of “evolution theory in cerebrospinal fluid dynamics” and minor pathway hydrocephalus in developing immature brain. Childs Nerv Syst. 2006;22(7):662–9. doi:10.1007/s00381-005-0020-4.

41. de Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in Medicine. 1st ed. New York: Cambridge University Press; 2011.

42. Haji FA, Dubrowski A, Drake J, de Ribaupierre S. Needs assessment for simulation training in neuroendoscopy: a Canadian national survey. J Neurosurg. 2013;118(2):250–7. doi:10.3171/2012.10.JNS12767.

43. Rothstein BD, Selman WR. Evaluating Simulation as a Teaching Tool in Neurosurgery. Virtual Mentor. 2015;17(1):33–36. doi:10.1001/virtualmentor.2015.17.01.medu1-1501.

44. Alaraj A, Luciano CJ, Bailey DP, et al. Virtual reality cerebral aneurysm clipping simulation with real-time haptic feedback. Neurosurgery. 2015;11 Suppl 2(1):52–8. doi:10.1227/NEU.0000000000000583.

45. Hamstra SJ, Brydges R, Hatala R, Zendejas B, Cook DA. Reconsidering Fidelity in Simulation-Based Training. Acad Med. 2014;89(3):387–392. doi:10.1097/ACM.0000000000000130.

46. Norman G, Dore K, Grierson L. The minimal relationship between simulation fidelity and transfer of learning. Med Educ. 2012;46(7):636–647. doi:10.1111/j.1365-2923.2012.04243.x.

47. Fried GM, Feldman LS, Vassiliou MC, et al. Proving the value of simulation in laparoscopic surgery. Ann Surg. 2004;240(3):518–525; discussion 525–528. doi:10.1097/01.sla.0000136941.46529.56.

48. Kestle JRW. Editorial: Measuring resident operative skills. J Neurosurg Pediatr. 2015:1–2. doi:10.3171/2015.2.PEDS1542.

49. McGaghie WC. Mastery Learning: It Is Time for Medical Education to Join the 21st Century. Acad Med. 2015;90(11):1438–1441. doi:10.1097/ACM.0000000000000911.

50. McGaghie WC. When I say … mastery learning. Med Educ. 2015;49(6):558–559. doi:10.1111/medu.12679.

51. Haji FA, Da Silva C, Daigle DT, Dubrowski A. From bricks to buildings: adapting the Medical Research Council framework to develop programs of research in simulation education and training for the health professions. Simul Healthc. 2014;9(4):249–59. doi:10.1097/SIH.0000000000000039.

52. Aronson L. The value of medical education programmes: what are the right questions? Med Educ. 2013;47(4):333–4. doi:10.1111/medu.12137.

Chapter 9

Future Directions

164

CHAPTER 9

165

Future directions

Clinical research

One of the main limitations in the care of hydrocephalus, is the lack of prospective randomized trials.1 One of the problems in setting up a randomized controlled trial is the process randomization; parents prefer endoscopic treatment in most cases. There are few prospective studies that have randomized patients, e.g. the shunt-design trial or the international infant hydrocephalus study.2,3 In a small prospective study comparing neuroendoscopic treatment of hydrocephalus with CSF shunt treatment shows better outcome for neuroendoscopic treatment, but the study’s design is poor.4 However, there is a larger study currently in progress on this same subject: the International Infant Hydrocephalus Study comparing cerebrospinal fluid diversion shunt with ETV in hydrocephalic children with aquaductal stenosis.3,5,6 We have shown that for patients with failure of initial ETV the ETVSS can be used for patient selection. The ETVSS may be further improved for patient selection of re-ETV by taking findings of previous endoscopic operation into account (severe adhesions may reduce chance of success of re-ETV). This could be assessed in a prospective study. We suggest for researchers contemplating conducting a prospective study on re-ETV to take the prior operative findings and the operative findings at surgery into account. It may be helpful to categorize the stoma closure patterns into: 1) ventriculostoma closed by scarring or gliosis; 2) ventriculostoma clearly narrower than at the end of the first ETV; 3) ventriculostoma patent, but with newly formed arachnoid membranes (not found during the first ETV) below the floor of the third ventricle in the basal cisterns.7 Patients with closure of ventriculostoma with gliosis or scar tissue may have lower chances of re-ETV success than patients with narrowing (or closure) of ventriculostoma by translucent membranes.8 The MR imaging findings, and decision analysis following these findings is a subject of interest as well: what to do with specific patients such as an asymptomatic patient with a true disappearance of previously present flow-void (flow void reverting to no flow-void) and the ventricles increase in size, and what if the ventricles do not increase in size? This is an interesting field of research as it is still not certain which treatment modality is superior (shunt or ETV); an intuitive next step would be to compare

166

CHAPTER 9

success rates of different treatments after the initial ETV fails, treat one group with re-ETV and the other group with a CSF shunt.We encountered wide varying practice on how neurosurgeons treated patients in which initial ETV failed in chapter 3, this prompted a survey study with international pediatric neurosurgeons as target population (International Society for Pediatric Neurosurgery) in which we intent to reveal the practice patterns, and subsequently to review the existing literature, and make suggestions for practice guidelines based on current data. Another important focus is age of the patient, as it seems that the majority of cases with secondary obstruction of the stoma are younger than 2 years.9,10 What pathophysiological concepts explain this phenomenon? The impact of pressure profile of a CSF shunt on cognitive development might be interesting; assess neurocognitive functioning and quality of life of low-pressure profile versus medium-pressure profile in the youngest category of hydrocephalic children. The outcome should then be assessed using reliable and validated outcome measures such as the Hydrocephalus Outcome Questionnaire.11 Simulation-based education

In chapter 8 we compare the opinion of residents and fellows on a physical simulator and a VR simulator. As a follow-up study we propose to compare training efficacy between simulators. It is necessary to conduct a study in which participants get trained on either simulator and evaluate the technical competence of trainees, and thus evaluate the efficacy of the simulators. We intent to start field-testing of the NEVAT, using it for evaluating residents, fellows and neurosurgeons in the operating room. Ideally conducting an additional study videotaping participants and have two raters evaluate each participant (inter-rater reliability), twice by the same rater on two separate occasions (test-retest reliability). An addition could be to evaluate the NEVAT scores of participants over multiple procedures on a simulator to see if we can track a learning curve; when does the participant reach a plateau? What is the

167

Future directions

plateau, and should all trainees reach this prior to commencing to operating on real patients? In the future simulation-based education might be incorporated into the Dutch neurosurgery curriculum. It may be a wise start to make an inventory of opinions of stakeholders (e.g. program directors, residents and consulting neurosurgeons) on organizing bootcamps for Dutch neurosurgery residents. It would be good to centralize courses (or bootcamps) to increase number of participants, making it more feasible and have more impact. Before starting to use standardized assessment instruments such as the NEVAT we should assess the feasibility of incorporating standardized evaluation of skills in the curriculum; are the stakeholders (i.e. program leaders, consulting neurosurgeons and residents) willing to use such assessment measures? Standardized objective assessment instruments such as the NEVAT or OSATS may be used as a basis for individualized feedback after operation to improve performance: which aspects are well developed, and which aspects could be focus for further training? This is not intended to replace constructive feedback during operations but to enhance one-on-one feedback regarding operative skills and to help measure performance over time.12 If such standardized evaluations will get incorporated, the effect of it should be analyzed after a period of using to assess what stakeholders think of it. Another next step in simulation-based education would be to create physical simulators for other procedures. Again, preferably first conduct a needs assessment to give an idea of what is important to simulate. We have created prototypes for simulation-based training of endoscopic colloid cyst removal and for choroid plexus coagulation, at this stage used for local courses at University of Toronto only. Performance assessments similar to the NEVAT can be developed and validated for these procedures to facilitate that, in the future, these procedures may be incorporated into a curriculum.

168

CHAPTER 9

References

1. Vinchon M, Rekate H, Kulkarni A V. Pediatric hydrocephalus outcomes: a review. Fluids Barriers CNS. 2012;9(1):18. doi:10.1186/2045-8118-9-18.

2. Kestle J, Drake J, Milner R, et al. Long-term follow-up data from the shunt design trial. Pediatr Neurosurg. 2000;33(5):230–236. doi:10.1159/000055960.

3. Kulkarni A V., Sgouros S, Constantini S. International Infant Hydrocephalus Study: initial results of a prospective, multicenter comparison of endoscopic third ventriculostomy (ETV) and shunt for infant hydrocephalus. Child’s Nerv Syst. 2016;32(6):1039–1048. doi:10.1007/s00381-016-3095-1.

4. Kamikawa S, Inui A, Kobayashi N, et al. Endoscopic treatment of hydrocephalus in children: a controlled study using newly developed Yamadori-type ventriculoscopes. Minim Invasive Neurosurg. 2001;44(1):25–30. doi:10.1055/s-2001-13587.

5. Sgouros S, Kulkharni A V, Constantini S. The International Infant Hydrocephalus Study: concept and rational. Childs Nerv Syst. 2006;22(4):338–45. doi:10.1007/s00381-005-1253-y.

6. Kahle KT, Kulkarni A V, Limbrick DD, Warf BC. Hydrocephalus in children. Lancet. 2015;55(11):502–507. doi:10.1016/S0140-6736(15)60694-8.

7. Wagner W, Koch D. Mechanisms of failure after endoscopic third ventriculostomy in young infants. J Neurosurg. 2005;103(1 Suppl):43–49. doi:10.3171/ped.2005.103.1.0043.

8. Etus V, Karabagli H, Geyik M, Celakil U, Gokbel A. Success of Repeat ETV Procedure According to Ventriculostoma Reclosure Pattern in Children. In: 43rd Annual Meeting of International Society for Pediatric Neurosurgery.; 2015.

9. Sacko O, Boetto S, Lauwers-Cances V, Dupuy M, Roux F-E. Endoscopic third ventriculostomy: outcome analysis in 368 procedures. J Neurosurg Pediatr. 2010;5(1):68–74. doi:10.3171/2009.8.PEDS08108.

10. Javadpour M, Mallucci C, Brodbelt A, Golash A, May P. The impact of endoscopic third ventriculostomy on the management of newly diagnosed hydrocephalus in infants. Pediatr Neurosurg. 2001;35:131–135. doi:10.1159/000050406.

11. Kulkarni A V., Shams I, Cochrane DD, McNeely PD. Quality of life after endoscopic third ventriculostomy and cerebrospinal fluid shunting: an adjusted multivariable analysis in a large cohort. J Neurosurg Pediatr. 2010;6:11–16. doi:10.3171/2010.3.PEDS09358.

12. Kestle JRW. Editorial: Measuring resident operative skills. J Neurosurg Pediatr. 2015:1–2. doi:10.3171/2015.2.PEDS1542.

169

Future directions

Chapter 10

Conclusions

172

CHAPTER 10

173

Conclusions

Conclusions

This thesis covered multiple aspects of patient selection for endoscopic third ventriculostomy (ETV) and training and assessment of ETV procedure. We found that in selected patients, an ETV can relieve a patient from hydrocephalus. The ETV success score (ETVSS) is a tool that can be used for patient selection for ETV and we have provided external validity evidence for the reliability and predictive accuracy of this score. Even though many rightly selected patients can benefit from an ETV, some patient will develop recurrent symptoms of hydrocephalus. In these patients the ETV has failed. The failure of ETV can occur via various mechanisms (e.g. closure of stomy site or formation of subarachnoid membranes). Once again, in rightly selected patients, after failure of an ETV, some patients can be helped with a repeat ETV (re-ETV). We found similar success rates for initial ETV and re-ETV procedures. The ETV procedure is technically demanding, endoscopic instruments is used and the neurosurgeons uses different hand-eye coordination skills than are required for open procedures. We have created a realistic, low-cost, silicone-based ETV simulator to teach neurosurgical trainees. This brain simulator may help increase familiarity with the camera skills, endoscopic instruments, and hand-eye coordination required to successfully perform an ETV. In a head-to-head comparison of a physical simulator with a virtual reality simulator we found that the two simulators each have their own advantages and disadvantages. The physical simulator can best be used for familiarizing the trainee with neuroendoscopic instruments and focus on developing manual dexterity and technical skills. The VR simulator can best be used for learning anatomy and decision-making for anatomical cues. We additionally developed a series of ETV assessment instruments; the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). Initially we created content for the NEVAT, together with a group of international experts on ETV and we subsequently provided validity evidence for the NEVAT. For validity evidence on internal structure we calculated interrater reliability and internal consistency of raters’ scores. Evidence of relationships with other variables was collected by comparing the ETV performance of experts, experienced trainees, and novices. Our results support the use of the NEVAT as a standardized method for evaluating neuroendoscopic competence in a simulation-based setting.

Chapter 11

Appendix

Appendix 1. Nederlandse samenvatting

We hebben een zestal studies gepresenteerd waarin verschillende aspecten van derde ventriculocisternostomie (3VC) werden belicht. Deze procedure houdt in dat een operateur middels endoscopische benadering een perforatie creëert in de bodem van de derde ventrikel in het brein. 3VC is een effectieve maar technisch complexe procedure met significante risico’s. Daarom is het belangrijk dat de juiste patiënten worden geselecteerd voor behandeling en dat de chirurg de techniek goed in de vingers heeft. De focus van ons onderzoek ligt op deze twee punten. In hoofdstuk 2 hebben we het in Toronto ontwikkelde score systeem geanalyseerd waarmee een voorspelling kan worden gedaan van de slagingskans van 3VC over een periode van zes maanden – de endoscopic third ventriculostomy success score (ETVSS). We hebben de ETVSS op een Nederlandse patiëntengroep toegepast en gevalideerd en aangetoond dat deze betrouwbaar de kans van slagen van de 3VC voorspelt. De ETVSS kan dus goed worden gebruikt voor patiënten selectie, hierbij is een patiënt met een hoge score een geschikte kandidaat voor 3VC. Bij sommige patiënten komen de klachten van hydrocephalus weer terug, in hoofdstuk 3 vonden we dat bij deze groep wederom de ETVSS kan worden toegepast voor het voorspellen van succeskans. Hierbij zijn enkele kanttekeningen, namelijk dat de slagingskans negatief wordt beïnvloed door aanwezigheid van subarachnoidale membranen in de pre-pontiene ruimte (geobserveerd tijdens de tweede 3VC) en eveneens dat de slagingskans negatief wordt beïnvloed door het postoperatief gebruik van een externe ventrikel drain. In hoofdstuk 4, hebben we een gestandaardiseerd beoordelingsinstrument ontwikkeld voor het evalueren van techniek van chirurgen die een 3VC procedure uitvoeren. Dit instrument bestaat uit een drietal checklists die samen of los van elkaar kunnen worden gebruikt: één is een procedure-specifieke checklist waarin de verschillende stappen van 3VC zijn opgenomen en kunnen worden gescoord, de tweede checklist bestaat uit de mogelijke fouten die kunnen worden gemaakt en de derde is een global rating scale waarmee meer algemene items zoals flow van operatie of samenwerking met assistenten kunnen worden gescoord. Samen heten deze lijsten de Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). In hoofdstuk 5 beschrijven we de validatie van de NEVAT in simulatie-setting.

178

CHAPTER 11

Hierbij hebben we onder andere vergeleken in hoeverre de scores van twee beoordeelaars overeenkwamen als zij één iemand beoordeelden met de NEVAT. De lijsten blijken in de simulatie-setting te kunnen worden gebruikt om beginners, van participanten met geringe ervaring en experts te onderscheiden. De endoscopische behandeling vereist andere hand-oog coördinatie dan open chirurgie. Het leren van 3VC vergt dan ook oefening. Om een risico-vrije trainingsomgeving te creëren hebben we met 3d-printtechtniek een herbruikbaar siliconen namaakbrein gemaakt. De ontwikkeling van dit namaakbrein is beschreven in hoofdstuk 6. In hoofdstuk 7 vergeleken we het synthetische namaakbrein met een virtual reality namaakbrein. Hier werd gevonden dat beide systemen hun eigen voordelen en nadelen hebben: waar het synthetische model beter kan worden gebruikt voor het leren omgaan met de apparatuur en oefenen van hand-oog coördinatie bij endoscopie kan het virtual reality model beter worden gebruikt voor het eigen maken van intra-ventriculaire anatomie en anatomische herkenningspunten.

179

Appendix

Appendix 2. Lay Summary & samenvatting voor leken

Lay Summary

One of the treatment options for hydrocephalus is using an endoscope to puncture the floor of the third ventricle in the brain. This procedure is called an endoscopic third ventriculostomy (ETV). It is an effective but technically demanding procedure with significant risks. It is important to select the right patient for this procedure and that the surgeon masters the technique. The focus of this thesis is on these two issues. For patient selection, we tested a score system with which chances of ETV success can be predicted, the ETV success score (ETVSS). A patient with a high score is a good candidate. We analyzed the predictive accuracy of the ETVSS in a Dutch cohort and found that the score predicts success reliably. However, some patients develop recurrent symptoms of hydrocephalus; additional treatment is needed. We found that for these patients, again, the ETVSS accurately predicts the chance of success.The endoscopic technique requires different hand-eye coordination skills than open procedures and mastering ETV requires training. A pilot needs to train extended periods in a flight-simulator before getting responsibility over an airplane. Why is this not the same for a surgeon? To facilitate this and to create a risk-free training environment we used 3d-printtechnology to create a silicon replica of a brain. We also developed a standardized assessment instrument to evaluate the surgeons’ skills and technique. Maybe in the future, neurosurgeons will be required to score well on such an assessment instrument before getting the responsibility over an actual patient.

Samenvatting voor Leken

De behandeling van hydrocephalus (waterhoofd) is chirurgisch. Een van de behandelingsopties is middels een kijkoperatie (endoscopie) een gaatje te maken in de bodem van derde hersenkamer. Deze procedure heet een derde ventriculocisternostomie (3VC). Het is een effectieve maar technisch complexe procedure met significante risico’s. Het is belangrijk dat de juiste patiënten worden geselecteerd voor behandeling en dat de chirurg de techniek goed onder controle heeft. De focus van ons onderzoek ligt op deze twee punten. Voor patiënten selectie hebben we een score systeem getest waarmee de kans van

180

CHAPTER 11

slagen van de 3VC kan worden voorspeld, de endoscopic third ventriculostomy success score (ETVSS). Een patiënt met een hoge score is een geschikte kandidaat. We hebben de ETVSS op een Nederlandse patiëntengroep toegepast en vonden dat de test de slagingskans vrij nauwkeurig voorspelt. Bij sommige patiënten komen de klachten van hydrocephalus weer terug, bij deze groep kan wederom de ETVSS worden toegepast voor voorspellen van succeskans. De endoscopische behandeling vergt andere hand-oog coördinatie dan open chirurgie. Het leren van 3VC vergt dan ook oefening. Een piloot oefent eerst in een simulator alvorens verantwoordelijkheid te krijgen over een vliegtuig. Waarom geldt hetzelfde niet voor een chirurg? Om deze mogelijkheid te faciliteren en een risico-vrije trainingsomgeving te creëren hebben we met 3d-printtechtniek een siliconen namaakbrein gemaakt. Daarnaast hebben we een gestandaardiseerde toets voor evalueren van techniek ontwikkeld. Misschien dat in de toekomst wordt verwacht dat een neurochirurg-in-opleiding eerst goed scoort op een soortgelijke toets alvorens de verantwoordelijkheid te krijgen over een patiënt.

181

Appendix

First Checklist: Procedural Steps A. Set-up of Endoscope and instruments 1. The camera was oriented to ensure the view was in the upright position before commencing the procedure 2. The camera was focused prior to commencing the procedure 3. The irrigation system was checked to ensure appropriate function 4. The appropriate irrigation solution was used (e.g. isotonic saline at body temperature) 5. Supportive equipment (e.g. articulating arm) was set up appropriately at the beginning of the procedure 6. The endoscopic instruments were checked to ensure appropriate function (e.g. instruments go down easily, full closure of alligator forceps, functioning monopolar cautery, etc.) 7. The endoscope was checked to ensure appropriate function (e.g. checked for rounded edges of scope, smooth walls of sheath, etc.) B. Exposure 8. The image guidance was appropriately set-up and used to plan the cortical entry point, target and trajectory to reach the Foramen of Monro and floor of the 3rd ventricle (optional) 9. The position and size of the skin incision and burrhole were appropriate for the individual patient (i.e. at or anterior to the coronal suture and lateral to the midline) 10. The trajectory used to access the lateral ventricle was appropriate 11. The ventricle was tapped with a smaller brain needle before passing the larger sheath/trocar (optional) 12. The sheath/trocar was advanced into the ipsilateral frontal horn to an appropriate depth in the ventricle 13. Orientation and position in the ipsilateral frontal horn was confirmed using anatomical landmarks 14. Hemostasis was appropriately maintained throughout the procedure C. Navigation 15. The endoscope was maintained in the selected trajectory 16. The endoscope was navigated through Foramen of Monro into 3rd ventricle 17. The anatomy of the third ventricular floor was correctly identified 18. An appropriate ventriculostomy site was selected D. Ventriculostomy 19. An initial perforation at the ventriculostomy site was made using an accepted method (e.g. blunt perforation, etc.) 20. The ventriculostomy was widened to ensure patency using an accepted method (e.g. forceps or Fogarty balloon dilation) E. Confirmation of adequate ventriculostomy 21. The adequacy of the ventriculostomy was assessed by visualizing bidirectional movement of 3rd ventricular floor 22. The endoscope was advanced to the ventriculostomy to visualize the pre-pontine cistern and confirm that no additional membranes were blocking CSF flow 23. If Lillequist membranes were present, they were perforated using an appropriate technique F. Closure 24. The ventricle was refilled with irrigation solution to remove air 25. Upon removing the endoscope, the fornix was inspected to ensure no significant damage during procedure 26. The burrhole was appropriately covered (e.g. with cap, gelfoam, acrylic, bone dust etc.) 27. The skin was closed in water tight fashion

Second Checklist: Procedural Pitfalls

A. Improper set-up

1. Rotated camera 2. Unfocussed image 3. Image blurred by lens: debris / function 4. Wrong temperature or osmolality of irrigation solution 5. Inappropriate checking or set-up of endoscopic instruments or supports (e.g. poorly functioning alligator forceps, improper set-up of the articulating arm) B. Improper entry / trajectory

Appendix 3. Three initial lists for online survey

182

CHAPTER 11

6. Malposition of cortical entry (too far anterior/posterior or lateral/medial), resulting in abnormal orientation upon entry into ventricle 7. Endoscope inserted at improper trajectory (too far anterior/posterior or lateral/medial), resulting in an inability to advance into the lateral or 3rd ventricle without damaging adjacent neural structures C. Poor exposure 8. Inability to insert endoscopic apparatus due to small skin exposure or inadequate hemostasis 9. Difficulty accessing cortical surface or ensuring adequate trajectory of trocar/sheath due to inadequate bony exposure 10. Unnecessary neural damage due to advancing trocar/sheath without establishing a track with a smaller instrument 11. Endoscope inserted to an inappropriate depth (e.g. too shallow or too deep) 12. Failure to establish orientation and position in lateral ventricle D. Traction injuries, failure to identify anatomy 13. Tearing of ependymal vessels due to excessive endoscope movement 14. Damage to vascular structures (septal/thalamostriate veins, choroid plexus) as endoscope advanced through FOM (resulting in excessive bleeding or requiring cautery) 15. Excessive traction on fornix upon advancement into 3rd ventricle 16. Failure to appropriately identify anatomy of 3rd ventricular floor, resulting in inappropriate selection of ventriculostomy site 17. Obstruction of irrigation outflow causing raised ICP E. Technically inadequate ventriculostomy 18. Failure to fenestrate the 3rd ventricular floor 19. Inadequate size of fenestration 20. Inappropriate placement of fenestration 21. Technically unsafe fenestration, e.g. excessive movement, rough handling of tissues, or failure to abort procedure when appropriate to do so F. Failure to recognize technically inadequate fenestration 22. Did not recognize or attempt to correct lack of bidirectional flow 22. Did not check for membranes or other obstructions to CSF flow into pre-pontine cistern 23. Failed to open membranes of Lilequist if present G. Improper closure 24. Collapse of ventricles after release of too much CSF 25. Traction injury to fornix upon removal of endoscope 26. CSF leak due to inadequate closure of burrhole or skin incision

Third list: Global Rating Scale

Item and Anchors 1. Preparation for the procedure (Rating Anchors: 1 = Did not organize or set-up equipment well; had to stop procedure frequently to prepare or fix equipment; 3 = Equipment generally organized; occasionally had to stop and prepare or fix equipment; 5 = All equipment neatly organized, prepared and ready for use) 2. Respect for tissue (Rating Anchors: 1 = Frequently used unnecessary force on tissue or caused damage; 3 = Careful handling of tissue but occasionally caused inadvertent damage; 5 = Consistently handled tissues appropriately with minimal damage) 3. Time and motion (Rating Anchors: 1 = Many unnecessary moves; 3 = Efficient time and motion, but some unnecessary moves; 5 = Clear economy of hand movement and maximum efficiency) 4. Instrument handling (Rating Anchors: 1 = Repeatedly makes tentative or awkward moves with instruments; 3 = Competent use of instruments but occasionally appeared stiff or awkward; 5 = Fluid movements with instruments and no stiffness or awkwardness)

183

Appendix

5. Knowledge of instruments (Rating Anchors: 1 = Frequently asked for wrong instrument or used inappropriate instrument; 3 = Knew names of most instruments and used appropriate instrument; 5 = Obviously familiar with instruments and their names) 6. Flow of operation (Rating Anchors: 1 = Frequently stopped operating and seemed unsure of next step in the procedure; 3 = Demonstrated some forward planning with reasonable progression of procedure; 5 = Obviously planned course of operation with effortless flow from one step to the next) 7. Use of assistants (Rating Anchors: 1 = Consistently used assistants poorly or failed to use assistants; 3 = Appropriate use of assistants most of the time; 5 = Strategically used assistants to the best advantage at all times) 8. Knowledge of specific procedure (Rating Anchors: 1 = Deficient knowledge. Required specific instruction at most steps of operation; 3 = Knew all important steps of operation; 5 = Demonstrated familiarity with all steps of the operation) 9. Overall performance (Rating Anchors: 1 = Very poor; 3 = Competent; 5 = Clearly superior)

184

CHAPTER 11

Year of residency:______ Date:______

Please rate the following: Strongly Strongly Disagree Disagree Neutral Agree Agree

The camera view is comparable to what you would see in a 1 2 3 4 5 real surgical scene

Performing the ventriculostomy on the floor of the 3rd 1 2 3 4 5 ventricle of the model feels like it does in real reality

The simulator matches actual tissue properties closely 1 2 3 4 5

The bleeding looks realistic 1 2 3 4 5

This model helps to develop camera skills needed for ETV 1 2 3 4 5

This model helps to develop hand-eye coordination 1 2 3 4 5 needed for ETV

The ventriculostomy task is a valuable training exercise 1 2 3 4 5

Use of this model will increase resident competency when 1 2 3 4 5 used to train residents prior to their first ETV

I would be interested in using this model to train residents 1 2 3 4 5

Comments (what you liked/disliked, suggestions for improvement, etc):

..................................................................................................................................................

..................................................................................................................................................

Appendix 4. Feedback on brain simulator

185

Appendix

Appendix 5. Simulator assessment - VR v physical

1

Simulator Assessment (NeuroTouch): 1. Please rate the following aspects of the virtual ETV simulator you just tried in terms of

its realism compared to an ETV performed on a live patient in an operating room:

1 = strongly disagree; 2 = disagree; 3 = neither agree nor disagree; 4 = agree; 5 = strongly agree

Anatomy 1 2 3 4 5 The surface anatomy was realistic and appropriately detailed for choosing an entry point/trajectory to insert the endoscope

The intra-ventricular anatomy was realistic and had the appropriate detail required for navigation to the third ventricular floor

The anatomy of the 3rd ventricular floor (landmarks to guide perforation point, visualization of the cistern, etc.) were realistic and had the appropriate detail required to select and perform the ventriculostomy

Comments regarding anatomy:

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Instrument Handling 1 2 3 4 5 The handling of the endoscope was realistic The handling of the endoscopic tool was realistic The haptic (tactile) feedback from the simulator was realistic The response of the tissue to manipulation by the endoscope/endoscopic tool was realistic

Comments regarding instrument handling:

________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Content of Procedure 1 2 3 4 5 The steps required to complete the task were representative of the steps required to complete the real procedure

The skills required to complete the task were representative of the skills required to complete the real procedure

This task was technically challenging for me Comments regarding procedural steps and skills:

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

186

CHAPTER 11

2

Overall Task Fidelity (Realism) 1 2 3 4 5 The simulation suspended disbelief The simulator environment is realistic of the real-life situation (e.g. look and feel of the endoscope and tool handles, position of the patient head, position of the rest of the equipment, look and feel compared to a real OR, etc.)

Real life factors, situations and variables were built into the simulation scenario

Overall comments (please also indicate what can be done to improve the simulator as a tool for teaching ETV technical skills):

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Virtual simulator overall assessment: 2. Would you use this simulator for your personal training, or the training of

residents/fellows in your program, on ETV? Yes No

a. If no, please explain what would need to be changed in order for you to use this tool: ______________________________________________________________________________________________________________________________________________________________________________________________________

b. What alternate scenarios (e.g. variations in anatomy, complications, or additional endoscopic procedures) would you like to see included in the simulator? ______________________________________________________________________________________________________________________________________________________________________________________________________

187

Appendix

3

Simulator Assessment (Physical Model): 3. Please rate the following aspects of the physical ETV simulator you just tried in terms

of its realism compared to an ETV performed on a live patient in an operating room:

1 = strongly disagree; 2 = disagree; 3 = neither agree nor disagree; 4 = agree; 5 = strongly agree

Anatomy 1 2 3 4 5 The surface anatomy was realistic and appropriately detailed for choosing an entry point/trajectory to insert the endoscope

The intra-ventricular anatomy was realistic and had the appropriate detail required for navigation to the third ventricular floor

The anatomy of the 3rd ventricular floor (landmarks to guide perforation point, visualization of the cistern, etc.) were realistic and had the appropriate detail required to select and perform the ventriculostomy

Comments regarding anatomy:

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Instrument Handling 1 2 3 4 5 The handling of the endoscope was realistic The handling of the endoscopic tool was realistic The haptic (tactile) feedback from the simulator was realistic The response of the tissue to manipulation by the endoscope/endoscopic tool was realistic

Comments regarding instrument handling:

________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Content of Procedure 1 2 3 4 5 The steps required to complete the task were representative of the steps required to complete the real procedure

The skills required to complete the task were representative of the skills required to complete the real procedure

This task was technically challenging for me Comments regarding procedural steps and skills:

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

188

CHAPTER 11

4

Overall Task Fidelity (Realism) 1 2 3 4 5 The simulation suspended disbelief The simulator environment is realistic of the real-life situation (e.g. look and feel of the endoscope and tool handles, position of the patient head, position of the rest of the equipment, look and feel compared to a real OR, etc.)

Real life factors, situations and variables were built into the simulation scenario

Overall comments (please also indicate what can be done to improve the simulator as a tool for teaching ETV technical skills):

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Physical simulator overall assessment: 4. Would you use this simulator for your personal training, or the training of

residents/fellows in your program, on ETV? Yes No

a. If no, please explain what would need to be changed in order for you to use this tool: ______________________________________________________________________________________________________________________________________________________________________________________________________

b. What alternate scenarios (e.g. variations in anatomy, complications, or additional endoscopic procedures) would you like to see included in the simulator? ______________________________________________________________________________________________________________________________________________________________________________________________________

Physical vs. Virtual Simulation: 5. Which of the two simulations that you tried today do you feel is a better training tool?

Please explain your answer. ________________________________________________________________________________________________________________________________________________________________________________________________________________________ ________________________________________________________________________________________________________________________________________________________________________________________________________________________ ________________________________________________________________________________________________________________________________________________________________________________________________________________________ ____________________________________________________________________________________________________________

189

Appendix

Appendix 6. Curriculum vitae

PersonaliaSurname BreimerChristian name Gerben Eise Address Lamonggracht 10, 1019 RE AmsterdamPhone 06 13753070 E-mail [email protected] of birth and -place 10 February 1988, WageningenMarital status Single Nationality Dutch

Education

University of Groningen Medicine (Master’s degree 2014) Attended general studies modules Philosophy (2007-2008) Pre-university education Pantarijn, Wageningen, VWO 6, degree 2006 Profile: Physics and Health (addition Economy)

Professional experience in Medicine

Neurosurgery resident, Universitair Medisch Centrum Groningen, May – December, 2014

Internship at Department of Pathology, Netherlands Cancer Institute – Antoni van Leeuwenhoek, Amsterdam, January – February, 2015

Resident Clinical Pathology, Academisch Medisch Centrum, Amsterdam, July, 2015 – present

PhD student, 2012 – 2017. As a medical student I started doing research on several aspects of hydrocephalus in a pediatric population in the period 2010-2014, this resulted in five published papers, one revised resubmitted manuscript, and two draft manuscript. The first paper was on low-pressure valves in a shunt for diversion of cerebrospinal fluid from ventricle to the abdomen. The second one was on a predictive model for estimating chances of treatment

190

CHAPTER 11

success of endoscopic third ventriculostomy (ETV), a treatment modality for hydrocephalus. In the final year of medical school I went to Canada to conduct research in the Hospital for Sick Children in Toronto. Here I developed a synthetic brain simulator for ETV (paper 3). The simulator is now in use for neuroendoscopy training of the neurosurgery residents at the University of Toronto Neurosurgery department. Simultaneously, I developed the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT) (paper 4). This is a series of assessment tools to assess performance of neurosurgical trainees doing the ETV procedure. The NEVAT was subsequently validated during an international hands-on workshop on cerebral and ventricular neuroendoscopy in Naples, Italy (January 2014) (paper 5). In 2014 and 2015 I gathered data during neuroendoscopy courses organized at the surgical skills lab at Mount Sinai Hospital in Toronto, for a comparison study between a virtual reality and physical neuroendoscopy simulator (submitted manuscript 6). In 2015 I started working on an retrospective nationwide cohort study in which the effectivity of repeated endoscopic third ventriculostomy is evaluated (draft manuscript 7). After this we set out to reveal practice patterns after initial ETV fails (draft manuscript 8)

Other professional experience

Trainer youth teams field hockey 2004 – 2006 (learned to motivate people to participate and enhance skills)Voluntary work Humanitas ‘buddy project’ 2009 – 2010 (learned how to provide structure to study and reduce loneliness) Medisch Spectrum Twente Benefietcommissie, 2012 (we raised 25.000 Euro for a new cargo van for ‘de Huifkar-Meerik’, a school for students with learning disabilities to facilitate the gardening project)

Courses and workshops

International hands-on workshop on cerebral and ventricular neuroendoscopyTropical medicine course Statistics SPSS (certificate) Field hockey referee (certificate)

191

Appendix

Publications

1. Breimer GE, Sival DA, Hoving EW. Low-pressure valves in hydrocephalic children: a retrospective analysis. Childs Nerv Syst. 2012;28(3):469–73. Available from: http://www.ncbi.nlm.nih.gov/pubmed/22205533

2. Breimer GE, Sival DA, Brusse-Keizer MGJ, Hoving EW. An external validation of the ETVSS for both short-term and long-term predictive adequacy in 104 pediatric patients. Childs Nerv Syst. 2013;29(8):1305–11. Available from: http://www.ncbi.nlm.nih.gov/pubmed/23644629

3. Breimer GE, Bodani V, Drake JM. Design and evaluation of a new synthetic brain simulator for endoscopic third ventriculostomy. J Neurosurg Pediatr. 2015;15(1):82–8. Available from: http://www.ncbi.nlm.nih.gov/pubmed/25360853

4. Breimer GE, Haji FA, Hoving EW, Drake JM. Development and content validation of performance assessments for Endoscopic Third Ventriculostomy. Childs Nerv Syst. 2015;31(8):1247-59. Available from: http://www.ncbi.nlm.nih.gov/pubmed/25930722

5. Breimer GE, Haji FA, Cinalli G, Hoving EW, Drake JM. Testing reliability and validity of the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). Operative Neurosurgery. 2016 doi: 10.1227/NEU.0000000000001158

6 Breimer GE, Haji FA, Bodani V, Cunningham MS, Lopez-Rios AL, Okrainec A, Drake JM. Simulation-based Education for Endoscopic Third Ventriculostomy: A Comparison between Virtual and Physical Training Models. Operative Neurosurgery. 2016. doi: 10.1227/NEU.0000000000001317

7 Breimer GE, Dammers R, Buis DR, Delye H, Woerdeman P, Hoving EW. Repeat endoscopic third ventriculostoy in pediatric patients: the Dutch experience. Submitted. 2016

8 Breimer GE, Warf B, Cinalli G, Drake JM, Kulkarni AV, Constantini S, Schroeder HWS, Roth J, Tamburrini G, Hoving EW. Current Practice of Treatment After Failure of Initial Endoscopic Third Ventriculostomy: A Survey Study and Review of Literature. Work in proces. 2017

Presentations

Oral presentations

European Society for Pediatric Neurosurgery Congress, May 2012, Amsterdam, the Netherlands (“The use of low-pressure valves in hydrocephalic children under two years of age.”)American Association of Neurological Surgeons - Pediatric Section Meeting, December 2013, Toronto, Canada (“Fidelity Study of a New Synthetic Simulator for Endoscopic Third Ventriculostomy.”)European Society for Pediatric Neurosurgery Congress, May 2014, Rome, Italy (“An external validation of the ETVSS for both short-term and long-term predictive adequacy in 104 pediatric patients.”)European Society for Pediatric Neurosurgery Congress, May 2014, Rome, Italy (“Fidelity Study of a New Synthetic Simulator for Endoscopic Third Ventriculostomy.”)

192

CHAPTER 11

European Society for Pediatric Neurosurgery Congress, May 2016, Paris, France (“Validity evidence for the neuro-endoscopic ventriculostomy assessment tool (NEVAT)”)European Society for Pediatric Neurosurgery Congress, May 2016, Paris, France (“Repeat endoscopic third ventriculostomy in pediatric patients: the Dutch experience”)

Poster presentation:

Canadian Neurological Sciences Federation 49th Annual Congress, June 2014, Banff, Canada (“A physical simulator for intraventricular neuroendoscopy – validation and performance assessment”)

193

Appendix

UITNODIGING

voor het bijwonen van de verdediging van mijn proefschrift

Improving outcomes in pediatric endoscopic third ventriculostomy

through outcome analysis and surgeon training

door

Gerben Breimer

op woensdag 11 januari 2017 om 11:00 uur

in de Aula van het Academiegebouw, Broerstraat 5 te Groningen.

Aansluitend wordt een lunch georganiseerd.

Paranimfen:

Arthur de Jager

&

Wico Breimer

info:[email protected]

Improving outcomes in pediatric endoscopic third ventriculostomy through

outcome analysis and surgeon training

Gerben Breimer

Improving outcom

es in pediatric endoscopic third ventriculostomy through outcom

e analysis and surgeon training G

erben Breim

er

Breimer cover fin.indd 1 29-11-16 13:39

improving outcomes in pediatric endoscopic third ... · arthur de jager & wico breimer info:...

Documents