Austin Neurosurg Open Access. 2014;1(1): 1004.
Paramita Das, Tarini Goyal, Andrew Xue, Sruti Kalatoor and Daniel Guillaume*
Department of Neurosurgery, University of Minnesota, USA
*Corresponding author: Daniel Guillaume, Department of Neurosurgery, University of Minnesota, D429 Mayo Memorial Building, 420 delaware Street S.E. MMC96, Minneapoilis, MN 55455, USA
Received: March 18, 2014; Accepted: April 18, 2014; Published: April 21, 2014
In medicine there is an increasing emphasis on efficiency, error reduction, and training within limited hours. Simulation training has played a role in neurosurgical training. With improvements in haptic technology and visual displays, virtual reality surgical simulators can offer a tool for surgical planning, and training in a safe environment. In this review we discuss the current state of medical simulation, the integration of simulation into current neurosurgical residency training, specific neurosurgical simulation experiences, the potential benefit of simulation and the future of neurosurgical simulation.
Keywords: Neurosurgical Simulation; Endoscopic; Hemostasis
Our primary objective in neurosurgery is to provide high–quality patient care that is safe and effective. Over the course of our careers, we strive to achieve excellence in all aspects of the care we provide. When it comes to educating future neurosurgeons, our goal is to provide them with the knowledge and technical skills needed so they can go on to provide safe high–quality care to their patients.
Surgical training has evolved over the last several decades and surgical simulation continues to play a growing role. Three major forces driving neurosurgical simulation development are: regulations in surgical resident training, higher demands for quality and safety, and higher expectations from patients and families.
The first factor leading to use of surgical simulation in resident training has been recent changes in resident educational regulations, the greatest being the 80–hour per week resident work hour limit instituted from the Accreditation Council for Graduate Medical Education. Because residents must now spend less time in the hospital while obtaining their education in the same number of years, training programs need to be more efficient in methods of educating and teaching surgical dexterity to residents. Additionally, there is a need for objective measurements of competence. With advances in surgical simulation technology, the potential exists for a means to collect data that may be useful in measuring surgical technical ability.
A second driving force promoting simulation training is a desire from health–care facilities and payers to improve the quality and safety of our specialty. Because simulation can allow acquisition and refinement of surgical techniques in an educational environment free from the pressures, demands and risks of patient care, there is an opportunity for residents to acquire and perfect technical skills prior to operating on an actual patient.
A third issue leading to an increasing role of simulation in resident education has occurred as patients and families have developed a higher expectation from physicians and a lower tolerance of complications. Patients will want to know if their neurosurgeon is competent and experienced. Hospitals expect greater efficiency in the operating room. There is a growing expectation that surgical simulation is a part of neurosurgical training, just as flight simulation is part of a pilot’s training.
In the future, we expect simulation to play an increasingly large role in education and training of neurosurgical residents. The value of simulation will increase as technology improves, and as simulators can more closely mimic a true surgical field. Work in education and simulation research will need to focus on validation studies that carefully evaluate the usefulness of simulation as a tool that actually improves our ability to operate.
In addition to resident education, neurosurgical simulation has developed a role in continuing education and in pre–surgical planning. For planning of operations on specific patients, imaging data can be loaded onto a variety of simulation devices including even the Apple IPad, for anatomic study and planning. Other potential roles exist in anatomic study, surgical rehearsal of uncommon procedures, and credentialing.
In this review we discuss the current state of medical simulation, the integration of simulation into current neurosurgical residency training, specific neurosurgical simulation experiences, the potential benefit of simulation and the future of neurosurgical simulation.
Simulation in medical education is not a new concept, and in fact cadaveric dissection as a form of surgical simulation has been used as a primary means of education for hundreds of years. The earliest non cadaver and non animal models in medical training started in the 1950s when mannequins were introduced into medical training to simulate basic cardiopulmonary resuscitation . Likewise, throughout history a primary means of medical education has been in scenario simulation, where the trainee is asked to think through and describe all of the steps in order of an activity such as management of the patient with intracranial hypertension, or the surgical steps needed for clipping a posterior communicating artery aneurysm. This has been the basis for the oral board exam in our specialty. Surgical simulators, which are designed to evaluate technical ability of the trainee, are relatively new and currently in the early stages of development.
Simulation is presently used in many different ways in the education of neurosurgical trainees. Photographic rendering, virtual interaction, anatomic models and haptic devices are the core components of today’s simulators . Haptics refers to the feedbackof sensory information. It is an integration of what is seen with a realistic sensation of what is felt . Medical educational simulators are generally described as being one of 4 basic types: physical, virtual reality (VR), web–based and hybrids (Table 1). The majority of those used in neurosurgical simulation to date are of the physical and virtual reality type.
Direct manipulation and contact
Provides actual manipulation and tactile feedback
Not reusable, cadavers not always realistic (no bleeding, etc.)
Aneurysm clipping, craniotomy for trauma and tumor, placement of EVD
Reusable, collect metrics, continually modify and update models. More accurate representation of surgical procedure,
Initial cost of purchase, haptics not as good as cadavers for drilling
Evaluate self-assessment and decision making
Self-assessment in neurosurgery (SANS) module,
Inexpensive, can evaluate knowledge and decision-making
No haptic controls or tactile feedback
Combination of other types
Table 1: Neurosurgery Simulator Models.
The so–called physical simulators include cadaver models, animal labs and manikins. Traditionally, these have been at the forefront of surgical training and education. Cadaveric dissection in some ways remains a “gold standard” for cranial or skull base exposures or manipulation of the soft tissues and bone of the spine. Unfortunately, the supply of cadavers is limited and the procurement and maintenance of cadavers and animal models for surgical dissection and teaching is expensive. Issues related to the ethical treatment of animals raise important questions as to the real need and educational value of animal labs. There is debate however animal labs are the only models that can realistically teach techniques like homeostasis, and other technical aspects requiring living vascularized tissues. Fresh frozen cadavers not only poorly replicate the fine operative manipulation of live vascular brain tissue but they are also poor models for approaches involving ventricular anatomy and cannot replicate pathological conditions such as brain tumors or aneurysms.
Computer–generated graphics has enabled the creation of virtual reality (VR) based simulators that have the ability to recreate human anatomy in a virtual space without the need for physical models. These types of simulators address the problems related to limited availability of cadaver models, ethical and legal issues related to animal models, and poor representation of cadavers for representing living vascular tissue. The initial neurosurgical simulators did not contain haptic feedback mechanisms and instead used imaging data to create representations of surgical approaches or specific pathologies. The Dextroscope (Bracco, Princeton, New Jersey USA) developed by Kockro et al. uses a VR environment in which the operator reaches behind a mirror into a computer generated stereoscopic three dimensional images and moves and manipulates the object in real time with natural 3–D hand movements .
General surgery training programs have been at the forefront of incorporating simulation into their residency. Laparoscopic surgery is a rather new area of general surgery and the technique can bedifficult to master. This has led to the development of laparoscopic training simulators. VR simulation for laparoscopic surgery has been shown to be beneficial and is actively used in general surgery residency education [5,6]. Gurusamy et al reviewed 23 validationstudies with 612 participants on virtual laparoscopy simulators and showed that operative time was decreased, accuracy was increased, and general errors were decreased .
The introduction of haptic feedback into free hand neurosurgical simulators is more challenging from a technological standpoint than laparoscopic simulators, as those instruments are limited in their degrees of freedom. A free hand surgical simulator must have a haptic device that freely moves with 6 degrees of freedom and responds with a high degree of spatial and temporal accuracy to subtle variations in the relative position of the user and virtual space. One of the earliest neurosurgical simulators was designed to emulate ventricular catheter insertion based on the Virtual Brain Project . This concept was further developed and is the basis of the Immersive Touch system developed at the University of Illinois in Chicago (Immersive Touch, Westmon, Illinois, USA). This system has been applied to other task oriented applications such as ventriculoperitoneal shunt insertion, pedicle screw insertion, and vertebroplasty.
The NeuroTouch (Figure 1), a VR simulator developed by the National Research Council of Canada, includes a series of cranial and endoscopic modules, with spinal modules in development. The NeuroTouchCranio, a VR simulator for select cranial microsurgery procedures, uses stereovision and bimanual tool handles with forcefeedback, including aspirator, cavitron ultrasonic surgical aspirator (CUSA), bipolar, and micro scissors. Training modules for the cranial portion are based on brain tumor patients and measure performancemetrics providing an objective assessment of technical skills. The NeuroTouch Endo is a VR simulator using the same software for endoscopic procedures. It generates a realistic endoscopic view including lens distortion effects, blurring, and tissue deformation. The endoscopic portion includes modules for endoscopic third ventriculostomy and navigation in the nasal cavity with performance metrics.
Figure 1: The NeuroTouch Neurosurgical Virtual Reality simulator. The NeuroTouch Neurosurgical Simulator (A), developed by the National Research Council of Canada, is a VR simulator with stereoscopic view of a 3–dimensional environment (B), and haptic feedback. Shown are 2–dimensional images from the simulator depicting the Foramen of Monro for endoscopic third ventriculostomy (C) and tumor removal with suction and bipolar instruments (D).
The use of simulation in neurosurgical training is playing a larger role in resident education today. For teaching surgical technique, a large number of models and options are available. These simulation devices are designed to supplement teaching of such skills as drilling, sewing, dissecting, control of hemostasis, tumor removal, suction, endoscopy, placement of screws, ventricular catheter placement, and angiography among others.
Drilling techniques: Drilling is an integral technique used in all types of neurosurgery. Many simulation approaches exist for teaching this. Physical models continue to be a primary tool used for training junior residents in drilling burr holes, a critical skill with a risk of dural injury. Practice on physical and animal bones (freeze–dried bone scapula) helps teach trainees how to avoid the risk of injuring the underlying neural tissue.
There also exist VR options for drilling simulation. The NeuroTouch, described earlier (Figure 1), allows for manipulation of the tools and instruments most commonly used during endonasaltranssphenoidal surgery, including drills, debrider, suction, cautery, curettes, and forceps. The bilateral use of both hands also mimics the actual handling of tools during transsphenoidal surgery. The NeuroTouch Endo includes a 4 mm microdrill with a spherical drill burr, which is used to enter the sphenoid sinus. This simulator has structured levels of difficulty by varying the size of the nasal passages. The standard technique in using the simulator involves using a microdrill to enlarge the sphenoid ostia, and then entering sellarturcica through one or both nostrils .
Other simulators exist to teach other skull base techniques, including drilling the petrous bone and other middle–fossa structures. For example, the Barrow Neurological Institute developed a 3–D surgical simulator called the interactive virtual dissector. This simulator is useful in teaching residents how to drill the petrous bone and to identify critical structures .
Ventriculostomy: One of the most common and essential procedures practiced in resident training is placement of a ventricular catheter. Simulation–based training for ventriculostomy early in residency may help the residents develop these technical core skills faster and with less risk to patients. There currently exist several commercially available ventriculostomy simulators. Physical simulators involve using a physical model to represent the skull, dura, and ventricular anatomy. A second approach involves VRbased ventriculostomy simulation. Several VR–simulators have programs that are useful in educating residents in this commonly used technique, including the Immersive Touch (Chicago, Illinois) and NeuroTouch (National Research Council of Canada). Scoring for ventriculostomy performance using these VR–simulators is typically measured by entry point for the burr hole, catheter trajectory, lengthof catheter inserted, and time to complete procedure. In some cases, level of technical difficulty can be adjusted and can be measured to observe the learning curve of different levels of trainees. In order to measure overall performance, other performance measures including knowledge of anatomy and simulation familiarity can help reinforce concepts .
Craniotomy: Current craniotomy simulators can be either web or VR based, and tend to focus more on specific intracranial pathologies, rather than on basic techniques like planning incisions, burr holes, and extent of bone removal in a particular craniotomy [10–12]. Incision planning and basic concepts of craniotomy are still taught using physical models. VR–based craniotomy simulators allow for unlimited practice in a model with in–built scoring metrics, for unbiased evaluation. These simulators tend to have higher up–front cost and imperfect haptic feedback.
Physical craniotomy simulators have the most realistic portrayal of tool handling and potential complications, such as variant skull anatomy. These models can only be used a limited number of times, and are associated with lower up–front costs but greater ongoing maintenance. A physical simulator that portrayed all 5 layers of the scalp, as well as overlying muscle and blood supply, was used in a course for 12 residents at a large national training conference. There was overall improvement in trainees’ efficiency, knowledge of anatomy, and dexterity as judged by supervising faculty . The course also required residents to recognize and appropriately respond to surgical complications. This teaching model demonstrates both content and face validity by appropriately distinguishing the more experienced users. However, the small number of study subjects and probable variability in the evaluation of dexterity make it difficult to make conclusions about the simulator’s predictive validity, or the correlation between simulator skills and clinical skills.
In order for neuro simulators to become an appropriate and useful teaching tool for craniotomy, predictive validity still needs to be established.
Endoscopic Endonasal Approach: The earliest approaches to the pituitary region were through a subtemporal or transfrontal craniotomy. Cushing popularized the sublabialtransphenoidal approach to the pituitary in 1914 . However he abandoned this procedure due to difficulties with visualization. The transnasal transseptal approach was introduced in 1982 by Tucker and Hahn  and this procedure was recently further refined by the introduction of the rigid endoscope. For trainees it requires familiarizing oneself to the anatomy through a different perspective. Major concerns during surgery are identification of the internal carotid artery and optic nerves, and preservation of small cerebral vessels. Proficiency in traditional techniques, endoscopic experience with other procedures, and duration of practice do not predict proficiency .
Previous work by Bakker et al investigating endoscopic sinus surgery found that the skills that required manual dexterity were less difficult than the skills related to spatial orientation . Trainees often have more experience earlier on with the handling and the three dimensional picture provided by an operating microscope. The endoscope provides a view of anatomy with which trainees may be unfamiliar. Virtual endoscopy as a post–processing tool for radiological images was first described by Vining [18,19]. This ledto the beginning of development of training simulation for surgical procedures that may use an endoscope such as transnasal procedures and ventriculoscopy. Although solely visual feedback is useful for learning surgical anatomy the addition of haptic feedback creates a more realistic surgical experience.
Cadaveric dissection continues to be a mainstay of training in endoscopic procedures. However, the safe performance of this approach requires pliable tissues, which is difficult to simulate on embalmed surgical specimens [20,21]. By limiting the degrees of freedom that instruments are able to move, an endoscopic interface provides haptic fidelity without creating a compelling computational burden . The NeuroTouch endoscopic endonasal simulator discussed previously has a program for this type of training. Immersive Touch has a transnasal training module under development.
There continue to be adoptions of extensions of the transsphenoidal approach, with removal of bone along the tuberculumsellae and the planumsphenoidale, creating a need for practice environments even for the fully trained neurosurgeon. Further development of these training programs will be useful for more advanced training as well as personalized presurgical practice based on individual patient images.
Recreating the intraoperative complexity of cerebral angiography in a simulated setting poses a unique challenge for the burgeoning world of neurosimulation. Angiography is a field in which trainees may not have significant exposure during residency. Overall, angiography is considered to be an area of inadequate training. Neurosimulation is one strategy to help fill this apparent gap in training, and has the advantages of being an efficient training method that could potentially improve patient safety.
Currently, two main model types exist for angiography simulation. Electronic “augmented reality” simulators combine virtual reality with the use of real catheters and wires, giving the user haptic feedback. These simulators were used in a two–hour resident training course at a large national conference, after which residents had improved technical skills, measured via both objective and instructor–based evaluations . Studies using similar computerbased haptic simulators also showed improved resident efficiency and performance on visuospatial tests [24,25]. Drawbacks of these models center on the fact that no actual fluids are used. Thus, arterial pulsations are not visualized, and there may be stunted value in the haptic feedback of using catheters, wires, stents, and coils. Additionally, there are limitations to the feasibility of one–onone instruction on these simulators, whether they are used at large conferences or at individual residency sites.
The second type, flow models, give the user the opportunity to visualize the deployment of actual stents and coils in a real fluid environment, complete with arterial pulsations simulated with a water pumping system. Here, the disadvantage is in the lack of simulated fluoroscopy, and the inability of the user to practice endovascular catheter and wire skills. There are more models in development and some are large enough to include aortic and cervical vasculature.
Overall, angiography simulators currently do not generate intraoperative complications that require users to analyze and react to problems. They also cannot mimic procedural and anatomicvariability or provide perfected haptic feedback. However, several studies have shown that clinical experience in endovascular techniques correlates well with users’ simulator skills, suggesting some degree of comparability [26,27]. The future challenge lies in demonstrating the utility of simulation in improving trainees’ clinical technical skills, and, ultimately improving patient outcomes.
Neurosurgical simulation training has been developed on the national level to a great extent over the last few years. The Society of Neurological Surgeons (SNS) now sponsors a regional boot camp course that prepares all first–year neurosurgery residents in the USA in the 6 Accreditation Council for Graduate Medical Education (ACGME) competencies and for the minor surgical procedures they will be performing in their initial years of training. Simulation training in the boot camp course includes placement of intracranial pressure monitors, external ventricular drain placement, shunt taps, cervical traction placement, lumbar drains, management of intracranial hypertension, and surgical positioning. Recently there have been efforts by the SNS to create the next–level resident courses, also held regionally, which are more appropriate to second or third year residents. Simulation would play a major role in these advanced neurosurgical boot camp courses as well.
In general, the ACGME has noted benefits of simulation in medical education. Most data comes from simulation technology used in the field of general surgery. As already discussed, simulation has been widely utilized in the teaching of laparoscopic surgery to residents and attending who were trained in the pre–laparoscopic era. The use of laparoscopic simulation has been shown to improve technique, performance and the time needed to perform the procedure [5,28]. Seymour et al noted in a prospective randomized blinded study that residents trained on VR simulators were 29% faster with laparoscopic gallbladder dissection. The residents who did not participate in VR simulation were 9 times more likely to fail to make progress, 5 times more likely to injure the gallbladder or to burn non–targeted tissue, and overall mean errors were 6 times more likely . There is a need in neurosurgery for further investigation on the effects of simulation in education and training.
In 2010, the Congress of Neurological Surgeons (CNS) formed a committee, now known as the CNS Simulation committee, whose aim was to specifically address “how to maximize neurosurgical education to improve patient outcomes with the greatest efficiency and safety.” The mission of this committee is to develop a comprehensive neurosurgical simulation initiative to redefine the methodology of training neurosurgery residents and serve as the benchmark for future program development.
This much–needed committee has developed as a result of recent attention on safety and quality of care and analysis of patient outcome metrics. Analysis of current techniques for teaching surgical technique to residents, namely apprenticeship models, has brought up questions regarding proficiency and safety. How many cases does a resident need to perform before he or she is competent or proficient? Is there an optimal length of a surgical training program or does thisdiffers for each trainee because everybody learns at a different rate? How can we measure surgical technical skills?
Currently it is clear that there is no standard for teaching technical neurosurgical skills, and no objective criteria are in place to measure or evaluate operative technique. As discussed, the best available simulation methods include physical models, such as cadaver and animal laboratories, and VR–based simulation. All simulation models have the advantage over education in the operating room in that they provide training in a fabricated and controlled setting which allows the trainee an opportunity to make mistakes without compromising quality or safety. Most modern day VR simulators have the ability to objectively quantitate specific metrics, which can be used to evaluate certain aspects of technique.
Does proficiency on the simulator correlate to proficiency in the operating room? Simulation in neurosurgery is a relatively young field that still requires further investigation and validation. Currently, little information exists regarding the true benefit of neurosurgical simulators. It is not proven that this technology will help us train better neurosurgeons. We have not determined which simulation techniques result in improved technical skill and better patient outcomes.
The cost and effort to introduce simulation into neurosurgery training programs and the potential benefits of this type of education have not been assessed. As this technology is relatively young and in many cases still under development, costs are high and the benefit is unclear. Many prototypes have been developed, mostly in academic health centers with combined efforts from departments of neurosurgery and engineering. However, only few VR neurosurgical simulators are commercially available. Notable VR commercially available applications within neurosurgery include the Virtual Brain Project, Dextroscope, Robo–Sim, Immersive Touch, TempoSurg and NeuroTouch. Unfortunately, many of these young simulators are only available within North America. Most commercially available neurosurgical simulators are still undergoing robust developments and are improving as technology rapidly improves. This constant focus on research and development raises the cost of such simulators.
How much does it cost to create a simulation laboratory? Implementation of a 4–week general surgical skills curriculum in the University of Pennsylvania involved an initial expenditure of $4.2 million, $476,000 in annual expenses, and $12,500 cost per resident for that period. The Division of Neurosurgery at the University of Texas, Galveston TX, created a curriculum with 68 core exercises per year involving cadaver dissections, other physical models, and haptic/computerized sessions. Their analysis of cost, in a program of 1–resident–per–year, was $200 per hour of simulation laboratory training, compared to an operating room cost ranging from $2300 to $5500 for the first 30 minutes, to $926 to $2756 for each 30 minutes thereafter . Clearly there is variability in the cost of simulation training related to the types of simulation used, and time dedicated to simulation training. Importantly, the value of neurosurgical simulation in training of neurosurgical residents has not been systematically assessed and is difficult to study.
Over the past several years VR simulators have continued to improve in terms of haptic feedback and photographic rendering. As simulation more closely mimics what is seen in the operating room, we anticipate that it will continue to become more incorporated in surgical education. With this there will be a need for further validation studies of these training programs so they can continue to be refined to train safe and efficient surgeons. If there is correlation between surgical skill and performance on simulators, then they may serve not only in education but also in certification for surgeons. Another usage of these programs is that one may study effects of sleep deprivation or caffeine intake, scenarios that are seen in daily practice, on surgical skills.
We thank NeuroTouch (National Research Council of Canada) for supplying images of the NeuroTouch Neurosurgical Simulator.
- Robison RA, Liu CY, Apuzzo ML. Man, mind, and machine: the past and future of virtual reality simulation in neurologic surgery. World Neurosurg. 2011; 76: 419-430.
- Chan S, Conti F, Salisbury K, Blevins NH. Virtual reality simulation in neurosurgery: technologies and evolution. Neurosurgery. 2013; 72 Suppl 1: 154-164.
- Malone HR, Syed ON, Downes MS, D'Ambrosio AL, Quest DO, Kaiser MG. Simulation in neurosurgery: a review of computer-based simulation environments and their surgical applications. Neurosurgery. 2010; 67: 1105-1116.
- Kockro RA, Serra L, Tseng-Tsai Y, Chan C, Yih-Yian S, Gim-Guan C, et al. Planning and simulation of neurosurgery in a virtual reality environment. Neurosurgery. 2000; 46: 118-135.
- Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2004; 91: 146-150.
- Gurusamy K, Aggarwal R, Palanivelu L, Davidson BR. Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. Br J Surg. 2008; 95: 1088-1097.
- Phillips NI, John NW. Web-based surgical simulation for ventricular catheterization. Neurosurgery. 2000; 46: 933-936.
- Rosseau G, Bailes J, del Maestro R, Cabral A, Choudhury N, Comas O, et al. The development of a virtual simulator for training neurosurgeons to perform and perfect endoscopic endonasal transsphenoidal surgery. Neurosurgery. 2013; 73 Suppl 1: 85-93.
- Jabbour P, Chalouhi N. Simulation-based neurosurgical training for the presigmoid approach with a physical model. Neurosurgery. 2013; 73 Suppl 1: 81-84.
- Schirmer CM, Elder JB, Roitberg B, Lobel DA. Virtual reality-based simulation training for ventriculostomy: an evidence-based approach. Neurosurgery. 2013; 73 Suppl 1: 66-73.
- Acosta E, Liu A, Armonda R, Fiorill M, Haluck R, Lake C, et al. Burrhole simulation for an intracranial hematoma simulator. Stud Health Technol Inform. 2007; 125: 1-6.
- Yudkowsky R, Luciano C, Banerjee P, Schwartz A, Alaraj A, Lemole GM Jr, et al. Practice on an augmented reality/haptic simulator and library of virtual brains improves residents' ability to perform a ventriculostomy. Simul Healthc. 2013; 8: 25-31.
- Lobel DA, Elder JB, Schirmer CM, Bowyer MW, Rezai AR. A novel craniotomy simulator provides a validated method to enhance education in the management of traumatic brain injury. Neurosurgery. 2013; 73: S57-S65.
- Cushing H. Surgical experiences with pituitary disorders. JAMA. 1914; 63: 1515-1525.
- Tucker HM, Hahn JF. Transnasal, transeptal sphenoidal approach to hypophysectomy. Laryngoscope. 1982; 92: 55-57.
- Snyderman C, Kassam A, Carrau R, Mintz A, Gardner P, Prevedello DM. Acquisition of surgical skills for endonasal skull base surgery: a training program. Laryngoscope. 2007; 117: 699-705.
- Bakker NH, Fokkens WJ, Grimbergen CA. Investigation of training needs for functional endoscopic sinus surgery (FESS). Rhinology. 2005; 43: 104-108.
- Vining DJ, Shifrin RY, Hara AK. Virtual bronchoscopy. Radiology 1994; 193: 261.
- Vining DJ, Winston-Salem MD, Shifrin RY. Virtual colonoscopy. Radiology 1994; 193: 446.
- 20. Kassam, AB, Gardner P, Snyderman C, Mintz A, Carrau R. Expanded endonasal approach: full endoscopic, completely transnasal approach to the middle third of the clivus, petrous bone, middle cranial fossa, and infratemporal fossa. Neurosurg Focus. 2005; 19: E6.
- Neubauer A, Wolfsberger S, Forster MT, Mroz L, Wegenkittl R, Bühler K. Advanced virtual endoscopic pituitary surgery. IEEE Trans Vis Comput Graph. 2005; 11: 497-507.
- Spicer MA, Apuzzo ML. Virtual reality surgery: neurosurgery and the contemporary landscape. Neurosurgery. 2003; 52: 489-497.
- Fargen KM, Arthur AS, Bendok BR, Levy EI, Ringer A, Siddiqui AH, et al. Experience with a simulator-based angiography course for neurosurgical residents: beyond a pilot program. Neurosurgery. 2013; 73 Suppl 1: 46-50.
- Chaer RA, Derubertis BG, Lin SC, Bush HL, Karwowski JK, Birk D, et al. Simulation improves resident performance in catheter-based intervention: results of a randomized, controlled study. Ann Surg. 2006; 244: 343-352.
- Spiotta AM, Rasmussen PA, Masaryk TJ, Benzel EC, Schlenk R. Simulated diagnostic cerebral angiography in neurosurgical training: a pilot program. J Neurointerv Surg. 2013; 5: 376-381.
- Bech B, Lönn L, Falkenberg M, Bartholdy NJ, Räder SB, Schroeder TV, et al. Construct validity and reliability of structured assessment of endoVascular expertise in a simulated setting. Eur J Vasc Endovasc Surg. 2011; 42: 539-548.
- Van Herzeele I, Aggarwal R, Choong A, Brightwell R, Vermassen FE, Cheshire NJ. Virtual reality smulation objectively differentiates level of carotid stent experience in experienced interventionalists. J VascSurg. 2007; 46: 855-863.
- Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, et al. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg. 2002; 236: 458-463.
- Gasco J, Holbrook TJ, Patel A, Smith A, Paulson D, Muns A, et al. Neurosurgery simulation in residency training: feasibility, cost, and educational benefit. Neurosurgery. 2013; 73 Suppl 1: 39-45.