International Journal of Healthcare Simulation

Introduction

Operating theatre time is a costly resource, requiring efficient but safe surgical proceduralists. Training on laparoscopic simulators has potential to improve surgeons’ efficiency, but the translation of these benefits to clinical outcomes is often not realized [1]. This article describes introduction of a comprehensive laparoscopic simulation education program aiming to enhance operative efficiency in gynaecological procedures at a tertiary health service. We describe our approach to educational design, clinical integration and evaluation using an outcome logic model.

Background

Laparoscopic simulators have been demonstrated to be effective in training and assessing the skills of trainee surgeons in a teaching and training environment [2–6]. Virtual reality (VR) simulators, which offer the ability for trainees to rehearse entire surgical procedures, may be more efficient in skill development than simulators offering rehearsal of basic skills alone [7]. The Royal Australia and New Zealand College of Obstetricians Gynaecologists (RANZCOG) has now mandated that all trainee gynaecologists have access to box trainer laparoscopic simulators at their workplace [8]. However, simulation equipment such as VR laparoscopic simulators is an expensive investment and may not be accessible to all trainees. Even when VR training opportunities are provided, there is limited evidence of impact and transferability of skills to the operating room [1]. Training institutions are left with the dilemma of how to best integrate and utilize these education simulation tools.

Numerous strategies have been attempted to incentivize trainee engagement in laparoscopic simulation programs, such as providing portable take-home simulators, offering personalized feedback on performance, issuing credentialling certificates and utilizing online modules [8]. Despite these efforts, the level of engagement in many laparoscopic simulation programs remains low [8–10]. One potential solution may lie in credentialling junior surgeons based on skill assessments conducted on laparoscopic simulators. While laparoscopic simulation programs have primarily focused on training with the expectation of subsequent performance improvement, incorporating proficiency testing into the process could yield better results. By utilizing objective skill assessments, supervising surgeons could make more informed decisions regarding the appropriateness of trainees assuming primary operating roles.

Evaluation of training programs through the lens of clinical outcomes is challenging, but critical if our simulation is to have a translational impact [11]. Direct, causal links between educational programs and clinical outcomes are hard to establish. An outcome logic model may be an alternative approach to evaluation [12]. This approach depicts the path from program creation to implementation to outcomes and allows a comparison among the resources required for a program, the activities of the program and the results or outcomes [11,13,14].

We theorized that the promise of laparoscopic VR simulation training might be realized through closer integration of educational strategies with real-world operating theatre processes, including credentialling and supervision, and with evaluation through the lens of clinical outcomes. We introduced a comprehensive laparoscopic simulation education program to our institution with the primary goal of improving operating room theatre efficiency, and evaluated the program using an outcome logic model.

Methods

Evaluation approach

An outcome logic model was used to document the extent to which the simulation program attained its objectives [11]. An outcome logic model depicts the path from program creation to implementation to outcomes for participants and the institutions in which they learn/work [11,13,14]. The program objectives were selected by the research team as those that were predictive of desired feasibility and educational outcomes. To select the components for the model we took a pragmatic approach based on work previously presented and published on laparoscopic simulation programs. This informed our decision to incorporate components such as take-home laparoscopic simulation trainers, credentialling, approach to teaching sessions and outcome measures. A detailed outline is covered in Table 1.

Table 1:

The outcome logic model elements: inputs, activities, outputs and outcomes

Inputs: What resources were dedicated to or consumed by the program?
	● Faculty time
	● Trainee time
	● Facility space
	● Data storage on LAPSIM® VR
	● Non-consumed equipment
	○ LAPSIM® VR
	○ LAPSIM® VR operative software
	○ Take-home laparoscopic box trainers
	○ Instruments for take-home lap trainers
	● Consumed equipment
	○ Exercises for take-home lap trainers – EoSIM program
Activities: What does the program do with the inputs to fulfill its mission?
	● Development of online LAPSIM® instructional online learning module
	● Delivery of weekly LAPSIM® training sessions
	● Delivery of monthly surgical active learning sessions
	● Design and promotion of trainee LAPSIM VR credentialling
	● Design and delivery of LAPSIM® take-home lap box trainer program
Outputs: What are the direct outputs of the program activities?
	● Trainee use and overall time spent on the LAPSIM® VR
	● Salpingectomy procedural number on the LAPSIM® VR
	● Trainee attainment of LAPSIM® VR credentialling
Outcomes: What are the immediate and intermediate benefits for the trainees and institution during and after the program?
	● Survey of trainees and trainers
	● Retrospective review of patient laparoscopic salpingectomies – theatre time
	● Retrospective review of patient laparoscopic salpingectomies – primary operator rates

Results

Outcomes of the LAPSIM® VR program

Survey of trainees and trainers

Completed surveys were performed by eight trainees and eight trainers. Common perceptions are described here.

(i) Improvement in skills

Generally, the trainees thought that the practice they had performed on the LAPSIM® VR translated well to the real cases. When asked what they gained during the sessions trainees reflected on both technical skills and on team bonding. Interestingly many commented on the laparoscopic SIM sessions being an important opportunity to touch base with colleagues and support trainee team morale.

Hand eye coordination skills, camaraderie with colleagues (Trainee 1)

Very useful-particularly suturing and instrument use (Trainee 2)

(ii) Impact on supervision in the operating theatre

Only one trainee reported that trainers always asked if they had completed LAPSIM® VR credentialling. The remainder stated that they were never or rarely asked. There was variable opinion in the trainee cohort whether completion of VR salpingectomy credentialling made any impact on trainer decision to allow trainee primary operating in real salpingectomy cases.

Really well - some bosses asked if I had done the training before doing my own ectopic surgery (Trainee 4)

No correlation it seems. I never had a boss ask me what amount of simulation training I had before deciding whether or not I could be primary operator in a lap surgery (Trainee 5)

When asked whether they would allow a trainee to be primary operator if they had completed the LAPSIM® VR salpingectomy credentialling, six trainers agreed they would in principle support this.

Yes, but I probably would have anyway. If we start making this a requirement it will be a good carrot to do the lap sim (Trainer 5)

Trainers wanted to assess and appreciate themselves what the trainees were capable of rather than relying on certificates of competency from the laparoscopic simulation programs.

I talk to them about their previous laparoscopic experience and how they would aim to perform this operation (Trainer 1)

They are on the program and so there to be taught, or they are a promising PHO and deserve an opportunity (Trainer 2)

When asked if the LAPSIM® VR had any appreciable benefits or drawbacks to laparoscopic operating five trainers felt there was not yet discernible benefits or drawbacks observed. Only one trainer reported that registrars appeared to have improved dexterity after initiation of the program.

Retrospective review of patient laparoscopic salpingectomies – theatre time

There were 149 laparoscopic salpingectomies performed for ectopic pregnancy at GCHHS in 2020–2021. Table 2 reports numbers of procedures per year and complicated versus uncomplicated cases. Uncomplicated procedures included cases limited to a unilateral salpingectomy for a tubal ectopic pregnancy only. Complex cases included procedures requiring adhesiolysis and complicated ectopic pregnancies such as cornual, ovarian or heterotopic ectopic pregnancies. There were also cases where a decision was made to perform excision of endometriosis or further surgical procedures such as bilateral salpingectomy during the emergency operative procedure.

Table 2:

Procedural numbers for laparoscopic salpingectomy

	Total	Uncomplicated	Complicated
2020	70	52	18
2021	79	57	22

Laparoscopic salpingectomy for uncomplicated ectopic pregnancies

There were 109 cases of laparoscopic salpingectomy for uncomplicated ectopic pregnancy across the 2-year period. In 2020 the mean time for an uncomplicated salpingectomy for ectopic pregnancy was 52 minutes (range 12–88 minutes) versus 44 minutes in 2021 (range 16–96). For uncomplicated procedures, there was a reduction in operating time of 14 minutes between 2020 and 2021 (p < 0.001, 95% CI: 9–19 minutes) after adjusting for instrument and surgeon level. Operative time was recorded by a theatre nurse at the start and end of each surgical procedure on the electronic medical record and did not include anaesthetic induction, anaesthetic recovery or transfer in and out of the operating theatre.

In 2021 compared with 2020 all surgical groups O&G trainers, STP trainees, ITP trainees and PHO trainees had an improvement in mean operative time to perform uncomplicated and complicated salpingectomies as shown in Figure 2.

Figure 1:

Overall LAPSIM® VR statistics based on trainee surgeon level

When all ectopic pregnancies were considered including cases with added complexity – there was improvement in times across the cohort. After adjusting for complicated procedures and surgical technique there was evidence (p = 0.005) of a reduction in operating time between 2020 and 2021 (difference of 7 minutes, 95% CI: 2–12 minutes).

Retrospective review of patient laparoscopic salpingectomies – primary operator rates

Trainer gynaecologists did the majority of primary operating in both year cohorts as shown in Figure 3. Overall, across the 2-year period 81/149 (54.3%) cases had a consultant gynaecologist primary operator. In 2020 35 of the 70 cases (50%) were performed by consultants. In 2021 46 of 79 cases (58%) were performed by consultants. Surgeon level was adjusted for in the statical analysis of time for laparoscopic salpingectomies.

Figure 2:

Surgical procedural time – uncomplicated salpingectomy per surgeon level

Figure 3:

Overall salpingectomy number per surgical seniority

Discussion

Enhancing the efficiency of operating theatres requires a comprehensive approach beyond the mere acquisition of VR simulators to train surgeons. Simply purchasing simulation equipment does not guarantee improved trainee education or patient outcomes [1,8]. Achieving translational results necessitates careful consideration of the educational frameworks that support the effective use of simulation tools in training, and the integration of those frameworks within the realities of clinical practice. We focused on a specific surgical procedure taught using a mastery learning approach and assessed as part of a credentialling process. Our integrated, multifaceted LAPSIM® VR program was associated with quicker procedure times in the operating theatre, illustrating the potential for educational programs to improve clinical outcomes.

A core tenet of our program was credentialling – requiring trainees to achieve a pre-specified level of confidence prior to being granted primary operator status in real surgical cases. This was based on the assumption that this would lead to safer, faster procedures performed either by credentialled trainees or by their supervising surgeons if the trainees were not yet at the demonstrated level. Credentialling provides trainees with a clear goal and well-defined metrics to strive for. While many programs typically view competence credentialling as an endpoint, we challenged this approach by making it the initial step and primary goal. Whilst most trainees in our program achieved credentialling, there were significant number who did not. Whether this was an issue with training methods or learner motivation is unknown. Notably, the only junior PHO trainees who were primary operators for salpingectomies were those who had completed the credentialling process. Trainees were not forbidden to perform surgical cases as primary operator if they had not completed credentialling as ultimately the discretion of this decision was left to the surgical trainer.

Trainer trust of credentialling appeared to be problematic. Despite many trainers supporting the notion of primary operating after laparoscopic credentialling in surveys this didn’t translate to increased trainee operating. These findings are not isolated to our unit and echoed in research completed by Blackhall et al. [19] Primary operating within our trainer group increased after introduction of trainee credentialling. Determination of trainee ability to be primary operator appeared to rely on preoperative discussion between the trainer and trainee on how many procedures they had previously performed, surgical experience, steps to the operation and comfort level performing the surgery. Most trainees reported no correlation between completing laparoscopic credentialling and ability to be primary operator. This raises an important question – how do we engage our trainers to trust laparoscopic simulation credentialling?

There are nuances in understanding causal relationships within any complex service or educational intervention [20]. Using an outcome logic model may be a more useful approach to comprehensively evaluate simulation educational programs than traditional approaches that seek linear causality [11–14]. Specifically, it affords examination of questions beyond effectiveness of interventions, i.e. feasibility, efficiency, return on investment and more. Our work highlights the need for a greater degree of intentionality in educational program design when it is designed to operate within, and impact upon a health service. The connection between an outcome logic model and translational simulation lies in their shared focus on understanding and improving complex systems, particularly in the realm of program planning, implementation and evaluation [21]. The outcome logic model serves as a framework for comprehending how various components of a program can contribute to the desired outcomes. While the conceptual framework is provided by the outcome logic model translational simulation offers the necessary resources and toolkit for effective implementation.

Our findings present a promising outlook for educational programs aimed at enhancing operating theatre efficiency. To fully harness the potential of this program at our institution collaboration with other surgical specialities is imperative. The opportunity to optimize operating theatre efficiency may be even more significant with procedures such as laparoscopic appendectomy or cholecystectomy given the higher surgical operative volumes of these procedures. Additionally, further research is necessary to explore the factors influencing trainers’ decisions to allow trainee primary operating, barriers to trainee engagement in laparoscopic simulation credentialling, how often credentialling should be performed and explore trainer trust of simulated credentialling.

Conclusion

Introduction of a laparoscopic simulation VR credentialling program resulted in a reduction in operative time for laparoscopic salpingectomies in our institution. Educational programs should aspire to translational, patient-focused outcomes in their design and delivery. Credentialling may be a useful strategy to support clear and transparent communication in surgical goal setting, competence assessment, and directed feedback. Adopting an outcome logic model offers a promising approach to evaluate and complement translational simulation education programs more comprehensively.