International Journal of Healthcare Simulation

Introduction

As the COVID-19 pandemic entered its fourth wave in the fall of 2021, healthcare systems globally were straining to meet the surging demand of patients who required critical care resources [1]. The increased stress experienced by the healthcare system, coupled with the level of uncertainty present during the pandemic, brought the reality of having to potentially enact critical care triage (CCT; in Alberta represented by the CCT framework) [2,3]. As a result, it became crucial to be prepared for the worst-case scenario of overwhelmed critical care capacity and being able to ensure the most patients benefited from the life-saving critical care resources available [4].

It is essential to have planned and predetermined approaches for healthcare professionals to guide responses when allocating resources and ensuring the most equitable process is consistently followed [5,6]. The CCT framework provides a guide to execute clinical decisions; however, the conceptual operationalization of the CCT framework and preparation presents several challenges. While using evidence-based triage frameworks is pragmatic and provides a systematic approach to decision-making [7], there is uncertainty about its application from theoretical conceptualization into practice.

While many current articles describe the content of CCT frameworks [8,9], they have not described (a) the value in beta testing a CCT framework through online facilitated simulation prior to implementation to identify potential latent safety threats, and (b) have not had to consider the implications for time-sensitive decision-making in testing the framework across the geographical span of both rural and urban centers, ensuring equitable distribution of resources and access to critical care, regardless of patient location in a healthcare system. In addressing this imminent need, the simulation and human factors teams collaborated with provincial clinical experts in critical care to develop online facilitated scenarios, designed to identify potential latent safety threats that may have occurred.

The goal of this article is to describe the additive benefits of combining both a simulation and human factors methodology demonstrated through a quality improvement case study. Furthermore, the sharing of insights into how these unique approaches can be used for simulation-based testing, and providing recommendations for simulation practitioners who would want to design and enact similar exercises using online facilitation for broader organizational learning.

Background

This quality improvement case study took place in Alberta Health Services (AHS), the largest Canadian provincial fully integrated health system encompassing 106 acute care hospitals and 21 dedicated adult intensive care units that provide population-based care to an estimated population of 4.4 million spread across a large geographical area [10,11]. Within the provincial healthcare system, the Strategic Clinical Networks™ (SCNs) exist to advance improvements in specific areas of health and work provincially to develop integrated, sustainable solutions to complex, multidisciplinary challenges including one that focuses on critical care. The SCNs™ collaborate with patients, families, clinicians, operations, communities and researchers to address the most pressing healthcare challenges facing our organization [12,13].

In the spring of 2020 with the declaration of the COVID-19 pandemic, the AHS CCT framework was developed to ensure that triage decisions are ethical, fair and transparent, using a pre-developed and objective framework to guide healthcare professionals in the unthinkable task of CCT. As all individuals have equal moral worth, the principle of formal equality is used when several patients have an equal likelihood to benefit from critical care. Anchoring the triage process is the Capacity to Benefit: the best action when demand for scarce critical care resources exceeds supply, with the goal to save the greatest number of lives possible. Please see supplementary material for a summary of the AHS CCT triage framework [3]. The testing of the framework was further compounded by the provincial approach taken to support the vast geography contained under a single healthcare system. Equitable access to care and resources could not be limited by patient location. Leadership and clinicians were required to coordinate across large provincial services. Logistically, the unique requirements of using online facilitated simulation had to provide a modality where engagement of participants respected COVID-19 physical distancing and allowed for individuals to participate from all five zones of the province, including all levels of urban, rural and remote areas.

To ensure consistency and provincial perspective to CCT, the Critical Care SCN™ engaged and collaborated with the AHS provincial simulation and a provincial human factors program. Proactive testing of a CCT framework using simulation-based education and human factors is an emerging methodology that remains limited in the literature [4]. Human factors are best described as a body of knowledge about human abilities, human limitations and other human characteristics that are relevant to organizations and workplaces. When combined with simulation-based activities, applications can focus on the design of tools, systems, new processes, strategic planning and optimization of an inherently complex healthcare system [14,15]. In 2022, Kochel and colleagues used simulation alone to evaluate the acceptability, feasibility and reliability of an institutional triage framework based on German guidelines during the COVID-19 pandemic. They found the framework to be realistic and acceptable, while emphasizing the importance of interdisciplinary cooperation and education prior to having to use the framework [4]. Figure 1 visually shows how the triage framework would entail complex decisions through a team-based approach, using predetermined processes for selection and prioritization of adult patients into critical care beds across the province.

Figure 1:

Critical care triage protocol.

Methods

Development and planning of simulation exercises

Due to the expediated timeframes required to complete these simulation exercises, a modified methodology to the development and planning phases was implemented, incorporating elements from frameworks developed for Systems Focused Simulations and known project management approaches [16–21]. The project management plan started with the development phase, where an intake meeting included a needs assessment, clarifying the goals, objectives, scope, timelines and key stakeholders. Brainstorming sessions in subsequent meetings helped refine anticipated concerns, potential gaps and process issues that might arise and included these for testing [16–21].

Goals and overarching objectives

The overarching objectives of the CCT simulation exercises were threefold: (1) assess and define current resources, communication pathways, algorithms, cognitive aids and checklists, (2) identify gaps and mitigation strategies to better inform the operationalization of CCT and (3) develop familiarity and comfort with the process for CCT and validate role clarity and responsibilities. Specific detailed objectives for each scenario, which were determined by facilitators, are listed in Table 1. However, an unintended outcome was the learning that occurred from session to session as to how to best utilize and leverage the technology and approach to online facilitated simulation.

Table 1:

Summary of detailed objectives for each online simulation.

	Scenario 1	Scenario 2	Scenario 3
Simulation objectives	Enact the CCT consultation for admission Familiarize stakeholders with their roles and application of frameworks using CCT documentation Explore decision points for triaging patients: how to decide and manage the queue Communication and testing the process for electronic queue and patient prioritization	Educate and test the Triage Dispute Team: • Convening process • Agenda for meeting • Process for dispute resolution • Documentation • Communication back to stakeholders Allow Triage Coordinator to present case to the Decision Committee	Allow teams to work an end-to-end scenario using the documentation (Eligibility, Exclusion Criteria) and making triage decision Triage Coordinator Role • Communication between frontline providers and patient placement team • Presentation of case to Decision Committees • Documentation • Evaluate role education received prior to simulation Dispute resolution process – Convene Decision Committee Re-test bed allocation process from first simulation

Scope and timelines

Due to the constraints of urgency and the looming threat of having to enact the framework, the simulation was confined to the adult population only as COVID-19 at the time was primarily affecting adult patients. The pediatric inclusion criteria and response outlined in the protocol were considered out of scope. As well, the exercise would not be able to fully explore moral and ethical distress of both patient and staff and was excluded.

Timelines from the initial request for assistance to the first exercise were less than 1 week (17 September/23 September 2021). The second exercise occurred 4 weeks later, on 21 October 2021, with the final end to end exercise occurring on 18 November 2021 (see Figure 2).

Figure 2:

Timeline CCT request September to November 2021.

Key stakeholders

The key stakeholders both developed the framework, held a fundamental role in its operationalization and were crucial in the initial planning stages. For the simulation exercises, stakeholders fulfilled roles as facilitators, participants and observers. Facilitators included representative from the Critical Care SCN™, eSIM (Educate, Simulate, Innovate, Motivate) Provincial Simulation program, Human Factors (HF), and Referral, Access, Advice, Placement, Information & Destination (RAAPID), AHS’s single point of contact for physicians and healthcare providers to access appropriate and timely advice, referral, admission, repatriation and consultation for patients.

Participants and observers

Participants in the simulation exercises represented their professional role within the system, and included operational leaders, Integrated Operations Command Centre members, frontline clinicians (physicians and nurses), RAAPID staff, along with patient and family advisors. One week prior to the simulation exercises, each participant received an education package that included the CCT Framework, standard operating procedures, educational videos, documentation and resources pertinent to their role. To be fully prepared, there was an expectation of these participants to have reviewed all these resources prior to the simulations. Familiarity and understanding of the protocols and resources varied amongst the groups, as some aspects of these documents were in use as part of clinical practices (sequential scoring) and some participants had been involved in the design of the framework. Triage Coordinators attended two 1-hour online pre-education training sessions which included case scenarios intentionally scheduled at intervals prior to the simulation sessions that mimicked proposed timelines for activation of CCT.

Across all three simulation sessions, greater than 60 observers attended including Clinical Ethics, Emergency Disaster Management and stakeholders who contributed to the development of the CCT framework. Observers were used to identify and record system issues. Their feedback was elicited through the debriefing and post-simulation evaluation feedback forms. Table 2 provides an overview of the stakeholders involved in simulation exercises by role and role description.

Table 2:

Stakeholders involved in simulation exercises by role with role description.

Facilitators (N = 13)	Participants (N = 33)	Observers (N = >60)
Role	Role	Role
• Developed critical care process/content expertise • Developed patient scenarios and debriefing outline • Lead the simulation (SCN) • Monitored chat boxes • Supported the debriefing • Documented outcomes	• Scenario 1: Have intimate knowledge of CCT framework and played a role realistic to their clinical background • Scenario 2: Content experts in bed allocation eligibility • Scenario 3: Participants who were not involved in the development of CCT framework	• Provided clinical expertise • Medical legal implications • Ethical considerations • Patient and family perspective
Role description	Role description	Role description
• Simulationists • Human Factors Specialists • Critical Care Leadership (Educators, Management) • RAAPID Management	• Critical Care Leadership (Physicians, Nurses, Management Zonal and Provincial Executive) • Emergency Department physician leads and nurses • CC Triage Nurses • RAAPID leads (nurses, EMS)	• Critical Care Leadership Site Leadership for Critical Care and Emergency Departments (physicians and nurses) • Ethics • Patient Family Centered Care Management • Legal Representatives (AHS)

Results

The iterative process facilitated by human factors and simulation, allowed the SCNs to refine the CCT framework and documents by using them in a realistic manner with clinicians. Challenges with the documents were observed and addressed, and end-user feedback was incorporated into both the design of the documents and process itself. Critical dashboard design changes included the clear differentiation of patients who were ineligible for beds, as well as the addition of a free text status column to provide details such as transport requirements and site changes. This ensured greater situational awareness across the province.

Examples of additional reference documents included a visual aid for all provincial site-based critical care sites, outlining ICUs that offered specialty services such as Cardiac, Neurosurgery and Trauma. A CCT Quick Reference document and a Frequently Asked Questions tool provided supplemental resources for leadership and frontline clinicians. Communication issues in the handover processes from the triage coordinator to the eligibility review team became more structured with the addition of a formalized huddle agenda and handover tool with embedded scripts. Condensed workbooks were created to address the difficulty noted with scoring triage eligibility and variances in the frontline clinicians’ understanding of the Sequential Organ Failure Assessment (SOFA) score. This change allowed the Point of Care Triage team to better navigate through the ineligibility criteria. Additionally, inserting calculation tools into the assessment document facilitated a shared understanding of the patient’s condition. By explicitly labeling roles and responsibilities in the reference documents, team members and leadership gained an understanding of who would have decision-making power. In total, 50 safety threats were identified, 4 design changes were made to the dashboard displaying patient queue information, and 6 additional reference documents and 3 new documents were created to support end users completing CCT.

The sequential nature of the simulations allowed for an iterative design process to occur with the simulation process as well. The online facilitated simulation process was adapted to ensure that the simulation unfolded in a realistic manner. Data and dashboards were updated in real time based on participants’ decisions. Facilitators needed to be able to converse amongst each other, collect and collate participant and observer feedback, adapt the simulation and identify debriefing discussion themes, all as the simulation unfolded. To do this, these roles were spread amongst more facilitators: one to update dashboards in real time, one to organize observers’ feedback and multiple facilitators to keep the simulation going in a realistic manner. In the first simulation, one facilitator was assigned to run both the dashboards and provide patient information verbally as part of the scenario. This was not a manageable workload and the team transitioned in real time to have another facilitator take on the role of updating the dashboards and other visual presentation of data. The following sessions incorporated more equal division of labour among facilitators. Furthermore, in the first simulation the Zoom meeting chat function was used for both observers to enter comments and questions and to provide information relevant to the simulation. This caused some confusion and missed information relevant to the scenario. In future simulations, a separate messaging application was utilized for facilitators to communicate with each other. Observers were encouraged to share their findings in the debriefing checkpoints and on the observer, checklist submitted at the end (see Figure 3).

Figure 3:

Iterative process showing changes made to approaches between each of the three online simulations.

In summary the extensive data collected on the gaps and latent safety threats that were identified (highest risk and greatest impact) enabled validity testing for frameworks and documents, provided feedback on the education and training delivered to ensure the CCT framework met the highest ethical and patient care standards.

Discussion

The COVID-19 pandemic is a crisis that has significantly impacted the Canadian healthcare system [31]. As of March 2023, the virus has not been contained, and while vaccinations are underway in Canada, future logistical and distribution challenges mean COVID-19 is an ever-present concern [19]. Preparing for the unknown is extremely difficult given the unpredictability of multiple waves of the COVID-19 pandemic [32]. Valuable insights from this facilitated exercise highlighted the critical need for testing and validating the CCT tools within a strained healthcare system. Healthcare simulation has emerged as a valuable tool in preparing these teams for pandemic responses and potentially other emergency situations in a safe and controlled environment. The addition of human factors expertise augmented the simulation exercise by applying a human error lens to help identify gaps in the process and potential for confusion in the CCT documentation and tools.

The global pandemic emphasized the importance of leadership and staff preparedness in times of crisis [33] and the value of testing the CCT framework prior to activation as a strategy to support emergency management (EM). While expertise in the creation and development of CCT framework, design, frameworks and resources has been previously well described [34–36], there has been scarcity of literature describing examples of applications of simulation and human factor approach to test and evaluate the real-time execution of newly developed CCT frameworks. The literature, however, highlights how EM methodology plays a key role in preparing Canadian healthcare organizations for disasters and emergencies [33]. During the COVID-19 crisis, healthcare systems formed integrated operations committees, fashioned to mimic EM structure [34]. These committees helped to maintain plans while ensuring logistics and communication systems functioned effectively in times of crisis. Much like emergency disaster preparedness exercises [33,34], the online facilitated scenarios were designed to prepare clinical operational leaders, with frontline medical experts making complex care decisions through a team-based approach. This was accomplished by using prioritization documents and clear processes that informed revisions to the framework and education modules, development of cognitive aids and the need for additional re-testing following those revisions. Regrettably, empowering and training healthcare leaders in EM techniques can be problematic, as they typically lack the specific knowledge, experience and preparedness in managing large-scale crises effectively [35]. Once trained, keeping the mastery level of knowledge of disaster EM, wanes in the time that occurs from the last time it was practiced or enacted. Recommendations from this facilitated exercise would include future annual exercises for the CCT, to maintain leadership preparation, familiarity of the protocol and system readiness.

The COVID-19 pandemic has forced many healthcare systems to adopt online solutions. This case study had to pivot and adapt to the challenges of a provincially integrated system, across multiple units, departments, hospitals and geographic zones. Simulation can be adapted to these online environments, providing healthcare providers with similar levels of training and preparation as in-person simulation exercises. With technology, these simulations exercises can be accessed from anywhere, allowing healthcare providers to participate in training programs even during periods of lockdown, quarantine and geographical barriers. Our online facilitated simulations build on the approach of Dogan et al. (2021), that have used ‘Visually Enhanced Mental Simulation’ (VEMS) where teams rehearse all the steps of scenario in an interactive environment without physically doing it, using both a ‘think aloud technique’ and team decision-making approach [36]. Using the VEMS technique, our participants were able to enact what a CCT response would look like without having the pressures of real patient decisions. Applying these complex decisions across the geographical span of both rural and urban centres, ensured equal distribution of resources and access to critical care, regardless of patient location in the province. Through the beta testing of the CCT framework using online facilitated large-scale simulation and engagement of stakeholders across the entire healthcare system, the team was able to develop recommended actions (fix, change, improve) for time-sensitive decisions based on the highest risk and highest impact to the healthcare system.

Overall, healthcare simulation is a crucial tool for preparing healthcare teams for pandemics and other emergency situations, developing leadership skills and facilitating online facilitated learning. As the world continues to navigate the challenges of the COVID-19 pandemic and beyond, simulation will remain a vital component of healthcare training and preparedness.

Limitations

This study is subject to some identified limitations. Only 33 participants and 60 observers took part in the simulations, thus limiting access to data and generalizability to all relevant clinical groups across Alberta. The results of the simulations exercises have been generated through a set of clinician responses and assumptions with one Canadian provincial healthcare system; therein, care should be taken in generalizing these findings to other settings and contexts.

Further, the quality improvement project used a cross-sectional design and included a convenience non-probability sample which may have resulted in sampling and selection bias of participants and feedback on the CCT framework. All simulations sessions were held remotely, which may have influenced the rapport developed between the facilitators and participants and potentially influenced the data obtained in the debriefing. Also, participants may have had some degree of recall bias, recalling either only incredibly positive or extremely negative experiences during the simulation session. In addition, the limitations to the realism of the online facilitated simulation exercise could not fully replicate what clinicians in the real world would have to face, balancing the care of current patients while also being involved having to treat for incoming and current critically ill patients. Limitations of the scope of the study were due to the constraints of urgency and the looming threat of applying the framework. These confines included not being able to assess the pediatric component of the CCT due to COVID-19 primarily effecting adult patients and fully exploring moral and ethical distress of both patient and staff. Further studies and research into CCT frameworks could explore using simulation to evaluate pediatric inclusion criteria and decision-making. Another limitation of the simulation exercise design was the potential for confirmation bias concerning the appropriate level of educational preparation received on the CCT framework. At the time of the testing, many leaders, educators and clinicians in the participant roles engaged in the development of the process and familiar with the roles, responsibilities and documents. (Prior experience with CCT framework: Scenario 1 N = 50%, Scenario 2 N = 83%, Scenario 3 N = 50%). Pressures on the frontline prevented the team from pulling human resources away from the bedsides of real patients. The design of the exercise was also constrained by having to condense the timeframe into a 2-hour exercise, so may or may not have accurately reflected real-time decision-making.

Finally, a key assumption of this case study was that the changes made to the framework and documents following the simulations could have direct impact on the framework. As these items were never operationalized in a real context because of declining system stress, we can only extrapolate that the themes and latent safety threats identified in the three simulations could improve clinician ease, understanding and application of the CCT.

Conclusion

In the event of a pandemic, preparedness is crucial. No one has been trained for a situation in which our healthcare system is strained to meet the surging demands of patients who require the resources of critical care. The overall pattern of the coronavirus pandemic so far has been a series of COVID-19 waves: surges in new cases followed by declines, new variants of concern and no predictable end in sight. Therefore, it is imperative that healthcare professionals are supported to meet the decision-making challenges they may face.

The bridging of both simulation and human factors teams brought expertise in preparing systems for real-time testing of the CCT framework, resources, efficacy and usability. It is anticipated that the organization’s learnings from these novel simulations give a more realistic appreciation of what clinicians would experience if the CCT framework became operational, allowing for a proactive approach to ensure healthcare system preparedness and patient safety. Furthermore, this experience has informed simulation sessions that use online facilitated simulation and future large-scale simulations will benefit from the learnings from this project.