The COVID-19 pandemic motivated simulation educators to attempt various forms of distance simulation in order to maintain physical distancing and to rapidly deliver training and ensure systems preparedness. However, the perceived psychological safety in distance simulation remains largely unknown. A psychologically unsafe environment can negatively impact team dynamics and learning outcomes; therefore, it merits careful consideration with the adoption of any new learning modality.
Between October 2020 and April 2021, 11 rural and remote hospitals in Alberta, Canada, were enrolled by convenience sampling in in-person-facilitated simulation (IPFS) (n = 82 participants) or remotely facilitated simulation (RFS) (n = 66 participants). Each interprofessional team was invited to attend two COVID-19-protected intubation simulation sessions. An
There was no statistically significant difference between RFS and IPFS total scores on the
Psychological safety can be established and maintained with RFS. Furthermore, in this study, RFS was shown to be comparable to IPFS in improving psychological safety among rural and remote interdisciplinary teams, providing simulation educators another modality for reaching any site or team.
Simulation-based education (SBE) witnessed a rapid adoption of distance simulation as a result of the physical distancing requirements and travel restrictions brought on by the COVID-19 pandemic [
Previous studies have suggested that high-quality feedback is more important than the method of debriefing [
The aim of this study is to compare RFS with standard IPFS in establishing and maintaining participant psychological safety through a two-arm controlled experimental design. This question helps to inform the use of RFS for low-resource, geographically isolated and cost-conscious situations to deliver simulation for learning and systems improvement [
A total of 11 rural or remote hospitals in Alberta, Canada, were enrolled by convenience sampling in an experimental controlled trial. Six hospitals were enrolled in the standard IPFS arm, and five hospitals were enrolled in the RFS arm. Hospitals located greater than a 2-hour drive away from the nearest facilitator were automatically offered RFS with the ability to switch to IPFS if requested. Of the 11 hospitals enrolled, none requested to switch arms during the trial. Local institutional approval was obtained prior to enrolling each hospital in addition to obtaining approval from the Health Research Ethics Board of Alberta (HREBA.CHC-20-0057).
Rural and remote hospitals’ participants were recruited through a rural Alberta nursing educator e-mail list. Interested rural educators acted as the local site champion and further recruited local participants through a combination of methods, including e-mails, physical sign-up sheets, posters and snowball sampling. Site champions were also tasked with technical and equipment set-up of the
Each hospital was offered to invite their staff to take part in at least one of two simulation sessions of the same
An
Each simulation session (IPFS or RFS) was completed within 2 hours including a 30-minute technology set-up period. For each session, the facilitators followed a standardized protocol: 1) introduced the study, 2) administered the pre-intervention
The study introduction and pre-briefing were scripted for standardization and allowed only minor changes in word choice or speaking style [
The measurement of the primary outcome variables was the participants’ perceived psychological safety rating both pre- and post-simulations, as measured by the sum score of six self-reported items of psychological safety at each time point. These questions were taken from the first construct of
The pre- and post-intervention data were pooled into IPFS and RFS groups for between-group analysis and were not matched by individual participants. Prior to analysis, the Likert-scale items on both the abbreviated and complete Edmondson Psychological Safety survey were coded as never = 0, sometimes = 1 and always = 2, with negatively worded items appropriately reverse-coded prior to creating sum scores as a measure of total psychological safety rating for each instrument.
Due to normality violations, differences between IPFS and RFS groups for the
Prior to exploring the abovementioned between-group differences, the Scheirer–Ray–Hare test, a non-parametric alternative to a factorial analysis of variance (ANOVA), was used to examine if an interaction effect existed between timing of the simulation (pre/post) to the type of simulation (RFS/IPFS) on the total psychological safety score of the
The self-reported data from the site champions for the
All analyses were conducted in R Version 4.1.2 [
Potential harms were mitigated through the use of rigorous inclusion and exclusion criteria and informed consent process. Specifically, patient harm was avoided by excluding hospitals where simulation sessions would negatively impact patient care either due to patient care priorities, or critical shortage of staffing. Unintended consequences with the use of video (e.g. use for performance reviews or evaluation) were mitigated through secure video storage on a password-protected drive which was only accessible to the research team in compliance with data research requirements by the research ethics board. While the video recordings show identifiable participants, all data from the study have been aggregated such that any linkage between type of professional or other demographic information has been anonymized.
In total, 148 participants were enrolled throughout the 11 rural and remote hospitals through 20 simulation sessions between November 2020 and April 2021. Of the 148 participants, approximately, 55% (n = 82) were included in the IPFS arm and 45% (n = 66) in the RFS arm.
Baseline demographic characteristics of the site champions and participants in both IPFS and RFS arms were collected as part of the pre-session surveys (
Site Champion and Participant Demographic Characteristics
Site Champion | ||
---|---|---|
Nurse |
||
3 |
5 |
|
Time in Practice (years, %) |
||
Prior Simulations (number, %) |
||
Nurse |
||
34 |
45 |
|
Time in Practice (years, %) |
||
Prior Simulations (number, %) |
IPFS – in-person-facilitated simulation, RFS – remotely facilitated simulation
Prior to exploring between-group differences, the Scheirer–Ray–Hare test, a non-parametric alternative to a factorial ANOVA, indicated that no significant interaction effect existed between the timing of the simulation (pre/post) and the type of simulation (RFS/IPFS) on the total psychological safety score of the
Analysis of between-group difference (Wilcoxon rank-sum test) at baseline, i.e. pre-intervention revealed that there was no statistically significant difference between RFS (Mean = 9.32, SD = 2.15) and IPFS (Mean
Abbreviated Edmondson Psychological Safety Instrument Post-Scores by Question
Question | Mean/SD | p-value |
---|---|---|
1. When someone makes a mistake in this team, it is often held against him or her. | RFS; M=1.77, SD= .49 |
|
2. In this team, it is easy to discuss difficult issues and problems. | RFS; M=1.72, SD=.54 |
|
3. In this team, people are sometimes rejected for being different. | RFS; M=1.85, SD=.44 |
|
4. It is completely safe to take a risk on this team. | RFS; M=1.50, SD=.59 |
|
5. It is difficult to ask other members of this team for help. | RFS; M=1.67, SD=.66 |
|
6. Members of this team value and respect each others’ contributions. | RFS; M= 1.85, SD=.40 |
Statistically significant (p < 0.05), SD – standard deviation
Furthermore, for participants in both IPFS and RFS arms, there was a statistically significant difference in the
Secondary outcomes compared RFS and IPFS on self-reported measure of psychological safety based on the
A signed-rank test using the Pratt method to handle zero differences revealed that there was a statistically significant increase in psychological safety from the pre-test scores (Mean = 30.73, SD = 5.33) to post-test scores (Mean = 34.64, SD = 5.90) on the
Psychological safety can be established and maintained with RFS. Furthermore, RFS is equivalent to IPFS in improving psychological safety among rural and remote interdisciplinary teams [
RFS falls within the growing field of telesimulation which could benefit from standardized systematic methodology for successful implementation [
This study supports the use of virtual facilitation in SBE by establishing potential equivalence of psychological safety with virtual facilitation compared with traditional in-person facilitation. While the reliability of self-reported survey data is often questioned due to its subjective nature, psychological safety is inherently subjective and self-defined by the participant. Therefore, this study benefits from the ability to match a subjective tool with subjective outcome measures.
This intervention results from the combination of recently available technology and immediate need for remote learning [
Recent adoption of telesimulation has generated many questions regarding quality, feasibility, implementation strategies and outcomes [
The literature identifies a key feature of successful distance education as having facilitators who are skilled in facilitation and technology management [
While there has been a bourgeoning amount of literature describing telesimulation to support rural medical education in response to the COVID-19 pandemic, we are also seeing a shift in the application of remote facilitation to include urban centres, outreach programs and non-clinical groups [
A limitation of this experimental trial is the inability for complete randomization of hospitals due to geographic constraints of the rural and remote environment. This trial was conducted across vast geography where winter road conditions and travel restrictions precluded facilitators from driving to farther hospitals such that smaller remote hospitals were disproportionately enrolled in the RFS arm. Trial findings may reflect a difference in geography and simulation experience and not the facilitation method. This limitation was mitigated by enrolment of several geographically closer sites in the RFS arm during the peak of the pandemic when in-person learning events were not possible. This limitation of convenience sampling may explain the differences between IPFS and RFS participant and site champion baseline demographics with the IPFS arm having slightly more simulation experience and being comprised of more trainees when compared with the RFS teams (
Another limitation of this trial is the variability between facilitators who may have different facilitation styles. We attempted to mitigate this limitation by selecting experienced facilitators with similar practice backgrounds (rural family physicians and rural nurses) who have all undergone the same standardized training in facilitation and psychological safety; however, inter-facilitator differences are unavoidable. Of the team of seven facilitators, four facilitated both IPFS and RFS sessions, while one facilitated only IPFS, and two facilitated only RFS.
A further limitation of this trial is that the validated Edmondson Psychological Safety instrument was abbreviated for expeditious participant data collection and to increase survey completion rates which may have impacted the validity and reliability of the
However, the
Another limitation of the study’s design was that, due to resource limitations, the facilitators did not assign individual participant codes to each participant, limiting the ability to match samples of pre and post data for both RFS and IPFS sessions. Consequently, researchers were unable to match pre and post data at the individual participant level, which is required to conduct a more powerful paired samples statistical analysis.
Our study advances this existing understanding of psychological safety in telesimulation by providing empirical evidence through an
As the field of telesimulation grows within SBE, there will be a need to establish best practice standards [
The authors thank Drs. Gavin Parker and Sheena CarlLee for their comments and insights.
SR, KS, MJ, ADM, NT, SW, and TC collected data; SR, SR, and AK analyzed the data; all authors contributed to the experimental design and writing of the manuscript.
This research was funded by the Rural Health Professions Action Plan (RhPAP) of Alberta and in-kind funding through the Alberta Health Services Provincial Simulation Program (eSIM).
The data analysis is available upon request.
This research protocol was approved by the Health Research Ethics Board of Alberta (HREBA) – Community Health Committee (CHC) (ethics identification HREBA.CHC-20-0057).
None declared.
Questions | Answers |
---|---|
When someone makes a mistake in this team, it is often held against him or her. | Never / Sometimes / Always |
In this team, it is easy to discuss difficult issues and problems. | Never / Sometimes / Always |
In this team, people are sometimes rejected for being different. | Never / Sometimes / Always |
It is completely safe to take a risk on this team. | Never / Sometimes / Always |
It is difficult to ask other members of this team for help. | Never / Sometimes / Always |
Members of this team value and respect each others' contributions. | Never / Sometimes / Always |
Questions | Answers |
---|---|
When someone makes a mistake in this team, it is often held against him or her. | Never / Sometimes / Always |
In this team, it is easy to discuss difficult issues and problems. | Never / Sometimes / Always |
In this team, people are sometimes rejected for being different. | Never / Sometimes / Always |
It is completely safe to take a risk on this team. | Never / Sometimes / Always |
It is difficult to ask other members of this team for help. | Never / Sometimes / Always |
Members of this team value and respect each others’ contributions. | Never / Sometimes / Always |
Problems and errors in this team are always communicated to the appropriate people (whether team members or others) so that action can be taken. | Never / Sometimes / Always |
We often take time to figure out ways to improve our team’s work processes. | Never / Sometimes / Always |
In this team, people talk about mistakes and ways to prevent and learn from them. | Never / Sometimes / Always |
This team tends to handle conflicts and differences of opinion privately or off-line, rather than addressing them directly as a group. | Never / Sometimes / Always |
This team frequently obtains new information that leads us to make important changes in our plans or work processes. | Never / Sometimes / Always |
Members of this team often raise concerns they have about team plans or decisions. | Never / Sometimes / Always |
This team constantly encounters unexpected hurdles and gets stuck. | Never / Sometimes / Always |
We try to discover assumptions or basic beliefs about issues under discussion. | Never / Sometimes / Always |
People in this team frequently coordinate with other teams to meet organization objectives. | Never / Sometimes / Always |
People in this team cooperate effectively with other teams or shifts to meet corporate objectives or satisfy customer needs. | Never / Sometimes / Always |
This team is not very good at keeping everyone informed who needs to buy in to what the team is planning and accomplishing. | Never / Sometimes / Always |
This team goes out and gets all the information it possibly can from a lot of different sources. | Never / Sometimes / Always |
We don’t have time to communicate information about our team’s work to others outside the team. | Never / Sometimes / Always |
We invite people from outside the team to present information or have discussions with us. | Never / Sometimes / Always |
Members of this team help others understand their special areas of expertise. | Never / Sometimes / Always |
Working with this team, I have gained a significant understanding of other areas of expertise. | Never / Sometimes / Always |
The outcomes or products of our work include new processes or procedures. | Never / Sometimes / Always |