International Journal of Healthcare Simulation

Eligibility criteria

Due to the relative novelty of the subject, we included for the qualitative descriptive analysis all published and unpublished works regardless of article type, language or date of publication.

For the quantitative analysis, we included only randomized clinical trials. We based the search strategy on the PICO (population, interventions, comparisons and outcomes) model. The population was adult healthcare learners of any sex and specialty who underwent SBE using deception and were compared to other learners who did not experience deception. To be included, a trial had to use a defined outcome with an objective measurement tool relating to learners’ performance.

Information sources and search strategy

Because deception use in SBE in healthcare is an emerging field, we searched several resources to maximize the inclusion of all relevant studies. We searched Medline, PubMed, Embase, Cochrane, Cinahl and Mednar from conception up to the date of search (December 2023) and looked at the reference lists of included studies and previously published reviews. We undertook forward citation tracking using SCOPUS to identify further eligible studies or reports.

We built the search strategy with a medical librarian (AF) and conducted the search in Medline and applied it to other databases accordingly. Our systematic search strategy combined three concepts: deception, simulation-based training and the medical field (in a broad sense). We captured the ‘simulation training’ as a concept by including common analogous concepts such as ‘computer simulation’, ‘computer-assisted instruction’, ‘programmed instruction’, ‘simulated patient’, ‘standardized patient’ and consistent keywords with truncation such as interactive training, virtual or augmented reality, and computer modelling. For the ‘deception’ concept, we broadened the search terms to include ‘misconduct’, ‘false’, ‘deceiving’, ‘lie’, ‘trick’, ‘emotion’, ‘surprise’, ‘ethics’ and ‘cognition’. Finally, to focus our search on healthcare education, we combined topics of ‘internship and residency’, ‘clinical clerkship’, ‘medical education’, ‘nursing education’, ‘health occupations students’ and corresponding keywords. The detailed search strategy is provided in Appendix 2.

Selection process

The database search identified 9840 articles in the literature. Reference list checking and hand search identified an additional eight articles that were imported to Zotero. After duplicate removal, two researchers (JS, AK) independently reviewed titles and abstracts of the remaining 4964 records and discussed inconsistencies until obtaining consensus. A total of 4935 were excluded after the initial screening. Next, they independently screened the remaining 29 full-text articles for inclusion. In case of disagreement, a consensus was reached by discussion. When necessary, a third researcher (NS) was consulted. A total of 17 texts were excluded for an irrelevant subject, discussion of deception out of the simulation context, discussion of other aspects of simulation or assessment of unannounced simulation patients. At the end, 12 articles were included, from which two were randomized controlled trials (RCTs) and used for the quantitative analysis. The other articles were included in the qualitative analysis and consisted of five reviews (Table 3), one prospective study, one mixed methods study, one guideline text, one letter to the editor and one response to the latter. A PRISMA flow diagram was created to illustrate this process (Figure 1).

Figure 1:

PRISMA diagram showing the detailed search strategy in relation to the effect of deception in simulation-based education in healthcare.

Data collection process

A data extraction sheet was developed and piloted on five randomly selected included articles and then refined. After finalizing the extraction sheet, one reviewer (AK) performed the initial data extraction for all included articles, and a second reviewer (JS) checked all proceedings. A third reviewer (NS) reviewed data extraction and resolved conflicts. We contacted study authors when the articles contained insufficient or unclear information.

Data items

Data were collected from the 12 included articles in tables. For each article, the following items were recorded when applicable: author, year, study type, study population and sample size, aspect of deception described, randomization process, procedure simulated, deceptive action and its variable, outcome and effect.

Risk of bias and quality assessments

We evaluated the quality of evidence with the Medical Education Research Study Quality Instrument (MERSQI tool). It was developed and validated to measure the methodological quality of educational research studies. It is a six-domain score graded over 18 points, which evaluates study design, sampling, type of data, evaluation instruments, data analysis and outcome [8]. We assessed the risk of bias in the articles included using the Cochrane risk of bias (RoB) tool [9]. Two reviewers independently did both appraisals (AK, JS), and discordance was resolved by discussion until a consensus was reached.

Effect measures and synthesis methods

Descriptive synthesis was used for the articles that were included in the qualitative analysis concerning benefits, risks, origin, success contributors, ethical issues, and determinants of deception in SBE in healthcare.

The two RCTs included in the quantitative analysis had similar designs. The outcome variable was the maximal modified Advocacy-Inquiry Scale (mAIS): a score that measures individuals’ response strength when challenging authority. It is a 6-point ordinal scale ranging from 1 (say nothing), the lowest response, to 6 (attempts to take over the case or other decisive action), the highest. This modified scale (Appendix 3) was validated in previous studies [10,11]. Multiple observations for the same outcome (repeated measurements) were conducted, and the maximal value was assigned for each learner as outcome measurement to allow the capture of any minimal benefit in terms of AIS modification. All participants had the same number of repetitive measures; thus, we did not find any risk in outcome measurement from this strategy. Measurements were made on the same scale across the included studies.

We used the mean difference (MD), a standard statistic that measures the absolute difference between the mean value in two groups. We searched each group for the mean value of the outcome measurement, standard deviation (SD), and the number of participants to perform the meta-analysis for continuous data using the MD. However, the authors of the two RCTs reported only medians, interquartile ranges (IQRs) and ranges (Table 1).

Table 1:

Outcomes (mAIS), as Friedman et al. reported in the RCT

	Intervention					Control
Study	Median	IQR	Min	Max	N	Median	IQR	Min	Max	N	CI 95%	P
Friedman, 2015	3.0	2.2–4.0	1.0	5.0	17	3.5	3.0–4.5	2.6	6.0	17		0.06
Friedman, 2022	4.0	3.0–4.0	2.5	5.0	22	5.0	4.5–5.1	4.0	5.5	22	0.7–1.4	<0.001

IQR: interquartile range; CI: confidence interval; mAIS: modified Advocacy-Inquiry Scale; RCT: randomized controlled trial.

We tried to contact the corresponding author of both articles without success. Consequently, we used Wan and colleagues’ approach to approximate SD and to impute a missing mean value from the sample size, the median, range and IQR of each group in the two studies [9,12] (Appendix 4).

Calculations for mean differences, confidence intervals and the quantitative analysis were conducted using RStudio 2021.09.0 + 351 ‘Ghost Orchid’ Release, and the ‘meta’ package (Appendix 5). Results are presented in Table 2.

Table 2:

Estimated means and standard deviations of mAIS by the Wan and colleagues’ methodology [11]

	Intervention			Control			Difference
Study	Mean	SD	N	Mean	SD	N	MD	CI
Friedman, 2015	3.05	1.28	17	3.81	1.09	17	−0.76	[−1.56; 0.04]
Friedman, 2022	3.69	0.72	22	4.84	0.43	22	−1.15	[−1.50; −0.80]

SD: standard deviation; MD: mean difference; CI: confidence interval; mAIS: modified Advocacy-Inquiry Scale.

Origin of deception

Deception can be applied in two ways: by omitting information or other aspects of the environment (equipment, personnel) or by providing false information or faulty equipment [13,19]. Deception of learners results from a misunderstanding of the educational model due to insufficient preparation for the activity. It can also result from unexpected content during the activity, like altered equipment, unexpected action of a confederate or a deterioration of the patient’s condition. Furthermore, it may be caused by the absence of immediate consent between learners and educators (e.g. in situ unannounced simulation) [1].

Determinants of deception

The learners’ experience level is an essential determinant of the planning of the educational activity [18]. Thus, the level of complexity, fidelity and realism of the simulation activity has to be modulated based on the need, previous experience and learning outcomes of the targeted learners [1]. Content and face validity of the initial situation are essential concepts that can influence learners’ decision-making processes and interventions during SBE activities. A highly realistic situation could be helpful to get the highest performance from experts. Reliability is another crucial element in ensuring similar learning experiences between groups. Consistent use of a particular approach, technology, detailed scripting of key scenario elements and setup help the trainers to respond to the variability in learners’ decisions and actions. Additionally, the pre-determination of interventions from simulated participants and difficulty level is also as important.

A triangular model of simulation fidelity was presented by Kyaw Tun et al. It concerns the patient representation, the healthcare facilities and the clinical scenario, with an outer circle corresponding to the deception needed to increase the level of realism [23]. More recently, a different perspective was proposed to be applied to each simulation-based activity and includes four key elements: environmental, patient, semantical and phenomenal (Table 5). Each dimension can have its level of fidelity and can be supplemented by some level of deception to reach the intended level of realism and learning objectives [1].

Quantitative analysis

Only two RCTs were conducted among second-year anaesthesia residents in the same institution and were eligible for the conduction of the meta-analysis. Overall, 78 learners were included and randomized using a computer-generated list on a 1-to-1 ratio to either intervention or control groups. Both studies approached deception through an authority-challenging situation during a difficult airway crisis but using a different variable (Table 4). Both RCTs evaluated learners’ performance using mAIS, and other non-objective secondary outcomes (feeling of learners during the activity). In both studies, the deceptive intervention aimed to make the learners challenge the authority (superior’s wrong decisions). While Friedman et al. in 2015 used the strict communication behaviour of a consultant anaesthetist as a deceptive intervention, Friedman et al. in 2022 used role-play as part of the study team to induce deception between learners. In the former, authors reported no significant effect of the consultant’s behaviour on trainees’ ability to challenge authority. In the latter, authors found that the intervention group demonstrated less-effective mean challenge scores and a lower number of individual challenges. In both studies, debriefing (during which the deception intervention was disclosed) followed the simulation-based activity. None of the learners felt upset from the interventions and everyone felt it was a helpful experience.

Table 4:

Key characteristics of the studies included in the quantitative analysis

Study	n	Population	Randomization	Procedure simulated	Deceptive action	Variable	Outcomes	Effect
Friedman, 2015 [9]	34	Second-year anaesthesia trainee	Computer-generated list	Life-threatening airway crisis	Challenging the authority (consultant anaesthetist)	Superior’s interpersonal behaviour with trainees (strict vs. open communication dynamics)	Primary: - Maximal PS (mAIS) Secondary: - Feelings of the subject	No
Friedman, 2022 [5]	44	Second-year anaesthesia trainee	Computer-generated list	Rapidly deteriorating difficult airway	Challenging the authority (consultant anaesthetist)	Superior’s role (subject in the study vs. acting a part in the scenario)	Primary: - Maximal PS (mAIS) Secondary: - Average mAIS by participant - number of challenges by participant - number of participants who made no challenges at all	Yes

PS: performance score; mAIS: modified Advocacy-Inquiry Scale.

Table 5:

Determinants of simulation fidelity summarized from Alinier and Oriot [1] and Calhoun et al. [3]

Dimensions of simulation	Elements	Examples of manipulations related to deception
Environment	Training venue Equipment Furniture	• Specific positioning • Background noise • Intentional alteration
Patient	Full body simulator Organ simulator Physiological data	• Technological limitation (e.g. absence of haptic feedback) • Disproportionate physical features • Alarm sounds of the monitor • Making a task more complicated than it is normally • Control of the simulator’s response
Semantical	Relation to clinical reality Embedded scenario participants Time modulation	• Modification of the amount of information disclosed during briefing and debriefing • Evolving of the simulation based on the learner’s actions (rapid or slow evolution) • Absence of recovery despite appropriate actions • Intentional errors or behaviours (taking wrong actions/decisions, oppressive attitude) from embedded participants (clinician known or unknown to participants, simulated relatives, or one of the participants themselves) • Adjustment of patient deterioration of recovery depending on time and learners’ confidence level • Allowing the patient to die or not
Phenomenal	Participants’ level of engagement	• Not informing learners that they are taking part in an SBE activity (in situ simulation activity)

SBE: simulation-based education.

Quality of evidence

The two RCTs included in the meta-analysis were scored using the MERSQI tool. It was developed and validated to measure the methodological quality of education research studies over six domains, with a maximal score of 18 [8]. A score of 12 or higher indicates high study quality [24]. The MERSQI scores of the two included RCTs (detailed in Table 6) are 14.5, indicating their high evidence quality.

Table 6:

Evaluation of the quality of evidence of articles included in the quantitative analysis by the MERSQI score

Study	Friedman et al., 2015		Friedman et al., 2022
MERSQI item	Answer	Score	Answer	Score
Study design	RCT	3	RCT	3
Sampling: institutions	1 institution	0.5	1 institution	0.5
Sampling: response rate	≥75%	1.5	≥75%	1.5
Type of data	Objective	3	Objective	3
Validity evidence for evaluation instruments	Content Internal structure	1 1	Content Internal structure	1 1
Data analysis: sophistication	Beyond descriptive analysis	2	Beyond descriptive analysis	2
Data analysis: appropriate	Data analysis appropriate for study design and type	1	Data analysis appropriate for study design and type	1
Outcome	Knowledge/skills	1.5	Knowledge/skills	1.5
Total	14.5		14.5

MERSQI: Medical Education Research Study Quality Instrument; RCT: randomized controlled trial.

Effect of the interventions

Both analyses of common and random effects of the studies included in the quantitative analysis show a negative effect of the use of deception on the performance of the trainee, which is statistically significant with a mean difference of −1.09 (CI [−1.41; −0.77]) in the intervention group. The estimates of the measures of heterogeneity (τ ² = 0, I² = 0%) and the test for heterogeneity (P -value = 0.38) indicate that there is no statistical heterogeneity between the studies (Figure 2).

Figure 2:

Forest plot examining the effect of deception use in simulation-based education in healthcare.

Accordingly, as there is no evidence of heterogeneity, and as both fixed and random effects are similar and show strong evidence of an effect, we conclude that there is strong evidence that the use of deception negatively affects the performance of learners.

Risk of bias

The risk of bias of the two RCTs was assessed using the Cochrane RoB tool (Table 7). Both studies had similar bias concerns regarding the randomization process, reporting of outcomes and missing data. Our assessment was based on the details provided in the texts and could not be completed with information from the authors due to a lack of response. For randomization, it was stated in both studies that the allocation sequence was random and completed using computer randomization. However, it was not clearly stated if it was concealed until participants were enrolled and assigned to interventions. In both studies, the authors did not display the baseline characteristics of the groups, so eventual baseline differences that could have affected the results could not be evaluated. During reporting of outcomes, multiple eligible measurements were done, and the highest was chosen, regardless of the value of the other measurements. This questions the representativity of the recorded outcome value and may lead to a biased estimation of the effect of the intervention. Consequently, the overall risks of bias in both studies indicate some concerns.

Table 7:

Risk of bias summary showing the review authors’ judgments about each risk of bias item for each included study

Study	Bias arising from random sequence generation (selection bias)	Bias arising from allocation concealment (selection bias)	Bias arising from blinding of participants and personnel (performance bias)	Bias arising from blinding of outcome assessment (detection bias)	Bias arising from incomplete outcome data (attrition bias)	Bias arising from selective reporting (reporting bias)	Overall risk of bias
Friedman et al., 2022	Low risk	Some concerns	Low risk	Low risk	Low risk	Some concerns	Some concerns
Friedman et al., 2015	Low risk	Some concerns	Low risk	Low risk	Low risk	Some concerns	Some concerns