Recently, simulation-based education (SBE) has been evidenced as an effective form of pedagogy in mental health and care settings, through consistent improvements in self-efficacy and technical and non-technical skills. A key component of SBE is post-simulation debriefing. Debriefing involves educators turning into facilitators and guiding participants through reflective discussions; however, there is no single debrief model used across simulation training. Debrief models have been previously evaluated, but not directly compared. This paper investigated whether there is a significant difference between self-efficacy scores of participants debriefed using the Diamond model and a modified Pendleton’s during SBE.
Participants included 751 healthcare professionals who attended various simulation training courses between September 2017 and August 2019. Participants completed pre- and post-course questionnaires using the Human Factors Skills for Healthcare Instrument. Pre- and post-data were screened using Mahalanobis distance and Levene’s test and data were analysed using paired-samples
Significant differences in human factors scores were found for the Diamond debrief model only. No significant improvements were found for the Pendleton’s model.
Results suggested a benefit to using the Diamond model over the Pendleton’s model during simulation debriefs, due to a significant improvement in self-efficacy scores. These findings contribute to the gap in literature around direct comparison of debrief models and support studies where the Diamond model has yielded significant improvements in human factors skills previously.
This research contributes to the gap in literature around comparing different debrief models within simulation training to analyse impact on participants’ self-efficacy.
It provides support for simulation training as an effective teaching method to improve healthcare professionals’ technical and non-technical skill sets.
It lays the foundations for further longitudinal or explanatory research to unpack why the Diamond debrief model is more effective than other models.
In recent decades, our understanding of mental health needs for the general population has developed significantly, particularly regarding the overlap between mental and physical health [
Recently, simulation-based education (SBE) has emerged as a powerful tool facilitating experiential learning and the acquisition of both clinical and reflective skills, generating various positive outcomes in healthcare [
Referred to as one’s belief in their ability to perform activities and tasks successfully [
Moreover, self-efficacy has been noted as a key component in major behaviour change models [
Key to effective healthcare and clinical proficiency is the cognitive and social skills required to manage the demands of different clinical and high-pressure situations and work collaboratively within these. Such skills include decision-making, leadership, situational awareness and teamwork, often referred to as non-technical skills. It has been argued, however, that these skills termed as strictly ‘non-technical’ differ across different healthcare sectors and disciplines. Communication, for example, particularly in mental healthcare settings, is a mediator for diagnosis and treatment, suggesting its role as both a technical and non-technical skill. As such, Reedy et al [
Much of the literature has identified the most important component of SBE to be post-simulation debriefing [
However, there is no single debrief style used across SBE. Two popular models include Pendleton’s debrief model [
Both the Diamond and Pendleton models have been proven effective at improving non-technical skills, though they have not been directly compared [
This research paper aims to begin addressing this gap in the literature, where these models of debriefing have not been overtly directly compared. We aim to answer the following research question:
Is there a significant difference between the skills gained when using these two highly adopted debrief models?
Participants consisted of 792 clinical and non-clinical professionals who attended various simulation training courses conducted in South London between September 2017 and August 2019.
Demographic information: A general range of demographics was collected including profession, age, self-identified gender and career stage.
HFSHI [
A quantitative research design was adopted using pre–post measures in the form of a survey with a validated tool to collect data.
The simulation training courses that participants attended were related to a wide variety of mental healthcare topics, knowledge and skill sets. Depending on the course, these could last for between 1 and 4 days and consisted of specific learning objectives. The majority of training courses consist of a full 1-day session. As the training courses and dates were pre-pandemic, all training was held in Maudsley Simulation’s training centre.
At the beginning of each training session, all participants were briefed on the process of simulation and the structure of the day ahead. Following this, any questions were answered, and relevant notice was given to pre-warn for any distressing content that may be covered during scenarios and discussions throughout the day. Each training day consisted of numerous clinical simulated scenarios (between 6 and 10) using simulated patients (SPs), and each lasting for around 10 minutes. Each of these scenarios were outlined in a manual that all facilitators and SPs follow, were related to specific learning objectives and were designed to focus on certain skills or knowledge areas. The SPs were also briefed before the course and received specialist, comprehensive training from Maudsley Simulation’s quality assurance course, as well as guides and rough scripts for each of their roles. These briefs were produced by course facilitators comprising of multi-professional faculty and were designed to ensure SPs had the appropriate knowledge and skills to portray the patients they were representing accurately. The SPs would usually portray between 1 and 3 patients throughout different scenarios during each training day. Facilitators were generally consistent across all training courses and trained to the same standards in debriefing to ensure they each delivered the same quality of teaching.
Training sessions and scenarios covered a multitude of topics and needs reflective of the course title; for example, some courses targeted specific professions and their training needs (e.g. Mental Health Crisis in the Emergency Department Simulation training), while other courses were designed for a large variety of staff across different settings who may encounter someone who has mental healthcare needs (e.g. Perinatal Mental Health Simulation training). Each session and scenario was created and developed by a team of clinical experts in the field including doctors, psychiatrists, clinical psychologists, mental health nurses, subject matter experts and lived experience individuals. Please see Appendix A for an example of a simulated scenario faced by the participants. Courses followed a mixed structure of didactic teaching led by clinical experts or facilitators, and simulated scenarios consisting of interactions between the SP and training participants. Course facilitators would assign 1–2 participants to each scenario to allow them to play their role as they would in their professional setting. Those not involved in the scenario would observe using a live audiovisual link.
Each simulation scenario would then conclude with a 40-minute in-depth debrief, involving all the training delegates and the course facilitators. The types of debrief used included both an adapted version of Pendleton’s [
For data collection and evaluation purposes, ahead of each training course a detailed information sheet was shared with all participants outlining the use of any data collected for research, and written informed consent was obtained. All participants were also asked to complete a pre-course and post-course questionnaire. These included questions around course learning objectives, course-specific questions and human factors questions using the HFSHI scale. Data were collected for individual, internal course evaluations, which is standard practice for all participants engaging in training courses.
A paired-samples
A total of 792 participants started the pre-course survey. Of these, 780 specified their gender; 70.8% were female (
Age ranges
Age range | <20 | 20–24 | 25–29 | 30–34 | 35–45 | 45–55 | 55< |
---|---|---|---|---|---|---|---|
1 | 66 | 234 | 144 | 182 | 118 | 46 |
Job roles
Job role | Nursing | Midwifery | Medicine | Allied health professional | Other non-clinical professional | Other clinical professional |
---|---|---|---|---|---|---|
352 | 23 | 254 | 74 | 51 | 38 |
Career stage
Career stage | Qualified | Student | No response |
---|---|---|---|
664 | 90 | 38 |
Seven hundred and ninety-two clinical and non-clinical professionals attended various simulation courses conducted in South London between September 2017 and August 2019 and completed the pre- and post-course questionnaires. From this, participants (
The HFSHI sum difference between pre- and post-scores was screened using Mahalanobis distance to identify participant outliers. There were two degrees of freedom, which equated to a chi-square value of 34.61 at
Prior to any statistical analysis, Levene’s test was conducted to ensure the assumption of homogeneity was not violated,
There was a minimum change of −48 and a maximum of 50 (
While we recognize and have considered the risk of potential bias with excluding participants, screening for participant outliers where participants may have misread questions, for example, and excluding participants who did not complete questionnaires was deemed a necessary measure in order to accurately match participant data pre- and post-course.
A paired-samples
Statistics for both debrief groups
Debrief model | Pre-course | Post-course | Effect size ( |
|||
---|---|---|---|---|---|---|
Mean | SD | Mean | SD | |||
Diamond | 96.83 | 12.19 | 101.08 | 14.70 | <0.001 | 0.31 |
Pendleton’s | 92.03 | 12.28 | 91.16 | 14.82 | 0.399 | N/A |
Results indicate that mean HFSHI scores were significantly improved after simulation training amongst the participants that were debriefed using the Diamond model only; there was no significant improvement found for the Pendleton’s model. A significant increase in HFSHI scores for participants who experienced the Diamond model was shown to have a small–medium effect size. Given the limited existence of literature comparing debrief models, particularly within simulation, this result does suggest it is worth exploring further. Our results imply an underlying benefit of utilizing the Diamond model over the Pendleton’s model due to the improvement in self-efficacy scores. Although the finding is not a replication of anything in the current literature, as the direct comparison of debrief models is an exploratory piece of research, it does support research where the Diamond model has been used and significant improvements in Human Factor skills have been reported quantitatively and qualitatively [
In addition, the implication of these findings is congruent with literature where the Pendleton’s model has been criticized for its lack of deeper analysis, explanation [
Nonetheless, it is important to note that the sample size of each group differed greatly, with the Diamond debrief group having over twice as many participants than the Pendleton’s group, which may have an impact on the power to detect change. A potential reason for this imbalance, however, may be related to the criticisms noted in the use of Pendleton’s model as a debriefing technique and an underlying preference towards the more structured and in-depth process of the Diamond model for SBE. It is important to note, however, that for both groups, the duration of scenarios did not change, and both groups covered similar courses including both clinical and non-clinical staffing groups, of different stages of qualification.
However, the vast sample size generally puts our research at an advantage in the literature; not only was the sample representative of a large age range working within healthcare, but a vast range of healthcare professions and mental health training topics were also represented in the data. Improvement in self-efficacy when the Diamond model was used was observed across a range of simulation courses that have dissimilar learning objectives, indicating that the model is effective regardless of course type or topic. This also suggests that the Diamond model style of debrief can be applied to various healthcare workers training and learning needs. However, it would be beneficial to also explore the efficacy of the model in non-healthcare SBE.
There are other limitations to this research that must also be considered. As discussed previously, self-efficacy is not a direct measure or predictor of behavioural change, but only a factor which can influence this and so while our results do evidence that the Diamond debrief model yields higher self-efficacy scores, we cannot assume a direct causal link to behaviour change. While we justified self-efficacy as a proxy for perceived skill, caution must be taken with directly applying these findings. Indeed, future research is needed to explore longitudinal aspects of our findings to assess whether the model has any long-term impacts on learning, behaviour change and perceived self-efficacy. Moreover, explanatory research looking into debriefing specifically would be beneficial for further unpacking
It is also important to recognize that this study was an opportunistic secondary analysis of data collected from simulation courses run previously. Therefore, groups were not able to be matched, and there was increased potential for confounding factors. Factors including previous attendance at a simulation training and more exposure to debriefing amongst some participants may have impacted improved scores. For those who have attended more training, have higher levels of expertise, knowledge or experience, they may already have a higher perceived self-efficacy, or contribute more to debriefing, also potentially impacting scores. Previous research has explored the possibility of gender differences impacting HFSHI scores [
Despite these limitations, our findings make an important contribution to the gap in SBE literature where debrief models are not extensively compared. For the wider simulation community, and within mental health simulation training, these results indicate the benefits of using the Diamond debrief over other potential debrief models on the improvement of non-technical skills.
None declared.
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Data are available on request.
Ethical approval was awarded by the Psychiatry Nursing and Midwifery Research Ethics Subcommittee at King’s College London, PNM 13/14-179.
The authors declare no conflict of interest.
Setting: GP, Neurology Clinic | ||
Requires: 1 actors | ||
Jemma Jacobs is a 30-year-old female, who has a history of tonic clonic seizures since childhood. Her seizures have worsened recently, which neurologists have worked up & feel to be secondary to non-epileptic seizures, so they want to refer her to a psychiatrist. The work up has included an MRI & 24-hour video EEG which showed the patient having these episodes without seizure activity. | From course: 1 participants | |
She developed seizures as a teenager. She loses consciousness and then feels tired afterwards. She was started on medication (Lamotrigine (pronounced la-mot-ra-gine) 100 mg once daily) and the seizures were well-controlled on this and has not had a seizure for a number of years. Then a few months ago, she started to have seizures, they were a little different and felt as if she was separated from the world, but she managed to control some of the seizures and she can communicate with people during it. Sometimes she falls to the ground. These are happening multiple times a day and impact on the care she provides for her mother. | Set Up & Props: GP Letter |
|
The patient has been back to see her neurologist who has stated that these new ‘attacks’ do not sound the same as epilepsy, which, in his opinion, remains well controlled. | Participant task: The task for the participant is to begin to take a history of presenting complaint. | |
She currently does not work as she had to stop to look after her mother who is unwell at the moment with an early onset dementia. She is also married with two young children, ages 6 & 8, a boy and girl. Her husband is very busy, works as a management consultant in the City. The mother now has carers twice day, but the patient had moved her mother into her home to help feed her and look after her between carer visits. The mother can manage some things herself, but the patient is having to do more for her than before. She still recognizes the patient, but the patient fears the point when she cannot. The patient has a sibling who is helping out now that the patient’s seizures are getting worse, but has not previously been helping out up until now. Her husband also helps out a bit more with household responsibilities and kids when the patient is unwell. The patient claims that she is happy to care for her mother as well as the kids and does not recognize that this is stressful. She previously worked as an office manager. She had a difficult time at school and was bullied as an adolescent. | ||
She is sleeping poorly, as she is worried about her mother, appetite is gone, but her intake is fine as she eats with her mother. | ||
She does have some friends whom she sees occasionally but not much as she you look after her mother and is quite isolated. | ||
Unsure what to look forward to as only looking after her mother most of the time. | ||
She stays away from all drugs and alcohol b/c her mother used to drink a lot when the patient was a child. | ||
She tends to be almost emotionless, detached from her feelings of stress. | ||
Overall, the patient doesn’t really agree that this is psychological, but rather that this is something medical & that she requires further medical investigation. She is resistant to psychological suggestions & brings the discussion back to medical interventions. | ||
One observer assigned to 2–3 items done well
One observer assigned to 2–3 items they would have done differently
One observer assigned to the golden moment
Facilitator checks in with participant, then group