International Journal of Healthcare Simulation

Why a synthesis?

Although there are broadly similar published works relating to debrief interaction and learning for participants, the exact methods, foci of analysis and contextual factors differ across studies. Currently, studies provide little accumulated understanding of the topic but rather discrete knowledge blocks which are limited in their reach to the debriefing community. A qualitative synthesis aims to bring such isolated studies together to form new understandings of the field under scrutiny, to clarify inconsistencies, to inform practice and policies and to define future research agendas, particularly when the evidence is complex and is undertaken in different contexts [12–18]. The aim of a synthesis is to identify ‘what is known from multiple perspectives and reveal different factors, dimensions and explanations’ (p40) [19] for that existing knowledge, while also helping to illuminate a path for future research, practicum and scholarship.

Campbell et al. argue that when examining and synthesizing qualitative research, there is a risk of ‘destroying the integrity of individual studies’ and articulates a concern of ‘thinning out the desired thickness of particulars’ (p3) [12] – akin to a loss of context of the original studies [20]. Thus, to synthesize qualitative literature, particularly studies which form discrete knowledge blocks, the contextual elements during synthesis must be maintained. This is in part, because ‘meaning is sometimes inseparable from the data and not usually generalisable beyond it’ (p253) [21] unless readers engage in etic analysis.

Why meta-ethnography?

Meta-ethnography, as an interpretative approach, maintains contextual elements whilst also mandating an emic approach to surface a ‘new interpretation or theory that goes beyond the findings of any individual study’ (p8) [12]. Noblit and Hare [22] first described this method in response to a failed attempt by others to synthesize original research on desegregation in schooling, where an aggregative approach resulted in the loss of the richness of the primary data and diluted the uniqueness of individual study sites. Taking a new interpretative stance, Noblit and Hare were able to demonstrate how new meanings can arise from seemingly similar studies by directing and undertaking a different analytic approach. Meta-ethnography enables new research questions to be identified and to ascertain the knowledge status of the field under study [12,20], and is thus an apt approach to explore the field of healthcare simulation debrief interactions.

Bearman and Dawson [21] argue that a qualitative synthesis relies on the series of expert judgements made by the researcher to enable conclusions to surface from the collective studies of interest. Indeed, meta-ethnography presents conclusions based upon the interpretation of the researcher – the judgement decisions the researcher makes are woven into the synthesis. Thus, to achieve new meaning, the process is deeply interpretative rather than aggregative [16]. This interpretative layer, formulated through seven phases as outlined by Noblit and Hare, extends beyond the original interpretation of the authors of the included studies [21], permitting the collective knowledge to be identified and progressed in parallel [12,23,24].

Can a qualitative synthesis be ‘systematic’?

Noblit and Hare’s [22] original conception of meta-ethnography did not suggest the process of synthesis should be systematic, and the original formulation was constructed so a group of similar studies could be synthesized. Further supporting Noblit and Hare’s original position, Dixon-Woods et al. [25] and Toye et al. [26] argue that sampling should continue until theoretical saturation is reached; this indicates that the researcher has more control of what is included, guided to an extent by their interest. This may hold true where there is already a significant amount of published research in a field. However, in a relatively small and embryonic field, a point of saturation is unlikely to be reached.

To ensure we can identify as much of the published research as possible, to fulfil the aims of a meta-ethnography – which includes knowledge synthesis, identifying knowledge gaps and potential research avenues – it is prudent to employ a method to searching which is simultaneously comprehensive and manageable. It is here where frameworks traditionally used in positivist systematic reviews facilitate a structured approach to searching and article inclusion. Specifically, these frameworks typically include detailing a clearly formulated question with transparent and established frameworks for identifying, selecting, appraising and collecting data [27].

Positivist systematic reviews are further structured to ensure that analysis is undertaken in accordance with accepted standards; the approach is often clearly defined in logical sequential steps which are intended to be replicable, and transparency is conceptualized as the detailed documentation of procedures to enable reproduction of findings. It is here where there is perhaps the most clear departure with the theoretical basis of qualitative work. The systematic nature of a qualitative review is not related to (nor accepting of) positivist notions of the nature of knowledge and how knowledge can be synthesized. Instead, the concept of systematic here refers to the ways in which we decide how to search and what should be included in a review, and to make the process of undertaking a review manageable. It is systematic in that it uses accepted frameworks for this process, many of which have been used and continue to be used for qualitative work. There has also been a drive within the medical education community to improve standards of qualitative synthesis through the use of frameworks [17] in response to historic concerns of poor reporting [16]. We thus define transparency as detailing the explicit steps taken in question formulation, searching strategy and article inclusion decisions, and reporting in accordance with accepted guidelines. Current reporting guidelines include PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [27,28] and the recently published eMERGe guidance [29], which also overlaps with the ENTREQ statement (Enhancing Transparency in Reporting the Synthesis of Qualitative research) [14].

Further, the protocol functions here not to argue that analysis is rigid, or is akin to following a recipe (a charge sometimes levelled against positivist work). Instead, it is to explicate the analytical steps of meta-ethnography, so the reader can follow the interpretative journey of the researcher, with cognizance that the researcher is deeply integral to analysis and the final research results are necessarily imbued with researcher decisions and identity. This demonstrates that the work undertaken aligns to the original ethos of Noblit and Hare’s seminal work, which we hope will provide the reader with a degree of confidence in the final interpretations produced.

In summary, the function of this protocol, and the intention of taking a systematic approach, is to be transparent regarding how the final articles selection came to be, explicating the intended analytical route – accepting the iterative and interpretative nature of the work – and presenting this in frameworks, including meta-ethnography-specific frameworks, which are advocated by the research community.

Aim & research question

The aim of this review is to systematically search, examine and synthesize literature surrounding debrief interactions using a meta-ethnographic approach. The specific research question to be addressed through the synthesis is: How are interactions in simulation debriefing related to participant learning?

This will inform the community by:

• a. Identifying and sharing what is currently known about interactional factors and their relationship to knowledge;
• b. generating new knowledge regarding debriefing practice;
• c. understanding the relationship between debriefing evidence and practice; and
• d. identifying knowledge gaps where future research could be beneficial.

Phases in a meta-ethnography

The synthesis will be conducted aligning to the seven phases outlined by Noblit and Hare [22] which are presented in Table 1, with key practice points specified for each phase. This protocol fully addresses the first two phases, and elements of phase 3: data extraction fields and consideration of quality appraisal. The protocol also outlines the steps which will be taken in phases 4–6 and detail the intended analytical approach used in meta-ethnography.

Table 1:

Seven phases of meta-ethnography

Phase	Key practice points
Phase 1: Getting started	Select & constitute research team Scope & research question Review registration
Phase 2: Deciding what is relevant to the initial interest	Search strategy Study selection: inclusion criteria Study selection: exclusion criteria Study selection: screening & PRISMA Flowchart Define quality appraisal framework
Phase 3: Reading the studies	Define data extraction fields Data extraction Quality appraisal Decide route of analysis±
Phase 4: Determining how studies are related	Analysis I: Grouping studies, identifying main metaphors
Phase 5: Creating reciprocal translations	Analysis II: comparing studies, developing new conceptual insights – ‘translations’
Phase 6: Synthesizing translations	Analysis III: synthesizing translations* Analysis IV: develop line of argument+
Phase 7: Expressing the synthesis	Conclusions I: presenting the findings Conclusions II: subjective and field interpretation of findings

This table outlines the seven phases of meta-ethnography as originally stated by Noblit and Hare. We have detailed key practice points we intend to follow for each phase.

^± Route of analysis: decide if the meta-ethnography is to be a reciprocal translation synthesis, refutational synthesis or line of argument synthesis or combination of above.

*Not essential if number of included studies is small and no disparate groups of translations.

⁺ Not essential if translations do not yield new information.

Phase 1: Getting started

To undertake a transparent synthesis, each step must be clearly detailed and rationalized and the first step involves generating a research question [20]. Historically, as evident in Noblit and Hare [22], there has been little guidance on how to structure such question which will later serve as the basis for searching. The eMERGe guidance (criteria 3) requires a ‘well-defined’ and ‘focused’ question to be essential, especially for later stages, yet does not stipulate details on how it should be formulated [29].

Advocates of a systematic approach argue it should be structured within a standardized question framework and ensuring such formulation considers the contextual nature of the field [19]. Table 2 defines the research question according to the PICOS (population, intervention, comparison, outcome, setting) framework, which also forms the basis for both search terms (Table 3) and inclusion and exclusion criteria. Whilst this approach may have originated from positivist disciplines, the framework and utility are not tied to this realm; an indiscriminate approach may make it difficult to decide where and how articles should be searched for with a risk that the pool of potential articles generated being too small or too big.

Table 2:

PICOS framework

P	Population (who)	Simulation participants
I	Intervention (what)	Debrief
C	Comparison (against)	None
O	Outcome	Interactions associated with learning
S	Setting (context)	Healthcare simulation

Research question presented in PICOS question framework.

Table 3:

Search terms

Simulation	Debrief	Interaction	Learning
Simulat*	Debrief*	Interact*	Learn*
scenario*		Conversation*	Know*
train*		Observ*	Transform*
		discussion*	Outcome
		Discourse*	behaviour*
		Discursive	behavior*
		Linguistic	chang*
		ethno*	skill*
		talk*	attitud*
		question*	result*
		narrative*	reflect*
		stor*
Search string: (simulation* OR scenario* OR train*) AND (debrief*) AND (interact* OR conversation* OR observ* OR discussion* OR discourse* OR discursive* OR linguistic OR ethno* OR talk* OR question* OR narrative* OR stor*) AND (learn* OR know* OR transfer* OR outcome* OR behaviour* OR behavior* OR chang* OR skill* OR attitude* OR result* OR reflect*)

The search terms have been derived from PICO question framework and a corresponding search string generated. ‘Setting’ (from ‘PICOS’) has been excluded from search terms so as not to inadvertently filter out relevant articles where ‘healthcare’ is not included. This is considered in the inclusion/exclusion criteria.

Reviews are often conducted in teams of two or more individuals; to maximize efficiency and to increase rigor. Within meta-ethnography, there are differing perspectives regarding individual and collaborative approaches, especially when the number of potential studies to be included is anticipated to be high [28]. Considering the process of meta-ethnography is deeply interpretative, the work should be seen as original research. As such, and in keeping with Noblit and Hare’s approach [22], this synthesis will be undertaken with a researcher and supervisor team (authors of this protocol).

Phase 2: Deciding what is relevant to the initial interest

This phase explicates our focus of the synthesis by listing the inclusion and exclusion criteria and undertaking a process of searching, screening and appraising literature [28]; the intention is to be transparent about how articles have been included and why [30], and has been advocated in eMERGe guidance [29]. The STARLITE mnemonic (Table 4) was devised based upon a systematic review conducted by Booth [30] examining the reporting of searches within systematic qualitative reviews; its construction, based upon a systematic literature review, is robust and its utility is demonstrated in this context.

Table 4:

Search strategy

S	Sampling strategy	Comprehensive search: peer-reviewed articles only
T	Type of studies	Qualitative research with analysis of naturalistic data Studies focused on debrief interaction captured via: • participant observation • video recording • audio recording
A	Approaches	Electronic searches (see list below) Citation snowballing Hand-searching following journals: • Simulation in Healthcare • Advances in Simulation • BMJ Simulation & Technology Enhanced Learning • Clinical Simulation in Nursing
R	Range of years	No limits applied
L	Limits	Functional limits: humans, subjects aged 18 or over, full-text availability
I	Inclusion and exclusion criteria	See Tables 5 and 6
T	Terms used	See Table 3
E	Electronic sources	• British Education Index (BEI) • CINAL • Cochrane • Education Abstracts • ERIC • ERICESCBO • ProQuest • PubMed • SCORPUS • Web of Science

The search strategy is detailed using the STARLITE framework.

Study selection: inclusion and exclusion criteria

The criteria for inclusion and exclusion (Tables 5 and 6) should begin with the original question in mind and should be structured based upon the domains of interest [17]. The field of empirical debrief interaction research remains primitive, and thus it is important to include all published studies, irrespective of the publication date. Whilst synthesis methods exist for the interpretation of studies with different methodologies, there is an inherent risk of weak conclusions being drawn from data which is generated from different epistemological positions [20,25]. For this reason, interpretive data using methodologies aligned with such different positions will be excluded; only interaction analysis studies are included.

Table 5

: Inclusion criteria

Inclusion criteria	Rationale
All languages	Emerging field; important not to exclude studies which are potentially influential Considering analysis is based on second-order constructs, if these are in another language, they can be translated whilst preserving meaning.
All date range	Emerging field; important not to exclude studies which are potentially influential
Peer-reviewed journals	Keep the process of searching systematic, rigorous and manageable
Conference proceedings	Emerging field; important not to exclude studies which are potentially influential Full-text search for abstracts identified will be performed
Simulation-related debriefing	Scope is limited to simulation; excluding other industries and clinical debriefing
Undergraduate and postgraduate participants	Emerging field; important not to exclude studies which are potentially influential Reflects simulation practice; simulation is available to both undergraduate and postgraduate courses
Faculty of all experience	Emerging field; important not to exclude studies which are potentially influential Reflects simulation practice; simulation is facilitated by educators of varying experience

Study inclusion criteria and corresponding rationale.

Table 6

: Exclusion criteria

Exclusion criteria	Rationale
Full text not available	Unable to gather full details of methods, analysis or results
Conference proceedings without corresponding article*	Outside scope of research; conference proceedings are excluded
Non-simulation debriefing	Outside scope of research; debriefing only related to simulation included
Simulation event analysis	Outside scope of research; focus is on debrief, not simulation event
No debrief interaction analysis*	Lack of interpretative data; analysis of naturalistic data must occur
Non-healthcare simulation debriefing	Outside scope of research
Debriefs involving patients contributing to the discussion	Atypical simulation debrief; ethical issues
Debrief focused on CSID/reducing PTSD	Atypical simulation debrief; ethical issues
Classroom size debriefs (c. >20)	Atypical simulation debrief
Single faculty single participant debrief*	Atypical simulation debrief
Theoretical articles	Lack of interpretative data
Content analysis	Lack of interpretative data
Exclusively interview/focus group-based study	Analysis of naturalistic data must occur
Methods not reported/unclear*	Unable to gather full details of methods, analysis or results
Outcomes not stated	Unable to gather full details of methods, analysis or results
Partial/incomplete data reporting±	Unable to gather full details of methods, analysis or results
Survey studies/data	Analysis of naturalistic data must occur
Document analysis	Outside scope of research; focus is on debrief interaction
Analysis for purpose of evaluation	Outside scope of research; focus is debriefing for learning
Studies involving participants < 18 years of age	Atypical simulation debrief; ethical issues
Faculty only debriefs	Atypical simulation debrief; data must include faculty–participant interactions
Participant only debriefing (e.g. self-debriefing)	Atypical simulation debrief; data must include faculty–participant interactions
Non-face-to-face debrief (i.e. virtual reality, distance or tele-debriefing debrief)	Atypical simulation debrief
Descriptive study (e.g. course or debrief description)*	Lack of interpretative data; analysis of naturalistic data must occur
Debrief assessment using tools*	Outside scope of research.

Study exclusion criteria and corresponding rationale; CSID, critical stress incident debriefing; PTSD, post-traumatic stress disorder; ± includes different types of data grouped during analysis.

*Denotes criteria added or modified after initiating initial searches.

Sawyer et al.’s [2] non-systematic literature review of debrief practice concluded that facilitator-guided post-event debriefing was the most reported method. If this then represents the community’s perspective on ‘ideal’ debrief practice, it is prudent, to further our knowledge in this field, to synthesis literature based around the practice which is occurring internationally. For example, debriefs involving participants numbers exceeding c.20 are excluded, as the conversations and interactions at this level are likely to be very different to those seen more commonly in smaller groups [31]. Similarly, the field of virtual reality and online debriefing have only recently emerged, and these represent different interactional approaches. The exclusion criteria ensure that papers with common ground are selected, as this is central to meta-ethnographic work [32].

Phase 3: Reading the studies

Noblit and Hare [23] argue that for the researcher to become sensitive to the detail of the text and to be able to extract the meaning, multiple readings are required. Multiple readings of the same article are achieved inherently: firstly, upon selecting for inclusion in the synthesis; a second time to highlight key aspects; detailed reading during the third time to extract study details including contextual elements [24]; a fourth time to extract data relating to quality appraisal; the fifth time when identifying, extracting and interpreting metaphors; then further readings when translating studies into one another. These multiple readings serve to allow the researcher to view the text from different perspectives, thereby surfacing different readings [22,28] whilst simultaneously preserving the context when data is being extracted [20]. Figure 1 outlines these three distinct data extraction episodes: the first involves extracting study details, contextual factors and key features; the second is extracting data for quality appraisal; and the third involves extracting data in the form of metaphors, undertaken in preparation for reciprocal translation in phase 5. It is important to note here that the process of extraction is, itself, a form of interpretation, and marks the interpretative work involved in meta-ethnography.

Figure 1:

Data extraction episodes.

Data is extracted at multiple junctures using bespoke templates adapted from existing frameworks and incorporating key contextual detail. Choices made from the outset in phase 2 influence future data extraction.

Figure 2:

Analytical steps.

Noblit and Hare’s original approach included four detailed analytical steps. At the core of this is the process of reciprocal translation (phase 5). Phases 3 and 4 function to familiarize the researcher with the data and to group this in preparation for phase 5. Phase 6 is a further analytical step where newly formed concepts are synthesized, and a new theory is proposed to account for the data.

Data extraction method

There is no consensus of best practice for data extraction and management in meta-ethnographies. Atkins et al. have used standardized forms for collecting information regarding main themes, methods, quality and details of ethics [20]. Parry and Land [31], when undertaking a synthesis of conversation analysis studies, focused on the type and amount of data, and the detail and depth of data analysis reported. It is critical that as much detail as possible is extracted from the primary studies, so as not to lose the context in which the results are interpreted [33]. As such, a data extraction template (see Table 7 for data extraction fields) will capture both second-order constructs and detailed contextual factors [12].

Table 7:

Data extraction fields

Paper details	Author; year; journal; language; study period; country research conducted; study aim(s)/question(s); study background
Participant Demographics	Speciality; undergraduate/postgraduate; year; ages; sex (number); simulation experience
Faculty Demographics	Speciality; experience; sex (number); ages
Simulation Course Details	Simulation modality; simulation description; number of participants/courses; number of participants/simulation event
Debrief Details	Number of faculty/debrief; debrief model/approach; debrief time/session; video use in debrief; debrief number
Method and Analysis Details	Data capture method; secondary data capture method; total debrief time; transcription details; materials transcribed; analysis method
Study Findings	Main findings; additional findings/author conclusions
Limitations/Reflexivity	Author-cited limitations; identified limitations
Miscellaneous	Additional notes; further references to review from bibliography (for citation snowballing)

The data extraction fields are specific to the types of research included and centred around simulation-based education demographic details. Data will primarily be captured in phase 3.

Protocol for data analysis: phases 4–6

A key component of the synthesis is to understand how studies are related to one another, and more specifically, how the interpretations one author makes of the phenomenon in question is similar or different to that of other authors [12]. Noblit and Hare label these connections as assumptions [23] and these are surfaced based on common or frequently occurring metaphors [24]. Thus, phase 4 will be concerned with specifying what these connections are and identifying the main groups.

Phase 5 will be conducted in line with Noblit and Hare’s original framework, which offers an option to take one of three different approaches: reciprocal translation (when studies are similar); refutational synthesis (when studies implicitly or explicitly refute each other); and, line of argument synthesis (when studies build a larger theory of an issue) [22]. Reciprocal translations also form the first phase of a line of argument synthesis (phase 6). Thus, a line of argument, in accordance with Noblit and Hare’s approach, is a second phase of analysis. It is here, where Britten et al. argue that the line of argument is a ‘middle-range theory’ (p214) [24] achieved by ‘…making a whole into something more than the parts alone imply’ (p28) [22] which France et al. (2019) state, after consultation with Noblit, produces ‘a new “storyline” or overarching explanation of a phenomenon’ (p10) [32].

Since inception, this framework has evolved, and recently France et al. (2019) have stated that researchers should actively seek to undertake both reciprocal and refutational analysis (which they also label as ‘deviant’ data) in heterogenous studies, as this facilitates a fuller understanding of the phenomenon [32]. France et al. (2019) [32] have provided further guidance on the analytical phases, echoing Noblit and Hare’s stance that the process is iterative with overlapping phases. Such an iterative approach is not restricted to meta-ethnographies and is seen as a key principle in other qualitative synthesis methods [33]. The key aspect of this approach is to derive meaning from multiple cases in a systematic and transparent approach [21], ‘relating knowledge and showing its relevance in establishing its meaning’ (p99) [23].