Virtual reality (VR)-based, artificial intelligence (AI) driven conversational agents to train de-escalation skills

Following the implementation of findings from their recent study [1], the Western Sydney Local Health District (WSLHD)/University of Sydney research team are now testing a new iteration of their VR-based AI driven verbal de-escalation trainer.

The VR-based simulation places the user in front of a patient showing aggressive behaviours who will respond either positively or negatively depending on what the user says. If a user says something that would be inflammatory, the conversational agent will become increasingly frustrated with corresponding animations. Conversely, saying de-escalatory statements will lower the frustration level.

Findings from the initial research study highlighted limitations in the complexity of the responses the agent was able to provide. The original response system was programmed with predicted responses drawn from the AI algorithm which were hardcoded into the application. To combat these limitations, the team have now implemented a direct link to the AI algorithm being utilised to drive the agent’s conversation, broadening the responses available.

While not without its challenges, such as keeping the conversation on track to meet the educational outcomes, the research team is excited at the potential improvements this innovation could deliver. The team have also expanded the number of available scenarios to increase the versatility of the application. You can also read about this research team’s work published in IJoHS:

1. Moore, N., et al., Designing Virtual Reality–Based Conversational Agents to Train Clinicians in Verbal De-escalation Skills: Exploratory Usability Study. JMIR Serious Games, 2022. 10(3): p. e38669.

2. Moore, N., et al. Designing virtual reality experiences to supplement clinician Code Black education. International Journal of Healthcare Simulation. 2022;1(Supplement SRSIS 1):S12-S14.