Echocardiography (echo) use in hospitalized patients has become an invaluable tool, particularly in the critically ill. Basic echo (Point of Care Ultrasound [POCUS]) helps to differentiate causes of shock and more advanced techniques using Doppler can identify cardiac or valvular dysfunction . Previously, echo was an imaging modality performed only by cardiologists and sonographers; however, more recently, doctors from many specialties are now competent both in echo as well as general ultrasound. These tools help in the daily management of patients where there can be rapid changes in clinical state . Echo has the advantage of being done at the bedside, providing immediate feedback of results and being completely safe. There are however some challenges in that it takes time to master the understanding of the image quality and assessment as well as the associated pathologies. Hence, any tools that can aid in developing the technical and clinical skills required may be greatly beneficial in the overall echo learning curve.
The emergence of Virtual Reality (VR), Mixed Reality (MR) and Augmented Reality (AR) utilizing powerful stand-alone hardware presents new and compelling solutions to supplement clinician education . One of the key limitations of traditional clinical training is that it requires individuals and teams to be present at a location at a given time, which is both expensive and limits accessibility . In contrast, these technologies allow the creation of immersive training environments that can be accessed anywhere and at any time through the use of low-cost portable headsets.
A significant drawback with the use of the current VR platforms for “hands-on” training is the lack of true haptic (physical touch) feedback. However, it is possible to use MR to allow the user to physically interact with digitally enhanced objects in real life . In essence, this means that the VR simulation can be overlaid on a physical object allowing haptic interactions to occur. This strategy was employed to develop the MR Sineva echocardiography simulator.
The Sineva simulator utilizes the VR pass-through functionality of the Oculus Quest 2 Head Mounted Display (HMD) (available from https://store.facebook.com/au/quest/products/quest-2, Meta, California, USA). This allows the user to perform an echo using a 3D-printed replica ultrasound probe on a physical portable chest model, thereby creating an immersive tactile-enabled MR experience.
To enable a realistic tactile feedback system to promote the appropriate motor skill acquisition, the Sineva development team are utilizing customized 3D-printed accessories for the MR experience including the manikin and ultrasound probe overlay for the Quest 2 controller (Figure 1). Accessories are printed in polylactic acid which is a lightweight plastic and the manikin is coated in silicone to simulate soft tissue. A novel synchronization process is implemented within the simulator to ensure precise alignment of physical accessories with the MR experience.
Once the user has logged in, synchronized the hardware and donned the HMD, they are able to see their physical surroundings with a digitally enhanced overlay of the controller and the chest. The overlay of the user’s controller shows an ultrasound probe with the ultrasound plane visible. The overlay of the chest can be adjusted to show a realistic replica of a patient’s chest or the underlying anatomy (e.g. only solid organs and ribs visible). The centre of the users’ field of view shows the ultrasound image with all appropriate ultrasound adjustments possible (Figure 2).
A number of novel features have been developed and optimized for use within the simulator to enhance the user experience. These include curated MR recordings to teach basics of view acquisition and pathology interpretation, live-labelling of structures on the virtual ultrasound screen, a custom ‘image rating’ system allowing users to perform self-assessment of acquired images, live guidance for probe orientation and position and fully adjustable echocardiographic pathologies. The objective of these systems is to enable users to acquire skills in echocardiography without the need for a skilled mentor being present.
The Sineva echocardiography simulator is developed using the Unity development platform (https://unity.com/) and deployed as a stand-alone application for use with the Meta Quest 2 headset. Multiple proprietary supplemental systems also support the core MR experience to facilitate web-based report management and streamlined user authentication. The simulator and supplementary systems have been exclusively developed by Dr. David Bairamian and Dr. Daniel Bairamian.
To ensure training anatomical accuracy, initial technical art was reconstructed from CT coronary angiography DICOM images. These were then progressively refined with close feedback from echocardiography experts to ensure anatomical accuracy and realism.
Initial feedback from a limited number of users has been positive. Formal evaluation of the application is planned as we move into a broader deployment strategy.
The application has been trialled in several locations across Sydney, NSW. Anecdotal feedback from the clinicians and educators has been positive that this is a useful tool which meets its purpose and has broad applicability. This user feedback has informed the iterative development of the application resulting in a comparatively low-cost yet high-fidelity simulator which will increase the accessibility of this training.
Future steps for the Sineva echocardiography simulator include:
Daniel and David Bairamian are the designers and developers of the Sineva simulator and have founded a company for its potential future commercialization. David and Daniel contributed to the manuscript by describing the innovation, supporting technology and providing the figures. The majority of the publication was written by the lead author and remaining team.