Virtual and Augmented reality for Biomedical applications
Abstract
The article is created as part of a curriculum assignment for Medical Electronics. In this article, I would like to provide a glimpse of the Virtual and Augmented reality usage in Biomedical applications.
Introduction
Virtual Reality and Augmented Reality for training and treatment using medical electronics.
Virtual Reality (VR) and Augmented Reality (AR) have emerged as effective tools in the field of medical electronics for training and treatment purposes. They have the potential to provide a simulated and controlled environment for medical personnel to train and practice various medical procedures, and for patients to undergo therapy and rehabilitation.
In the field of medical training, VR and AR can offer a safe and controlled environment for medical professionals to practice complex procedures without the risk of harming a patient. Medical students and residents can use VR and AR simulations to develop their skills in performing surgeries, identifying medical conditions, and practicing communication and decision-making skills.
For example, medical students can use VR to explore and interact with virtual organs, allowing them to better understand anatomy and the effects of medical procedures. Surgeons can also use AR to overlay virtual images onto the patient’s body during surgery, improving the accuracy and safety of the procedure.
In the field of medical treatment, VR and AR can be used for pain management, anxiety reduction, and rehabilitation. Patients can use VR and AR simulations to distract themselves from pain and anxiety during medical procedures. VR can also be used to create immersive and interactive environments for rehabilitation and physical therapy, helping patients regain strength and mobility.
For example, patients with neurological conditions such as stroke or Parkinson’s disease can use VR to perform specific exercises that help to improve their motor function and coordination. VR can also be used to simulate real-life scenarios, allowing patients to practice and improve their social and communication skills.
Pictorial representation of Virtual and Augmented Reality
Current advancements in Virtual Reality and Augmented Reality for medical training and treatment
Virtual Reality (VR) and Augmented Reality (AR) technologies have made significant advancements in recent years in the field of medical training and treatment. Some of the current advancements are:
Surgical Training: VR and AR technology are being used to train medical professionals in performing surgeries. For example, the Osso VR surgical training platform uses VR to simulate surgeries and provide feedback on the performance of the trainee.
Medical Imaging: VR and AR technology are being used to visualize medical imaging data in a more immersive way. For example, the EchoPixel True 3D Viewer allows physicians to interact with patient-specific 3D models of organs or tissues.
Pain Management: VR technology is being used to distract patients from pain during medical procedures. The Samsung VR app, for example, provides immersive experiences that can help patients manage pain and anxiety.
Rehabilitation: VR technology is being used for rehabilitation and physical therapy. For example, the MindMotion GO system uses VR to provide exercises and feedback for patients recovering from stroke.
Telemedicine: AR technology is being used to facilitate remote consultations between physicians and patients. For example, the Vuzix M400 Smart Glasses allow physicians to see what the patient sees in real-time, facilitating remote diagnosis and treatment.
Mental Health: VR technology is being used for mental health treatment, such as exposure therapy for anxiety disorders. The Limbix VR platform, for example, provides virtual environments to help patients confront and overcome their fears.
conclusion, VR and AR technology have made significant advancements in the field of medical training and treatment, and their potential continues to expand as technology advances. These technologies have the potential to revolutionize the way medical professionals are trained and patients are treated.
Working principles of VR &AR
Virtual Reality (VR) is a process of visualizing a computer-generated environment in an interactive manner using software and hardware.8 The experience involves total immersion in the virtual environment, allowing the user to act in the virtual world as they would in the real world .VR devices obtain input from the user through a combination of head tracking, controllers, hand tracking, voice, joysticks, on-device trackpads, or buttons. VR headsets use two lenses to create a stereoscopic 3D image by projecting a pair of two-dimensional (2D) images, one to each eye, with a slight difference in perspectives. In addition, VR headsets have a wide FOV of 90°–210° and a frame rate of at least 90 frames per second to increase immersion.
VR devices obtain input from the user through a combination of head tracking, controllers, hand tracking, voice, joysticks, on-device trackpads, or buttons. VR headsets use two lenses to create a stereoscopic 3D image by projecting a pair of two-dimensional (2D) images, one to each eye, with a slight difference in perspectives. In addition, VR headsets have a wide FOV of 90°–210° and a frame rate of at least 90 frames per second to increase immersion.
Augmented reality (AR): smartphone-based AR. The smartphone augments the brain’s sketch in the real world captured by the camera by overlaying the brain’s virtual image.
VR- and AR-based visualization
VR- and AR-based visualization of scientific experimental imaging data, tools for surgery and anatomy, and collaborative interfaces for education and telehealth
- Digital whole-slide visualization and navigation using an HMD in VR and a web-based browser for whole-slide imaging on a desktop
- Visualization of a user demonstrating a neuron tracing tool. For example, TeraVR can visualize whole-brain imaging data in VR and reconstruct neuron morphology at different regions of interest (ROIs)
- Visualization and navigation of a 3D scanning electron microscope (SEM) image using VisionVR software by arivis.
- Physicians can use AR to rotate certain anatomy during brain surgery and cardiac surgery to get full visualization to better perform, plan, and explain their surgeries.
- Studying anatomy using VR can help physicians visualize and explain medical processes to other health professionals. A medical student visualizes multiple organs and organ systems in VR.
- AR pens can be used to get a 3D image to help students better visualize and study concepts.
- VR can be used for clinical assessments where the doctor and affected individual can enter a virtual world to receive a checkup.
Biomedical VR-AR case studies
1. Visualization of 3D and highly multiplex protein images in single cells (VR based)
Biomedical applications use HMDs and VR applications for 3D visualization. Software packages for visualizing a 3D dataset in VR can be developed using computation platforms such as Unity. As mentioned previously, ConfocalVR is one such software developed by Immersive Science to visualize confocal microscopy images.32 This software can help users understand cellular architecture and the distribution of proteins and molecules through immersive 3D visualization.
The dataset used for visualization contains 3D subcellular co-detection by indexing (CODEX) images acquired through multiplex imaging of DNA-barcoded antibodies to target 20 cellular markers.56 CODEX data were developed by a spinning-disk confocal microscope with a 60× objective lens, providing diffraction-limited optical images across 25–30 depth slices of a 5-μm cancer tissue sample. The resultant high-resolution CODEX datasets were visualized for three different regions on a microarray sample. The 3D scene contained single-cell distributions from individuals with chronic lymphocytic leukemia (CLL), Hodgkin’s lymphoma (HL), and natural killer (NK)/T cell lymphoma. Image processing algorithms were used for background subtraction and registration, and the resulting images were stored in the Tag Image File Format (TIF or .tif). The TIF files were converted into Neuroimaging Informatics Technology Initiative file format (NIfTI or .nii) using ImageJ to visualize these 3D data using the ConfocalVR software.
ConfocalVR software allows users to control image viewing parameters such as lighting, image depth, opacity, image quality, colors (RGB), intensity range. Users can also grab the 3D volume using the controllers to scale and rotate the image. In addition, the parameters used during visualization can be saved in a text file. This approach helps to reproduce the visualization results in the future.
2. AR for neurosurgical planning and execution (AR Based)
Several recent studies have employed AR for surgical planning and execution in interventions involving the head, neck, and spine.AR was used for visualizing presurgical neurovascular anatomy before endovascular intervention.However, many of these studies have been demonstrative and exploratory; far fewer have quantified the effect of AR-based approaches on outcomes. A notable exception focused on AR navigation for spine fixation.In that study, 20 individuals receiving screw placement surgery were treated using an AR surgical navigation (ARSN) technique. Specifically, bone entry points, “bulls eye” views along the screw axis, and instrument navigation queues were displayed via AR during surgery. Twenty other individuals received screw placements via a more conventional free-hand (FH) technique. A total of 262 screws were placed in ARSN-based interventions, and 288 screws were placed in FH-based interventions. Both groups were composed of similar numbers of screw placements in the thoracic and lumbosacral vertebrae. The same surgeon performed all operations. The Gertzbein scale was used to assess screw placement accuracy on a postoperative basis via imaging, and grades 0 and 1 were categorized as accurate. In addition to accuracy, procedure time, blood loss, and length of hospital stay were also quantified. The results demonstrated that the share of clinically accurate screw placements was higher for the ARSN cohort than the FH cohort; this was statistically significant (93.9% versus 89.6%, p < 0.05). The proportion of screws placed without a cortical breach was also higher for the ARSN cohort (more than twice as high, in fact) compared with the FH cohort (63.4% versus 30.6%, p < 0.0001). Statistically significant differences were not observed for the procedural outcome parameters that were quantified. Nevertheless, this study demonstrated that AR-based surgical navigation for spine fixation holds great promise for enhancing screw placement accuracy.
Conclusion and future perspectives
Most of the educational applications of VR and AR seem to have construct and predictive validity, with the acquired skills to be transferable to real situations. However, the credibility of several of these studied might be questionable. First of all, many of them are not randomized studies, with cohorts of different characteristics and inadequate number of participants (less than 30 in most studies). Moreover, there are few studies for each simulator, and there are no similar standards in their design, so that they could be summarized and directly compared. In addition, many of the studies referring to specific simulators become quickly outdated as they do not take into account the simulators’ continuous upgrades. More randomized control trials, comparing the effect of VR training against no training, other simulation-based training, or different VR training systems, are needed. The samples must be larger to strengthen the results and the designs of different studies . Moreover, the extent of the decay of the skills over time must be elucidated. When these properties, along with the cost factors, are clarified, then we can examine the way that VR and AR can be officially incorporated in medical education curricula.
However, the published literature suggests a positive educational impact. VR/AR training displays certain advantages toward other simulation techniques. Although expensive to buy, VR/AR simulators provide a relatively costless opportunity for reproducible training under various environments and difficulty levels. Moreover, they do not raise ethical issues, compared with other animal and living tissue simulation models. They provide immersion for the user and the ability to perform complete procedures, in contrast with partial task trainers. Multiple studies have shown a favorable impact of VR/AR trainers on inexperienced trainees, and we can intuitively assume that they are technically evolving in high pace as the technology progresses. Future improvements could include the integration of olfactory stimuli. Odors can be used as diagnostic tools or even to recreate stressful conditions (e.g., in a combat or in the operating room) with greater realism . Medical informatics is also an evolving field. Medical data will be visualized more clearly and impressively with VR/AR technology. Users will be able to dive into statistical plots and reports, watch them in 4D (in 3D space and time), manipulate them, and “wander” around them. Although significant progress has been made, there is still a need for more processing power, higher resolution, better design of the scenarios, and more advanced haptic devices in order to achieve highly realistic environments
