Augmented Reality for Surgical Applications | OpenSight

Quick Nav

Augmented Reality for Surgical Applications

The Novarad® Opensight® augmented reality system is a unique pre-surgical planning solution built upon the company’s vast experience with image processing. Using OpenSight, clinicians can interactively create 2D, 3D, and even 4D images of patient anatomy, accurately overlaying them directly onto the patient’s body in order to enable accurate registration for surgical planning. Combining patented virtual tool technology with an integrated targeting system enables physicians to see the patient in real time while simultaneously visualizing into the patient, resulting in a more accurate surgical plan.

Contact Us

Combining 3D imaging and augmented reality to see patients and their imaging studies simultaneously.

Unlike previous preoperative planning systems that use virtual reality, OpenSight utilizes the Microsoft HoloLens® headset, which enables clinicians to see 3D patient images in augmented reality while concurrently viewing the patient. Augmented reality also enables physicians to see real-world surroundings at the same time, avoiding the disorientation and confusion inherent with virtual reality.

OpenSight permits an improved understanding of anatomic relationships because the images are co-localized to the patient and continuously aligned using our innovative Persistent Registration™ technology. Preoperative planning with OpenSight enables physicians to highlight relevant anatomy and avoid critical structures while positioning virtual tools and guidance systems to more accurately plan a surgical approach. Multiple headsets can be shared among users, improving the training of less experienced residents. A teaching version of the software allows medical students to perform virtual dissections on cadavers.

The OpenSight system significantly improves my precision in the operating room. Specifically, it allows me to determine exactly where to place my incision thereby allowing me to keep the incision small – minimizing pain and improving overall cosmesis – and also shortens my operative time by allowing me to expose the surgical target more precisely and quickly.”
-Dr. Babak Sarani, MD, FACS, FCCM, Director of Trauma and Acute Care Surgery and Co-Medical Director of Critical Care at the GW Hospital, and Professor of Surgery and Emergency Medicine at GWU School of Medicine.

Preoperative Guidance

The use of medical imaging guidance and navigation to assist with invasive procedures has evolved over many decades as radiology itself has advanced, enhancing patient comfort, quality of care and outcomes. As medial imaging has matured and expanded, so have preoperative surgical guidance techniques. A notable advancement was 3D freehand biopsy guidance, which built upon previous preoperative planning developments. With 3D freehand, tiny application sensors are combined with scanhead localization sensors and a tracking needle to guide ultrasound as it tracks the distal tip of a needle and the scan planes. This allows clinicians to visualize in real time the patient anatomy and needle tip position with unparalleled accuracy, resulting in more accurate biopsy procedures.

OpenSight ushers in a completely new world of preoperative planning, overcoming critical pre-surgical and surgical obstacles and has the potential to improve accuracy, increase operational efficiency and decrease clinician mistakes. The result is improved outcomes, including minimized patient morbidity and mortality.

The Next Generation Solution

While earlier technology was limited in both scope and efficacy, OpenSight is different: it enables users to display, manipulate and evaluate 2D, 3D, and 4D digital images acquired from various modalities, including computed radiography (CR), computed tomography (CT) and magnetic resonance imaging (MRI) — all in real time. The solution was designed to visualize 3D imaging holograms of the patient, on the patient, for preoperative localization and preoperative planning of surgical options. It achieves this by enabling users to segment previously acquired 3D data sets, overlay them, and register the data sets with the same patient anatomy in order to enhance preoperative analysis. The overriding goal is to help doctors better understand the patient’s anatomy and pathology in the context of anticipated surgical procedures.

 OpenSight Technology

OpenSight tools and features include window level, segmentation and rendering, registration motion correction, virtual tools, alignment and the capability to measure distance and imaging intensity values, such as standardized uptake value. OpenSight displays measurement lines, annotations and regions of interest.

The 3D holograms are created by a refractory system in the OpenSight device using a combination of the HoloLens hardware and OpenSight technology for 3D image display and rendering. Images are visible-rendered of the object in the OpenSight device and streamed in 2D format from the Novarad server via wireless communication, which is encrypted for maximum security and patient privacy. Registration of the patient to an image data set, such as MRI or CT, is performed by the OpenSight device, which contains infrared ranging cameras that can map the surface geometry of an object. Placing visual tracking tags on the patient allows for even higher accuracy and follows patients movement.

The camera within the headset has ranging and localizing technology, which maps the surrounding environment–including the patient. It determines where objects are and creates mesh surface maps of these objects to determine 3D positioning. The 3D images are then rendered and surface shells of the patient’s skin are matched to the augmented reality device when the headset is on and in use. Patient and anatomical movement are continuously monitored and compensated for–registration does not require expensive infrared tracking devices in order to perform registration. Anatomy images and the patient being evaluated will only register if the data match, minimizing the potential for preoperative use on the wrong patient with the wrong images. Image capabilities include tumors, other masses, appendix, heart, kidney, bladder, stomach, blood vessels, arteries, nerves and more.

OpenSight Uses

Novarad OpenSight enables a broad range of uses, all designed to provide physicians with the maximum amount of patient information prior to surgery. Uses include:

  • Capability to mark the appropriate entrance point or angle, trajectory and location for needle placement into the patient in order to extract a foreign body, or to place a pedicle screw.
  • Capability to assist operating physicians as they localize anatomy prior to intervention.
  • Superimposition of an anatomic atlas upon the patient’s anatomy, allowing clinicians to more readily identify structures that would need to be treated or avoided during a surgical procedure.
  • Enables surgical trainees to visualize both the internal anatomy from cross-sectional imaging such as CT, MRI, or PET scanning superimposed on a patient prior to actual surgery, providing invaluable 3D understanding of a surgical approach.
  • Enables clinicians to perform surgical procedures that are exceedingly complex and require a much greater depth of understanding in order to execute, such as repair of congenital heart malformations where complex three-dimensional vascular anatomy makes surgical treatment difficult.
  • Enables colorization of target organs as well as outlining or annotation in the medical images.
  • Allows surgeons to better understand complex anatomy and disease processes by taking the data-rich information they already have and providing it in a more accessible format through holographic imaging.
OpenSight allows the user to visualize the area of interest even before making an incision. Direct projection of these holographic reconstruction based on pre-operative CT/MRI can potentially increase the safety, and the efficacy of surgical procedures by reducing the amount of x-ray radiation utilized, accuracy of spine instrumentation placement, and visualization of key anatomic structures during brain surgery. When holographic models can be stereotactically synced, and if these are anatomically accurate, technology such as this can significantly improve the current state-of-the-art navigation system. -Jang W. Yoon, MD, MSc
Complex Spine Fellow
University of Miami

U.S. Pat. No. 9892564

U.S. Pat. No. 10010379

OpenSight is 510(k) cleared for its intended use.