Immersive medical imaging visualization with Specto Medical and UnitySpecto Medical is a Swiss software company, founded in 2022, as a spin-off from the University of Basel, Switzerland. Specto Medical has evolved into a promising healthtech venture. Achieving regulatory milestones has been critical to Specto’s journey from research labs to operating rooms. With backing from Johnson & Johnson’s JLABS incubator and a successful seed round, the company recently achieved FDA 510(k) clearance for clinical use, with CE marking on the horizon.In this article, discover how Specto Medical is redefining pre-surgical planning and patient communication through immersive 3D visualization — and the journey behind their FDA-cleared XR platform built on Unity that’s transforming medical imaging into an interactive, intuitive experience.Note: This article contains medical images that may be uncomfortable for some.In a pre-surgical planning session, a neurosurgeon puts on a VR headset and steps inside a patient’s brain scan. A complex tangle of blood vessels is displayed in three dimensions around them and a vascular abnormality, a so-called aneurysm, is quickly identified. What was once a stack of grayscale computed tomography (CT) slices is now an immersive 3D environment, and the surgeon can literally walk around the aneurysm to examine it from every angle. This isn’t a far-off prototype; it’s a daily reality with Specto, a cutting-edge medical imaging visualization platform built with Unity.From complexity to clarityIn the high-stakes world of neurosurgery, clarity is everything. Today's medical imaging technology produces incredibly detailed scans, but these rich datasets are often underutilized in surgical planning and patient communication. Traditional 2D visualizations require surgeons to mentally reconstruct complex 3D anatomies from flat slices—a process that's both cognitively demanding and prone to misinterpretation. Meanwhile, patients frequently struggle to comprehend their diagnoses when presented with grayscale images and medical jargon.Specto addresses these challenges by transforming the raw imaging data into intuitive and interactive 3D visualizations that both medical professionals and patients can easily understand. For surgeons, this means reduced cognitive load and enhanced spatial understanding. For patients, it provides an accessible window into their own conditions, replacing anxiety and confusion with clarity. Transforming medical imaging into actionable insightSpecto turns that promise of clarity into action by offering a suite of practical tools that bring medical imaging to life in three dimensions. From immersive visualization and surgical planning to remote collaboration and hospital integration, the solution supports clinicians at every step—from diagnosis to the operating room.1. High-fidelity medical visualization in XRAt its core, Specto enables health care professionals to visualize traditional medical imaging data, such as CT and MRI, in an interactive XR environment within seconds. Using direct volume rendering with real-time interaction, Specto eliminates the need for lengthy image analysis, 3D reconstructions or surface mesh creation. Radiological data is rendered on-the-fly via ray marching, preserving all the rich detail of the original scan.Physicians can adjust visualization parameters dynamically to highlight different tissues or structures and isolate the organs or pathology of interest. The volume renderer is optimized to maintain high frame rates even with complex datasets, ensuring a smooth and comfortable experience in extended XR sessions. 2. Surgical planningGoing beyond visualization, Specto includes interactive tools for preoperative planning. Surgeons can annotate structures of interest, use measurement instruments to gauge distances or angles between anatomical landmarks, and virtually position implants within the patient's anatomy to assess their fit—all before making a single incision in the operating room.For surgeons, one of the most transformative aspects is the ability to visualize complex anatomy with true depth perception. The mental gymnastics of reconstructing a 3D picture from flat CT or MRI scans is no longer necessary. The patient-specific anatomy is rendered in 3D, at scale, right before their eyes. This enhanced spatial awareness improves surgical confidence and can result in a déjà vu effect in the operating room.3. Online collaborationSpecto supports multi-user sessions in a shared virtual environment. Multiple surgeons, radiologists, or students can join the same 3D scene and collaboratively examine a case together. In an educational setting, an instructor can guide residents through anatomy in VR. In a clinical context, surgeons across different locations and expertise areas can meet in a shared virtual space to discuss a surgical approach.These collaborative features also enhance patient education. Being told about a diagnosis in technical jargon or looking at fuzzy black-and-white scan images can be intimidating and confusing for patients.Using Specto, care givers can show patients what the problem is inside their body and how it will be fixed in an intuitive format that requires no medical background to understand. This transparency helps patients become more active participants in their care journey rather than passive recipients of treatment.4. Hospital integrationSpecto can seamlessly integrate with the hospital imaging and archiving systems, allowing surgeons to quickly pull scans into the visualization environment. This integration saves precious minutes during morning surgical briefings, as clinicians can immediately access the patients' scans and discuss the case in a virtual 3D environment. Powering precision with Unity’s real-time 3D platformTo create this real-time medical visualization platform, Specto's developers chose Unity as their foundation—a decision that has paid dividends in agility and capability.1. Rapid prototyping and iterationIn the early prototyping phase, the team needed to experiment with various volumetric rendering techniques and interaction concepts in VR. Unity's development environment provided ideal tools for rapid iteration, particularly for shader development. Unity’s hot-reloading capability allows shader code to be updated in real-time without recompiling the entire application, creating an efficient workflow for refining volume rendering algorithms. This feature proved invaluable when fine-tuning the ray marching shaders that power Specto's visualization.2. High-Performance graphics and integrationUnity's extensible graphics pipeline enabled Specto's team to implement their custom ray-marching renderer while still benefiting from the core capabilities of a robust, production-proven rendering system. They leveraged custom compute shaders for GPU-accelerated volume processing, achieving the performance needed for medical visualization in VR.The profiling tools built into Unity helped identify and resolve performance bottlenecks, resulting in a smooth experience where surgeons can instantly adjust tissue visibility or apply cutting planes and segmentations to highlight internal structures. 3. Cross-platform XR supportOne of the most important strengths of Unity is its broad platform and device support. With Unity, the team can deploy Specto to various hardware without dedicating significant engineering resources on integrating new APIs. The application supports VR headsets like Meta Quest 3 and AR devices like HoloLens 2. Unity abstracts many hardware differences, including the underlying graphics backends used by different headsets, so adding support for new devices is relatively straightforward. This flexibility future-proofs Specto as new XR hardware emerges. 4. Unity ecosystem benefitsBuilding on Unity allowed Specto to tap into a mature and well-supported development ecosystem. The availability of resources on Unity Asset Store such as prebuilt assets, tools, and third-party packages significantly accelerated the team’s progress—freeing them to focus on their core innovations rather than reinventing foundational components.For example, Specto used Mirror, a third-party multiplayer framework, to quickly implement collaborative features without building a networking layer from scratch. Other off-the-shelf resources, such as simple 3D models, debugging tools, and environment assets, helped streamline development and prototyping.Unity’s ubiquity also means broad compatibility with third-party services and hardware integrations, while support for the .NET Standard enables access to trusted libraries for encryption, advanced logging, and other essential backend functionality. Trusted by surgeons, validated on the front lines of careThe most compelling validation comes from the surgeons using the software.Prof. Dr. med. Raphael Guzman, Head of Neurosurgery at University Hospital Basel, and Dr. med. Thorsten Gehrke, Chief Physician at Helios Clinic in Hamburg, attest how using Specto has reduced surgery time and improved safety through more precise, minimally invasive approaches.Shaping what’s next in surgical careAs XR technology advances—with lighter headsets, sharper displays, and longer battery life—the experience of using Specto will only become more seamless and impactful. It’s easy to envision a future where reviewing scans in VR is as routine for surgeons as reading case notes, and where patients receive personalized 3D tours of their anatomy as part of standard care.By combining medical expertise with Unity's real-time 3D technology, Specto Medical is helping to bridge the gap between complex medical data and intuitive understanding—enabling clinicians to make more confident decisions and empowering patients with clearer insights into their health.Immersive healthcare is no longer on the horizon; it’s here—and it’s already shaping better outcomes for both surgeons and patients.