A 3D Augmented Reality Guidance System for Minimally Invasive Surgery Using the Unity Game Engine (MAY_U26CMP)

About the Project

Are you interested in applying cutting-edge AI and augmented reality (AR) to real-world healthcare challenges? This PhD studentship offers an exciting opportunity to develop next-generation imaging technology that could transform how clinicians perform minimally invasive procedures.

X-ray fluoroscopy is a cornerstone of modern medicine, providing real-time imaging to guide devices such as catheters during procedures. However, it only offers two-dimensional (2D) views, making it difficult to interpret complex 3D anatomy. In addition, repeated exposure to X-ray radiation presents risks for both patients and healthcare professionals. This project addresses these challenges by developing an innovative AR framework that enhances clinical decision-making while improving safety.

You will design and build a system that overlays patient-specific 3D anatomical models onto live X-ray images, effectively bringing depth and clarity to existing imaging. These models will be reconstructed from routine clinical scans using tools such as 3D Slicer, and integrated into a real-time AR environment developed with Unity 3D game engine and OpenCV.

The research centres on two key challenges: real-time tracking surgical devices and developing intelligent 2D–3D registration methods. You will explore AI-driven approaches to fuse imaging data, track surgical instruments, and align virtual models with live fluoroscopy for precise spatial guidance.

You will join a supportive research team, working alongside fellow PhD students and collaborating with clinical researchers at King’s College London. With access to state-of-the-art AI workstations and strong interdisciplinary support, you will gain valuable experience in medical imaging, computer vision, and machine learning.

[1] Xi, L., et al. Catheter detection and segmentation in X-ray images via multi-task learning. Int J CARS 21, 163–173, 2026.

[2] Y. Ma, et al. An Integrated Platform for Image-Guided Cardiac Resynchronization Therapy. Physics in Medicine Biology. 57(10), 2953-68, 2012

[3] Y. Ma, et al. Real-time registration of 3D echo to x-ray fluoroscopy based on cascading classifiers and image registration, Physics in Medicine Biology, 66(5), 2021.

The School of Computing Sciences (https://www.uea.ac.uk/about/school-of-computing-sciences) provides a vibrant research environment for conducting Computing and allied research and training. We collaborate with multi-national companies such as Apple, BT, the National Trust and Aviva, research institutes in the Norwich Research Park (https://www.norwichresearchpark.com), as well as other universities and industries in the UK and overseas. We are also members of the Turing University Network, a group of 65 UK universities working together to advance world-class research and build skills for the future.

The successful candidate will also be expected to contribute to Tutor activities for laboratory support on our BSc and MSc Courses in Artificial Intelligence, Data Science, Computing Sciences and Cyber Security commensurate with their core expertise, within the working hours permitted for full-time Postgraduate Researchers.

Funding Notes

This PhD project is in a competition for a Faculty of Science funded studentship. Funding is available to UK applicants and comprises ‘home’ tuition fees, an annual stipend for 3 years and a research training support budget.