NeRF for 3D Reconstruction from X-ray Angiography: Possibilities and Limitations
No Thumbnail Available
Date
2023
Journal Title
Journal ISSN
Volume Title
Publisher
The Eurographics Association
Abstract
Neural Radiance Field (NeRF) is a promising deep learning technique based on neural rendering for three-dimensional (3D) reconstruction. This technique has overcome several limitations of 3D reconstruction techniques, such as removing the need for 3D ground truth or two-dimensional (2D) segmentations. In the medical context, the 3D reconstruction of vessels from 2D X-ray angiography is a relevant problem. For example, the treatment of coronary arteries could still benefit from 3D reconstruction solutions, as common solutions do not suffice. Challenging areas in the 3D reconstruction from X-ray angiography are the vessel morphology characteristics, such as sparsity, overlap, and the distinction between foreground and background. Moreover, sparse view and limited angle X-ray projections restrict the information available for the 3D reconstructions. Many traditional and machine learning methods have been proposed, but they rely on demanding user interactions or require large amounts of training data. NeRF could solve these limitations, given that promising results have been shown for medical (X-ray) applications. However, to the best of our knowledge, no results have been shown with X-ray angiography projections or consider the vessel morphology characteristics. This paper explores the possibilities and limitations of using NeRF for 3D reconstruction from X-ray angiography. An extensive experimental analysis is conducted to quantitatively and qualitatively evaluate the effects of the X-ray angiographic challenges on the reconstruction quality. We demonstrate that NeRF has the potential for 3D Xray angiography reconstruction (e.g., reconstruction with sparse and limited angle X-ray projections) but also identify explicit limitations (e.g., the overlap of background structures) that must be addressed in future works.
Description
CCS Concepts: Computing methodologies -> Reconstruction; Applied computing -> Life and medical sciences
@inproceedings{10.2312:vcbm.20231210,
booktitle = {Eurographics Workshop on Visual Computing for Biology and Medicine},
editor = {Hansen, Christian and Procter, James and Renata G. Raidou and Jönsson, Daniel and Höllt, Thomas},
title = {{NeRF for 3D Reconstruction from X-ray Angiography: Possibilities and Limitations}},
author = {Maas, Kirsten W. H. and Pezzotti, Nicola and Vermeer, Amy J. E. and Ruijters, Danny and Vilanova, Anna},
year = {2023},
publisher = {The Eurographics Association},
ISSN = {2070-5786},
ISBN = {978-3-03868-216-5},
DOI = {10.2312/vcbm.20231210}
}