Neural Radiance Fields (NeRF) for 3D Reconstruction of Monocular Endoscopic Video in Sinus Surgery

Abstract

Abstract Objective To validate the use of neural radiance fields (NeRF), a state‐of‐the‐art computer vision technique, for rapid, high‐fidelity 3‐dimensional (3D) reconstruction in endoscopic sinus surgery (ESS). Study Design An experimental cadaveric pilot study. Setting Academic medical center. Methods Complete bilateral endoscopic sinus surgery was performed on 3 cadaveric specimens, followed by postsurgical nasal endoscopy using a 0° rigid endoscope. NeRF was utilized to generate 3D reconstructions from the monocular endoscopic video feed. Reconstructions were calibrated, scaled, and then co‐registered to postoperative computed tomography (CT) image sets to assess accuracy. Reconstruction error was determined by comparing ethmoid sinus measurements on NeRF reconstructions and CT image sets. Results NeRF‐based 3D scene reconstructions were successfully generated and co‐registered to corresponding CT images for 5 out of 6 cadaveric nasal cavity sides. The mean reconstruction errors and standard error of the mean (SEM) for ethmoid length and height were 0.17 (SEM 0.59) and 0.70 (SEM 0.44) mm, respectively. Conclusion NeRF demonstrates significant potential for dynamic, high‐fidelity 3D surgical field reconstruction in ESS, offering submillimeter accuracy comparable to postoperative CT data in cadaveric specimens. This innovative approach may ultimately augment dynamic real‐time intraoperative navigation through co‐registration of the 3D reconstruction with preoperative imaging to potentially reduce the risk of injury to critical structures, optimize surgical completeness and, thereby, improve surgical outcomes. Further refinement and validation in live surgical settings are necessary to fully realize its clinical utility.