Supervised Learning of Depth Estimation From Imperfect Rectified Stereo Laparoscopic Images of Liver Surface

Abstract

Depth estimation is crucial in surface reconstruction for laparoscopic minimally invasive surgery (MIS) navigation. The structured light depth recovery algorithm accurately restores pixel-level depth but is time-consuming and requires additional light sources. Using structured light 3D optical reconstruction and network training in porcine laparoscopic environments allows deep learning algorithms to achieve depth recovery in white light settings. Yet, using deep learning algorithms for binocular depth estimation in laparoscopic images may encounter challenges like system-induced calibration errors and significant object depth variations in the laparoscopic field of view, which can impact rectification and depth estimation accuracy. In this paper, a novel ACVNet++ algorithm based on vertical disparity correction is proposed. This method uses structured light reconstruction for depth estimation. which includes deformable convolution modules, vertical disparity correction weights, and edge-enhanced loss functions. A dataset is created for laparoscopic liver depth images, and the ACVNet++ algorithm is tested on this dataset. The experiments demonstrate that, compared to non-learning algorithms based on white light image features, the ACVNet++ algorithm reduces the average absolute error (MAE) and endpoint error (EPE) of depth prediction under white light conditions by 51% and 44%, reaching 2.46 mm and 2.56 pixels, respectively. The results indicate that the edge-enhanced ACVNet++ algorithm improves the ability for 3D surface reconstruction of the liver and holds potential for clinical application.