CasTensoRF: Cascaded Tensorial Radiance Fields for Novel View Synthesis

Abstract

Novel views synthesized from Neural Radiance Fields (NeRF) have reached remarkable rendering quality. However, a 5D radiance field volume is too large to be stored or directly rendered. In order to efficiently reconstruct and manipulate such a high-order tensor, we leverage inspirations from previous tensor decomposition methods, e.g. Tensorial Radiance Fields (TensoRF) and Hierarchical Tucker decomposition. And we propose a Hierarchical Vector-Matrix decomposition (HVMD) framework to learn a sparse approximation of high-order tensors. The proposed HVMD takes advantage of tensor separation and factorization properties and builds a hierarchical scheme that enables a better approximation of the high-order tensor with a very limited number of parameters. Our method achieves better-rendering quality than TensoRF in the NeRF-synthetic dataset given the same model size. The advantage gets more significant when the network parameter number becomes extremely small.