JPEG Pleno Light Field Encoder with Mesh based View Warping

Abstract

We introduce mesh-based view warping to the JPEG Pleno light field coding framework and replace the standardized sample-based forward warping and splatting of reference texture with mesh-based backward warping, which allows for a more disciplined interpolation of the reference texture for predicting the target view. Instead of coding depth maps with JPEG 2000, which is the default option of the JPEG Pleno framework, we employ a recent extension referred to as JPEG 2000 Part 17. This extension utilises breakpoints to describe discontinuity boundary geometry for the purpose of modifying the predict and update lifting steps in the vicinity of detected discontinuities. We directly decode breakpoints and corresponding DWT coefficients onto a mesh and describe a scheme to construct a single, consolidated mesh for a large group of views, borrowing information from multiple coded depth maps. Results show that the cumulative impact of all these modifications enable improved rate-distortion performance in comparison with the default operation of the JPEG Pleno encoder.