Learning-based Point Cloud Geometry Coding Rate Control

Abstract

Multimedia applications have been evolving towards providing users with more immersive and realistic experiences. A common way to model the light available for the users' eyes is the so-called plenoptic function – a powerful 7D representation of light. There are three main types of 3D representation models for the plenoptic function, capable of expressing the light information needed to offer 6-Degrees of Freedom (DoF) experiences, namely light fields, meshes, and Point Clouds (PCs). This paper focuses on PCs since they allow representing and processing objects directly in the 3D space, facilitating user interaction and navigation in a multitude of application domains. Since the illusion of real surfaces is provided by high-density point sets, a good quality of experience requires a rather large set of points to represent a single PC, thus originating huge amounts of data to be stored and/or transmitted. Consequently, PC Coding (PCC) with significant compression levels is a must to reduce the PC data to more manageable sizes and bring PC-based applications to practical deployment. The promising results for image coding led the Joint Photographic Experts Group (JPEG) to launch a standardization project especially targeting Deep Learning (DL)-based PCC, with a final Call for Proposals in January 2022. The best performing response to this call [1] became the JPEG Pleno Learning-based PCC Verification Model (VM), which is the seed codec for the final standard. In this codec, the rate may be controlled through a set of coding parameters, largely depending on the specific PC to code, notably its sparsity and homogeneity.