Adaptive energy-aware free viewpoint video transmission over wireless networks

Abstract

As consumer-level cameras are becoming cheaper and cheaper, the scene of interest can now be captured by multiple cameras simultaneously from different viewpoints. By transmitting texture and depth maps of the views captured using two adjacent cameras, users can synthesize any view in between using depth image based rendering (DIBR) technique such as 3D-warping. This provides users with continuous view angles and is key technique for Free Viewpoint TV (FTV), etc. In this scenario, multiple views must be delivered through bandwidth-limited wireless networks, how to adaptive reduce the source coding rate to fit the bandwidth becomes an important issue. The standard multi-view video codec H.264 MVC explores the inter-view and intra-view correlation to reduce source encoding rate. But all views are required at the client side for decoding any single view, which is apparently not transmission efficient. A better solution is using H.264 to differentially encode the two adjacent views separately, but still this requires double-folder bandwidth comparing to the traditional video streaming. Multiple description coding (MDC) encodes one view into multiple descriptions for robust transmission. This inspires another option: sending only part of each view. This can greatly reduce the source encoding rate, but interpolating skipped frames before synthesizing the middle view consumes mobile devices' precious battery life. Whether to send at a lower source encoding rate but with extra energy consumption or send at a high source encoding rate is nontrivial, and has not been studied in any formal way. In this paper, we investigated both source encoding rate reduction and energy consumption with the bandwidth constraint. We formulated the problem into an optimization problem, i.e., maximizing the system utility, which is defined as the received video quality minus the weighted energy consumption. Simulation results showed that our proposed scheme provides a significant advantage over competing schemes in typical network conditions.