Content-based video sequence representation

Abstract

The compact representation of video sequences is important for many applications, including very low bit-rate video compression and digital image libraries. We discuss here a novel approach, called generative video, by which video sequences are compactly represented in terms of their contents. This is achieved by reducing the video sequence to constructs. Constructs encode video sequence contents, such as, the shape and the velocity of independently moving objects, and the camera motion. Constructs are of two types: world images and generative operators. World images are augmented images incrementally generated. Generative operators, access video sequence contents and reconstruct the sequence from the world images. The reduction of a video sequence to constructs proceeds in steps. First, the shape of independently moving regions in the image is tessellated into rectangles. Second, world images are generated using the tessellated shape representation. This is described with an experiment using a real video sequence.