Generate optical flow with conditional generative adversarial network

Abstract

As the cGANs achieves great success on pix to pix problem [12], we proposed a new architecture based on cGAN to solve our optical flow estimation problem. Specifically, we propose a loss function which consists of an adversarial loss and a content loss. The adversarial loss is the pixel-to-pixel loss. We use a discriminator network which is trained to differentiate the ground-truth flow and the generated flow on pixel space. The content loss focuses on perceptual similarity of the ground-truth flow and the generated flow. Our architecture (FlowGan) contains a generator based on FlowNetS with Dense Block to make it deeper and a Markovian discriminator to classify image patch instead of the whole image. We train our network with FlyingChairs datasets and evaluated our network on MPISintel. FlowGan can get competitive results with practical speed.