Sain: Similarity-Aware Video Frame Interpolation

Abstract

Video frame interpolation (VFI) aims to synthesize an intermediate frame between two consecutive original frames. Most existing methods simply linearly combine the warped frames, leading to a loss of image texture. Since moving objects usually have similarities in consecutive frames, we propose a similarity-aware video frame interpolation method (SAIN) that searches patches with similar texture in the embedding space from input frames to extract features and capture image details. To gather the frame details and restore image texture, SAIN incorporates an implicit neural representation learning from similar patches to enrich image details and refine outputs in frame synthesis networks. Experiments demonstrate that SAIN preserves image texture and enhances interpolated image quality significantly.