Dual-Stream Generalized Face Forgery Detection Method Based on Intra-Inter Frame Self-Blending

Abstract

Although existing methods for detecting face forgery perform well within familiar source domains, they often suffer from overfitting, leading to a lack of generalizability in face forgery detection. As a result, their performance significantly decreases when faced with unfamiliar or unknown scenarios. To address this issue, this study proposes a dual-stream generalized face forgery detection method based on intra-inter frame self-blending. The intra-inter frame self-blending module is designed to prevent detector overfitting from unrelated identity information by leveraging inconsistencies within and between frames to generate a diverse and realistic forgery training set. This method enhances the generalizability of the detection model. Additionally, a detection model is developed as a dual-stream network using an RGB-frequency feature-enhancing module, which extracts and fuses RGB and high-frequency features within the shallow layers of the network to capture forged artifacts. This method not only enhances the model performance but also alleviates the increase in model parameter size. Experiments are conducted against nine mainstream methods across four datasets, with the proposed model improving the AUC by 1.52% and EER by 1.5% on average in cross-dataset experiments. In addition, it ranks first or second in all four sub-datasets of different manipulations in cross-manipulation experiments. These results demonstrate that the proposed method achieves excellent generalizability in face forgery detection.