Quality-based Artifact Modeling for Facial Deepfake Detection in Videos

Facial deepfakes are becoming more and more realistic, to the point that it is often difficult for humans to distinguish between a fake and a real video. However, it is acknowledged that deepfakes contain artifacts at different levels; we hypothesize a connection between manipulations and visible or non-visible artifacts, especially where the subject’s movements are difficult to reproduce in detail. Accordingly, our approach relies on different quality measures, No-Reference (NR) and Full-Reference (FR), over the detected faces in the video. The measurements allow us to adopt a frame-by-frame approach to build an effective matrix-based representation of a video sequence. We show that the results obtained by this basic feature set for a neural network architecture constitute the first step that encourages the empowerment of this representation, aimed to extend our investigation to further deepfake classes. The FaceForensics++ dataset is chosen for experiments, which allows the evaluation of the proposed approach over different deepfake generation algorithms.