Yihao Huang (Contact)
The creation and the manipulation of facial appearance through deep generative approaches, known as DeepFake, have achieved significant progress and promoted a wide range of benign and malicious applications, eg, visual effect assistance in movie and misinformation generation by faking famous persons. The evil side of this new technique poses another popular study, ie, DeepFake detection aiming to identify the fake faces from the real ones. With the rapid development of the DeepFake-related studies in the community, both sides (ie, DeepFake generation and detection) have formed the relationship of battleground, pushing the improvements of each other and inspiring new directions, eg, the evasion of DeepFake detection. Nevertheless, the overview of such battleground and the new direction is unclear and neglected by recent surveys due to the rapid increase of related publications, limiting the in-depth understanding of the tendency and future works.
To fill this gap, in this paper, we provide a comprehensive overview and detailed analysis of the research work on the topic of DeepFake generation, DeepFake detection as well as evasion of DeepFake detection, with more than 191 research papers carefully surveyed. We present the taxonomy of various DeepFake generation methods and the categorization of various DeepFake detection methods, and more importantly, we showcase the battleground between the two parties with detailed interactions between the adversaries (DeepFake generation) and the defenders (DeepFake detection). The battleground allows fresh perspective into the latest landscape of the DeepFake research and can provide valuable analysis towards the research challenges and opportunities as well as research trends and directions in the field of DeepFake generation and detection. We also elaborately design interactive diagrams (http://www.xujuefei.com/dfsurvey) to allow researchers to explore their own interests on popular DeepFake generators or detectors.
This webpage (in beta version) provides interactive diagrams as an auxiliary to the figures in the paper.
The full-resolution videos for "Identity Swap" and "Expression Swap" categories in Figure 1 of the paper are shown here to better illustrate the DeepFake phenomenon.
Example 1 (subtle)
Example 2 (subtle)
Example 3 (less subtle)
Example 4 (less subtle)
Example 1 (subtle)
Example 2 (subtle)
Example 3 (less subtle)
Example 4 (less subtle)
The chord diagram represents the comparison among the existing detection methods. The node indicates the method for DeepFake detection and the link represents that one of the work is served as the baseline in evaluation. The baselines include with and without the peer reviewed works and typical CNN models. Explore the interactive diagram by hovering the cursor on top of the links or moving the nodes around.
The chord diagram represents the comparison among the existing detection methods. The node indicates the method for DeepFake detection and the link represents that one of the work is served as the baseline in evaluation. The baselines include with and without the peer reviewed works and typical CNN models. Explore the interactive diagram by hovering the cursor on top of the links or clicking the nodes or moving the nodes around.
The "Battleground". The Sankey diagram shows the interactions between various DeepFake detection methods (right column) and various DeepFake generation methods (left column). Both of the generation and detection methods are sorted by the release time and labeled with the corresponding years. The four colors represent the different types of detection methods. Blue is Type-I (spatial based) methods, green is Type-II (frequency based) methods, yellow is Type-III (biological signal based) methods, and red is Type-IV (others) methods. Explore the interactive diagram by hovering the cursor on top of the links or moving the nodes around.
Relation pairs of the image-based and video-based DeepFake generation methods that are simultaneously evaluated by some DeepFake detection methods. Explore the interactive diagram by hovering the cursor on top of the links or moving the nodes around.
The humanity may reach a stage where DeepFakes have become so genuinely looking that they are beyond human and machine’s capability to distinguish from the real ones, a
"DeepFake singularity", if you will. If this day is inevitable, be it a utopia or a dystopia, perhaps a more interesting era is upon us. Are we brave enough to embrace it?
International Journal of Computer Vision (IJCV), 2022.
@article{fx_ijcv22_dfsurvey,
author={Felix Juefei-Xu and Run Wang and Yihao Huang and Qing Guo and Lei Ma and Yang Liu},
title={{Countering Malicious DeepFakes: Survey, Battleground, and Horizon}},
journal={International Journal of Computer Vision (IJCV)},
year={2022}
}