dorsal/arxiv
View SchemaDeepfake detectors are DUMB: A benchmark to assess adversarial training robustness under transferability constraints
| Authors | Adrian Serrano, Erwan Umlil, Ronan Thomas |
|---|---|
| Categories | |
| ArXiv ID | 2601.05986vv1 |
| URL | https://arxiv.org/abs/2601.05986 |
| License | http://creativecommons.org/licenses/by/4.0/ |
Abstract
Deepfake detection systems deployed in real-world environments are subject to adversaries capable of crafting imperceptible perturbations that degrade model performance. While adversarial training is a widely adopted defense, its effectiveness under realistic conditions -- where attackers operate with limited knowledge and mismatched data distributions - remains underexplored. In this work, we extend the DUMB -- Dataset soUrces, Model architecture and Balance - and DUMBer methodology to deepfake detection. We evaluate detectors robustness against adversarial attacks under transferability constraints and cross-dataset configuration to extract real-world insights. Our study spans five state-of-the-art detectors (RECCE, SRM, XCeption, UCF, SPSL), three attacks (PGD, FGSM, FPBA), and two datasets (FaceForensics++ and Celeb-DF-V2). We analyze both attacker and defender perspectives mapping results to mismatch scenarios. Experiments show that adversarial training strategies reinforce robustness in the in-distribution cases but can also degrade it under cross-dataset configuration depending on the strategy adopted. These findings highlight the need for case-aware defense strategies in real-world applications exposed to adversarial attacks.
{
"annotation_id": "5fcee3d6-c14d-4e7c-aba4-cb815a0503e9",
"date_created": "2026-02-17T05:53:05.064000Z",
"date_modified": "2026-02-17T05:53:05.064000Z",
"file_hash": "1901b8c599a020ea90b3061f941fff7880275fe9c2a3610d6e8f50965c65950d",
"private": false,
"record": {
"abstract": "Deepfake detection systems deployed in real-world environments are subject to adversaries capable of crafting imperceptible perturbations that degrade model performance. While adversarial training is a widely adopted defense, its effectiveness under realistic conditions -- where attackers operate with limited knowledge and mismatched data distributions - remains underexplored. In this work, we extend the DUMB -- Dataset soUrces, Model architecture and Balance - and DUMBer methodology to deepfake detection. We evaluate detectors robustness against adversarial attacks under transferability constraints and cross-dataset configuration to extract real-world insights. Our study spans five state-of-the-art detectors (RECCE, SRM, XCeption, UCF, SPSL), three attacks (PGD, FGSM, FPBA), and two datasets (FaceForensics++ and Celeb-DF-V2). We analyze both attacker and defender perspectives mapping results to mismatch scenarios. Experiments show that adversarial training strategies reinforce robustness in the in-distribution cases but can also degrade it under cross-dataset configuration depending on the strategy adopted. These findings highlight the need for case-aware defense strategies in real-world applications exposed to adversarial attacks.",
"arxiv_id": "2601.05986",
"authors": [
"Adrian Serrano",
"Erwan Umlil",
"Ronan Thomas"
],
"categories": [
"cs.CV",
"cs.CR"
],
"license": "http://creativecommons.org/licenses/by/4.0/",
"title": "Deepfake detectors are DUMB: A benchmark to assess adversarial training robustness under transferability constraints",
"url": "https://arxiv.org/abs/2601.05986",
"version": "v1"
},
"schema_id": "dorsal/arxiv",
"source": {
"execution_id": "1469f4c4-1ba3-4f93-9401-08b6b82adeb2",
"id": "arXiv Dataset",
"type": "Model",
"variant": "snapshot-2026-01-17",
"version": "0.1.0"
},
"user_id": 1000002
}