A Training-Free Proactive Defense Against Partial Speech Manipulation via Self-Embedding Steganography

Yigitcan Özer🔗
Zhe Zhang🔗
Wanying Ge🔗
Xin Wang🔗
Junichi Yamagishi🔗

Abstract. Partial deepfake speech, where only limited segments of an utterance are synthesized or manipulated, poses a significant challenge to existing deepfake detection systems. As the proportion of spoofed regions decreases, passive detectors become increasingly unreliable, and accurate detection and restoration remain challenging. In this paper, we revisit audio steganography from a new perspective and propose its use as a proactive defense against partially deepfaked audio. In particular, we consider a self-embedding strategy in which a clean speech signal embeds a compressed representation of itself, enabling post-hoc extraction of reference content. We demonstrate how existing audio steganography methods can be repurposed to support detection of partial deepfakes through codec-based restoration. Experiments on a benchmark dataset show that the proposed approach complements passive defenses. Remarkably, the proposed method operates without any training, providing a robust and data-efficient alternative for partial deepfake detection.

Pipeline: input speech is codec-encoded and self-embedded (publisher); an attacker partially manipulates the published audio; the receiver decodes, reconstructs, and computes a mismatch score to detect and localize the manipulation, then repairs the affected span. — The self-embedding pipeline. The publisher compresses the speech with a neural codec and hides the resulting tokens inside the audio itself. The receiver extracts these tokens, reconstructs a reference, and compares it with the received audio: a mismatch reveals that the speech was manipulated. The samples below go one step beyond the Interspeech paper, which only covers detection. They also localize and repair the manipulated interval, using the mechanism from our follow-up work on top of the same SNAC-based setup [1].

Listen to five manipulations

Each example below: the authentic utterance, the partially manipulated (attacked) version an attacker publishes, and the repaired version reconstructed once the manipulated span is localized. These examples come from the AV-Deepfake1M dataset [2], which is based on VoxCeleb2 [3].

TTS replacement

utt id00018_Bg3MvcwUF_M_00028

Manipulated span380 ms

Originaljobs plan as the president's calling it is more of the same continuing unemployment benefits for 99 weeks

Manipulatedjobs plan as the president's calling it is more of the different. continuing unemployment benefits for 99 weeks

Authentic

Watermarked

Attacked

Recovered (no repair)

Repaired

Deletion

utt id02497_sOC9GrVewX0_00187

Removed600 ms

OriginalYeah. That's certainly discordant and commenting on action movies. But you know, it's one guy taking on the mob. So it's still linear.

ManipulatedYeah. That's certainly discordant and commenting on action movies. But you know, it's one guy on the mob. So it's still linear.

Authentic

Watermarked

Attacked

Recovered (no repair)

Repaired

Direct replacement

utt id01931_T3R8JkaPWLY_00277

Manipulated span1140 ms

OriginalAnd if I could do better than that or manage to get a point for the team, I think it would be like a victory.

ManipulatedAnd if I could do better than that or get to get a point for the team, I think it would be like a victory.

Authentic

Watermarked

Attacked

Recovered (no repair)

Repaired

Insertion · same speaker

utt id01778_ka0446tibig_00137

Inserted530 ms

OriginalHold them more accountable, make those standards higher. You have people that practice law in our lawyers and go to school for eight years, but you can become a cop in six months and don't have to have the same amount of training as a cosmetologist.

ManipulatedHold them more accountable, make those standards higher. You have people that practice law in our lawyers and go to school for eight years, years, but you can become a cop in six months and don't have to have the same amount of training as a cosmetologist.

Authentic