Minor Manipulations, Major Threat: An Overview of Partially Fake Speech

Speech can easily be manipulated through techniques, such as text-to-speech synthesis, voice conversion, replay, tampering, adversarial attacks, and more. However, when the manipulation is applied only to a minor portion of an audio, the remaining real segments can have a dominant influence upon human listeners and make machine detection extremely challenging. Therefore, there is an urgent need to explore such a scenario, where synthetic speech is embedded within otherwise real audio. The primary objective of this webinar is to review research efforts aimed at defending against such partially fake audio. The presenter will focus on relevant databases, explainable analyses, and three core tasks: (1) Spoof Detection: Whether the utterance is spoofed? This task aligns with the common task in the spoofing community distinguishing whether an audio is real or fake. (2) Spoof Localization: When do spoofs happen? This task aims to determine the location of spoof segments within audio. (3) Spoof Diarization: What attacks when? This task not only locates the spoofed segments but also discriminates the specific spoofing techniques employed.