ElevenLabs recreated Monica’s voice from an old home video, allowing her to speak at her daughter’s wedding. The company says its technology has restored voices for over 11,000 people and aims to help one million worldwide.
Key points
Restored voice: Monica’s speech at wedding using recreated audio.
Scale: technology used for more than 11,000 people; commitment to assist one million voice-loss patients.
Sources
Continuous Audio Thinking for Large Audio Language Models adds latent workspace
Continuous Audio Thinking (CoAT) introduces a continuous latent workspace for organizing acoustic information before response generation. CoAT is processed in a single prefill and does not add autoregressive decoding cost. Benchmarks on Qwen2-Audio, Qwen2.5-Omni-7B, and Audio Flamingo 3 show performance gains across audio reasoning, understanding, music classification, speech emotion, and transcription.
Key points
Continuous thinking block processed in a single prefill, no additional autoregressive decoding cost.
Gains reported on three LALMs: Qwen2-Audio, Qwen2.5-Omni-7B, and Audio Flamingo 3 across a broad benchmark suite.
Sources
AU-Harness: Open-Source Toolkit for Efficient Unified Audio-LLM Evaluation
AU-Harness is an open-source toolkit that speeds audio-LLM evaluation by 77% compared with existing toolkits. It adds multi-turn dialogue support and standardized evaluation protocols for fair model comparison.
Key points
77% reduction in evaluation processing time over existing toolkits
Provides unified, scalable evaluation framework with multi-turn dialogue support
Sources
Riverside Agents launch AI production team for video
Riverside Agents introduces an AI production team built only for video, featuring roles named Producer, Editor, Clipper, and Marketer. The team claims to handle preparation, full editing, moment cutting, and live publishing.
Key points
Producer preps the session
Editor creates the full edit
Clipper cuts the best moments
Marketer gets it live
Sources
Why Full-Duplex Speech Models Collapse in Long Conversations
Why Full-Duplex Speech Models Collapse in Long Conversations presents a new benchmark for long-duration dialogue. It shows that existing full-duplex models degrade or exhaust context within minutes and proposes two fixes that lower collapse in hour-long replays.
Key points
Full-Duplex Long-Horizon Benchmark (FD-LHB) evaluates five recent models on long replayed conversations and live interaction with an LLM partner.
Initialization-Anchor Pinning (IAP) and Duplex Repetition Penalty (DRP) reduce conversational collapse in replays up to an hour without retraining.
Sources
Noisy Target Training Meets Drifting for Weakly-Supervised Speech Enhancement
Noisy Target Training Meets Drifting for Weakly-Supervised Speech Enhancement adds a Sinkhorn drifting loss to NyTT. The loss aligns enhanced speech distribution with unpaired clean speech in latent space. Experiments on VoiceBank-DEMAND and CHiME-3 real recordings demonstrate the combination’s efficacy.
Key points
Combines Noisy Target Training with a Sinkhorn drifting loss.
Uses only unpaired clean speech as the target distribution.
Evaluated on VoiceBank-DEMAND and CHiME-3 real recordings.
Sources
X-Dubbing: Speech-to-Speech Translation under Temporal and Visual Constraints announced
X-Dubbing introduces a joint time-aware speech-to-text translation and video-aware text-to-speech system for cross-lingual dubbing. The method consists of a Dubbing Translator that creates temporally structured scripts and a Voice Actor that renders them in sync with the source video. It requires no paired dubbing data, training on speech-text and monolingual audio-visual data respectively.
Key points
Dubbing Translator generates temporally structured scripts.
Voice Actor renders scripts synchronized with the source video using a language-agnostic timing budget.