High-fidelity, multilingual, zero-shot speech generation. Provide a reference recording once, then change the text freely, even switching languages, while the speaker identity remains clearly recognizable.
Audio8-TTS-0.6B (Preview) is a model for high-fidelity, multilingual, zero-shot speech generation. A reference recording is needed only once; new text can change freely, even across languages, while the voice identity remains clear.
The main model contains 601,159,424 parameters, excluding the neural audio codec shipped with the model. It is designed for GPU serving and supports paged attention, dynamic batching, reference-audio encoding, waveform decoding, and an OpenAI-compatible speech interface through SGLang Omni.
DualAR: separate semantic and acoustic detail
A spoken sentence is governed by two time scales. Slow AR advances one semantic token per codec frame, determining text alignment, pauses, and the overall direction of the sentence. Fast AR reads Slow AR hidden states and fills the acoustic codebooks along RVQ depth within the current frame. With separate responsibilities, long-range structure and local fidelity do not compete for one token sequence.
Figure 1: Target text is always present; reference audio and its transcript are optional conditions. Slow AR determines the frame-level plan, Fast AR traverses the multi-layer RVQ, and the codec reconstructs the waveform.
About 21.5 frames per second; up to 2,048 packed text/audio positions
2,048 samples correspond to about 46.4 ms. Slow AR makes about 21.5 semantic decisions per second, and Fast AR fills each decision with ten layers of residual information. Earlier codebooks establish a rough structure; later codebooks add timbre, articulation, and spectral texture. This depth-wise token budget is key to balancing coherence and detail.
Start with one reference voice
Zero-shot cloning does not require fine-tuning for every speaker. The system encodes a short reference recording together with its accurate transcript, then sends the pair as conditions to DualAR when generating new text. The reference transcript is part of the condition, not a note beside the recording; clean audio and accurate text usually produce the most stable results.
Loading selected audio samples...
Multilingual and cross-lingual
0.6B covers Chinese, Dutch, English, French, German, Italian, Japanese, Korean, Polish, Cantonese, and Spanish. A Chinese reference recording can guide an English sentence, and an English reference can guide Chinese: the text language changes, while the voice anchor remains.
SOTA-class performance at a compact scale
In public Seed-TTS and CV3 comparisons, 0.6B is one of the smallest main models included while staying in the leading group for English WER, Chinese CER, and multilingual error rates. Different projects use different test sets and normalization, so these results should be treated as public reference results.
Evaluation
Public result
Seed-TTS
EN 1.506 WER / 63.2 SIM; ZH 0.950 CER / 73.1 SIM
CV3
ZH 3.205; EN 3.128; Hard-ZH 10.535
Seed-TTS: lower WER/CER is better; higher SIM is better.CV3: public multilingual error-rate comparison.
The Audio8 repository provides an SGLang Omni adapter with paged attention, dynamic batching, reference-audio encoding, waveform decoding, and an OpenAI-compatible /v1/audio/speech endpoint. In a warm single-stream test on one NVIDIA H20, generating 128 frames had p50 latency of 0.691 s and RTF of 0.116; cold start and compilation time were excluded.
Preview boundaries and responsible use
0.6B is in Preview. Test experimental languages and dialects on the data and hardware of the target application. Zero-shot cloning requires a clean reference recording, accurate text, and speaker authorization; disclose generated audio where appropriate and evaluate accuracy, safety, and legal compliance before production use.
