A 170M-parameter zero-shot speech model designed for edge deployment
The smallest zero-shot TTS worth running.
A 170M-parameter zero-shot speech model designed for edge deployment.
0.1B targets local CPUs, low-memory devices, and offline environments. It compresses zero-shot voice conditioning into a 170M-parameter main generative model and provides a complete inference path around ONNX Runtime. Its value is retaining a usable voice-generation pipeline within a compact budget.
0.1B retains the same semantic-to-acoustic handoff as 0.6B. Slow AR advances semantic tokens frame by frame; Fast AR reads its hidden states and completes the acoustic codebooks across ten RVQ layers. The difference is that Slow AR uses the Falcon-H1 hybrid backbone, combining attention layers with Mamba 2 state-space layers and reducing width to 512 while preserving the contract between long-range state and within-frame acoustic decoding.
| Component | Configuration |
|---|---|
| Main model | 170M parameters, excluding the audio decoder |
| Slow AR | 24 layers, width 512, 8 attention heads, 2 KV heads; Mamba d_state=64 |
| Fast AR | 4 layers, width 512 |
| Acoustic representation | 10 codebooks, 4,096 entries each |
| Neural audio codec | 44.1 kHz, 2,048 samples per model frame; about 120M parameters |
| Context | Up to 2,048 packed text/audio positions |
170M describes the main generative model; the separate codec decoder still counts toward storage and memory when producing a waveform.
Audio8-TTS-0.1B ONNX INT8 is more than quantized weights. It combines the Slow AR and Fast AR token-by-token graphs, an FP16 audio codec, explicit loop state, and a CPU inference interface into a path scheduled by ONNX Runtime. After downloading the model, CUDA, PyTorch, and Transformers are not required.
| Module | Runtime form | Role |
|---|---|---|
| Slow AR | INT8 token-by-token graph | Explicitly passes attention and Mamba states; valid-prefix attention limits computation to the valid prefix |
| Fast AR | INT8 token-by-token graph | Generates ten acoustic codebooks frame by frame |
| Audio encoder / decoder | FP16 | Registers reference timbre and reconstructs 44.1 kHz waveforms |
| Output waveform | FP32, mono | Passed to a local file, PCM stream, or service interface |
On a Linux x86_64 test platform, ordinary inference after loading uses about 0.4 GiB. On an 8-thread host, Slow AR takes about 19 ms per token and Fast AR about 8 ms per frame; these figures vary with the ONNX Runtime build, allocator, thread count, and hardware.
Ordinary synthesis needs only Slow AR, Fast AR, and the codec decoder. The codec encoder is loaded only when creating a new voice, converting a reference recording and accurate text into reusable voice codes; after registration it can be unloaded before normal synthesis resumes. Voice registration is a one-time lifecycle operation, not a resident cost for every request.
From model files to the service boundary, the CLI, local HTTP, streaming PCM, voice registration, and OpenAI-compatible interface are all covered.
0.1B primarily targets Chinese and English; other languages are experimental. INT8 quantization can change the sampled token sequence, so the ONNX version should be evaluated separately from the original PyTorch checkpoint. Zero-shot cloning still requires a clean reference recording, accurate text, and speaker authorization.