OmniVoice is a single-stage non-autoregressive diffusion TTS model (Qwen3-0.6B backbone, 8-codebook MaskGIT + DAC vocoder). This document covers quantization selection, diffusion step tuning, and audio quality benchmarks.
# Basic synthesis
./build/rs-tts-offline -m models/omnivoice-f16.gguf \
-t "Hello, this is a test." -o output.wav
# With voice cloning
./build/rs-tts-offline -m models/omnivoice-q4_k_m.gguf \
-t "Your text here." --ref reference.wav --ref-text "transcript" \
--n-steps 32 -o output.wav
# Chinese synthesis
./build/rs-tts-offline -m models/omnivoice-q4_k_m.gguf \
-t "你好,这是一段测试文本。" --lang zh --n-steps 32 -o output.wav| Model | Size | bpw | Quality | Speed | Recommendation |
|---|---|---|---|---|---|
| F16 | 1552 MB | 16.0 | ★★★★★ | ★★★ | Baseline, development |
| Q8_0 | 1073 MB | 8.5 | ★★★★★ | ★★★★ | Near-lossless |
| Q5_K_M | 870 MB | ~5.5 | ★★★★★ | ★★★★ | High quality daily use |
| Q4_K_M | 823 MB | ~4.5 | ★★★★ | ★★★★ | Recommended default |
| Q3_K | 696 MB | ~3.5 | ★★★½ | ★★★★ | Good quality/size balance |
| IQ3_XXS | 687 MB | ~3.06 | ★★★ | ★★★★ | Minimum viable for TTS |
| Q2_K | 635 MB | ~2.6 | ★★ | ★★★★ | Degraded, use with caution |
| IQ2_XXS | 606 MB | ~2.06 | ★ | ★★★★ | Not recommended |
Key insight: IQ2_XXS (2.06 bpw) fails on OmniVoice TTS regardless of imatrix calibration. The 28-layer Qwen3 backbone has insufficient parameter redundancy to absorb 2-bit weight noise, causing logit collapse in MaskGIT codebook sampling. IQ3_XXS (~3.06 bpw) is the practical minimum for intelligible output. See Why IQ2_XXS fails below.
MaskGIT iteratively unmasks acoustic tokens. More steps = more refinement opportunities.
| Steps | F16 Quality | IQ3_XXS Quality | Speed (RTF) | Use Case |
|---|---|---|---|---|
| 8 | ★★★ | ★★ | ~0.3× | Fast preview |
| 16 | ★★★★ | ★★★ | ~0.5× | Development testing |
| 32 | ★★★★★ | ★★★½ | ~1.0× | Default (FP16/Q4_K+) |
| 50 | ★★★★★ | ★★★★ | ~1.5× | IQ3_XXS / Q3_K |
| 64 | ★★★★★ | ★★★★½ | ~2.0× | IQ3_XXS最佳质量 |
| 128 | ★★★★★ | ★★★★★ | ~4.0× | Maximum quality (slow) |
F16 / Q8_0 / Q5_K_M / Q4_K_M: --n-steps 32
Q3_K / IQ3_XXS: --n-steps 50
Q2_K: --n-steps 64 (still degraded)
OmniVoice uses Classifier-Free Guidance. Quantized models benefit from adjusted CFG scale and temperature to compensate for reduced logit contrast.
The engine auto-applies quantization-aware CFG adjustments based on detected weight precision:
| Quant Level | CFG Scale | Temperature |
|---|---|---|
| F16 / Q8_0 | 2.0 | 1.0 |
| Q4_0 / Q4_1 / Q4_K | 2.5 | 0.9 |
| Q2_K / IQ2 / IQ3 / IQ4 | 3.0 | 0.85 |
These are applied automatically — no user configuration needed.
Real-Time Factor = synthesis time / audio duration. RTF < 1.0 means faster than real-time.
Test machine: Apple M1 Pro, GPU (Metal). Sentence: "The quick brown fox jumps over the lazy dog."
| Model | Steps=16 | Steps=32 | Steps=50 |
|---|---|---|---|
| F16 | 0.73 | 0.90 | 1.31 |
| Q5_K_M | 0.74 | 1.09 | 1.62 |
| Q4_K_M | 0.61 | 1.11 | 1.54 |
| IQ3_XXS | 0.61 | 1.04 | 1.53 |
| Q3_K | 0.69 | 1.09 | 1.59 |
| Q2_K | 0.64 | 0.98 | 1.44 |
| Model | Steps=16 | Steps=32 | Steps=50 |
|---|---|---|---|
| F16 | 0.52 | 0.81 | 1.11 |
| Q5_K_M | 0.53 | 0.92 | 1.43 |
| Q4_K_M | 0.52 | 0.87 | 1.30 |
| IQ3_XXS | 0.50 | 0.87 | 1.33 |
| Q3_K | 0.51 | 0.89 | 1.44 |
| Q2_K | 0.48 | 0.82 | 1.46 |
All RTF values measured on Apple M1 Pro (Metal GPU), 8 threads. Lower is faster; <1.0 = faster than real-time. IQ types (IQ3_XXS) are slightly faster than K-quants at equivalent step counts due to optimized Metal matrix-multiply kernels.
RTF data will be filled after benchmark completion.
Click ▶ Play to listen in your browser. Compare quality across models and step counts.
| Model | Audio |
|---|---|
| F16 | ▶ Play |
| Q4_K_M | ▶ Play |
| IQ3_XXS | ▶ Play |
| Q2_K | ▶ Play |
| Model | Audio |
|---|---|
| F16 | ▶ Play |
| Q5_K_M | ▶ Play |
| Q4_K_M | ▶ Play |
| IQ3_XXS | ▶ Play |
| Q3_K | ▶ Play |
| Q2_K | ▶ Play |
| Model | Audio |
|---|---|
| F16 | ▶ Play |
| Q4_K_M | ▶ Play |
| IQ3_XXS | ▶ Play |
| Q3_K | ▶ Play |
| Q2_K | ▶ Play |
| Model | Audio |
|---|---|
| F16 | ▶ Play |
| Q4_K_M | ▶ Play |
| IQ3_XXS | ▶ Play |
| Model | Audio |
|---|---|
| F16 | ▶ Play |
| Q5_K_M | ▶ Play |
| Q4_K_M | ▶ Play |
| IQ3_XXS | ▶ Play |
| Q3_K | ▶ Play |
| Q2_K | ▶ Play |
| Model | Audio |
|---|---|
| F16 | ▶ Play |
| Q4_K_M | ▶ Play |
| IQ3_XXS | ▶ Play |
| Q3_K | ▶ Play |
Provide a reference audio clip to clone the speaker's voice:
./build/rs-tts-offline \
-m models/omnivoice-q4_k_m.gguf \
-t "Your text to synthesize." \
--ref speaker_reference.wav \
--ref-text "transcript of the reference audio" \
--n-steps 32 \
-o output.wavRequirements:
- Reference audio: WAV format, any sample rate (will be resampled to 24kHz)
- Reference text: exact transcript of what the speaker says in the reference clip
- For best results, use ≥3 seconds of reference audio with clean speech
Control voice characteristics via --instruct:
./build/rs-tts-offline -m models/omnivoice-q4_k_m.gguf \
-t "Your text." --instruct "female, young adult, high pitch, fast speed" -o output.wavEffective instruct descriptors: male/female, young adult/middle-aged/senior,
low pitch/moderate pitch/high pitch, slow speed/fast speed.
For activation-aware quantization with IQ1/IQ2 types:
# Step 1: Collect imatrix on the F16 model
./build/rs-imatrix -m models/omnivoice-f16.gguf \
-o imatrix-f16.dat --n-steps 16 --gpu
# Step 2: Quantize with imatrix
./build/rs-quantize models/omnivoice-f16.gguf \
models/omnivoice-iq3_xxs.gguf iq3_xxs --imatrix imatrix-f16.datNote: Even with imatrix, IQ2_XXS fails on OmniVoice. IQ3_XXS is the practical minimum.
At 2.06 bits-per-weight, each 1024-dimensional hidden vector carries only ~2109 bits of information. The MaskGIT codebook classification (8 codebooks × 1025 classes each) requires precise logit separation that IQ2_XXS cannot provide:
-
Grid too coarse: IQ2_XXS maps weights to a small precomputed lattice; each weight can only take ~4 distinct values. Across 28 transformer layers, the accumulation of quantization noise destroys the fine-grained differences needed for codebook selection.
-
CFG amplifies noise: Classifier-Free Guidance subtracts uncond from cond logits and multiplies by CFG scale (2.0–3.0). This 2-3× amplification applies to quantization noise as well as signal.
-
No parameter redundancy: With only ~300M backbone parameters, there is no excess capacity to absorb the information loss from 2-bit quantization. Large LLMs (7B+) tolerate IQ2_XXS because they have orders of magnitude more redundancy.
-
imatrix cannot fix this: Importance weighting helps by preserving sensitive columns, but at 2.06 bpw the fundamental information capacity is below the task requirement.
- Long text (>100 chars): Set
--n-stepshigher. RoPE positions beyond the training context (4096) produce unfamiliar embeddings. More steps give the model more chances to self-correct. - Chinese + English mixed: Use
--lang zhfor Chinese-dominant text,--lang enotherwise. The language token affects the style embedding. - Reproducibility: Use
--seed <N>for deterministic output. - GPU vs CPU: GPU (Metal/CUDA) is ~2-3× faster. For IQ types, GPU benefit is larger because the IQ matrix multiply kernels are heavily optimized.