Multi-Voice & LLM-Assisted Audiobook Generator

A Python tool to convert EPUB, TXT, and Markdown books into multi-character dramatized audiobooks using Chatterbox TTS (voice cloning) and a local LLM (LM Studio, Ollama, or any OpenAI-compatible API) for character discovery and dialogue attribution.

---

Features

• 3-Stage Workflow:
  1. Character Discovery & Analysis (--mode analyze): Automatically scans chapters with a local LLM to extract characters, estimate line frequency, classify gender, and categorize into archetypes (male_young, female_elderly, guard, eccentric, etc.).
  2. Human-Editable Voice Mapping (voice_map.json): Map discovered characters and archetypes to clean reference WAV samples (~5–15s).
  3. Dialogue Attribution & Synthesis (--mode synthesize): Segments text into sequentially attributed speaker lines and generates chapter audio with cloned voices.
• Character-Batched Synthesis: Dialogue chunks are grouped by character voice during TTS generation to minimize model conditioning overhead, then stitched back together in strict chronological sequence.
• Intermediate Caching: Dialogue segmentation results are saved per chapter in <output_dir>/cache/, so LLM parsing is not re-executed upon resuming or tweaking audio.
• Smart Pacing & Silence Control: Dynamic micro-pauses for sentence breaks (0.35s), speaker transitions (0.30s), and paragraph/scene breaks (0.85s).
• Audio Normalization: RMS and peak loudness normalization to maintain uniform volume across disparate reference voice clips.
• Single-Voice Mode (--mode single): Classic narration mode using a single reference WAV file.
• Flexible File Formats: Full support for .epub, .txt, and .md ebooks.

---

Installation

1. Clone & Install Dependencies

git clone <repo-url>
cd audiobook
pip install -r requirements.txt

Requirements

chatterbox-tts>=0.1.0
torchaudio
torch
ebooklib
beautifulsoup4
pydub
tqdm
openai

Note on Chatterbox TTS: Requires PyTorch with CUDA or CPU support. For multilingual / Czech support, custom weights such as t3_cs.safetensors can be used.

---

Quickstart Guide

1. Stage 1: Analyze Characters

Point the tool to your ebook and your running local LLM (e.g. LM Studio on port 1234 or Ollama on port 11434):

python audiobook_generator.py \
  --mode analyze \
  --input "books/alice_in_wonderland.epub" \
  --output-dir "./audiobook_output" \
  --llm-url "http://localhost:1234/v1"

This scans the book and generates audiobook_output/voice_map.json.

---

2. Stage 2: Configure Voice Map

Open the generated voice_map.json and adjust the paths to your reference WAV samples:

{
  "default_narrator": "voices/narrator_neutral.wav",
  "characters": {
    "Alice": {
      "tier": "main",
      "gender": "female",
      "archetype": "female_young",
      "voice": "voices/alice.wav",
      "dialogue_frequency": 42,
      "notes": "Protagonist"
    },
    "The Mad Hatter": {
      "tier": "main",
      "gender": "male",
      "archetype": "eccentric",
      "voice": "voices/hatter.wav",
      "dialogue_frequency": 18,
      "notes": "Eccentric character"
    }
  },
  "archetype_fallbacks": {
    "male_young": "voices/generic_young_man.wav",
    "male_elderly": "voices/generic_old_man.wav",
    "female_young": "voices/generic_young_woman.wav",
    "female_elderly": "voices/generic_old_woman.wav",
    "guard": "voices/generic_guard.wav",
    "eccentric": "voices/generic_eccentric.wav",
    "neutral": "voices/narrator_neutral.wav"
  }
}

---

3. Stage 3: Synthesize Multi-Voice Audiobook

Synthesize the final chapter MP3 files:

python audiobook_generator.py \
  --mode synthesize \
  --input "books/alice_in_wonderland.epub" \
  --output-dir "./audiobook_output" \
  --voice-map "./audiobook_output/voice_map.json" \
  --llm-url "http://localhost:1234/v1" \
  --device cuda \
  --language cs

---

Alternative: Single-Voice Mode

To generate a simple single-narrator audiobook without multi-voice dialogue splitting:

python audiobook_generator.py \
  --mode single \
  --input "books/alice_in_wonderland.epub" \
  --ref-audio "voices/narrator_sample.wav" \
  --output-dir "./audiobook_output"

---

CLI Options Reference

Option	Flag	Default	Description
Mode	--mode, -m	synthesize	Operation mode: analyze, synthesize, or single
Input File	--input, -i	(required)	Path to .epub, .txt, or .md book file
Output Dir	--output-dir, -o	./audiobook_output	Directory for audiobooks, caches, and voice map
Voice Map	--voice-map, -v	<output_dir>/voice_map.json	Path to voice_map.json configuration
Ref Audio	--ref-audio, -r	None	Reference WAV file for single-voice mode / narrator fallback
Cache Dir	--cache-dir	<output_dir>/cache	Directory for dialogue segmentation JSON cache
LLM URL	--llm-url	http://localhost:1234/v1	Base URL of local LLM (LM Studio / Ollama / OpenAI API)
LLM Model	--llm-model	local-model	LLM model identifier
Device	--device, -d	cuda (if available)	Compute device (cuda or cpu)
Language	--language	cs	Language code for Chatterbox synthesis (e.g., cs, en)
Max Chars	--max-chars	300	Max character length per TTS sub-chunk
Sentence Pause	--pause-sentence	0.35	Silence between sentences (seconds)
Speaker Pause	--pause-speaker-switch	0.30	Silence between speaker turns (seconds)
Paragraph Pause	--pause-paragraph	0.85	Silence at paragraph/scene breaks (seconds)
No Normalize	--no-normalize	False	Disable RMS audio level normalization
Overwrite Audio	--overwrite	False	Overwrite existing chapter MP3 files
Overwrite Cache	--overwrite-dialogue	False	Re-run LLM dialogue parsing, ignoring cache

---

License

MIT License.