VoiceHarvester

A small drag-and-drop desktop app that pulls a clean, isolated voice out of your video and audio files and saves it as a WAV that's ready to feed into AI voice-cloning tools (ElevenLabs, Play.ht, RVC, XTTS, and similar).

Drop in as many files as you like — it processes them in a batch and can even stitch them into one combined sample for better cloning results.

---

Hear them again — one command

If what you have is a few seconds of someone you miss — even a single 4–5 second phone video — this is the path:

python3 hear.py mom_clip.mp4

It extracts the audio, decides honestly whether the clip even needs cleaning (a genuinely quiet clip is kept raw — heavy processing adds artifacts a clean recording never had), keeps only the cleanest window of their speech, saves an ideal 24 kHz reference — and plays their actual voice for you.

python3 hear.py mom_clip.mp4 --say "Good morning, beta"

adds a spoken line in that voice afterwards (needs XTTS installed; it is always announced as the clone — never confused with the real recording).

What the research and our own ears agree on: clean beats long. Four good seconds outperform minutes of noisy tape; XTTS only reads the first ~12 seconds of a reference anyway. If the clip has music or a noisy room under the voice, install Demucs (pip install demucs) and hear.py will isolate the voice first. If you have two or three short clips, build a reference from each — cloning engines can average them into a steadier voice.

---

What it does

For each file it:

1. Extracts the audio track.
2. Isolates the spoken voice — using Demucs (deep-learning separation) if you install it, otherwise an ffmpeg denoise/clean-up chain that always works.
3. Cleans and normalizes the result to a mono 44.1 kHz 16-bit WAV, the format cloning services prefer.

Output files are named <original>_voice.wav. With the merge option on, you also get combined_voice_sample.wav.

---

Requirements

ffmpeg must be installed (the app uses it under the hood):

• macOS: brew install ffmpeg
• Windows: winget install Gyan.FFmpeg
• Linux: sudo apt install ffmpeg

Python 3.9+, then optionally install the extras:

pip install -r requirements.txt

• tkinterdnd2 enables drag-and-drop. Without it, the app still works via the Add files... button.
• demucs (commented out in requirements) gives the best voice isolation but is a large download. Enable it by uncommenting the line and re-running pip.

---

Run it

python app.py

Then drag files onto the window (or click Add files...), pick an output folder, and click Extract Voice.

Command-line version

For folders or automation, skip the GUI:

python cli.py /path/to/videos -o out/ --merge

Build the ideal clone reference (recommended)

refine.py goes further than plain extraction — it turns a short, imperfect clip into the best possible reference for XTTS-v2 zero-shot cloning, which only needs ~6 seconds of clean speech:

python refine.py mom_birthday.mov --out mom_reference.wav

What it does, and why each step matters for cloning a specific person:

1. Isolates the voice (Demucs) and transcribes it (Whisper) into segments.
2. Separates speakers by timbre, not just pitch — so a mother and a daughter in the same clip aren't merged (the classic failure). Target one with --speaker A.
3. Scores every segment (SNR, voiced-ratio, clipping, length) and keeps only the single cleanest contiguous window — clean beats long for XTTS.
4. Renders 24 kHz mono, gently normalized (XTTS's sweet spot, not broadcast loudness which dulls timbre), and prints a quality report + warnings so you know the sample is good before cloning.

It prints JSON (ok, duration, snr_db, speaker, report, warnings) for automation, and degrades gracefully if Demucs/Whisper aren't installed.

---

Tips for good cloning results

• Even ~6 seconds works for zero-shot cloning (XTTS-v2) — clean beats long. refine.py finds the cleanest window for you automatically.
• Prefer clips with little background music or noise (Demucs handles a lot).
• Multiple short clips? Refine each and keep the highest-SNR result, or use cli.py --merge to combine.

---

A note on consent

Cloning a real person's voice should be done with their consent, or — for a memorial — in line with what your family is comfortable with. Reputable cloning services require you to confirm you have the right to use the voice.