Audio clip lab: AI Voice dataset builder

A browser-local tool for building TTS training datasets from video. You open a local video, record clips of one person's voice off its playback, trim them, auto-transcribe them, tag their emotion via an LLM, and write the result to a folder on disk in LJSpeech-style layout (wavs/audio_NNN.wav + metadata.csv + a richer manifest.jsonl). Goal is a natural, expressive voice, not identity cloning, so clips carry dual emotion labels (free-text + a 13-class taxonomy) and prosody metrics (pitch/loudness/speaking rate).

- src/ audio-classifier-core, a UI-agnostic TS library (video control, audio capture, trimming/WAV encoding, Whisper transcription, DeepSeek emotion tagging, File System Access disk writes). No backend.

- app/ Vite + React UI on @mellon-design/react.

- training/ Python scripts that turn the exported dataset into Parler-TTS fine-tuning data (proof-of-concept ran end-to-end on Colab; pipeline validated, audio quality not there yet — dataset was only ~9.3 min).

engine/ a local Python/FastAPI service (separate process, own venv, not in the npm workspace) that automates "find every other clip of this same speaker in the video," so you don't scrub the whole thing manually.

Stack: FastAPI + uvicorn + pydantic, torch/torchaudio 2.5.1 (pinned), pyannote.audio 3.x, scikit-learn, demucs 4.1.0, plus ffmpeg as a subprocess. All heavy imports are lazy.

Modules:

- main.py routes: health, enrollment/negative CRUD, profiles, POST /scan + job polling, per-segment audio, /process/isolate-vocals.

- pipeline.py pyannote used as three separate models rather than the bundled diarization pipeline: VAD → overlapped-speech detection (those regions get dropped) speaker embedding. Plus WAV slicing, embedding clustering, and Demucs vocal isolation.

- scan.py one scan job: extract audio → VAD → drop overlaps → embed each region → cluster embeddings → score each cluster centroid against enrollment and negative references by cosine similarity.

- enrollment.py negative.py — two reference stores (positive examples, rejected voices), each clip persisted as .npy embedding + .wav.

- profiles.py named speaker profiles, one enrollment/negative store per person.

- config.py every tunable via SPEAKER_ENGINE_* env vars.

Results come back as a reviewable queue in the UI (accept/reject/discard on a segment track), never auto-saved, accepts fold back into enrollment, rejects into the negative store, so detection sharpens as you go.