Overview
This pipeline takes a classroom recording (teacher + ~30 students) and produces three outputs: a cleaned final.mp3 with background noise removed and only on-topic speech kept, a transcript.txt with speaker-labeled (TEACHER / STUDENTS) timestamped entries, and a summary.md with chaptered summaries and bullet key points per chapter.
No external API calls, no cloud services. Models are loaded and unloaded one stage at a time to fit within ~10GB VRAM and ~7GB system RAM, targeting under ~3 hours for a 2-hour recording.
Pipeline Stages
Five sequential stages — one model in VRAM at a time:
- Denoise — DeepFilterNet strips background noise while keeping all speech
- Diarize — pyannote 3.1 splits speech into
TEACHERvsSTUDENTS(most-frequent speaker = teacher; no individual student identification) - Transcribe — faster-whisper
large-v3with forced language (en/fr as selected) - Analyze — Qwen2.5-7B-Instruct via llama.cpp: per-segment relevance verdict (drops off-topic chatter and wrong-language speech), chapter boundaries, and bullet summaries
- Cut final audio — ffmpeg trims and concatenates only the relevant segments from the denoised audio, encoding to MP3
Implementation
Everything is packaged as Docker containers. The stack requires an NVIDIA GPU with ≥10GB VRAM and nvidia-container-toolkit. A Hugging Face token is needed for pyannote's gated diarization models.
Login protection via AUTH_USERNAME / AUTH_PASSWORD in the environment configuration.
Technologies
Interesting Details
- Models loaded sequentially to stay within ~10GB VRAM budget — nothing runs concurrently
- Qwen2.5-7B analyzes each transcribed segment and drops off-topic chatter and wrong-language segments before the final cut
- Chapter boundaries are detected automatically, not from slides or timestamps
- First run downloads model weights (~15-20GB disk) including DeepFilterNet, pyannote, faster-whisper large-v3, and Qwen2.5-7B GGUF