The newest member of the suite is Scribe -- live transcription with AI-generated meeting minutes. You speak, it writes. You stop, it summarizes and extracts action items. Simple in concept. Not simple in execution.
I thought WebSocket audio would be straightforward. I was wrong. What followed was a deep dive into the kind of debugging session that reminds you why software engineering is equal parts persistence and humility.
The symptoms were clear: connect, hit record, speak for thirty seconds, hit stop. Nothing. The server logs showed a steady stream of "LOOP 47, buffer=0.0s -- skipping." The audio was leaving the browser. It was not arriving at the server.
First assumption: the audio graph was not wired correctly. The ScriptProcessorNode was connected to a gain node set to zero, which Safari helpfully optimized away entirely. The callback never fired. Fix: wire it to a MediaStreamDestination instead. Now the callback fired. But the server still saw nothing.
Second assumption: binary WebSocket transport works everywhere. It does not. Safari (and some versions of Starlette) handle raw ArrayBuffer payloads differently than Chrome. The server received ASGI event dictionaries with empty payloads instead of the expected bytes. Fix: encode audio as base64 strings on the client and decode them on the server. Suddenly the buffer had data. Real data. 22.4 seconds of it.
Then came the stop problem. Click stop, and the connection would drop before the final Whisper pass could complete. The transcript status was stuck at "error" in the database. Every single time. The issue was that faster-whisper is CPU-bound and blocks the async event loop. While the server was transcribing, it could not process the disconnect gracefully. Fix: run Whisper in a thread pool via asyncio.run_in_executor, and run Ollama cleanup, summary, and action extraction in the same pool. The event loop stays responsive. Transcripts finalize. Results stream back.
Then came the disconnect-during-processing problem. If you closed the browser while Ollama was still generating, the background task would get garbage-collected and the transcript would never be saved. Fix: a module-level task registry that keeps strong references to background tasks until they complete.
The final architecture:
- Client captures 16kHz mono audio, encodes each frame as base64, sends over WebSocket.
- Server extracts ASGI text/bytes payloads from Starlette event dicts.
- Every 2 seconds, Whisper runs in a thread pool on accumulated audio and streams partial transcripts back.
- On stop, the full buffer gets a final pass, then the text is sent to Ollama for cleanup, summary, and action extraction -- all in the thread pool.
- If the client disconnects mid-processing, the task registry keeps the finalization alive.
- The result: cleaned transcript, summary, and bullet-point action items -- all local, all private, all yours.
The first successful transcription was just a test sentence. "This is a test. Please ignore." I did not ignore it. I stared at it for a solid minute. It represented thirty loops of debugging, two browser engines, one ASGI specification rabbit hole, and a stubborn refusal to give up.
The gap between "it should work" and "it works" is where you actually learn. Scribe taught me more about WebSocket binary transport, asyncio event loops, and Safari Web Audio quirks than any tutorial ever could.
Scribe is live at ownscribe.ifthenelse.net. It connects to MyOwnLifeOS so your Pomodoro timer knows what you actually decided to do in your last meeting. Because knowing what you said is only useful if you act on it.