Live speech-to-text is the one feature where I use WebSockets instead of Server-Sent Events, because audio flows up while text flows down. The OCR & Speech Workspace does live microphone transcription with a post-recording refinement pass, and the architecture below is what made it feel instant rather than laggy.
The data flow
- The browser captures microphone audio, resamples to the rate the model expects, and sends small chunks — a few hundred milliseconds each — over a WebSocket.
- The server buffers chunks into windows, runs the streaming model, and sends back partial transcripts as they stabilise.
- When the user stops, the server runs a second, slower pass over the whole recording and replaces the live text with a refined final transcript.
Partial versus final
Streaming models revise their guesses as more audio arrives. Show partial text in a lighter style and lock segments once they are final; users tolerate words changing for a second, not paragraphs rewriting themselves. Mark segment boundaries with silence detection so the refinement pass has clean units to work with.
Latency budget
- Chunk size sets the floor: 300 ms chunks mean nothing appears for at least 300 ms.
- Network round trip and model time add to it; keep the model warm and the connection persistent.
- Aim for under a second to the first partial; beyond two seconds, users start repeating themselves.
The refinement pass is not optional
Live transcripts are optimised for speed, not accuracy. A post-recording pass with a stronger model, punctuation restoration and speaker-aware formatting turns a stream of words into a document someone will actually read. In the workspace, that final transcript is what gets indexed for document-scoped chat.
Details that matter
Ask for microphone permission at the moment of need with an explanation, show a live level meter so the user knows audio is flowing, handle reconnects without losing the buffer, and let people edit the final text. Speech features live or die on these details, not on the model.
GenAI engineer and AI product builder with 2+ years shipping production-grade AI systems: RAG pipelines, fine-tuned LLMs, hybrid retrieval and multi-modal apps across vision, speech and OCR, architected end to end from ingestion to deployment. Leads engineering for SCULT INDIA's 14-member team, founded the 500+ member TechVerse Enclave community and has mentored 200+ students. Three hackathon first prizes: Changethon 2025 (IIT Roorkee), Product Genesis at Vividhotsava 2025 (CSJMU Kanpur) and BYTEBATTLE (MeetKats).
Open to GenAI roles, hackathon judging, mentorship sessions and guest talks at colleges. Email pranjulrathour41@gmail.com.
