Discussion on "Offline speech-to-text for Flutter, with no server and no API key"

Ishanga Vidusha · 2026-10-19T18:11:50.056Z

At work I build real-time transcription with cloud speech APIs. They are very good. They also need a network connection, an account, a key, a bill, and a decision about whose servers your users' voice

The audio never leaving the phone is a big deal for apps handling private stuff. Nice to see offline speech getting fast enough to feel usable on a normal phone.

Keeping the audio on the phone removes the whole question of whose servers the voice goes to, that alone makes on-device worth the accuracy tradeoff for a lot of apps. Getting live text to settle without flickering is harder than it looks.

Thanks! The flicker was the part that took the longest to get right. It’s surprising how much the last word of each read changes.

One edge case I’d be curious about is what happens when the audio route changes mid-session. For example, a user starts dictating through a Bluetooth headset, unplugs it halfway through a sentence, and the phone switches back to its built-in mic with a different sample rate.

On iOS, a route change can affect the sample rate and channel configuration, so it’s not just a microphone restart problem. How does transcribe_cpp handle that transition? Does it rebuild the capture pipeline while preserving the current utterance, or does it finalize the sentence and start a fresh one? That distinction matters quite a bit for hands-free transcription, especially during long sessions.

The "whose servers your users' voices go to" framing is exactly the decision point. I hit the same one with documents instead of voice - official letters people shouldn't upload anywhere - and ended up with a local model on CPU for the same reason you ended up on-device: the constraint turns out to be the feature. The 40x-real-time number on a mid-range phone kills the assumption that local always means slow. Curious about the voice detection part: did you have to tune it per language or per environment, or does the default hold up in noisy places?

Thank you! No per language tuning. Silero VAD does not report the language of the audio, only whether it is speech or not, so the same settings work for all languages. I ran long public recordings (earnings calls and podcasts) through the real pipeline with the defaults and they stood up to added noise. The tough case is when other people are talking in the background. That is speech, so the detector carries on. Steady noise like fans or traffic is okay. If your environment needs something different, then every threshold is in EndpointerConfig.

The two-read agreement rule raises an interesting boundary: what happens if a word agrees twice, then the completed sentence changes it? Does the final pass correct the displayed text, or is the committed prefix fixed? A small example with a long pause in the middle of a phrase would help distinguish stable display from final transcription accuracy.

Good question. The two answers are different on purpose. While you’re speaking, the live text only grows: once a word has matched across two reads it’s never removed, even if a later read changes its mind. That’s what keeps the live text from flickering. When the sentence ends, the whole utterance is decoded again in one pass, and that result replaces the live text completely. So if anything was wrong in the live text, the final result corrects it.

On the long pause: if you stop for more than about 400 ms, that counts as the end of the utterance. The first half gets its final result straight away, and the rest starts a new paragraph. A mid-phrase pause can therefore split a sentence in two, and the first half is decoded without the words that come after it. That’s the trade for getting text quickly. A longer hangover setting means fewer splits but a later final result.

Thanks for the detail. Background speech is exactly the case that bites in practice, and good to know the thresholds are all exposed in EndpointerConfig. I will take a look.

Thanks, let me know how it goes if you try it.

One thing I’d be careful about here is the timing of the final decode. Say the Bluetooth input is running at 16 kHz mono, and the route switches to the built-in mic at 44.1 kHz while the decoder still has unprocessed frames from the previous utterance. If the VAD and endpointer reset immediately, how do you make sure those buffered frames are actually decoded before the old state is discarded?

I’d probably treat the route change as a boundary between two audio segments: stop accepting frames from the old capture session, drain its pending audio through the existing resampling and decode path, emit a final result if there’s enough speech to recover, and only then reset the endpointing state for the new input. Otherwise, even if the ring buffer survives the switch, the sentence can still disappear because the decoder never gets a chance to finish it.

There’s another edge case worth thinking about: what happens if a route change lands while the capture callback is still delivering buffers in the old format? For example, a Bluetooth headset disappears, the input switches to the built-in mic, and a delayed callback from the old tap arrives after the new resampler has been configured.

If both streams write into the same ring buffer, I’d want some explicit guarantee that frames from different input configurations can’t be interpreted using the wrong sample rate or channel layout. One approach would be to tag buffers with a capture-generation ID and format metadata, then reject stale callbacks while keeping the old segment available for final decoding. That would also make rapid headset connect/disconnect cycles easier to handle without accidentally mixing audio across route changes. Does the current capture layer already guard against that race, or is that something you’re planning to address alongside utterance preservation?

Fixed in 0.1.3, now on pub.dev. Testing it showed two problems, not one. On iOS, re-activating the audio session during the change was refused, so the mic could stop entirely; a route change now rebuilds only the engine. And the sentence in progress is now finished instead of dropped.

The order you describe is how it works now. The ring only ever holds 16 kHz mono, because each tap callback converts before it writes. So on a route change the audio already buffered from the old route is drained into the endpointer first, the open sentence is emitted as a final result and queued for the decoder, and only then is the endpointer reset for the new input. The decoder runs separately, so the reset can't cancel a decode that's already queued.

On stale callbacks: the old tap is removed and the engine stopped before the new converter is built, so a buffer in the old format shouldn't reach it. There's no explicit check, though, and checking each buffer's format against the converter's input is cheap, so I'll add one.

Search Hashnode

Offline speech-to-text for Flutter, with no server and no API key

Responses14