Voice Input for Bookkeeping: Where the Flow Breaks

You can talk to a bookkeeping app now. Say "forty euros lunch with a client yesterday, cash" and get a booked expense, or hold the microphone and dictate a question about your VAT deadline. That part works, and it is the least interesting thing about it.

The interesting part is how little of the 2026 voice stack you need to make it work, and how much of the effort goes into things that have nothing to do with speech recognition. We shipped voice capture on two surfaces this year, and the failures were never in the transcript. They were in a file path, a permission prompt, a tap that was too short, and a model upgrade that quietly removed a feature we depended on.

Can you do bookkeeping by voice?

For capture and for questions, yes. For unattended commands, no, and not because the model cannot parse them.

Speech is an input method for text. It is not an authorisation mechanism. A spoken sentence that turns into a booking has to pass through a state a human can read and correct, because there is no undo on a filed VAT return and because a transcript is exactly the place where "nineteen" becomes "ninety". So both of our surfaces do the same thing: transcribe, show the text, and let the person press send. On the messaging surface the bot literally echoes what it heard back to the user before it answers.

That single rule removes most of the risk from a voice feature, and it is also why we do not need the parts of the industry stack everyone is shipping.

The two voice modes, and which one you want

There are two ways to get speech into an app, and they are not variations of the same thing.

We use both. Dictation is right when someone is composing a message and wants to see it appear. The voice note is right for "forty euros lunch, cash", where the user says one thing, stops, and expects a result. The interesting implementation detail is that our voice-note recorder uses the on-device recognition module as a recorder rather than a recogniser: it asks for the microphone, meters the level for the waveform, and persists the audio, while the actual transcript comes from a server model. Audio is captured at 16 kHz, PCM 16-bit, because that is what speech models want and because shipping 48 kHz stereo over a mobile connection is a waste of everybody's time.

What actually breaks in a voice flow

Four things, none of them the model.

Permissions, on a fresh install only. Dictation asked for microphone access before starting; the voice-note path assumed it was already granted. On a device where it had never been granted, the recogniser emitted a not-allowed event, no audio file was ever produced, and the UI sat there. The fix is trivial and the lesson is not: every entry point into audio has to request permission itself, because your test device granted it weeks ago.

The audio file does not exist when you think it does. The recording URI arrives on a start event, but the native writer is still holding the file. Reading it then gives you a path to something incomplete. We keep the early path only as a fallback and consume it after the final end event, when the writer is done. If your voice feature works on a simulator and fails on a real phone, this race is the first thing to check.

Tap and release. A user who taps the record button and lets go immediately sends a header with no audio in it. The transcription service rejects it, correctly, and the naive implementation reports a server error, retries, and fails again with the same bytes. It is a 4xx, not a 5xx: the input is the problem, the retry is pointless, and the right response is "that was too short, try again" rather than an incident in your error tracker.

The latency budget, which is a chain and not a number. The client caps a recording at two minutes. The server call has an eighteen-second timeout, chosen to sit under the request-handler ceiling of twenty-five seconds so a slow transcription fails cleanly instead of being killed mid-request. Pick those three numbers together or the outer one will decide for you, and it will decide by dropping the user's recording after they spoke for ninety seconds.

The model upgrade that cost us language detection

This is the part I would want to read in someone else's postmortem.

Our transcription originally used a model whose verbose response format returns the detected language and the duration of the audio, derived from the audio itself. For a product used in German, English and Polish, that language code is not a nicety: it decides which language the assistant answers in. When we moved to a newer, better transcription model, that response format was rejected outright. The newer model transcribes more accurately and tells you less.

So the language now comes from a stopword heuristic over the resulting text, which is a downgrade, and an honest one: a short utterance with no stopwords in it gets no language at all, and we fall back to the interface language. Meanwhile the streaming vendors have gone the other way and made this a headline feature. Deepgram's Flux Multilingual reached general availability on 29 April 2026 with ten languages and dynamic switching inside a single conversation, and AssemblyAI's multilingual streaming model, published 12 November 2025, covers six languages in one model at a flat hourly rate regardless of language.

Which is genuinely useful, and mostly not for us. Look at what the rest of that stack is optimised for: Flux was built around knowing when a speaker has finished a turn, and AssemblyAI shipped a streaming diarization upgrade on 4 May 2026. Turn detection and speaker separation are the hard problems of a live, interruptible, multi-party conversation. A bookkeeping capture flow has one speaker, no interruptions, and an utterance that ends when a thumb leaves a button. Buying the conversational stack for that is paying for a problem you do not have.

What the usage actually looks like

One number, with its caveat attached, because I would rather publish a small honest signal than a big vague one.

On our messaging-bot surface, where a user can send text, a photo or a voice note, slightly more inbound messages are voice notes than typed text. That is from a couple of hundred inbound messages in an early cohort, so treat it as directional and not as a benchmark. It still surprised me: given a free choice of input on a device built for typing, these users talk.

The second thing our data says is about our own instrumentation. In the chat tables, a spoken message is indistinguishable from a typed one, because what we store is the transcript. That is correct for privacy and inconvenient for product decisions: we cannot answer "how many bookings started as speech" without adding a flag we never added. If you are building this, put the origin on the message before you ship, not after you want the number.

Frequently asked questions

How do I stop typing expense data by hand?

Capture at the moment of spend rather than in a monthly session, and use the input that is fastest for the situation: a photo when there is a document, voice when there is not. A spoken sentence like "forty euros lunch with a client, cash" carries amount, category hint, and payment method in one breath, which is faster than any form. Then confirm the parsed result, because the correction is the only manual step left.

Can I book an expense by voice in German?

Yes, and language handling is the thing to check rather than recognition quality. Modern transcription handles German including numbers and currency perfectly well; the failure is when the app answers in the wrong language, or normalises a decimal comma into a decimal point. Ask a vendor what happens with a short utterance where the spoken language differs from the interface language, because that is the case that quietly breaks.

Is voice input accurate enough for bookkeeping numbers?

Accurate enough to propose, not to file unattended. Amounts and dates come back reliably, but "nineteen" and "ninety" are one vowel apart in several languages and a transcript is where that collapses. Any voice flow touching tax-relevant records should show the parsed values for confirmation before writing them, which also gives you a clean audit trail of who approved the booking.

What happens to the audio recording afterwards?

In our flow the recording is uploaded, transcribed, and then the local file is deleted, so what persists is the text and not the audio. That is worth asking any vendor explicitly, because voice notes about business expenses routinely contain third-party names, and audio kept "for quality" is a copy of personal data you did not need to keep. We take the same position on our own model traces, which store a hash of the payload rather than the payload.

Does voice input work without an internet connection?

Live on-device dictation can, depending on the platform and the installed language. Recorded voice notes cannot, because the transcript comes from a server model. In practice this matters less than it sounds: the booking itself needs the server anyway, so an offline transcript would only queue up behind the same connection.

Conclusion

Voice in a bookkeeping app is a solved problem sitting behind four unsolved engineering details, and the details are permissions, a file handle, a too-short recording, and a timeout chain. None of them appear in a speech model comparison.

The strategic bit is knowing which parts of the 2026 voice stack to skip. Turn detection, diarization and dynamic language switching are the right investments for a conversational agent on a phone call. For someone standing outside a restaurant saying "forty euros, lunch, cash", the right investment is a recording that reliably reaches a transcript, a transcript the person can see, and a send button they press themselves. We built the same discipline into how the agent collects documents it is missing, and it holds here too: the interesting engineering is almost never the model.