Every AI meeting notes tool on the market makes the same promise about transcription, and the promise is almost always an adjective. Accurate. Reliable. Blazingly precise. Ask what the adjective is measured against, and the conversation usually changes subject — which is strange, because transcription accuracy is one of the few qualities in this product category that has been measured the same way for decades. The yardstick exists, the arithmetic is public, and any vendor with a test set could report a number tomorrow. This guide explains how that yardstick works: what character error rate and word error rate actually measure, why the difference between the two matters more than most product comparisons admit, what an error rate leaves out, and how to evaluate any transcription tool — including ours — without taking anyone's adjectives on faith.
What an Error Rate Actually Measures
Every error-rate metric, whatever its unit, rests on the same comparison. You take the transcript the system produced — call it the hypothesis — and lay it against a reference transcript, a version of the same audio written out and carefully checked by humans. Then you count the disagreements between the two, and they come in exactly three kinds. A substitution is something that came out wrong, whether a garbled term or the wrong participant name entirely. A deletion is something the system dropped, from one mumbled word to a whole clause spoken away from the microphone. An insertion is the reverse, something standing in the transcript that nobody actually said. Add those three counts together, divide by the length of the reference, and you have an error rate. Zero means the two transcripts match perfectly; every point above zero is a fraction of the meeting a reader would have to repair by hand before trusting the record.
This is the same edit-distance logic that powers spellcheckers and file-diff tools, applied to speech, and two properties make it a good yardstick. First, it charges every direction of failure in the same currency — dropping a sentence, mangling a product name, and inventing a plausible-sounding clause all cost the metric something, so a system cannot earn a good score by guessing confidently. Second, it is symmetric across systems: two different tools can be scored against the same reference and compared on equal footing, which is exactly the property marketing copy lacks. Comparability is what makes this metric boring to a landing page and valuable to an engineer.
One subtlety is worth internalizing before you read any vendor's claim. Because insertions count toward the total, a system that pads its transcript with words that were never spoken is actively punished, and the rate can climb past the ceiling a casual reader might assume it has. That behavior is a feature rather than a bug, and it matters most in meetings. The failure people actually fear from AI transcription is not the dropped filler word; it is the action item that appears out of nowhere, or a decision paraphrased into something nobody agreed to. In the arithmetic, that entire class of failure lives in the insertion count. You will also see vendors flip the framing and publish an accuracy score instead, which is simply the errors subtracted from a perfect score — identical information, friendlier packaging. Whichever way a number is framed, the question that decides whether it means anything is the same: measured how, and on what.
WER and CER: Two Rulers for Two Kinds of Language
Word error rate, or WER, runs that arithmetic on whole words, which makes it the natural fit for English and other languages that mark word boundaries with spaces. The reference length is a word count, each misheard word is one substitution, and the result reads as the share of a meeting's words that would need human attention. WER has been the standard reporting metric for English speech recognition for decades, and if a tool will mostly hear English, it is usually the number to ask for.
Character error rate, or CER, runs the identical arithmetic on characters instead of words, and for Chinese, Japanese and other languages written without spaces it is the fairer ruler — for a structural reason worth understanding rather than memorizing. To compute WER on Chinese, you must first decide where the words are, because the writing system does not mark those boundaries. Word segmentation is a modeling decision, and reasonable evaluators can segment the same sentence differently; change the segmentation and you change both the reference length and what counts as a one-word error. Two teams applying different conventions can produce different WER figures from identical transcripts, which is precisely the ambiguity a benchmark should not have. CER sidesteps the argument entirely, because a character count is unambiguous — and in languages where a single character often carries a full word's meaning, the character is simply the honest unit of measurement.
CER earns its keep twice over in meetings that mix languages, which describes a growing share of the world's meetings. A product review in Mandarin that leans on English technical terms, an English standup interrupted by a vendor name in Japanese, a client call that slides between dialects — whenever the language of a sentence changes, word-based counting inherits every assumption the tokenizer made about where one language's words end. Character-based counting just keeps counting. CER also grades mistakes the way readers experience them: a term captured with one wrong character is a small, recoverable error rather than a whole word struck out, which is closer to how a person actually absorbs a slightly flawed transcript.
There is one reading skill that ties this section together: the two numbers are not on the same scale, and this is the most common way accuracy comparisons mislead. Because an English word averages several characters, the same transcript will generally show a smaller figure under CER than under WER, since the errors are spread across a finer unit. A vendor quoting CER beside a rival quoting WER is therefore not offering a comparison at all — it is two numbers measured with different rulers. Before comparing anything, confirm which metric each figure uses; a vendor who cannot answer has itself supplied a data point.
What the Number Leaves Out
An error rate is a statement about a test set, not about your meetings, and the distance between those two things is where most buying mistakes happen. The first variable is vocabulary. Speech recognition systems fail first on what they have never seen — internal product names, project codenames, industry jargon, the surnames of the specific people in your room. A benchmark built from generic business conversation will flatter a system that your quarterly roadmap review will embarrass, and a metric run on the former tells you little about the latter.
The second variable is the audio itself. A headset microphone close to a speaker's mouth and a laptop at the far end of a conference table are different problems, and the second one arrives with room echo, air conditioning, chair scrapes and the remote participant dialing in from a moving car. Accents and speaking rate move scores too, as does the single hardest condition in the field — overlapping speech. When two people talk at once, or an audience supplies a steady pattern of short back-channel agreements, even strong systems lose words, and lively meetings are made of exactly this material. A figure earned on clean, read-aloud, single-speaker audio tells you almost nothing about how a tool will fare in a room where three people finish each other's sentences.
The third variable is the scoring convention, which stays invisible until two benchmarks disagree. Was punctuation included in the reference, or stripped before scoring? Was capitalization normalized? Were numbers scored as spoken — fifty-two dollars — or as written, digits with a currency sign? Was the reference transcript itself checked once or twice, and by whom? Are speaker labels, the question of who said what, inside the metric at all, or handled as a separate problem the headline number never touches? Every one of those conventions can move a reported figure without the underlying system changing at all. This is why the conditions belong next to the score every single time, and why a naked number deserves the same skepticism as a naked adjective.
Why Most Vendors Publish No Numbers
Browse enough AI meeting notes product pages and a pattern emerges: transcription quality is described in qualitative terms — highly accurate, remarkably reliable, impressively precise — while published accuracy figures are rare across the category. There are understandable reasons. A meaningful benchmark takes real work to build and keep current; a published number invites direct comparison with every competitor; and any single figure is easy to attack for the conditions it omits, so the commercially safe move is to let adjectives do the work. The market-level result is that the one measurable quality in this category becomes a matter of belief, and adjectives inflate precisely because nothing checks them.
The practical response is not to wait for the industry to standardize; it is to ask better questions whenever a number does appear. What test set was it measured on — whose meetings, recorded how, with how many speakers, read speech or real conversation? Which languages, under which metric, WER or CER, and with what normalization? And under what recording conditions — headset-quality audio or far-field rooms, overlapping speech or none? A vendor who has genuinely done the evaluation work can answer all three in a couple of sentences. A vendor who changes the subject has told you something too.
How to Run Your Own Evaluation
The benchmark that matters for your buying decision is your own meeting, and building a small version of it is an afternoon's work. Record one genuinely representative meeting, with everyone's consent, and run the identical audio through each tool you are comparing. Then take the densest few minutes of the transcript — the stretch where names, numbers and decisions cluster — and correct it by hand against the recording. Count what you had to fix: substituted terms, dropped words, and anything that stands in the transcript but was never said in the room. Even a rough count, applied identically across tools, is more decision-useful than any landing page, because it inherits your vocabulary, your room and your speakers.
While you correct, watch for the failure modes that actually cost time rather than the ones that look worst. A transcript that faithfully renders every filler word is technically worse and practically fine; a transcript that confuses two teammates' names is worse than its error count suggests, because the damage lands on attribution — who committed to what, and who owes whom a follow-up. Numbers, dates and action-item verbs deserve the same suspicion, since a shifted digit is a small error in the arithmetic and a large one in an invoice. This is also the moment to check the layer above the transcript: a summary is only as trustworthy as it is traceable, and good notes should let you connect every claim in the summary back to the sentence that was actually said.
Keep the recording. Models update quietly, engines get retrained, product pages get redesigned — and the saved audio file becomes a private regression suite you can rerun whenever a vendor ships an update. If the transcript of your reference meeting drifts over time, you will know before it turns into a misremembered decision, and you will know it in the only currency that matters, which is your own meetings measured on your own terms.
How Today Measures Transcription Quality
Today Recording records meetings on Mac and iPhone, and it is opt-in by design: when a calendar event approaches, a live activity asks whether you want to record, and one tap starts it. The transcript it produces is treated as raw material rather than the deliverable. Notes draw on the context around the meeting — which project it belongs to, who was present, what was decided last time — and when a name, number or relationship cannot be confirmed, the note flags the uncertainty instead of papering over it with a guess. That posture is what earns a transcript a second read, because a tool that admits the edge of its knowledge is a tool you re-check rather than blindly re-trust.
Underneath that behavior sits the discipline this guide has been describing. Internally, transcription quality is measured with CER, chosen deliberately because the product serves meetings in Chinese, Japanese and mixed-language settings where a character-level metric is the fair ruler. Every change to the transcription pipeline, from a new model to an adjustment in audio preprocessing, runs against a regression suite before it ships, and an improvement that holds on one kind of meeting but degrades another does not ship. The entire point of keeping an internal metric is to make decisions against measurements instead of adjectives, release after release.
Now the numbers can go on the page — with their conditions attached, because that is the argument of this whole guide. Our most recent evaluation (September 2026) took five and a half hours of long, overlapping multi-speaker meetings from AISHELL-4, a public benchmark, and ran a four-way comparison: Today's full transcription pipeline against the raw output of three speech providers, scored with MeetEval, an open evaluation tool. The headline result is about who spoke when: speaker-attribution error (DER) for Today's pipeline came in at 22.2%, against 26.5% for raw provider output — a 4.3 percentage-point improvement from the separation stage alone. Text error held steady across the same comparison, within half a percentage point of the raw baseline (22–25% CER depending on alignment method). The conditions are deliberately hard: overlapped speech in long meetings is where transcripts actually break. And one boundary we will state ourselves — phone-microphone and weak-network capture are not covered by this benchmark yet; when they are, those numbers land in this paragraph. Until then, the standing invitation holds: measure us against your own meetings.
Accuracy Is the Foundation, Context Is the Building
A perfect transcript of an hour-long meeting is still an hour of reading. What turns it into something worth paying for is the layer built on top — discussion separated from decisions, owners attached to action items, next steps surfaced without prompting, and last month's decision flagged when this month's meeting reverses it, rather than the two being quietly merged. That layer can only be as good as the transcript underneath it, which is why accuracy is where an evaluation should start. It is equally true that the evaluation cannot end there, because a flawless record with no structure is a filing problem wearing a productivity costume.
If that distinction is the kind of thing you care about, it is worth reading what an AI personal agent means — the pattern of a tool that carries context across your work instead of processing one file at a time. Meeting transcription is the sensory layer of that pattern, and the systems that treat the transcript as evidence and memory as context are the ones building something more durable than a typing service. Accuracy is what makes the evidence admissible; context is what makes it useful.
That is the standard Today holds itself to, measured with the ruler this guide has taught you to read. Transcription quality tracked with CER, every pipeline change regression-tested, and public figures that will arrive with their conditions attached rather than standing naked. If your weeks run on meetings, try Today Recording — and judge the results the way this guide taught you to judge everything else, against your own recordings and your own meetings.