Picture an insurance contact center that evaluates a speech-to-text vendor on a single number: word error rate. The vendor benchmarks at 5% WER, meaning 95% word accuracy, which beats the competition and wins the deal. Six months in, the analytics team is frustrated. The transcripts look accurate but the downstream analytics — compliance detection, topic categorization, sentiment — are performing worse than expected.
Dig into the failures, and the 5% of words the engine gets wrong are often not randomly distributed. They tend to concentrate exactly where it matters: product names, policy terms, proper nouns, numbers, and the domain-specific vocabulary the analytics depend on. The engine can nail the common words — “the,” “and,” “I’d like to” — and miss “deductible,” “rider,” “underwriting,” and the specific product SKUs. 95% word accuracy, with the misses landing on the words that carry the meaning.
This is the trap of evaluating speech-to-text on word error rate. WER treats every word as equally important. In a contact center, the words are catastrophically unequal in importance, and the engine that wins on WER often loses on the only accuracy that matters.
Why Word Error Rate Is the Wrong Metric
WER counts the percentage of words transcribed incorrectly, treating all words identically. A missed “um” costs the same as a missed policy number. For general transcription this is reasonable. For contact center analytics it’s actively misleading.
The reason is that the value of a transcript isn’t in its words — it’s in what you do with it. If you’re running compliance detection, the only words that matter are the ones tied to the compliance requirement. If you’re categorizing call topics, the only words that matter are the topic signals. If you’re detecting product mentions for cross-sell analysis, the only words that matter are the product names. WER averages across all words and tells you nothing about accuracy on the words your use case depends on.
This is why two engines with identical WER can produce wildly different analytics performance. The one that’s accurate on common words and weak on domain vocabulary will benchmark identically and perform far worse in production.
What Actually Determines Transcription Value
For contact center speech analytics, several accuracy dimensions matter more than aggregate WER.
Domain vocabulary accuracy. How well does the engine handle your industry’s specific terms, your product names, your acronyms? This is usually the single biggest determinant of analytics quality and it’s completely invisible in a generic WER benchmark.
Named entity accuracy. Numbers, names, dates, amounts, account references. These carry disproportionate meaning and are disproportionately hard for general engines, which are trained on conversational speech where these appear less frequently.
Speaker separation accuracy. In a two-party call, knowing who said what is often as important as what was said. A compliance disclosure only counts if the agent said it. An engine that transcribes accurately but attributes poorly is useless for agent-level analytics.
Robustness to real conditions. Accents, background noise, telephony compression, crosstalk, emotional speech. Benchmark WER is usually measured on clean audio. Production audio is messy, and the accuracy gap between clean and messy conditions varies enormously between engines.
The Adaptation Question
The biggest lever on transcription accuracy for a specific contact center isn’t the base engine — it’s adaptation to the specific domain.
A general engine handles general speech well. A general engine adapted to your vocabulary, your products, your acronyms, and your typical call patterns handles your calls dramatically better. The accuracy improvement from domain adaptation on the words that matter often dwarfs the difference between competing base engines.
This is why the procurement question “what’s your WER” is the wrong question. The right questions are: how does the engine adapt to our domain, how quickly, and what’s the accuracy on our specific high-value vocabulary after adaptation. Vendors who lead with WER are usually answering the question that flatters them, not the one that predicts your outcome.
Five Things You Can Do This Week
1. Build a domain vocabulary test set. List your 50 most important terms — products, policy language, compliance phrases. Test your current engine’s accuracy specifically on these. The result will likely differ sharply from the headline WER.
2. Audit speaker separation accuracy. Pull 20 transcripts and check whether agent and customer turns are correctly attributed. Agent-level analytics depend entirely on this.
3. Test on your worst audio, not your best. Evaluate accuracy on heavily accented calls, noisy calls, and emotional calls. That’s where the engines separate.
4. Trace one analytics failure back to the transcript. When compliance detection or topic categorization misses, check whether the root cause was the transcript. It often is, and it’s usually a domain-vocabulary miss.
5. Ask vendors about adaptation, not WER. How does the engine learn your vocabulary, how fast, and what accuracy does it reach on your terms specifically. The answer predicts your outcome better than any benchmark.
A transcription engine that’s 95% accurate on words and 40% accurate on the words you actually need is worse than useless — it’s confidently wrong in exactly the places you’re relying on it. Word error rate measures the easy part. The hard part, and the valuable part, is the vocabulary your business runs on.
