Your reading path
Start with the right questions.
From principle to practice
A small normalization test with large consequences
Open the worked exampleTest the representation before comparing corpus totals
A buyer may receive audio labeled “Spanish” or “Arabic” without enough information to interpret the exact text targets. The immediate engineering questions are whether the labels match the recordings, whether the writing convention is explicit and whether the data loader preserves the intended characters. A corpus can have many hours and still fail this first intake.
Take a deliberately varied sample before bulk processing: different files, source families, speakers, language labels and known difficult text patterns. Use the same small fixture through decoding, normalization, tokenization, training-data preparation and scoring. Record where the representation changes instead of assuming every stage treats the text identically.
Inspect a public card as a schema example, not a supplier guarantee
Google’s FLEURS dataset card exposes language metadata and both raw and normalized transcriptions. These fields demonstrate useful distinctions to preserve when inspecting another source. They do not establish a universal schema or guarantee that another collection has the same coverage.
A language identifier, a recording location and a speaker’s self-described language background are different fields. The archive may support some and not others. Preserve unknown values and document how a label was assigned. Do not fill a locale field by guessing from an accent or treat a language code as proof of regional representativeness.
A small normalization test with large consequences
The following authored test fixture uses hypothetical source text; it is not a transcript from an available dataset. Its purpose is to surface choices that can otherwise remain hidden until evaluation.
- Test string or event
- “12/03/2026” spoken as a date
- Decision to document
- Locale, intended spoken form and reference representation
- What can go wrong
- A text-normalizer imposes the wrong day/month interpretation
- Test string or event
- A product name embedded in another language
- Decision to document
- Preserve the name, mark language change if supported, define scoring
- What can go wrong
- A normalizer translates or deletes the term the product must recognize
- Test string or event
- A word with a meaningful diacritic
- Decision to document
- Character preservation, Unicode normalization and tokenization
- What can go wrong
- Different text encodings create apparent recognition errors
- Test string or event
- Two valid written variants
- Decision to document
- Allowed variants and adjudication policy
- What can go wrong
- One acceptable output is counted as wrong without a stated convention
- Test string or event
- An incomplete self-correction
- Decision to document
- Verbatim treatment and uncertainty marking
- What can go wrong
- A cleaned target erases words that remain audible
Interpret the fixture as a contract between pipeline stages
If one preparation tool writes a number as words while the scorer expects digits, the resulting error may be a convention mismatch rather than a recognition failure. Decide which representation belongs in the source transcript, model target and scoring view. Keep the original available so the transform is reversible or at least explainable.
The Transformers ASR task guide illustrates model-specific preparation, including a particular vocabulary convention. Follow the selected processor’s requirements rather than copying a case transformation into every language. What is harmless for one tokenizer can damage another script or task.
Keep audio input choices separate from language claims
Inspect the actual source sample rate, channel layout and encoding. Prepare the model input as required while retaining the source identity and timing relationship. Resampling should not silently rewrite metadata to claim that the source was recorded at the derived rate.
For code-switching, decide whether the example has one dominant language label, span-level labels, multiple labels or an explicitly unknown status. Those choices support different tasks. A mixed utterance should not disappear from evaluation merely because the loader assumes exactly one language code per file.
An illustrative intake record
The record below is an authored schema example. It keeps what the source says separate from what the preparation pipeline does. Its locale is intentionally unknown; inventing a plausible region would make the record less trustworthy.
{
"segment_id": "sample-017",
"source_language": "es",
"source_locale": null,
"label_method": "source-provided language label; sample review pending",
"raw_text": "La entrega es el 12 de marzo.",
"normalized_text": "la entrega es el doce de marzo",
"normalizer_version": "project-rule-set-1",
"source_sample_rate_hz": 48000,
"model_input_sample_rate_hz": 16000,
"source_group": "recording-family-04",
"split": "development"
}Score each language with its own assumptions visible
Whisper’s official documentation reports substantial variation in recognition performance by language. A single aggregate score can therefore conceal both source gaps and scoring errors. Report the metric, text convention, example count and relevant source conditions for each tested group.
Where word segmentation is ambiguous or inconsistent, define a suitable scoring unit with qualified language review rather than assume space-delimited English-style WER is automatically comparable across scripts. Keep the same evaluation identities through revisions. Do not interpret a small language slice as a precise population estimate.
Release an intake report that another engineer can repeat
The useful output is not a reassuring language checklist. It is a record of what passed, which transformations were applied and what remains unresolved. A failed test should lead to a correction, an explicit exception or a decision not to use that subset for the intended task.
- Identify the sampled source groups, language labels and unsupported metadata.
- Retain raw text, normalization rules and processor/model versions.
- Record decode failures, transcript mismatches and script-handling defects separately.
- Define treatment of code-switching, variants, uncertainty and non-speech events.
- Report scores and sample counts by language and relevant conditions.
- Agree which unresolved fields or errors block a larger delivery.
Questions buyers ask
Sources & research notes
Public documentation reviewed 2026-11-13. Provider descriptions and published terms are attributed below. Published facts are attributed to their sources. Worked files are labeled as illustrative or as an inspected editorial source; neither implies commercial dataset clearance.
- FLEURS dataset cardGoogle · Reviewed 2026-11-13
The card exposes language metadata, raw and normalized transcriptions, and identifies CC BY 4.0. Benchmark coverage is not proof of representative spontaneous dialogue or commercial supplier inventory.
- Transformers automatic speech recognition task guideHugging Face · Reviewed 2026-11-13
ASR maps audio to text. The example pairs audio/transcription, resamples to the selected model input rate, and evaluates WER. One implementation recipe, not universal corpus specifications, data-hour requirements, or annotation-accuracy thresholds.
- Whisper official repository and model documentationOpenAI · Reviewed 2026-11-13
Whisper supports recognition, translation and language identification; reported recognition performance varies by language. Model and software documentation does not license external recordings or establish guaranteed adaptation performance.
