Code-switching speech datasets for ASR evaluation Code-switching ASR benchmarks: natural versus prompted speech, mixed-script references, scoring rules, source rights and pilot acceptance. Code-switching ASR datasets evaluate recognition when more than one language is used within an utterance or conversation. They need explicit switching definitions, script and normalization rules, speaker/session relationships and scoring that does not hide errors at language boundaries. A set of separate monolingual recordings is not the same target, and synthetically assembled speech should remain distinguishable from naturally occurring dialogue. This guide focuses on cross-source recognition benchmarks, separate from acquiring a multilingual call or podcast archive. Fiund can discuss source-specific evaluation material and reference preparation, with availability, permissions and terms confirmed for the selected collection. Primary task Recognize mixed-language speech Core distinction Natural switching, prompted bilingual speech and synthetic assembly Reference design Scripts, language spans, normalization and uncertainty Important split Speaker, session and related source groups Price status Source access and language-qualified reference scope Define the switching behavior before selecting data Code switching can occur across turns or inside one utterance. Borrowed terms, proper names and abbreviations can create ambiguous boundaries, so the benchmark needs an operational definition. A buyer asking for bilingual data should specify which behavior it wants rather than assuming every mixed vocabulary example qualifies. A two-person conversation can also involve one monolingual speaker in each language. That is a useful interaction but a different recognition condition from a single person switching repeatedly. Preserve these patterns in the manifest. An aggregate language list cannot describe the switch structure or its difficulty. Natural, prompted and synthetic source classes Natural conversation captures switching that arose during the interaction. Prompted speech can control coverage but may reflect the collection instructions. Synthetic text or audio can add examples with a known construction method. These source classes should be compared explicitly instead of silently merged. Synthetic assembly can introduce artifacts at boundaries, while prompted reading can change prosody and vocabulary. Those are experimental properties to document, not reasons to reject every such resource. A held-out set intended to represent actual user conversation should describe how its source behavior matches that target. Public benchmarks worth comparing [CS-Dialogue’s original paper](https://arxiv.org/abs/2502.18913) reports 104 hours of spontaneous Mandarin-English dialogue from 200 speakers. [ASCEND’s official card](https://huggingface.co/datasets/CAiRE/ASCEND) exposes language, session, topic and speaker fields for another Mandarin-English resource, withCC BY-SA 4.0 metadata. Their current release terms and annotation details should be reviewed directly. [SwitchLingua’s paper](https://arxiv.org/abs/2506.00087) describes a multilingual synthesis framework and an associated speech collection. It should not be summarized as wholly spontaneous dialogue. [FLEURS](https://huggingface.co/datasets/google/fleurs) and [VoxPopuli](https://github.com/facebookresearch/voxpopuli) provide broader multilingual controls, but multilingual breadth alone does not make them equivalent code-switching tests. Evidence of a specific data need A [public original-bilingual-clip request](https://www.reddit.com/r/datasets/comments/1x2sv7q/request_paid_selfpromotion_60_bounty_for_30/) illustrates the need for meaningful speech in both languages rather than a shared name alone. It is one qualitative request with its own conditions, not a representative survey or a large commercial procurement. Potential users include multilingual recognizer developers, bilingual voice agents and speech-translation teams. The relevant benchmark differs by application. A recognizer may need literal words, a dialogue system may need turn context and a translation model may need a separate semantic target. No category page establishes a purchase commitment from a named company. Reference scripts and normalization A benchmark must decide whether words appear in native script, transliteration or another agreed representation. Keep the literal reference separate from any normalized scoring form. Proper names, borrowed terms and mixed abbreviations need examples in the policy so reviewers do not invent inconsistent rules. Tokenization matters when one language is naturally scored at a different unit from another. Document the unit and normalization for each span and the way they combine into a mixed-language score. A result without that policy can be difficult to compare even when two teams use the same audio. Language boundaries need a review convention Not every token has an unambiguous language assignment. A widely borrowed term may be pronounced within the surrounding language, and a name can occur in several languages. Define how the benchmark handles those cases and preserve uncertain or shared-category labels where appropriate. Span-level language labels should be tied to the selected audio/text version. If a transcript is corrected, update dependent language spans rather than leaving an old alignment. A single file-level mixed label can help filter candidates but does not describe where switches occur or whether the annotations are suitable for boundary-level analysis. Coverage beyond a total-hour figure Useful coverage measures can include language pair, switch direction, switch frequency, within-turn versus between-turn patterns, topic and speaker/session diversity. Count monolingual context separately from actual switching regions when it matters to the task. A long mixed-language file can contain only one brief switch. Coverage item Recommended evidence Why it helps Language pair/direction Reviewed source labels and spans Distinguishes different switching conditions Switch structure Within-turn, between-turn or borrowed term Prevents a broad mixed label from hiding the target Speaker/session Stable anonymous groups Supports independent evaluation Source class Natural, prompted or synthetic Keeps construction artifacts visible Channel/domain Capture setup and topic Exposes nonlanguage confounders Reference status Script/normalization policy and review Makes scores interpretable Scoring should reveal, not erase, boundary difficulty Report the exact scoring implementation and the treatment of scripts, punctuation, numbers and alternative spellings. A mixed error measure can be useful when languages use different natural units, but its construction should be explicit. Keep per-language and switching-region diagnostics where they answer the application question. A semantically tolerant score can complement literal recognition scoring, but it should not silently replace it. A transcript that preserves approximate meaning may still fail a task requiring names or exact numbers. Choose the primary metric before inspecting results and retain enough outputs to explain substitutions, deletions and insertions. A hypothetical metric example Imagine a planning example with a reference containing 100 scored units after the agreed mixed-language tokenization. If an output hasfive substitutions, three deletions andtwo insertions, the illustrative error rate is 10%. Changing tokenization can change both the denominator and the alignment, so the policy must travel with the number. This is arithmetic, not a measured result for any model. If the application depends on switch boundaries, report that slice separately rather than letting abundant monolingual context dominate. Do not compare a character-based rate from one protocol directly with a word-based rate from another as if they were identical units. Audio context and source independence Keep enough preceding and following speech to interpret the switch. Cutting exactly at a language boundary can create an easier artificial segment or remove important acoustic context. If the product receives complete conversations, evaluate complete source units or document why prepared excerpts are appropriate. Group related segments, sessions and speakers according to the intended held-out question. Prompt templates and synthetic construction sources can also create relationships. Randomly splitting clips from one conversation is not evidence of generalization to new speakers or new switching behavior. Record unknown source links as limitations. Provider and access comparisons A public benchmark can supply a defined research reference under its current conditions. A custom collection provider can target specific language pairs and switching instructions. An archive source can supply naturally occurring behavior but may require more selection and review. The same bilingual label should not obscure those differences. Ask whether the proposed source already exists, how switch behavior was established and what reference preparation is included. A research paper’s publication license does not itself settle all underlying recordings or participant permissions. Confirm the actual files, source terms and intended use before treating a resource as commercially available. Pricing and a hypothetical pilot model No reliable public average for commercially usable code-switching ASR evaluation data was established here. Source access, language-qualified transcription, boundary labeling and independent review need their own scope. A small public collection bounty should not be converted into an average per-hour benchmark price. For a planning-only ten-hour pilot, invented placeholders might allocateUSD 1,000 for scoped source access, USD 1,600 for mixed-script reference preparation, USD 1,000 for independent language-boundary review andUSD 600 for packaging/scoring reconciliation: USD 4,200 total. These are transparent assumptions, not supplier prices or a Fiund quote. Pilot acceptance should challenge the policy Include natural switches, monolingual controls, a borrowed term, a proper name, an ambiguous boundary and an utterance with different script choices. Review the source class and full context before deciding how it should be labeled. A clean obvious bilingual sentence alone does not test the rubric. Reconcile audio, text, language spans and scoring output. Document disagreements and revise examples before scaling. A pilot report should distinguish file validity, label quality, permission review and model results. Success in one layer does not establish the others, and no benchmark should be described as universally complete for every language pair. Critical terms and release reproducibility A mixed-language transcript can preserve broad meaning while getting a product name, number or short command wrong. If those errors matter to the application, define an additional critical-term review rather than assuming a semantically tolerant metric captures them. Keep the literal reference and the semantic interpretation separate so readers can understand what improved. Package the release with a stable source manifest, annotation policy, normalization rules and split definition. Record which source and reference versions were scored and how uncertain language spans were handled. A later text correction should have a version identifier rather than silently changing the benchmark. This makes repeated experiments comparable without implying that every source relationship or possible spelling has been perfectly resolved. Switch frequency and accepted volume Report both complete source duration and the regions that demonstrate the required switching behavior. A long recording with one borrowed term should not silently satisfy a brief for sustained bilingual interaction. Preserve monolingual context when useful,but explain how acceptance counted it. This keeps a headline hour total tied to observed behavior rather than a broad file-level label. Discuss a code-switching brief with Fiund Email jaeden@fiund.com with language pairs, switching patterns, domains, recording conditions and the intended task. State whether natural dialogue is essential or prompted material can fit. Include reference scripts, normalization and held-out requirements if already defined. Fiund can discuss candidate sources and a bounded preparation plan. Availability, permissions, pricing and delivery are confirmed for selected material. No largest publicly priced code-switching dataset transaction was verified here; research scale, funding rounds and broad speech partnerships should not be relabeled as a corpus sale. Is a multilingual corpus automatically code-switched? No. It may contain separate monolingual recordings. The benchmark needs evidence of the switching behavior required by the task. Should proper names count as a language switch? Define that convention explicitly. A shared name alone may not represent the meaningful mixed-language speech the application needs. Can synthetic data be included? It can be useful when permitted and clearly labeled. Preserve its construction method and evaluate separately where source artifacts could affect results. Is one mixed error rate enough? Often additional diagnostics are useful. State the tokenization and normalization, then examine switching regions and language slices if they matter to the product. What should a Fiund pilot establish? That candidate sources and the reference policy support the specified behavior on representative inspected examples. It should not imply guaranteed access, complete coverage or a model-performance promise. Sources CS-Dialogue original paper: https://arxiv.org/abs/2502.18913 Google FLEURS official dataset card: https://huggingface.co/datasets/google/fleurs Meta VoxPopuli official repository: https://github.com/facebookresearch/voxpopuli Bilingual clip collection bounty: https://www.reddit.com/r/datasets/comments/1x2sv7q/request_paid_selfpromotion_60_bounty_for_30/ SwitchLingua original paper: https://arxiv.org/abs/2506.00087 ASCEND official dataset card: https://huggingface.co/datasets/CAiRE/ASCEND