Speech datasets with speaker demographic metadata: sourcing and evaluation

Start with the question the metadata must answer

A team evaluating speech recognition across age bands needs a different collection from one studying regional pronunciation or training an expressive voice model. Write the evaluation question before choosing a list of attributes. Explain why each requested field changes the analysis and which decisions will be made from it. Collecting every possible field creates review work without necessarily improving the experiment.

Separate source description from model output. A recording can have a language label, an acoustic channel description and optional speaker attributes, each established through a different process. Do not turn a model's guess about accent, age or gender into verified participant metadata. If inferred attributes are intentionally part of research, retain the inference method and uncertainty as a separate derived layer; do not substitute them for source-reported facts.

Read a metadata field as a claim with a provenance

The Common Voice maintainer's CorporaCreator field documentation explicitly distinguishes contributor-reported age, gender, accents and language variant. That is useful evidence about that documented schema; it is not a guarantee that every release or recording has those fields populated. Inspect the release and locale actually selected.

The official FLEURS card provides a different field structure, including language and a gender class label. A buyer should examine its documented meaning rather than silently translating it into a richer demographic profile. Its read-speech focus also matters: a demographic distribution in read utterances is not equivalent to the same distribution in spontaneous dialogue.

A practical delivery should attach three pieces of information to each optional attribute: its value, its basis and its record/version reference. A missing value can mean not asked, not supplied, withheld, removed or not applicable. Those states have different consequences for recruitment and analysis, so choose an explicit vocabulary before merging sources.

Count speakers, duration and sessions independently

A corpus with many hours can still contain only a few recurring voices. Report unique speakers, unique sessions, accepted speech duration and the distribution across the attributes used in the study. Keep the denominator visible: an hours-weighted chart can conceal that one speaker provides most of a category's material.

Use a speaker-by-session matrix alongside duration totals. The same individual may appear in multiple shows, roles or source packages, and the same conversation may be distributed as alternate edits. A source-specific participant identifier can help with grouping, but identifiers from unrelated publishers should not be assumed to resolve the same person. Record unresolved identity relationships rather than claiming a globally unique speaker count.

For a held-out evaluation, determine whether the split should isolate speakers, recording sessions, household or organization groups, devices, or combinations of these. Choose the grouping for the actual deployment question. A metadata-balanced random clip split can still leak the same voice and room into training and evaluation.

Use a coverage matrix before commissioning a collection

Define required and optional cells, a minimum useful number of independent speakers per cell and what happens when a cell cannot be filled. Avoid promising that all combinations will be equally represented before recruitment feasibility is established. Attribute intersections can create very small groups even when each attribute looks balanced on its own.

Keep recording conditions in the same planning table. If every example in one demographic group comes from a telephone and every example in another comes from a studio microphone, a model comparison cannot cleanly attribute the difference to the requested attribute. Device, channel, topic and speaking style can be confounded with the group definition. Report these associations rather than describing the collection as representative of a population.

This planning approach follows the documentation purpose of Datasheets for Datasets: make composition, collection and intended use explicit. A buyer should still review their own use case, especially when attributes might affect consequential decisions; this guide does not establish that a proposed use is appropriate or permitted.

Worked example: eight fictional speakers and a missing-data decision

The downloadable CSV contains eight invented speaker IDs, illustrative English locale labels and duration values chosen solely to demonstrate a schema. It contains no recordings, identities or real permission records. Gender and accent are not provided; some age-band values are deliberately absent. These are meaningful missing states, not a signal to fill them from audio.

Suppose a project requires a pilot with at least two independent speakers for each agreed age-band and channel cell. A reviewer should first identify which rows have usable source-reported metadata, then check independence and channel coverage. A duration total cannot rescue a cell that contains only one person. If a required field is absent, the accepted choices are to obtain appropriate source documentation, change the agreed scope or exclude that record from the relevant slice—not to invent the missing value.

Download the illustrative speaker metadata CSV and field dictionary. Adapt the fields to a real project only after reviewing how the selected source obtained them. Keep evidence references separate from private forms or identity records.

Review a representative pilot without exposing participant records

Ask for a metadata dictionary, aggregate coverage report and appropriately accessible sample before requesting an entire archive. Review the same sample's audio and metadata together. Confirm that a speaker ID is stable across files, that duration rules match the supplied audio and that segment language changes are not hidden by a single recording-level language label.

Check missingness per source and per proposed group. A source with many optional values absent can still be useful for general acoustic variation, but it may not support the demographic analysis originally requested. Document the limitation in the proposal. Use restricted, agreed handling for any identity mapping or permission evidence; an initial sales email does not need private participant documents.

Acceptance should include a disagreement process. When two records conflict, determine which source/version is authoritative, how corrections propagate and whether affected evaluation results need recalculation. Do not silently overwrite an earlier value while keeping the same unqualified dataset version.

Pricing depends on the accepted coverage, not the number of columns

A quote may combine source access, participant recruitment, documentation review, recording, transcript preparation, metadata validation and quality assurance. Adding a metadata column to a file is easy; obtaining a documented value from an appropriate source can be a separate collection task. Compare the scope, not merely a per-hour number.

For a transparent planning example, eight coverage cells with four independent speakers and fifteen accepted minutes per speaker would require 32 speakers and eight accepted hours. Those are invented scope assumptions, not Fiund inventory, a representative demographic sample or a price estimate. The cost still depends on recruitment feasibility, retention, recording setup, allowed use and the review process.

Ask whether a proposed fee covers unique speakers, recorded hours, accepted speech hours or completed coverage cells. State how failed recruitment, rejected recordings and missing metadata are handled. Do not compare a public speech catalog list rate with a custom demographic recruitment project as if the deliverables were identical.

Write the sourcing brief so uncertainty remains visible

Send the model task, requested attributes, their purpose, recording conditions, required independence, target volume and acceptance criteria. Include which attributes may be unknown and whether the project can accept a source without them. Separate a preference for diversity from a measurable and reviewable coverage requirement.

Fiund can use that brief to review source fit and the work required to produce an appropriate proposal. This page does not promise any language, demographic group, speaker count or permission scope in current stock. For language-specific source and transcript requirements, read the multilingual speech solution; for technical background, see the speaker diversity guide.

Questions buyers ask