Your reading path
Start with the right questions.
From principle to practice
A worked correction sequence
Open the worked exampleA conversation is not one universal training example
For speech recognition, the target is what was said under a stated transcription convention. For diarization, the labels describe who spoke when. For a task-oriented agent, the target might be the current request state, an allowed tool action or an evidence-supported response. A collection can support the first while lacking the others.
This distinction affects source selection. A panel discussing travel contains travel vocabulary, but it is not automatically a dataset of successful bookings. A customer-service exchange can contain a request and an apparent resolution, yet the audio alone may not establish what happened in the underlying system.
Public research illustrates different target designs
The SpokenWOZ paper describes a task-oriented spoken-dialogue benchmark with cross-turn challenges. Its annotations are part of that dataset’s design. They should not be presumed to exist in a podcast archive just because both sources contain human conversation.
The Spotify Podcast Dataset paper describes episodes with ASR transcripts and varied formats. This offers a useful contrast: an automatic text layer can make an archive searchable without establishing human-reviewed state or outcome labels. Read the target definitions before comparing dataset sizes.
A worked correction sequence
This is an authored illustration, not a real customer call or recorded dataset sample. A user says, “Book Friday—sorry, Monday.” The other speaker asks, “Monday morning?” The user replies, “After two.” The same three turns produce different correct targets depending on the task.
- Task
- ASR
- Target for this illustration
- Retain the audible Friday-to-Monday correction under the chosen convention
- Evidence needed
- The exact audio and a reviewed text reference
- Task
- Speaker attribution
- Target for this illustration
- Keep the two participant identities consistent across the turns
- Evidence needed
- Supported speaker or channel evidence
- Task
- Dialogue state
- Target for this illustration
- Requested day becomes Monday; time constraint becomes after 2:00, with AM/PM and date context unresolved
- Evidence needed
- Turn order, interpretation rules and any relevant context
- Task
- Response generation
- Target for this illustration
- A suitable next response may clarify the actual date and available appointment
- Evidence needed
- The allowed task policy and available information
- Task
- Task success
- Target for this illustration
- A booking is completed only if an appropriate external record supports it
- Evidence needed
- A tool result, booking record or other defined outcome evidence
Why cleaning the transcript can change the task
A readable summary such as “The user wants Monday afternoon” is not a verbatim ASR reference. It removes the initial request, the correction and the clarifying question. That may be a useful summary target, but it should have its own type and lineage instead of replacing the original text.
The Transformers ASR guide provides an audio-to-text training example. In a conversation project, define any additional label layers explicitly. Stable turn and segment IDs let an engineer connect the transcript, speaker annotation and state update without pretending they are the same artifact.
Use label contracts to handle uncertainty
In the example, “Monday” lacks a calendar date unless other context supplies it. A labeler should not invent a date from the current time or assume that a request has been accepted. Represent the missing field and state the evidence required to resolve it. Similar issues arise with names, quantities, negation and repairs.
Write rules for ambiguous references, incomplete turns, overlapping words and unresolved outcomes. Include an adjudication route for disagreements rather than force every case into a clean success label. The difficult examples are often the most useful test of whether the annotation specification is actually actionable.
Diagnose a pipeline in stages
If a voice agent books the wrong day, inspect the stages separately. Did the recognizer miss “sorry”? Did the state updater fail to replace Friday? Did a tool call use the stale value? Did the system say “done” without a successful result? One end-to-end failure can have several causes.
A useful evaluation packet retains inputs and outputs at the relevant stages, with access and handling agreed for the project. It reports which layer was reviewed and which was generated automatically. Otherwise an apparently improved response model can conceal a worse transcript or an unsupported outcome label.
Select sources for the conversational evidence they retain
Complete exchanges are often more useful than isolated utterances when the target depends on a correction, question or earlier constraint. Keep source boundaries and related editions visible. A collection of short clips may be appropriate for recognition but unable to answer a long-context dialogue question.
Existing recordings, commissioned scenarios and generated conversations offer different evidence. Label the origin. A scripted scenario can cover a difficult policy branch but should not be described as a natural customer outcome. Evaluate on appropriate independent examples rather than treating a plausible script as empirical product validation.
Specify the handoff before expanding the corpus
A first brief should say which target layers are required and what will count as an accepted example. Fiund can discuss relevant sources and preparation against that brief; current availability and scope remain project-specific.
- Define the task, context window and selected source unit.
- List separate transcript, speaker, state, action and outcome fields as needed.
- Provide one worked positive example and one ambiguous or unanswerable case.
- State which labels are human-reviewed, automatic or inferred from another system.
- Group related sessions, speakers and derivatives to protect evaluation independence.
- Define acceptance by evidence and task behavior, not conversation hours alone.
Questions buyers ask
Sources & research notes
Public documentation reviewed 2026-11-13. Provider descriptions and published terms are attributed below. Published facts are attributed to their sources. Worked files are labeled as illustrative or as an inspected editorial source; neither implies commercial dataset clearance.
- SpokenWOZ: A Large-Scale Speech-Text Benchmark for Spoken Task-Oriented Dialogue AgentsSpokenWOZ authors · Reviewed 2026-11-13
The benchmark connects human spoken dialogues with task-oriented annotations and cross-turn challenges. Task-oriented dialogue is not interchangeable with podcast archives or a universal voice-agent evaluation.
- The Spotify Podcast DatasetSpotify research authors · Reviewed 2026-11-13
The published dataset contains roughly 100,000 episodes with ASR transcripts and varied podcast formats. ASR transcripts are not human-verified references; historical research access does not establish commercial AI-training rights.
- Transformers automatic speech recognition task guideHugging Face · Reviewed 2026-11-13
ASR maps audio to text. The example pairs audio/transcription, resamples to the selected model input rate, and evaluates WER. One implementation recipe, not universal corpus specifications, data-hour requirements, or annotation-accuracy thresholds.
