Your reading path
Start with the right questions.
From principle to practice
A schema for reproducible speaker activity
Open the worked exampleDiarization is not the same as transcription
Diarization assigns speaking activity to consistent speaker labels over time. Transcription identifies words. A system can get the words right but attribute them to the wrong person, or identify the speakers while missing a short utterance. A useful dataset treats those targets as related but distinct layers.
Overlap makes the distinction especially important. When two people speak simultaneously, a single active-speaker label cannot represent the full reference. Decide whether the task allows multiple labels at one time, whether backchannels count as speech and how uncertain activity is marked. The annotation policy should be chosen before a large collection is labeled.
What real requests and developer documentation show
A public diarization-data request specifically asks about overlapping speakers and noise, while another accented-speaker question seeks two-person recordings with different accents. These are qualitative needs, not verified purchases or estimates of market demand.
pyannote’s original project explicitly includes overlap detection, speaker change and embedding components. Its metric reference explains why evaluation needs a collar policy and can restrict scoring to annotated regions. Those primary technical sources support a detailed task specification without implying a Fiund model has been tested or certified.
Useful benchmark families
AMI is a 100-hour meeting resource with synchronized recording devices and annotations. It is useful for multi-person interaction but differs from telephone calls and edited podcasts. MagicData-RAMC offers Mandarin conversational material under noncommercial/no-derivatives terms. Source domain and access conditions belong beside any comparison.
AppTek’s call-center evaluation corpus supplies role and speaker metadata for commissioned service conversations. It is not automatically an overlap-complete diarization reference simply because speaker fields exist. Inspect the released annotation layer and intended task. No benchmark’s total duration guarantees the amount of manually reviewed overlap available for a new project.
Define the annotation target precisely
Speaker labels can be local to a recording or linked across a collection. Most within-recording tasks do not require personal identity. Use anonymous stable labels and include only identity relationships necessary and permitted for the experiment. A known job role such as agent is different from a speaker identity.
Decide whether laughter, whispered speech, nonlexical acknowledgments and distant voices are included. Some applications need all audible participants; others focus on a foreground conversation. Do not leave that decision to annotators file by file. A written policy with examples is more useful than an undefined requirement for perfect labels.
Overlap needs its own coverage measures
Total hours can conceal a collection with very little simultaneous speech. Request the overlap duration, number of overlap events, typical event length and speaker-count distribution where those figures can be established. Distinguish intentional collection targets from measured source properties.
A pilot should contain short interruptions, longer concurrent answers, laughter over speech and quiet backchannels if those are in scope. Clean alternation between speakers is a useful control but not a substitute for overlap. Preserve source context so a short sound can be interpreted rather than forcing every event into an isolated clip.
Reference timing and uncertainty
Human annotators cannot place every speech boundary at exact sample precision. Define the practical boundary convention and how uncertain regions are represented. A broad uncertain interval should not silently become nonspeech, because that changes both the reference and the score.
Use independent review or adjudication on difficult regions and preserve the decisions. If annotations begin from a model proposal, record that lineage and identify the human-reviewed subset. A recent automated export should not be represented as an independent gold standard without evidence of its preparation and review.
A schema for reproducible speaker activity
Package source recordings, interval annotations and the evaluation map together. The buyer should be able to tell which regions have a reference and which speaker labels can be compared within a file.
- Artifact
- Audio manifest
- Suggested content
- recording_id, source group, timebase, version
- Important distinction
- One source can have several mixes or encodings
- Artifact
- Speaker table
- Suggested content
- anonymous label, local/global scope, role if allowed
- Important distinction
- Role and personal identity are separate
- Artifact
- Activity intervals
- Suggested content
- start, end, speaker, event type
- Important distinction
- Several intervals can overlap in time
- Artifact
- Evaluation regions
- Suggested content
- included and excluded spans, reason
- Important distinction
- Unannotated is not automatically silent
- Artifact
- Review record
- Suggested content
- policy version, review status, adjudication
- Important distinction
- Automatic proposals are not gold references
- Artifact
- Scoring configuration
- Suggested content
- collar, overlap setting, mapping, aggregation
- Important distinction
- Two scores are comparable only under stated conditions
DER, JER and diagnostic measures
Diarization error rate commonly summarizes missed speech, false alarms and speaker confusion relative to the scored reference activity. Jaccard-based measures provide another view of speaker overlap. The exact implementation and mapping matter, so record the tool, version and configuration rather than comparing a bare percentage across papers.
The pyannote metric documentation supports collars and evaluation maps. A collar removes boundary regions from scoring; ignoring overlap changes the target again. Neither policy is inherently a complete description of application quality. Report what was scored and retain diagnostic counts so a change in one component does not disappear inside a single total.
Collars and overlap settings can change the result
Two systems evaluated on the same audio can appear different if one report excludes overlapping speech and another includes it. A boundary collar can also reduce the penalty for small timing disagreement. State these choices in the benchmark specification and in every result table.
For a hypothetical comparison, a buyer might request both an application-aligned primary condition and a stricter diagnostic condition. Those settings should be predeclared and interpreted separately. Do not choose the most favorable configuration after seeing which one improves the headline. A score is evidence about the stated protocol, not a universal property of a model.
Channels, mixtures and enrollment information
Channel-separated source audio can help annotation, but it can also leak information unavailable to the evaluated system. Decide whether the model receives a mono mixture, multiple channels, an exact speaker count or enrollment samples. A system with those hints answers a different question from one operating without them.
Keep reference-assistance artifacts separate from test inputs. If annotators use individual tracks to resolve overlap, document that method and ensure the submitted model sees only the allowed representation. All channels and derivatives of a source session should remain in the same split. Otherwise a model can encounter a closely related version during training.
Speaker counts and short turns
A diarizer can fail by merging similar voices, splitting one person into several labels or missing brief contributions. A collection should include the speaker-count and turn-length patterns relevant to deployment. Long single-speaker stretches alone do not establish performance on rapid conversational exchange.
Inspect short acknowledgments and returns after long pauses. Decide whether a speaker who appears only briefly belongs in the evaluated set and how unknown background voices are treated. These details affect both label consistency and interpretation of the result. They should not be left as undocumented exceptions in a preparation script.
Provider comparison and budget components
A source owner may supply appropriate recordings but no reliable interval labels. An annotation provider may label existing licensed audio but not grant rights to the recordings. A public benchmark may already have references, subject to its terms and domain limitations. Separate those roles when comparing proposals.
No public market average specifically for overlap-reviewed diarization data was established here. Request a quote for source access, candidate preparation, interval annotation, independent review and adjudication. The proportion of difficult overlap and number of active speakers can change effort even when elapsed hours remain identical. A per-hour quote should explain which review depth it covers.
An explicit hypothetical pilot model
Suppose a team plans eight accepted hours with several interaction structures. Using invented placeholders, it allocatesUSD 800 for scoped source access, USD 1,600 for first-pass interval labeling, USD 1,200 for independent overlap review andUSD 600 for adjudication and packaging. The modeled total isUSD 4,200. These assumptions are a planning illustration rather than supplier prices or a Fiund estimate.
Before scaling, measure the pilot’s actual annotation disagreements and repair effort. A collection with a similar duration but more brief turns or more simultaneous speakers can require different work. Keep accepted hours, overlap event counts, speaker groups and review status alongside the cost model so a headline rate does not obscure the deliverable.
Pilot acceptance and held-out design
Choose complete source recordings that contain the difficult events in the specification. Reconcile interval validity, label consistency, evaluation regions and the mapping to the selected audio version. Test the scoring pipeline on deliberately understandable examples before trusting aggregate outputs.
Group related sessions and speakers according to the experiment. A system intended for unseen people needs a different held-out boundary from one meant to track recurring participants. If speaker relationships are not available, document the limitation. Do not claim a speaker-disjoint benchmark simply because filenames were randomly assigned to partitions.
Reference availability by region
A recording can be valid while only some regions have a reviewed diarization reference. Preserve that evaluation map and report the scored duration separately from the whole file. Unreviewed regions should not silently become nonspeech or be counted as independently labeled hours.
Discuss diarization requirements with Fiund
Contact hello@fiund.com with the recording domains, languages, speaker counts, overlap patterns, allowed model inputs and scoring policy. Include whether the collection is for training or held-out evaluation and what independent reference review is required. Keep source acquisition and annotation expectations separate in the brief.
Fiund can discuss candidate source fit, a representative pilot and project-specific preparation. Availability, permissions, pricing and delivery are confirmed for selected material. No category page guarantees a model score, a universally complete reference or access to every named benchmark. Those claims require evidence from the actual collection and experiment.
Questions buyers ask
Sources & research notes
Public documentation reviewed 2026-11-12. Provider descriptions and published terms are attributed below. Published facts are attributed to their sources. Worked files are labeled as illustrative or as an inspected editorial source; neither implies commercial dataset clearance.
- AMI corpus original projectgroups.inf.ed.ac.uk · Reviewed 2026-11-12
Primary source reviewed November 12, 2026; check the current release and project-specific terms.
- MagicData-RAMC official OpenSLRopenslr.org · Reviewed 2026-11-12
Primary source reviewed November 12, 2026; check the current release and project-specific terms.
- AppTek Call-Center Dialogues official datasethuggingface.co · Reviewed 2026-11-12
Primary source reviewed November 12, 2026; check the current release and project-specific terms.
- Accented-speaker diarization requestwww.reddit.com · Reviewed 2026-11-12
Original public request; qualitative signal, not a verified purchase or search-volume measure.
- Overlap/noise diarization requestwww.reddit.com · Reviewed 2026-11-12
Original public request; qualitative signal, not a verified purchase or search-volume measure.
- pyannote.audio original projectgithub.com · Reviewed 2026-11-12
Primary source reviewed November 12, 2026; check the current release and project-specific terms.
- pyannote.metrics primary metric referencepyannote.github.io · Reviewed 2026-11-12
Primary source reviewed November 12, 2026; check the current release and project-specific terms.
