Fine-tune Whisper with custom data: a local training and evaluation workflow Plan local open-weight Whisper adaptation with grouped audio/transcript data, reproducible preprocessing and held-out scoring, separate from hosted API availability. Local Whisper fine-tuning adapts a selected open-weight checkpoint using paired audio and reviewed transcripts. It is separate from calling a hosted transcription API. Start with a reproducible baseline, permissioned task-relevant data, source-aware splits and a frozen evaluation policy. Train only when the experiment can test a defined failure without leaking the same speakers or recordings into evaluation. This guide provides an illustrative plan, not a trained model, measured improvement or promise that a particular hosted endpoint supports fine-tuning. Workflow scope Local checkpoint adaptation using an appropriate training stack API distinction Hosted whisper-1 lifecycle is a separate product question Evaluation first Baseline, grouped holdout and frozen scoring policy Worked files Illustrative grouped JSONL and an unrun evaluation plan Separate local checkpoint training from the hosted API A local workflow loads model weights and a compatible processor into infrastructure you control, then updates parameters using your training data. An API workflow sends requests to a hosted service. These differ in operational responsibility, data handling, customization and product lifecycle. Do not describe uploading a transcription file as creating a fine-tuned Whisper model, or assume that every model appearing in a hosted catalog exposes an equivalent local training service. As checked on 13 November 2026, the [official OpenAI Whisper API model page](https://developers.openai.com/api/docs/models/whisper-1) marks whisper-1 deprecated. The [official deprecations page](https://developers.openai.com/api/docs/deprecations) lists its hosted shutdown date as 26 February 2027 and documents a separate self-serve fine-tuning timeline. Those hosted notices do not establish that local open-weight training is technically impossible. The local workflow below is grounded in Hugging Face’s model implementation documentation, not a promise of access to an OpenAI hosted fine-tuning endpoint. Decide whether adaptation addresses the observed error Begin by inspecting real errors under the intended deployment conditions. A model may struggle with domain vocabulary, channel distortion, speaking style, language selection, segmentation or an inconsistent reference transcript. These do not all require parameter updates. A faulty resampling step, missing context or unsuitable chunking can make a strong model appear weak. Correct the pipeline and establish a stable baseline before deciding that more training is the right intervention. Write an experiment hypothesis that names the failure and the held-out evidence that would support improvement. For example, a domain-term project should evaluate the relevant terms as well as aggregate transcription error. Keep general-speech and difficult-condition checks so an apparent target-domain gain does not hide a regression. There is no universal minimum number of hours, default model size or guaranteed training duration for every language, source condition and adaptation objective. Prepare audio and transcripts as a paired record Choose a source where the proposed training and evaluation uses can be supported by appropriate documentation. Preserve source identity, recording session, relevant speaker relationships and preparation history. A public episode or downloadable recording does not automatically establish the scope for a new model-training use. Keep source review distinct from transcript correctness, and avoid placing private participant information or confidential agreement material in public manifests. The [Hugging Face fine-tuning tutorial](https://huggingface.co/blog/fine-tune-whisper) demonstrates paired audio/transcript preparation and a feature-extraction/tokenization workflow. Its published example is a dated implementation reference, not a universal recipe or a Fiund result. For your own data, verify that each target transcript corresponds to the delivered audio interval. Correct mismatched segments, missing words and invented text before training. A polished transcript that paraphrases the speech can be unsuitable when the objective requires faithful recognition. Freeze transcript policy before computing a baseline Specify how to handle punctuation, capitalization, filled pauses, repetitions, partial words, numbers, acronyms, non-speech and unintelligible regions. A verbatim reference and an edited publication transcript answer different questions. Keep the original reviewed text and any scoring-normalized version separately. Changing normalization after seeing model outputs can make two results appear comparable when the scoring target has changed. For code-switching or multilingual speech, document script and transliteration choices and how language labels are established. Do not silently translate a reference when evaluating transcription. Where a term or word is uncertain, use a defined review procedure rather than guessing from context. References are part of the measurement system: an incorrect transcript can penalize correct recognition or reward a model that reproduces an annotation mistake. Review a diagnostic subset before treating the metric as reliable. Group related recordings before making train and test splits [Download whisper-grouped-data-example.jsonl](/downloads/whisper-grouped-data-example.jsonl) is a four-row teaching fixture with invented paths and text, not included audio. Two records share a hypothetical speaker, session and source group and remain in training together. Separate illustrative groups form validation and test examples. All audio review states remain not performed and all permission states remain unestablished. The fixture is designed to make grouping explicit, not to claim these four rows are an adequate training collection. In a real archive, reconcile overlapping group relationships before assigning splits. A guest may appear in several sessions, and a session can contain several speakers. If the evaluation requires unseen speakers, connected relationships may need to be held together across episodes. Preserve clips, alternate edits and channel derivatives of the same recording in compatible groups. A random utterance split can otherwise reward memorization of familiar voices or source conditions while looking like independent evaluation. Split role What it is used for What to avoid Training Updating model parameters Including held-out source derivatives Validation Selecting settings and checkpoints Repeatedly treating it as untouched final evidence Test Final comparison under a frozen policy Tuning decisions based on its answers Diagnostic slices Investigating specific failures Confusing a selected slice with population accuracy Pin the processor and verify actual signal preprocessing The [Transformers Whisper documentation](https://huggingface.co/docs/transformers/model_doc/whisper) describes a processor combining audio feature extraction and tokenization. Its documented feature extractor expects mono input and uses a 16 kHz sampling-rate setting. Resample the signal correctly; changing a metadata field is not equivalent to resampling. Preserve the original source when needed and record the conversion so a later investigation can reproduce the prepared input. Check duration and segmentation against the actual implementation and chosen task. The older Hugging Face tutorial illustrates a short-segment preparation path; do not copy its truncation behavior blindly into long-form training or evaluation. A processor can discard audio beyond its configured limit while an untrimmed transcript still describes the discarded speech. Test a few prepared records end to end, including feature shapes, tokenized labels and the source interval, before scheduling a large run. Build a small reproducible training experiment Pin the checkpoint revision, processor files, software versions and relevant generation settings. Load the selected implementation’s model and processor consistently. Convert audio to input features and reference text to label tokens, then use a collator appropriate to the sequence-to-sequence task. Ensure padded label positions are handled according to the training implementation rather than treated as ordinary target words. Validate these choices on a small inspected batch. Choose batch size, accumulation, precision and training schedule based on the actual hardware and measured memory behavior. Avoid presenting one tutorial’s settings as a guarantee for another checkpoint or dataset. Save configuration and intermediate checkpoints under a reproducible run identifier. Track training loss, validation behavior and failure slices, while retaining a clear rule for selecting a candidate. This page has not executed a training run or produced a checkpoint, so it reports no convergence time or accuracy gain. Score paired predictions under one frozen policy The [NIST SCTK repository](https://github.com/usnistgov/SCTK) provides established speech-recognition scoring tools including sclite. The repository was marked archived/read-only when checked for this guide, so it is cited as a primary scoring reference rather than a promise of active maintenance. Record the chosen scoring implementation and version, reference normalization, hypothesis normalization and alignment settings. Word error rate summarizes substitutions, deletions and insertions relative to reference words; language-specific tokenization can materially affect interpretation. Compare baseline and candidate on the same held-out audio under matching decoding conditions. Inspect entity or domain-term errors where those matter, and include silence or non-speech diagnostics if unsupported insertions are a concern. Separate recognition quality from punctuation formatting, diarization and timestamp quality unless the task explicitly combines them. A lower aggregate metric does not establish better performance for every speaker, channel or language slice, and a small test does not support a broad production claim. Read the unrun evaluation plan and define acceptance [Download whisper-evaluation-plan-example.json](/downloads/whisper-evaluation-plan-example.json) records an explicitly unrun experiment with unset checkpoints and null metrics. It names the fields a project should freeze: model revision, software, grouping, reference policy, decoder configuration, slices and acceptance. It contains no invented baseline, customer outcome or measured training result. Use it to make the experiment reviewable before replacing the teaching values with a real approved source and run configuration. Define acceptance around the intended use, including meaningful target-domain improvement, tolerable regressions and operational constraints. Review paired errors rather than relying only on one percentage. If the result does not support the hypothesis, retain that outcome and examine source fit, transcript policy or preprocessing before simply increasing scale. Publish a model or public result only after the actual evaluation and source review justify the claim; this guide supplies a method rather than a shortcut to that evidence. Budget source preparation, compute and deployment separately Adaptation work can include source access or capture, transcript review, segmentation, resampling, grouping, pilot training, scoring, error analysis and deployment. Quote those activities with explicit units: accepted source hours, reviewed transcript segments, compute time or a defined experiment package. Raw recording duration is not the same as accepted supervised duration, especially when speech is overlapping, uncertain or poorly matched to the task. No representative commercial average or hardware runtime is asserted here. A Fiund inquiry can identify the domain, language, channels, intended use, source constraints and required evaluation evidence. Candidate data, preparation responsibilities and commercial scope are confirmed for the project. The useful next step is a small paired-source package and a frozen evaluation plan, allowing a team to decide whether local Whisper adaptation is justified before committing to a larger collection or deployment. Can I fine-tune whisper-1 through the hosted transcription API? This guide does not claim that capability. It describes a separate local checkpoint-training workflow. Check current official hosted product documentation and account availability before designing an API integration. Does the hosted Whisper deprecation prohibit local adaptation? The hosted notice describes an API product lifecycle. It does not by itself establish the feasibility or permitted use of a local checkpoint workflow; inspect the selected implementation, model terms and source data separately. Do the downloads contain audio or measured model results? No. They are an illustrative grouped manifest and an unrun evaluation plan. The named audio files do not accompany the fixture, no model was trained and no improvement is reported. Research method: Fiund research desk; public sources, explicit worked-example labels and project-specific availability review. Editorial contact: hello@fiund.com. Sources Whisper hosted API model documentation: https://developers.openai.com/api/docs/models/whisper-1 OpenAI API deprecations and self-serve fine-tuning timeline: https://developers.openai.com/api/docs/deprecations Transformers Whisper model documentation: https://huggingface.co/docs/transformers/model_doc/whisper Fine-Tune Whisper for Multilingual ASR with Transformers: https://huggingface.co/blog/fine-tune-whisper SCTK: NIST Scoring Toolkit primary repository: https://github.com/usnistgov/SCTK