ASR Whisper 51 African Languages¶

Overview¶

ASR Whisper 51 African Languages is an adaptation of Whisper Large v3 that supports Automatic Speech Recognition (ASR) for 51 languages widely spoken in Africa: English and French with African accents, Swahili, Afrikaans, Tswana, Kinyarwanda, Nigerian Pidgin, Luganda, Acholi, Lugbara, Ateso, Runyankole, Rutooro, Lumasaba, Lusoga, Rukiga, Akan, Amharic, Bambara, Bemba, Berber, Chichewa, Dagaare, Dagbani, Ewe, Fulani, Hausa, Igbo, Ikposo, Kabyle, Kalenjin, Kanuri, Kikuyu, Kwamba, Lendu, Lingala, Luhya, Luo, Malagasy, Ndebele, Oromo, Rukonjo, Ruruuli, Shona, Somali, Sotho, Thur, Wolof, Xhosa, Yoruba, and Zulu.

The model achieves lower Word Error Rate (WER) and Character Error Rate (CER) on most African languages when compared against proprietary and open-source alternatives like Gemini 3.5 Flash, GPT-4o Transcribe, and Meta's Omnilingual ASR (omniASR).

Task Type¶

Automatic Speech Recognition (ASR), Audio-to-Text.

Base Architecture¶

Whisper Large v3 (fine-tuned from openai/whisper-large-v3 and huwenjie333/whisper-v3-ft-af51).

Model Size¶

1.54B parameters.

Quantization Format¶

None. This is the full precision model.

Languages Supported¶

The model supports ASR for English, French, and the following 49 African languages:

Intended Use¶

This model is intended for Automatic Speech Recognition (ASR) applications across the 51 supported languages. Suitable use cases include:

• Transcribing audio to text in any of the 51 supported languages.
• Voice interface systems for local communities.
• Processing low-resource speech datasets for research and application development.

Known Limitations¶

• Speech recognition performance varies depending on the target language, speaker accent, and audio recording quality.
• Accuracy may degrade in noisy environments, with overlapping speakers, or when the spoken language deviates significantly from the training distribution.
• The model might exhibit hallucination or substitution errors, especially for highly low-resource languages.
• Verification and human-in-the-loop validation is recommended for high-stakes domains (medical, legal, financial).

How to Load / Run¶

You can try the model directly using this Colab Notebook, which supports transcribing audio samples from Hugging Face datasets, uploaded audio files, or your microphone.

For better accuracy, specify the language during generation. See the example below, which transcribes an audio sample from the Sunbird/salt Hugging Face dataset:

import transformers
import datasets
import torch

SAMPLE_RATE = 16000

LANGUAGE_TOKENS_WHISPER = {
    # Existing languages codes from Whisper
    "eng": 50259, "fra": 50265, "swa": 50318, "sna": 50324, "yor": 50325, "som": 50326,
    "afr": 50327, "amh": 50334, "mlg": 50349, "lin": 50353, "hau": 50354,
    # Overwrite unused language tokens
    "ach": 50357, "aka": 50356, "bam": 50355, "bem": 50352, "ber": 50351,
    "cgg": 50350, "dag": 50348, "dga": 50347, "ewe": 50346, "ful": 50345,
    "ibo": 50344, "kab": 50343, "kau": 50342, "kik": 50341, "kin": 50340,
    "kln": 50339, "koo": 50338, "kpo": 50337, "led": 50336, "lgg": 50335,
    "lth": 50333, "lug": 50332, "luo": 50331, "luy": 50330, "myx": 50329,
    "nbl": 50328, "nya": 50323, "nyn": 50322, "orm": 50321, "pcm": 50320,
    "ruc": 50319, "rwm": 50317, "sot": 50316, "teo": 50315, "tsn": 50314,
    "ttj": 50313, "wol": 50312, "xho": 50311, "xog": 50310, "zul": 50309
}

LANGUAGE_NAMES = {
    'Acholi': 'ach', 'Afrikaans': 'afr', 'Akan': 'aka', 'Amharic': 'amh', 'Ateso': 'teo',
    'Bambara': 'bam', 'Bemba': 'bem', 'Berber': 'ber', 'Chichewa': 'nya', 'Dagaare': 'dga',
    'Dagbani': 'dag', 'English': 'eng', 'Ewe': 'ewe', 'French': 'fra', 'Fulani': 'ful',
    'Hausa': 'hau', 'Igbo': 'ibo', 'Ikposo': 'kpo', 'Kabyle': 'kab', 'Kalenjin': 'kln',
    'Kanuri': 'kau', 'Kikuyu': 'kik', 'Kinyarwanda': 'kin', 'Kwamba': 'rwm', 'Lendu': 'led',
    'Lingala': 'lin', 'Luganda': 'lug', 'Lugbara': 'lgg', 'Luhya': 'luy', 'Lumasaba': 'myx',
    'Luo': 'luo', 'Lusoga': 'xog', 'Malagasy': 'mlg', 'Ndebele': 'nbl', 'Nigerian Pidgin': 'pcm',
    'Oromo': 'orm', 'Rukiga': 'cgg', 'Rukonjo': 'koo', 'Runyankole': 'nyn', 'Ruruuli': 'ruc',
    'Rutooro': 'ttj', 'Shona': 'sna', 'Somali': 'som', 'Sotho': 'sot', 'Swahili': 'swa',
    'Thur': 'lth', 'Tswana': 'tsn', 'Wolof': 'wol', 'Xhosa': 'xho', 'Yoruba': 'yor', 'Zulu': 'zul'
}

model_id = "Sunbird/asr-whisper-51-african-languages"
model = transformers.WhisperForConditionalGeneration.from_pretrained(model_id)
processor = transformers.WhisperProcessor.from_pretrained(model_id)

def transcribe_by_whisper(audio_array, language):
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    input_features = processor(
        audio_array, sampling_rate=SAMPLE_RATE, do_normalize=True, return_tensors="pt"
    ).input_features.to(device)

    lang_tok = LANGUAGE_TOKENS_WHISPER[LANGUAGE_NAMES[language]]
    transcribe_tok = processor.tokenizer.convert_tokens_to_ids("<|transcribe|>")
    notimestamps_tok = processor.tokenizer.convert_tokens_to_ids("<|notimestamps|>")
    forced_decoder_ids = [
        (1, lang_tok),
        (2, transcribe_tok),
        (3, notimestamps_tok),
    ]

    predicted_ids = model.to(device).generate(
        input_features,
        forced_decoder_ids=forced_decoder_ids,
        num_beams=1,
        do_sample=False,
    )
    transcription = processor.decode(
        predicted_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)

    print(transcription[0])

# Get some test audio
import huggingface_hub
huggingface_hub.login()

ds = datasets.load_dataset('Sunbird/salt', 'multispeaker-lug', split='test')
audio_column = 'audio'
ds = ds.cast_column(audio_column, datasets.Audio(sampling_rate=SAMPLE_RATE))
audio_array = ds[0][audio_column]['array']

# Specify a language
lang = 'Luganda'

transcribe_by_whisper(audio_array, lang)
# Ekikoola kya kasooli kya kyenvu wabula langi yakyo etera okuba eya kitaka wansi.

Performance Metrics¶

Our model is designed to excel on low-resource African languages such as Acholi and Ateso from Uganda, while achieving comparable performance on widely spoken African languages like Swahili, and African-accented English/French.

Below is a comparison of Word Error Rate (WER) and Character Error Rate (CER) measured for each language on the evaluation datasets. For ach, eng, lgg, lug, nyn, and teo, the dev splits of the multispeaker-* subsets in the Sunbird/salt dataset were used for evaluation. For the remaining languages, we used the first 50 examples from the dev split of each subset in the Training Datasets after filtering out clips longer than 30 seconds.

Visual Comparisons¶

Figure 1: Comparative WER and CER performance against proprietary and open-source models.

Figure 2: Word Error Rate (WER) vs. training hours.

Figure 3: Character Error Rate (CER) vs. training hours.

Training Datasets¶

The model was fine-tuned on a multilingual corpus covering 51 African languages, assembled from a range of publicly available and community-collected speech datasets. The table below lists each source dataset and the languages it contributes to training (ISO 639-3 code in parentheses). A language may appear in more than one source.

[!NOTE] African Next Voices dataset is drawn from two collection hubs: Kenya (Kikuyu, Kalenjin, Luo, Somali) and Southern Africa (Ndebele, Sotho, Tswana, Xhosa, Zulu).

Licence¶

Released under the Apache 2.0 license (matching the base architecture and training set details).

Hugging Face¶

https://huggingface.co/Sunbird/asr-whisper-51-african-languages

Last Updated¶

2026

Citation¶

@misc{sunbird_asr_whisper_51_african_languages,
  title        = {ASR Whisper 51 African Languages},
  author       = {Sunbird AI},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/Sunbird/asr-whisper-51-african-languages}},
}