huzy0 commited on
Commit
572cdbd
·
verified ·
1 Parent(s): 2fcf5a4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -5
README.md CHANGED
@@ -27,13 +27,13 @@ language:
27
 
28
 
29
 
30
- We introduce **MERaLiON-SpeechEncoder-2**, an update of [MERaLiON-SpeechEncoder-v1](https://huggingface.co/MERaLiON/MERaLiON-SpeechEncoder-v1) that greatly expands our pre-training data to **1.4 million hours** of unlabeled audio, with a **strong focus on Southeast Asian (SEA) languages and accents**. As a speech foundation model, it encodes speech into a general-purpose, multilingual acoustic representation that can serve as a high-performance backbone for a wide range of downstream tasks — including automatic speech recognition (ASR), speech translation, speaker and language identification, and emotion recognition.
31
 
32
  Unlike many existing models optimized for high-resource, Western languages, MERaLiON-SpeechEncoder-2 is designed from the ground up to reflect the linguistic diversity and complexity of Southeast Asia. See below for a full breakdown of the language coverage of our pre-training data. **This model can be finetuned on custom datasets, allowing developers to build speech systems tailored to their specific needs.**
33
 
34
  <p align="center">
35
- <img src="data1.svg" width="640"/>
36
- <img src="data2.svg" width="640"/>
37
  </p>
38
 
39
  ## Model Highlights
@@ -42,10 +42,10 @@ Unlike many existing models optimized for high-resource, Western languages, MERa
42
  With only 630M parameters (≈2.5 GB in memory), the model is easily deployable on most commercial GPUs, eliminating the need for distributed or large-scale compute setups.
43
 
44
  ### Natively multilingual
45
- Building on our [v1 release](https://huggingface.co/MERaLiON/MERaLiON-SpeechEncoder-v1) (which focused on English and Singlish), this version expands to include English, Chinese, Malay, Tamil, Thai, Indonesian, and Vietnamese, along with codeswitching support across these languages. Given the wide coverage of languages in the training corpus, it may also be applicable beyond the officially supported languages.
46
 
47
  ### Competitive performance on downstream speech tasks
48
- The model retains near state-of-the-art results on the SUPERB benchmark for English, and showcases strong multilingual capabilities deomnstrated through its integration into a [high-performance ASR system shown below](#Automatic Speech Recognition (ASR)).
49
 
50
  ### Innovative pre-training techniques
51
  MERaLiON-SpeechEncoder-2 was trained from scratch with an novel extension of the BEST-RQ self-supervised objective, by using more informative latent targets. We also adopted the Muon optimizer, which has previously only been shown to outperform the popular AdamW for LLM training. We find its advantages also carry over to speech-based models.
@@ -69,8 +69,18 @@ For details on background, pre-training, tuning experiments and evaluation, plea
69
  | MERaLiON-SpeechEncoder-V1 | 82.62 | 3.14 | 4.16 | 97.63 | 0.0590 | 91.09 | 5.18 | 5.06 | 68.02 | 98.60 | 88.99 / 23.89 |
70
  | MERaLiON-SpeechEncoder-2 | 82.72 | 3.40 | 4.96 | 97.57 | 0.0575 | 88.96 | 3.93 | 3.90 | 68.80 | 98.95 | 89.50 / 23.46 |
71
 
 
 
 
 
 
72
  ### Automatic Speech Recognition (ASR)
73
 
 
 
 
 
 
74
 
75
 
76
 
 
27
 
28
 
29
 
30
+ We introduce **MERaLiON-SpeechEncoder-2**, our next-generation multilingual speech encoder that was pre-trained from scratch on a greatly expanded corpus of **1.4 million hours** of unlabeled audio, with a **strong focus on Southeast Asian (SEA) languages and accents**. As a speech foundation model, it encodes speech into a general-purpose, multilingual acoustic representation that can serve as a high-performance backbone for a wide range of downstream tasks — including automatic speech recognition (ASR), speech translation, speaker and language identification, and emotion recognition.
31
 
32
  Unlike many existing models optimized for high-resource, Western languages, MERaLiON-SpeechEncoder-2 is designed from the ground up to reflect the linguistic diversity and complexity of Southeast Asia. See below for a full breakdown of the language coverage of our pre-training data. **This model can be finetuned on custom datasets, allowing developers to build speech systems tailored to their specific needs.**
33
 
34
  <p align="center">
35
+ <img src="data1.svg" width="620"/>
36
+ <img src="data2.svg" width="620"/>
37
  </p>
38
 
39
  ## Model Highlights
 
42
  With only 630M parameters (≈2.5 GB in memory), the model is easily deployable on most commercial GPUs, eliminating the need for distributed or large-scale compute setups.
43
 
44
  ### Natively multilingual
45
+ Building on [MERaLiON-SpeechEncoder-v1](https://huggingface.co/MERaLiON/MERaLiON-SpeechEncoder-v1) (which focused on English and Singlish), this version expands to include English, Chinese, Malay, Tamil, Thai, Indonesian, and Vietnamese, along with codeswitching support across these languages. Given the wide coverage of languages in the training corpus, it may also be applicable beyond the officially supported languages.
46
 
47
  ### Competitive performance on downstream speech tasks
48
+ The model retains near state-of-the-art results on the SUPERB benchmark for English, and showcases strong multilingual capabilities deomnstrated through its integration into a [high-performance ASR system shown below](#Automatic-Speech-Recognition-(ASR)).
49
 
50
  ### Innovative pre-training techniques
51
  MERaLiON-SpeechEncoder-2 was trained from scratch with an novel extension of the BEST-RQ self-supervised objective, by using more informative latent targets. We also adopted the Muon optimizer, which has previously only been shown to outperform the popular AdamW for LLM training. We find its advantages also carry over to speech-based models.
 
69
  | MERaLiON-SpeechEncoder-V1 | 82.62 | 3.14 | 4.16 | 97.63 | 0.0590 | 91.09 | 5.18 | 5.06 | 68.02 | 98.60 | 88.99 / 23.89 |
70
  | MERaLiON-SpeechEncoder-2 | 82.72 | 3.40 | 4.96 | 97.57 | 0.0575 | 88.96 | 3.93 | 3.90 | 68.80 | 98.95 | 89.50 / 23.46 |
71
 
72
+ SUPERB is an English-based benchmark for speech encoders covering a wide range of downstream speech tasks across domains such as recognition, detection, semantics, speaker, and paralinguistics, where each task is finetuned separately with a frozen encoder.
73
+
74
+ MERaLiON-SpeechEncoder-2 is competitive to state-of-the-art, improving slightly against our own v1 model on speaker and paralinguistic tasks.
75
+
76
+
77
  ### Automatic Speech Recognition (ASR)
78
 
79
+ <p align="center">
80
+ <img src="overall_wer.svg" width="620"/>
81
+ <img src="audiobench_wer.svg" width="620"/>
82
+ <img src="fleurs_wer.svg" width="620"/>
83
+ </p>
84
 
85
 
86