RESEARCH · RESEARCH · #1440
Language discrimination during HuBERT pretraining reduces the bilingual multilingual gap
Authors report that strengthening language discrimination in a controlled English–French HuBERT pretraining setup—via an auxiliary language classifier or per-language k-means targets—reduces the multilingual gap on phonetic, lexical, and prosodic measures while retaining cross-language sharing. Key results: phone-ABX fell from 11.6% to 10.4% (monolingual: 10.8%), sWUGGY rose from 52.1% to 56.7% (monolingual: 58.5%), and ProsAudit (lexical) rose from 68.9% to 72.9%; largest gains occur when discrimination is introduced in the first HuBERT iteration.
KEY POINTS
- Authors report that strengthening language discrimination in a controlled English–French HuBERT pretraining setup—via an auxiliary language classifier or per-language k-means targets—reduces the multilingual gap on phonetic, lexical, and prosodic measures while retaining cross-language sharing.
- Key results: phone-ABX fell from 11.6% to 10.4% (monolingual: 10.8%), sWUGGY rose from 52.1% to 56.7% (monolingual: 58.5%), and ProsAudit (lexical) rose from 68.9% to 72.9%; largest gains occur when discrimination is introduced in the first HuBERT iteration.
- This shows a practical intervention that narrows the performance gap between multilingual and monolingual self-supervised speech models, informing design choices for efficient multilingual pretraining.
WHY IT MATTERS
This shows a practical intervention that narrows the performance gap between multilingual and monolingual self-supervised speech models, informing design choices for efficient multilingual pretraining.