Code-switching detection from speech using language-selective dimensions in Whisper-small
The authors identify language-selective hidden-state dimensions in pretrained speech encoders and aggregate their frame-level activations to detect code-switching (CS) directly from audio. Using a small tuning set, the Whisper-small encoder achieved a mean F1 of 0.798 across 14 language pairs on CS-FLEURS, outperforming a CNN (0.776) and a HuBERT linear baseline (0.751).