Tech Meridian ← ENTITY INDEX
RU

COMPANY · ENTITY #175

Mixture-of-Experts (MoE)

Related event timeline, sources and context from the news index.

EVENT TIMELINE

3

MODELS · 1 SOURCE · Cohere

Cohere releases open-source "Transcribe Arabic" ASR model claiming best open-weight Arabic accuracy

Cohere published Transcribe Arabic, an open-source Arabic automatic speech recognition (ASR) model (Apache 2.0) based on its 2B ASR work. Cohere says the model achieves a 25.87 WER on the Hugging Face Arabic ASR leaderboard, outperforming Meta’s OmniASR-LLM-7B and OpenAI’s Whisper Large V3, and was preferred to Whisper in about 96% of human evaluation tests; weights are available on Hugging Face and via the Cohere API/Model Vault.

8.0

RESEARCH · 1 SOURCE · Cohere

Cohere introduces hardware-aware Dynamic Speculative Decoding (DSD) to adapt draft-token count to GPU constraints

Cohere presents Dynamic Speculative Decoding (DSD), an extension of speculative decoding that selects the number of draft tokens K based on hardware regime (memory-bandwidth vs compute-bound) and model type (dense vs MoE). DSD uses offline profiling of Acceptance Length (AL) and Inter-Token Latency (ITL) to choose an optimal K per condition, aiming to preserve SD speedups in production workloads and large-scale RL rollouts.

7.0

MODELS · 1 SOURCE · NVIDIA Developer

Dense vs. MoE Models: Active Parameters, Throughput, and When to Choose Each

An NVIDIA Developer post compares dense and Mixture-of-Experts (MoE) architectures, showing how a 30B-parameter model can activate only about 3B parameters per token and discussing the resulting capacity and throughput trade-offs; Nemotron 3.5 Lightning is used as an illustrative example.

6.0