Offline AI Modules (arXiv:2610.07026v1): voice‑first offline stack, hardware reference, and quantization benchmark
The Offline AI Modules arXiv paper presents a modular voice‑first offline architecture, a low‑cost hardware reference bill of materials, and a reproducible quantization and benchmarking pipeline for instruction‑tuned 2–5B parameter models aimed at African language communities. The authors report the first end‑to‑end benchmark across two hardware tiers (NVIDIA Jetson Orin NX — TierB, Raspberry Pi5 — TierA), evaluating three instruction‑tuned models across four quantization formats; they find Q4_K_M offers the best size‑to‑quality tradeoff (e.g., gemma‑4‑E2B‑it: 28.8 t/s decode throughput and 89.2% topic classification accuracy on TierB) and show all three models fit a 16GB memory budget on TierA, with ASR measured via Ethio‑ASR for Amharic and Oromo and multilingual quality assessed on MasakhaNEWS.