MODELS · 1 SOURCE · Google Research
Google researchers report that all next-word-prediction neural LMs in Gboard are now trained with federated learning and formal differential privacy guarantees, with over 30 on-device models across 7+ languages and 15+ countries. The deployed models satisfy (ε,δ)-DP with δ=1e-10 and ε ranging from 0.994 to 13.69; the post describes practical techniques used (DP-FTRL, secure aggregation, adaptive clipping, pretraining on C4) and states this is, to their knowledge, the largest production deployment of user-level DP and the first announced ε<1 for models trained directly on user data.