Scaling in Games: continued pre‑training study (arXiv:2610.28698v1)
This arXiv preprint reports a controlled study of continued pre‑training (CPT) for embodied agents across 120 training configurations varying model size (0.8B–27B), data volume (50–1,600 hours), and data diversity (Minecraft-only to multi‑world corpora). The authors fit an empirical scaling law and find that more data improves in‑corpus performance, diversity aids transfer to related worlds but gives limited gains in distant worlds, larger models improve data efficiency yet can generalize worse under single‑world training, and CPT alone likely cannot close gaps in distant virtual environments.