NVIDIA and AI4Bharat release Nemotron-pretraining-Indic-v1: 2.8T-token Indic corpus and open GPU data pipeline
NVIDIA, IIT Madras (AI4Bharat) and collaborators are releasing Nemotron-pretraining-Indic-v1, a synthetic pretraining corpus of about 2.8 trillion tokens spanning 22 Indian languages, together with an open, GPU-accelerated data-generation pipeline (including prompts, LLM workflows, and quality checks). The corpus includes native-script and romanized variants, preserves technical content (math, code, tables), traces outputs back to curated source documents, and was partially used to add 10 billion Indic tokens to continued pretraining of Nemotron 3.5 Super VL; the data and pipeline are being shared so others can reproduce and extend the workflow.