NVIDIA and AI4Bharat to Release 2.8-Trillion-Token Indic Pretraining Corpus and Open Pipeline

The corpus was developed by NVIDIA with IIT Madras, described in the announcement as India's premier engineering institute, and its AI4Bharat lab. AI4Bharat builds open-source datasets and models for Indian languages, including Sangraha, an open pretraining corpus for 22 Indian languages. The new release is described as a substantial expansion of open synthetic Indic training data available to the community.

The pipeline captures the full generation recipe, including prompts, LLM workflows, and quality checks, so developers can adapt it to their own source material and generate more data. The corpus includes a broad mix of curated English material, including mathematics, code, and question-answer content, brought into Indian languages. Every record traces back to a curated source document, and each output is checked against its input, so the synthetic text stays grounded in real information.

Mitesh Khapra, who heads AI4Bharat and made the 2025 TIME100 AI list, said Indian language technology trails English because 'we do not have enough data for Indian languages.' The release answers that with more data and an open recipe for generating more.

The corpus includes native-script data for all 22 languages and romanized text for 12, spanning 34 language-script configurations. For the 12 dual-script languages—Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Odia, Punjabi, Tamil, Telugu, and Urdu—the corpus has native-script and romanized data. For the other 10—Assamese, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri, Sanskrit, Santali, and Sindhi—it has native-script data. Native-script describes the writing system, and the text is still translated from English. Romanized means Indic-language text written in Latin script, not an English translation. For each of the 12 dual-script languages, romanization covers a subset of the accepted native-script translations.

The corpus contains 2,601.0 billion native-script tokens and 224.6 billion romanized tokens, using the Gemma-4 tokenizer. Document records include translations and script variants of shared sources; they are not counts of unique English documents. The token counts describe the retained corpus, including accepted rewrites, rather than a training run's token budget. Counts also depend on the tokenizer used, so corpus inventory and training-token counts should be read separately.

To build the corpus, the team started with documents from 15 Nemotron-CC-v2 subsets covering web text, question-answer material, mathematics, code, and other specialized content. They used NeMo Data Designer, NVIDIA's open-source framework for generating synthetic data, to run translation, quality checks, and targeted repair. The workflows combine document-level translation, quality assessment, and targeted repair, with checks to preserve content such as equations, tables, and code. Source identifiers are retained so developers can trace an output back to its original document. The data complements naturally authored Indic text, which remains essential for local knowledge and cultural context.

The team is sharing the open generation recipe so the community can reproduce the data and take the work further. Teams can use the corpus directly, adapt the prompts and quality checks, or bring their own source documents to the workflow. Continued pretraining for Nemotron 3.5 Super VL added 10 billion Indic tokens, drawn from a twelve-language subset of this corpus and natural Indic text. A technical report on the model covers the full story, and the team plans to build on this corpus in future Nemotron models.

Editor's Summary NVIDIA and IIT Madras' AI4Bharat plan to release Nemotron-pretraining-Indic-v1, a roughly 2.8-trillion-token corpus covering 22 Indian languages, along with an open pipeline for generating more data. The release is intended to expand pretraining resources for Indic language models and includes native-script and romanized data plus traceability and quality checks. Continued pretraining for Nemotron 3.5 Super VL already used 10 billion Indic tokens from a twelve-language subset.