NVIDIA has announced Nemotron-pretraining-Indic-v1, a synthetic pretraining corpus of roughly 2.8 trillion tokens across the 22 languages listed in India's Constitution. The post, published on Hugging Face on October 16, 2026, says the corpus is releasing soon. It was built with IIT Madras and its AI4Bharat lab, and the open pipeline behind it is meant to let others generate more data.
What is in the corpus
The corpus holds 2,601.0 billion native-script tokens and 224.6 billion romanized tokens. All 22 languages have native-script data; twelve also have romanized text, written in Latin script, which gives 34 language-script configurations. The source material comes from 15 Nemotron-CC-v2 subsets, including web text, question-answer content, mathematics and code. Every record keeps an identifier pointing back to its source document.
How it was made
For twelve languages, documents are translated in full with Gemma-4-31B-it, scored against an English source using a rubric inspired by Multidimensional Quality Metrics, and kept only at 75 out of 100 or higher. Lower-scoring outputs are repaired using the assessment feedback and rescored. That recovered 438.4 billion tokens already counted in the total. Romanized text is produced from accepted native-script translations, not directly from English. In an initial human comparison, NVIDIA says this gave fewer hallucinated additions and less residual English.
The other ten languages use Indic Translate, a document-level translation model from Bodhan AI, with the same 75-point threshold. They get native-script data only and no feedback-based rewriting.
Model results
Nemotron 3.5 Super VL received 10 billion Indic tokens in a continued-pretraining stage with a 100 billion token budget, drawn from a twelve-language subset of this corpus plus natural Indic text. Scores rose on all five reported benchmarks, by 4.98 to 9.82 percentage points. NVIDIA states that this reflects the combined synthetic and natural data, not the synthetic corpus alone. Against Nemotron 3 Super, the released model scores higher on five of six task benchmarks, while instruction following on IFEval 1.5 fell 2.04 points.
Open recipe and limits
The Nemotron Bharat translation plugin for NeMo Data Designer is described as open-source. It covers translation and evaluation prompts, optional rewriting, document windowing and script remapping. NVIDIA advises starting with a small sample of your own documents and refining prompts with native-language reviewers before scaling.
The post lists its own limits. Translated text does not replace text written natively in Indian languages or guarantee local cultural knowledge. Quality assessment is mostly automated, human review coverage varies, the ten-language workflow needs broader native-speaker validation, and source-ID checks remove processing duplicates but are not semantic deduplication.