NVIDIA and IIT Madras announce Indic-language pretraining data release
NVIDIA said it plans to release a 2.8-trillion-token corpus covering 22 Indian languages and an open data-generation pipeline developed with IIT Madras' AI4Bharat lab.
NVIDIA announced Nemotron-pretraining-Indic-v1, a planned release of approximately 2.8 trillion training tokens across 22 Indian languages, alongside an open GPU-accelerated data-generation pipeline.
The company said it developed the project with AI4Bharat, an Indian-language research lab at IIT Madras, and that the corpus and pipeline will be released soon.