OpenAI releases IndQA benchmark for Indian languages
The benchmark's 2,278 questions, drafted with 261 India-based domain experts, span 12 languages and 10 cultural domains including law, religion and cuisine.
- Benchmarks & progress
- Colour
OpenAI released IndQA, a benchmark of 2,278 questions across 12 languages — including Hindi, Bengali, Tamil, Telugu, Punjabi and Hinglish, the code-switched mix common in everyday Indian speech — built with 261 domain experts across India to test cultural and reasoning knowledge rather than translation ability alone. Questions spanned ten domains including law, religion, food, history and media.
OpenAI said the benchmark was meant to address a gap in existing multilingual evaluations such as MMMLU and MGSM, which it argued tested translated general knowledge rather than reasoning that depended on cultural context specific to India. It reported that model performance on IndQA had improved over time but still left substantial room for error, comparing GPT-5 Thinking against other frontier systems without publishing full numerical scores in the announcement.
The release fit a wider pattern of frontier labs building region-specific evaluations as they compete for users in large non-English-speaking markets, alongside product and infrastructure investments OpenAI and its rivals were making in India through 2025.