Publications
Legend: 🏠 = Conference, 📔 = Journal, 👓 = Workshop, * = equal contribution, † = equal mentorshipPreprints
Towards Inducing Document-Level Abilities in Standard Multilingual Neural Machine Translation Models
Varun Gumma*, Pranjal Chitale*, Kalika Bali
Preprint (2024)
HEALTH-PARIKSHA: Assessing RAG Models for Health Chatbots in Real-World Multilingual Settings
Varun Gumma, Anandhita Raghunath, Mohit Jain†, Sunayana Sitaram†
Preprint (2024)
Published Papers
Contamination Report for Multilingual Benchmarks
Sanchit Ahuja*, Varun Gumma*, Sunyana Sitaram
👓 EvalEval @ NEURIPS (2024)
PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data
Ishaan Watts, Varun Gumma, Aditya Yadavalli, Vivek Seshadri, Manohar Swaminathan, Sunayana Sitaram
🏠 EMNLP (2024)
Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technology
Rishav Hada, Safiya Husain, Varun Gumma, Harshita Diddee, Aditya Yadavalli, Agrima Seth, Nidhi Kulkarni, Ujwal Gadiraju, Aditya Vashistha, Vivek Seshadri, Kalika Bali
🏠 ACM FAccT (2024)
METAL: Towards Multilingual Meta-Evaluation
Rishav Hada*, Varun Gumma*, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram
🏠 ACL Findings (2024)
MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks
Sanchit Ahuja, Divyanshu Aggarwal, Varun Gumma, Ishaan Watts, Ashutosh Sathe, Millicent Ochieng, Rishav Hada, Prachi Jain, Mohamed Ahmed, Kalika Bali, SunayanaSitaram
🏠 NAACL (2024)
Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?
Rishav Hada, Varun Gumma, Adrian Wynter, Harshita Diddee, Mohamed Ahmed, Monojit Choudhury, Kalika Bali, Sunayana Sitaram
🏠 ACL Findings (2024)
MAFIA: Multi-Adapter Fused Inclusive Language Models
Prachi Jain*, Ashutosh Sathe*, Varun Gumma, Kabir Ahuja, Sunayana Sitaram
🏠 EACL (2024)
MunTTS: A Text-to-Speech System for Mundari
Varun Gumma, Rishav Hada, Aditya Yadavalli, Pamir Gogoi, Ishani Mondal, Vivek Seshadri, Kalika Bali
👓 ComputEL @ EACL (2024)
IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages
Jay Gala*, Pranjal A Chitale*, A K Raghavan, Varun Gumma, Sumanth Doddapaneni, Aswanth Kumar M, Janki Atul Nawale, Anupama Sujatha, Ratish Puduppully, Vivek Raghavan, Pratyush Kumar, Mitesh M Khapra, Raj Dabre, Anoop Kunchukuttan
📔 TMLR (2023)
An Empirical Study of Leveraging Knowledge Distillation for Compressing Multilingual Neural Machine Translation Models
Varun Gumma, Raj Dabre, Pratyush Kumar
🏠 EAMT (2023)
PAMMELA: Policy Administration Methodology using Machine Learning
Varun Gumma, Barsha Mitra, Soumyadeep Dey, Pratik Shashikantbhai Patel*, Sourabh Suman*, Saptarshi Das, Jaideep Vaidya
🏠 SECRYPT (2022)
