A Gender-intentional Framework for Evaluating AI-based solutions in India's Development Sector
Painel | Online
About the Event
While this year's Glocal theme focuses on AI in evaluation, this session offers a complementary lens: the evaluation of AI itself, from an equity and gender perspective. AI evaluation is largely trapped in a narrow technical frame -- measuring accuracy and engagement metrics while missing gendered risks and blindspots in design, deployment, sustained adoption, and developmental impact of AI solutions. When evaluations fall short, flawed tools get scaled. This session introduces a gender-intentional conceptual evaluation framework: grounded in theory and practical to apply. Through an expert conceptual presentation followed a practitioner panel, it walks evaluators and policymakers through what current approaches are missing, what is at stake, and what addressing it would require.
Drawing on field experiences from organisations at the intersection of AI, gender, and development in India, the session offers concrete learnings for anyone commissioning or conducting AI evaluations. Participants leave with: a gender-intentional checklist of evaluative questions & methods covering model evaluations, product evaluation, user testing, and outcome evaluation; practitioner accounts across health and agriculture domains; and a shared vocabulary for advocating for gender-intentional evaluation standards within their organisations.
Drawing on field experiences from organisations at the intersection of AI, gender, and development in India, the session offers concrete learnings for anyone commissioning or conducting AI evaluations. Participants leave with: a gender-intentional checklist of evaluative questions & methods covering model evaluations, product evaluation, user testing, and outcome evaluation; practitioner accounts across health and agriculture domains; and a shared vocabulary for advocating for gender-intentional evaluation standards within their organisations.
Speakers
| Nome | Título | Biography |
|---|---|---|
| Mahima Taneja | Associate Director - Research & MLE, GxD hub, LEAD at Krea | Mahima Taneja leads research and monitoring, learning & evaluation for the Gender x Digital (GxD) hub at LEAD. She brings over a decade of experience in research and evaluation, with work spanning gender equality, urban policy, WASH, and adolescent sexual and reproductive health. She will be presenting a framework on Gender Intentional AI evaluations, co-developed with the GxD hub team, and moderating the panel discussion. |
| Ruchit Nagar/Urvashi Wattal (TBC) | Co-Founder/MLE Lead, Khushi Baby | Ruchit Nagar is the CEO & Co-Founder of Khushi Baby, a digital health non-profit whose AI/ML-powered platform supports 70,000+ community health workers across Rajasthan, Maharashtra, and Karnataka to deliver maternal, child, and preventive health services. Khushi Baby’s MLE work grapples directly with challenges the framework addresses: auditing AI tools when users are predominantly low-literacy women ASHA workers, using shared devices, in low-connectivity settings. The speaker will be invited to share how they evaluate AI tools, what gender and equity gaps they encountered, and where the framework would have, or does help. |
| Kalika Bali (TBC) | Microsoft Research India | Kalika Bali is a Senior Principal Researcher at Microsoft Research India, working on NLP, multilingual AI, and gender-intentional datasets for Indian languages. Named to TIME100 AI 2023, she advocates for culturally grounded, gender-aware AI. Dr. Bali is invited to ground the session’s technical dimensions: what does representationally adequate training data look like for India’s linguistic and demographic diversity, and how should evaluators without specialist NLP expertise assess it? |
| Nilakshi Biswas (TBC) | NuSocia Translational Research Centre (NTRC) | Nilakshi Biswas heads NuSocia's Translational Research Centre, with expertise in evidence synthesis, theory of change, and public health policy. She previously contributed to impact evaluation research at 3ie and holds an MPH from George Washington University. Dr. Biswas is invited to bring the practitioner-to-policy bridge perspective: how do evaluation findings on AI equity gaps get translated into institutional change, and what advocacy levers exist for MLE leads and evaluation commissioners in India's development sector? |
| Aarushi Gupta (TBC) | Digital Futures Lab | Aarushi Gupta is Senior Research Manager at Digital Futures Lab, leading research on gender bias in Indian-language LLMs across healthcare and agriculture. She has presented at ACM FAccT and advised AI developers in South Asia and Africa on responsible AI practices. She will invited to share her reflections on science of AI evaluation. |
Moderators
| Nome | Título | Biography |
|---|---|---|
| Mahima Taneja | Associate Director - Research & MLE, GxD hub, LEAD | Mahima Taneja leads research and monitoring, learning & evaluation for the Gender x Digital (GxD) hub at LEAD. She brings over a decade of experience in research and evaluation, with work spanning gender equality, urban policy, WASH, and adolescent sexual and reproductive health. She will be presenting a framework on Gender Intentional AI evaluations, co-developed with the GxD hub team, and moderating the panel discussion. |