A Gender-intentional Framework for Evaluating AI-based solutions in India's Development Sector
Mesa redonda | En línea
-
Organizado por:
Gender and Digital Hub (GxD hub), LEAD at Krea University/IFMR
Sobre el evento
While this year's Glocal theme focuses on AI in evaluation, this session offers a complementary lens: the evaluation of AI itself, from an equity and gender perspective.
Currently, evaluation of AI-powered solutions are largely trapped in a narrow technical frame - measuring accuracy and engagement metrics while missing gendered risks and blindspots in design, deployment, sustained adoption, and developmental impact. When evaluations fall short, flawed tools get scaled.
This session introduces a gender-intentional conceptual evaluation framework, followed by a practitioner panel to respond to the framework and reflect on current AI evaluation practices and experiences. The session will walk evaluators and policymakers through what current approaches are missing, what is at stake, and what addressing it would require.
Drawing on field experiences from organisations at the intersection of AI, gender, and development in India, the session offers concrete learnings for anyone commissioning or conducting AI evaluations.
Participants will leave with: a gender-intentional checklist of evaluative questions & methods covering model evaluations, product evaluation, user testing, and outcome evaluation; practitioner accounts across health and agriculture domains; and a shared vocabulary for advocating for gender-intentional evaluation standards within their organisations.
Currently, evaluation of AI-powered solutions are largely trapped in a narrow technical frame - measuring accuracy and engagement metrics while missing gendered risks and blindspots in design, deployment, sustained adoption, and developmental impact. When evaluations fall short, flawed tools get scaled.
This session introduces a gender-intentional conceptual evaluation framework, followed by a practitioner panel to respond to the framework and reflect on current AI evaluation practices and experiences. The session will walk evaluators and policymakers through what current approaches are missing, what is at stake, and what addressing it would require.
Drawing on field experiences from organisations at the intersection of AI, gender, and development in India, the session offers concrete learnings for anyone commissioning or conducting AI evaluations.
Participants will leave with: a gender-intentional checklist of evaluative questions & methods covering model evaluations, product evaluation, user testing, and outcome evaluation; practitioner accounts across health and agriculture domains; and a shared vocabulary for advocating for gender-intentional evaluation standards within their organisations.
Presentador/a
| Nombre | Título | Biografía |
|---|---|---|
| Mahima Taneja, Deepti, Vaidehi Sahasrabhojanee | GxD hub, LEAD at Krea | The GxD team has developed a conceptual framework on Gender-Intentional AI evaluations. The authors will jointly present this framework to anchor the panel discussion, followed by remarks from the panelists on their experiences of evaluating AI solutions, and whether the framework speaks to the equity and gender concerns in the field. |
| Urvashi Wattal | Country Lead - Evidence and Impact, Khushi Baby | Urvashi Wattal is the Country Lead - Evidence and Impact at Khushi Baby. Khushi Baby's MLE work grapples directly with challenges the framework addresses: auditing AI tools when users are predominantly low-literacy women ASHA workers, using shared devices, in low-connectivity settings. The speaker will be invited to share how they evaluate AI tools, what gender and equity gaps they encountered, and where the framework would have, or does help. |
| Kalika Bali | Microsoft Research India | Kalika Bali is a Senior Principal Researcher at Microsoft Research India, working on NLP, multilingual AI, and gender-intentional datasets for Indian languages. Named to TIME100 AI 2023, she advocates for culturally grounded, gender-aware AI. Dr. Bali is invited to ground the session’s technical dimensions: what does representationally adequate training data look like for India’s linguistic and demographic diversity, and how should evaluators without specialist NLP expertise assess it? |
| Nilakshi Biswas | NuSocia Translational Research Centre (NTRC) | Nilakshi Biswas heads NuSocia's Translational Research Centre, with expertise in evidence synthesis, theory of change, and public health policy. She previously contributed to impact evaluation research at 3ie and holds an MPH from George Washington University. Dr. Biswas is invited to bring the practitioner-to-policy bridge perspective: how do evaluation findings on AI equity gaps get translated into institutional change, and what advocacy levers exist for MLE leads and evaluation commissioners in India's development sector? |
| Aarushi Gupta | Digital Futures Lab | Aarushi Gupta is Senior Research Manager at Digital Futures Lab, leading research on gender bias in Indian-language LLMs across healthcare and agriculture. She has presented at ACM FAccT and advised AI developers in South Asia and Africa on responsible AI practices. She will invited to share her reflections on science of AI evaluation. |
Moderador/a
| Nombre | Título | Biografía |
|---|---|---|
| Dr. Mahima Taneja | Associate Director - Research & MLE, GxD hub, LEAD | Dr. Mahima Taneja leads research and monitoring, learning & evaluation for the Gender x Digital (GxD) hub at LEAD. She brings over a decade of experience in research and evaluation, with work spanning gender equality, women's livelihoods, digital inclusion, urban policy, and WASH. She will be presenting a framework on Gender Intentional AI evaluations, co-developed with the GxD hub team, and moderating the panel discussion. |