Lessons learned from AI-supported coding of qualitative data

This session explores how AI-supported approaches can be used to systematically code, classify, and analyze large volumes of qualitative data in applied research and evaluation settings. Drawing on recent large-scale evaluation experiences in Africa, the presentation will compare two practical approaches: (1) human-in-the-loop coding using secure conversational AI tools, and (2) programmatic large language model (LLM) pipelines designed for high-volume processing.
Participants will gain insight into how these approaches perform across different data types (e.g., in-depth interviews and focus groups) and scales (tens vs. hundreds of transcripts). The session will focus on methodological design choices, including prompt development, coding framework standardization, batch processing strategies, and quality assurance protocols such as human validation and inter-rater reliability testing.
The discussion will highlight when AI-assisted qualitative coding is most effective—particularly in contexts with structured interview guides and clearly defined thematic frameworks—and where human expertise remains essential, such as interpreting nuance and synthesizing findings. Lessons learned will emphasize practical safeguards for responsible use, including iterative testing, controlled prompt use, and continuous validation workflows.
Overall, the session aims to provide a grounded, experience-based perspective on integrating AI into qualitative research workflows, helping evaluators assess trade-offs between scalability, consistency, and analytical depth.