AI and the Future of Evaluation: Practice, Judgment, and Systems

As artificial intelligence becomes increasingly embedded in development and evaluation work, many conversations remain either highly conceptual or focused only on tools. This session shifts the discussion from general possibilities to actual evaluation practice by exploring how AI is changing the day-to-day work of evaluators across the full evaluation lifecycle. Framed around a practical case-based example, the session will walk participants through how an evaluation of education and skill development programmes may have been conducted earlier, and how similar work can be approached now using AI-supported methods. Rather than presenting AI as a replacement for evaluators, the session will demonstrate how AI can augment human work across different stages of an evaluation—while also highlighting the continuing importance of professional judgment, contextual understanding, and methodological rigor. The session will feature a moderated relay-style conversation with practitioners playing different roles in the evaluation workflow. Each speaker will reflect on how their work has changed in specific areas such as designing theories of change and outcome frameworks, developing tools, reviewing or cleaning data, analysing findings, and preparing reports and knowledge products. Through this format, participants will see not only what tools are being used, but also what has changed in terms of speed, quality, synthesis, and decision-making. Using a practical example from education and skill training evaluations, the panel will illustrate how AI can support: the development of frameworks, outcomes, and indicators synthesis of prior evidence and programme documents improvement of tools and instrument design data handling, summarisation, and analysis report writing, presentation building, and communication of findings The session will also reflect on where AI adds genuine value, where caution is needed, and what this means for the future skill set of evaluators. Session Objectives demonstrate how AI can be used across the evaluation lifecycle in practical and grounded ways show how evaluation workflows are changing, not just which tools are available provide real examples of how different evaluation roles are using AI in their daily work compare “before” and “now” approaches to highlight changes in efficiency, quality, and output generation encourage critical reflection on where human judgment remains essential

As artificial intelligence becomes increasingly integrated into evaluation practice, much of the focus has been on improving efficiency, speed, and scale. However, evaluation is not only a technical process—it is fundamentally human, requiring contextual understanding, ethical judgment, and meaningful engagement with communities. This session explores the limits of AI in evaluation by asking a critical question: What aspects of evaluation cannot—and should not—be automated? Positioned as a counterpoint to AI-focused efficiency discussions, this panel will bring together diverse stakeholders from across the evaluation ecosystem to reflect on where human judgment remains essential for maintaining credibility, trust, and ethical integrity. Potential Speakers / Roles 1. an NGO leader or programme implementer with deep experience in field engagement and community-based work 2. a Monitoring & Evaluation (M&E) specialist or qualitative researcher focused on analysis, interpretation, and learning. 3. a CSR leader, donor representative, or policy advisor who uses evaluation evidence for decision-making. 4. a representative from a digital data or technology platform (such as SurveyCTO or similar) working at the intersection of AI and data systems. Through this multi-stakeholder perspective, the session will explore key dimensions where human involvement remains critical: Field engagement and trust-building, which shape the quality and authenticity of data Contextual interpretation, including cultural, social, and political nuances Ethical decision-making, including what to report, how to represent findings, and how to minimise harm Strategic use of evidence, where judgment is required to translate findings into meaningful action The session will also include an interactive component where participants reflect on differences between AI-generated outputs and human interpretation, highlighting how depth, nuance, and “so what” insights emerge through human engagement. Rather than positioning AI as a replacement for evaluators, the discussion will focus on defining a balanced, responsible approach—where AI supports evaluation processes while human judgment remains central.

AI is increasingly being used in evaluation, but most adoption in the social sector remains fragmented and task-based, focused on isolated uses such as report writing, data summarisation, or presentation development. There is limited clarity on how AI can be integrated into evaluation systems in a structured and meaningful way. In contrast, sectors such as technology and business have moved beyond using disconnected tools. They have embedded AI within SaaS-based platforms, integrated data systems, automated workflows, and real-time dashboards that support continuous data use and faster decision-making. These approaches enable data to flow across processes rather than remain siloed within reports. This session focuses on bridging this gap. It will explore how evaluation can move from tool-based use of AI to more integrated, system-level approaches, where data, analysis, and decision-making are more closely connected.