From Pilot to Practice: A Framework for Testing AI Tools in Evaluation
Roundtable | Online
-
Organized by:
Social Impact
- In partnership with: Millennium Challenge Corporation
About the Event
How do you decide when an AI tool is good enough to use in evaluation? Since 2023, Social Impact has run structured pilots comparing AI tools against manual approaches at multiple stages of the evaluation process. Social Impact will share findings from its most recent pilot with the Millennium Challenge Corporation (MCC), a multi-method comparative study of AI-assisted transcription, translation and qualitative coding. As evaluation commissioner, MCC will discuss how it weighs the benefits and risks of AI use, what it needs before trusting AI-assisted findings, and how its policies are evolving.
Speakers
| Name | Title | Biography |
|---|---|---|
| Dr. Kari Nelson | Technical Director, Social Impact, Inc. | Dr. Kari Nelson is a Technical Director at Social Impact with 25+ evaluations across energy, WASH, agriculture, health, and education. A mixed-methods specialist, she currently leads water and power evaluations and pilots AI for qualitative coding, translation, and transcription. |
| Julian Glucroft | Director, Dept. of Policy and Evaluation, Millennium Challenge Corporation | Julian Glucroft is a Director at MCC specializing in evidence-based project design and complex evaluations. With 15+ years of experience in M&E, policy and institutional reform, and the energy sector, he brings clear insights and practical solutions to development challenges. |
Moderators
| Name | Title | Biography |
|---|---|---|
| Mateusz Pucilowski | Chief Executive Officer, Social Impact, Inc. | Mateusz Pucilowski is CEO of Social Impact and an evaluation methodologist with 16+ years' experience. He is leading SI's AI integration initiative, developing systems that improve efficiency while maintaining analytical quality, and has overseen multiple pilots of AI-enhanced evaluation processes. |