Processing Large Amount of Text Data: A practical Example from a Meta-Evaluation using Computational Text Analysis

Evaluation systems generate large volumes of text—from project appraisal documents to evaluation reports—but these materials are rarely analyzed systematically to understand what drives evaluation outcomes.
In this webinar, we present a practical example of how we worked with such data in a meta-evaluation context.
Using a large sample of World Bank project and evaluation documents, we applied computational text analysis (CTA) to extract information on project design, focusing on how projects define and describe their intended beneficiaries. Based on this, we constructed simple indicators capturing whether interventions target change at the individual, organizational, or system level.
We then linked these design features to evaluation outcomes and explored the patterns that emerge.
The session walks through what we actually did—from data preparation to interpretation—highlighting both the potential and the limits of working with large-scale textual data. In doing so, it shows how such approaches can help bring more clarity to a central question in evaluation: to what extent are observed results shaped by how projects are designed in the first place.
Rather than presenting a general method, the session offers a transparent, experience-based account of using text analysis for evaluative learning, and invites participants to reflect on how similar approaches could be applied in their own contexts.