Adopting a nonlinear goals mindset in M&E: A case for loosening up as we strive for sustainable and inclusive outcomes
Roundtable | Online
-
Organizado por:
Crescenda Evaluation and Strategy
Sobre o evento
The field of evaluation can be highly innovative in its approaches but there’s still a tendency for many of us to get stuck in traditional, more rigid habits.The field is accustomed to generating detailed plans, drafting goals and expected outcomes that are SMART, and collecting data according to these predefined metrics.
While as guiding principles these things are critical to have, we usually rely on them to the point of these metrics being an end all, be all for our work. We can fall into overly defined, hyper-focused and hyperspecific metrics, which creates a kind of tunnel vision in our theories of change (ToC) and downstream processes. Blame the human desire for certainty or the need to meet funding requirements, or both, but this tunnel vision may not be serving our communities well.
For numerous reasons it is still commonplace to use the SMART goal structure, detailed logic models, and ToCs that contain very specific definitions of success to drive much of the evaluation, research, and learning process in development and social impact. This reliance could be a byproduct of wanting to achieve certainty and accountability but it also pushes us towards a traditional, linear definition of progress that can limit innovation, risk-taking, and learning opportunities.
A recent article from Ness Labs recalls a body of psychological research and states “the most significant breakthroughs often emerge from nonlinear paths. Yet, we cling to goal setting methods that promise certainty and control” (Ness Labs: Nonlinear Goal Setting). This is opposed to where M&E, impact-driven organizations, and many funders in development and impact want to go. So why do we keep falling into this limiting mindset?
Given the additional complexities of a rapidly changing impact landscape, there is a clear need to implement evaluation techniques for the unknown, unexpected, and emergent. Additionally, the slow pace of change that is inevitable in many development and social impact settings is not always compatible with the productivity-oriented SMART goal (in addition to often being incongruent with the desired outcomes and timelines of funders).
These are just some of the factors that prompt a rethink of how we define and measure success.
To highlight these issues and offer alternatives, this roundtable session will:
- Begin with an approximately 30-minute presentation of adapting nonlinear goal-setting to M&E in social impact spaces, including sharing tools and methods that evaluators can use in place of the traditional SMART structure and very detailed logic models when the outcomes aren’t or shouldn’t be super defined.
- We’ll cover leading project teams to a consensus on the benefits of less restrictive evaluation models (and certainly embracing mixed methods approaches)
- An outline of where SMART goals still make a lot of sense and where they may not.
- A second half Q&A-based discussion, with audience participation encouraged to share experiences, thoughts, and current practices.
While as guiding principles these things are critical to have, we usually rely on them to the point of these metrics being an end all, be all for our work. We can fall into overly defined, hyper-focused and hyperspecific metrics, which creates a kind of tunnel vision in our theories of change (ToC) and downstream processes. Blame the human desire for certainty or the need to meet funding requirements, or both, but this tunnel vision may not be serving our communities well.
For numerous reasons it is still commonplace to use the SMART goal structure, detailed logic models, and ToCs that contain very specific definitions of success to drive much of the evaluation, research, and learning process in development and social impact. This reliance could be a byproduct of wanting to achieve certainty and accountability but it also pushes us towards a traditional, linear definition of progress that can limit innovation, risk-taking, and learning opportunities.
A recent article from Ness Labs recalls a body of psychological research and states “the most significant breakthroughs often emerge from nonlinear paths. Yet, we cling to goal setting methods that promise certainty and control” (Ness Labs: Nonlinear Goal Setting). This is opposed to where M&E, impact-driven organizations, and many funders in development and impact want to go. So why do we keep falling into this limiting mindset?
Given the additional complexities of a rapidly changing impact landscape, there is a clear need to implement evaluation techniques for the unknown, unexpected, and emergent. Additionally, the slow pace of change that is inevitable in many development and social impact settings is not always compatible with the productivity-oriented SMART goal (in addition to often being incongruent with the desired outcomes and timelines of funders).
These are just some of the factors that prompt a rethink of how we define and measure success.
To highlight these issues and offer alternatives, this roundtable session will:
- Begin with an approximately 30-minute presentation of adapting nonlinear goal-setting to M&E in social impact spaces, including sharing tools and methods that evaluators can use in place of the traditional SMART structure and very detailed logic models when the outcomes aren’t or shouldn’t be super defined.
- We’ll cover leading project teams to a consensus on the benefits of less restrictive evaluation models (and certainly embracing mixed methods approaches)
- An outline of where SMART goals still make a lot of sense and where they may not.
- A second half Q&A-based discussion, with audience participation encouraged to share experiences, thoughts, and current practices.
Orador/a
| Nome | Título | Biography |
|---|---|---|
| Leslie Waller | Principal Consultant | Previously working as an epidemiologist and now consulting in the MERL space for health and environment programs, Leslie focuses on sound, appropriate, and trusted data and processes as a cornerstone of her work. While she's technically American, she currently resides in France. |
Resumo
While goal-setting outside of a linear path or plane is a nascent topic in many, if not all, fields, and therefore we're just exploring it's use, several conclusions did come from the roundtable.
1. There seemed to be interest in the topic, as well as acknowledgement of the need to embrace dynamic and/or complex settings where goals may be fluid or frequently need to change.
2. Many funders and donors still want to see clear impact from their dollars and seem less apt to embrace nuance and realizations that goals may need to change in some environments and timeframes.
3. There is no established framework for operating a monitoring and evaluation system or process in a "nonlinear goal" manner. This is more of a philosophical mindset change that accompanies using some of our existing tools differently. For example, we may not fully define a goal as we monitor and track data from outcome harvesting methodology. The theory of change may be much more broad and less detailed than we're used to, and we may need to be more explicit in narrative form our justifications for operating with a broad TOC.
A big follow-up action is continuing to communicate with funders and donors the utility, sometimes necessity, and often the inevitability of nuance, complexity and lack of clearcut cause and effect in our data. We live in a natural world with tons of factors that influence outcomes, so we have work to do to 1. figure out how to best understand and articulate this ourselves as evaluators and "the data people", and then 2. how to communicate that to the donor community (and likely public audiences as well).