Ambition versus Reality: Contrasting commitments to gender, equity and human rights in evaluation with results of an abstracts review at the AfrEA 11th Conference in Kigali, Rwanda, March 2024
Webinar (em inglês) | Online
-
Organizado por:
African Gender & Development Evaluators Network (AGDEN)
Sobre o evento
Introduction
Participants will discuss examples and learn how to incorporate gender, equity, human rights and social inclusion in their evaluation practice. By drawing on the lessons from a recent review of 344 evaluation abstracts accepted at the African Evaluation Association’s 11th Conference, participants will also gain a clearer sense of the challenges and barriers to these considerations and how to overcome those in their own work.
Background
The principles of gender and equity play important roles in evaluations across social, economic and environmental sectors. Evaluators as well as the commissioners, users and participants in evaluation value considerations of gender, equity, human rights and social inclusion, particularly in the theory of change, evaluation questions, design, analysis, interpretation and recommendations. The African Gender & Development Evaluators Network (AGDEN) conducted a comprehensive review of gender, equity and rights in all 344 abstracts accepted for the 2024 African Evaluation Association (AfrEA) 11th Conference held on 18-22 March 2024 in Kigali, Rwanda, following an established AGDEN model drawing on work by the UN Evaluation Group and UN Women that was applied in previous AfrEA conferences. Initial findings of this review were presented at the closing-day plenary session and formed the basis for the AfrEA 11th conference outcome statement. A full analysis is presented herewith.
Methods
All accepted conference abstracts were reviewed by professional AGDEN evaluators with experience in gender and equity considerations. Sampled abstracts were scored based on the strength (value=2), recognition (value=1) or absence (value=0) of gender and/or equity in five aspects: (1) formulation of the evaluation questions, (2) disaggregation of data and results, (3) analysis of power relationships by gender, either in the Theory of Change, results or interpretation, (4) examination of marginalisation and/or social status in the results or interpretation, and (5) specificity of gender and/or equity in evidence-based recommendations. Final scores for each abstract ranged from 0 to 10. Average scores for gender and equity were calculated for each strand and for type of abstract (paper, poster, roundtable, etc.), scaled to 100. Additionally, strands were rated based on the proportion of abstracts receiving positive scores on at least 4 aspects (“gender bright”), positive scores on 1-3 aspects (“gender aware”) or no positive score at all (“gender ignored”). Qualitative review of the abstracts yielded information about the application of gender and equity as well as missed opportunities Additionally, live scoring was conducted for a sample of oral presentations during the conference.
Results are presented by language of presentation and conference strand; additionally, results by presentation format (paper, poster, panel, etc.) were also examined but no appreciable differences were observed. Limitations to the study include a lack of information on the gender of the presenting author(s) and no access to the abstracts that had not been accepted at the conference, for comparison purposes.
Results
The AfrEA 11th Conference programme committee provided a database of all 344 accepted abstracts, of which 341 abstracts contained sufficient information for the gender and equity assessment. Francophone abstracts constituted 22.5% (n=77) and Anglophone abstracts were 77.5% (n=264), with comparable representation across all 12 conference strands.
The average score for Francophone abstracts was 9 out of 100 maximum, as compared to 20 for Anglophone abstracts. Among the 12 conference strands, average scores exceeded 50% for only three strands: the gender and equity strand itself, and two strands related to youth in evaluation. Notably, even Indigenous Evaluation and Made in Africa Evaluation scored relatively poorly on gender and equity. 18% of abstracts were rated as “gender bright”, and 65% ignored gender and equity in their entire presentation.
The number of categories in which each abstract received a positive score is an indicator for the breadth of consideration of gender and equity, from conceptualization to data collection and analysis through to presentation and interpretation.
Interpretation and use
Findings from the AfrEA/AGDEN gender review illustrate both progress on gender and equity considerations within evaluation abstracts to date, as well as the distance that the evaluation community must still cover to fulfil commitments to the commissioners, users and participants in evaluations. Given the immediacy and importance of the findings for AfrEA 11th conference participants, AGDEN will work closely with the AfrEA conference scientific and organising committees to present the current findings in plenary sessions and to shape potential uses of the results. Recalling previous results, particularly those shared at the plenaries of AfrEA 8th (Kampala) and AfrEA 9th (Abidjan), as well as the reiteration of AfrEA’s commitment to enhancing gender and equity at AfrEA 10 (online/Zambia), the AfrEA 11 gender/equity review will inform the conference outcome statement and will be shared with other regional evaluation associations that conduct similar assessments of gender and equity considerations in their participants’ abstracts.
In previous AfrEA conferences, 13% of abstracts scored 0 on gender/equity standards, and only 15% scored 8 or higher, on a scale of 0-10 based on the UNEG criteria. A majority of presentations scored in the lower half of standardised criteria. It is important to assess considerations of gender and/or equity in current evaluation abstracts. Past results informed the AfrEA Call to Action (2017) and conference outcome statement (2019). It is expected that the 2024 results will have a similar impact on the AfrEA 11th conference.
Discussion
To facilitate wider use of the findings from this gender, equity and rights review of evaluation abstracts, a gLocal discussion panel involving former AfrEA board members, senior evaluation experts, and stakeholders will be convened to delve deeper into the implications of the review findings.
Conclusion
For evaluation to be both useful and valid, the underlying assumptions must be made explicit and testable. Evaluations that ignore gender and equity embed multiple, implicit, false assumptions about the people in the evaluand sphere: primarily, that all genders share equally in the benefits and risks being evaluated, and that social factors driving inequities are somehow absent. Further, evaluation as a field operates either as an agent of change or a perpetuator of discrimination in gender and equity. The current results are sobering, in that such a large proportion of abstracts at the AfrEA conference do not distinguish effects by gender or equity, making a false assumption that people are treated identically. By prioritising gender, equity, and human rights considerations in evaluation abstracts, stakeholders can foster a more inclusive and impactful evaluation practice. The AGDEN review serves as a crucial step towards advancing these goals within the AfrEA community and beyond.
Participants will discuss examples and learn how to incorporate gender, equity, human rights and social inclusion in their evaluation practice. By drawing on the lessons from a recent review of 344 evaluation abstracts accepted at the African Evaluation Association’s 11th Conference, participants will also gain a clearer sense of the challenges and barriers to these considerations and how to overcome those in their own work.
Background
The principles of gender and equity play important roles in evaluations across social, economic and environmental sectors. Evaluators as well as the commissioners, users and participants in evaluation value considerations of gender, equity, human rights and social inclusion, particularly in the theory of change, evaluation questions, design, analysis, interpretation and recommendations. The African Gender & Development Evaluators Network (AGDEN) conducted a comprehensive review of gender, equity and rights in all 344 abstracts accepted for the 2024 African Evaluation Association (AfrEA) 11th Conference held on 18-22 March 2024 in Kigali, Rwanda, following an established AGDEN model drawing on work by the UN Evaluation Group and UN Women that was applied in previous AfrEA conferences. Initial findings of this review were presented at the closing-day plenary session and formed the basis for the AfrEA 11th conference outcome statement. A full analysis is presented herewith.
Methods
All accepted conference abstracts were reviewed by professional AGDEN evaluators with experience in gender and equity considerations. Sampled abstracts were scored based on the strength (value=2), recognition (value=1) or absence (value=0) of gender and/or equity in five aspects: (1) formulation of the evaluation questions, (2) disaggregation of data and results, (3) analysis of power relationships by gender, either in the Theory of Change, results or interpretation, (4) examination of marginalisation and/or social status in the results or interpretation, and (5) specificity of gender and/or equity in evidence-based recommendations. Final scores for each abstract ranged from 0 to 10. Average scores for gender and equity were calculated for each strand and for type of abstract (paper, poster, roundtable, etc.), scaled to 100. Additionally, strands were rated based on the proportion of abstracts receiving positive scores on at least 4 aspects (“gender bright”), positive scores on 1-3 aspects (“gender aware”) or no positive score at all (“gender ignored”). Qualitative review of the abstracts yielded information about the application of gender and equity as well as missed opportunities Additionally, live scoring was conducted for a sample of oral presentations during the conference.
Results are presented by language of presentation and conference strand; additionally, results by presentation format (paper, poster, panel, etc.) were also examined but no appreciable differences were observed. Limitations to the study include a lack of information on the gender of the presenting author(s) and no access to the abstracts that had not been accepted at the conference, for comparison purposes.
Results
The AfrEA 11th Conference programme committee provided a database of all 344 accepted abstracts, of which 341 abstracts contained sufficient information for the gender and equity assessment. Francophone abstracts constituted 22.5% (n=77) and Anglophone abstracts were 77.5% (n=264), with comparable representation across all 12 conference strands.
The average score for Francophone abstracts was 9 out of 100 maximum, as compared to 20 for Anglophone abstracts. Among the 12 conference strands, average scores exceeded 50% for only three strands: the gender and equity strand itself, and two strands related to youth in evaluation. Notably, even Indigenous Evaluation and Made in Africa Evaluation scored relatively poorly on gender and equity. 18% of abstracts were rated as “gender bright”, and 65% ignored gender and equity in their entire presentation.
The number of categories in which each abstract received a positive score is an indicator for the breadth of consideration of gender and equity, from conceptualization to data collection and analysis through to presentation and interpretation.
Interpretation and use
Findings from the AfrEA/AGDEN gender review illustrate both progress on gender and equity considerations within evaluation abstracts to date, as well as the distance that the evaluation community must still cover to fulfil commitments to the commissioners, users and participants in evaluations. Given the immediacy and importance of the findings for AfrEA 11th conference participants, AGDEN will work closely with the AfrEA conference scientific and organising committees to present the current findings in plenary sessions and to shape potential uses of the results. Recalling previous results, particularly those shared at the plenaries of AfrEA 8th (Kampala) and AfrEA 9th (Abidjan), as well as the reiteration of AfrEA’s commitment to enhancing gender and equity at AfrEA 10 (online/Zambia), the AfrEA 11 gender/equity review will inform the conference outcome statement and will be shared with other regional evaluation associations that conduct similar assessments of gender and equity considerations in their participants’ abstracts.
In previous AfrEA conferences, 13% of abstracts scored 0 on gender/equity standards, and only 15% scored 8 or higher, on a scale of 0-10 based on the UNEG criteria. A majority of presentations scored in the lower half of standardised criteria. It is important to assess considerations of gender and/or equity in current evaluation abstracts. Past results informed the AfrEA Call to Action (2017) and conference outcome statement (2019). It is expected that the 2024 results will have a similar impact on the AfrEA 11th conference.
Discussion
To facilitate wider use of the findings from this gender, equity and rights review of evaluation abstracts, a gLocal discussion panel involving former AfrEA board members, senior evaluation experts, and stakeholders will be convened to delve deeper into the implications of the review findings.
Conclusion
For evaluation to be both useful and valid, the underlying assumptions must be made explicit and testable. Evaluations that ignore gender and equity embed multiple, implicit, false assumptions about the people in the evaluand sphere: primarily, that all genders share equally in the benefits and risks being evaluated, and that social factors driving inequities are somehow absent. Further, evaluation as a field operates either as an agent of change or a perpetuator of discrimination in gender and equity. The current results are sobering, in that such a large proportion of abstracts at the AfrEA conference do not distinguish effects by gender or equity, making a false assumption that people are treated identically. By prioritising gender, equity, and human rights considerations in evaluation abstracts, stakeholders can foster a more inclusive and impactful evaluation practice. The AGDEN review serves as a crucial step towards advancing these goals within the AfrEA community and beyond.
Orador/a
| Nome | Título | Biography |
|---|---|---|
| Ravi Ram | AGDEN member | Dr. Ravi M. Ram is a monitoring, evaluation and learning and strategic planning professional, with special interests in evidence for social accountability and public policy. Based in Nairobi, Kenya, Ravi brings substantial experience in gender-responsive and equity-focused evaluation, drawing on UN Women/AGDEN guidance with a utilization-focused approach involving a range of development initiatives and integrated programming, including direct engagement with communities and policymakers in Africa and Asia/Pacific. Ravi brings extensive experience in the design and implementation of MEL systems for complex interventions, knowledge translation and evaluation leadership. Along with his methodological expertise in MEL, Ravi offers subject matter specialization in social development (health, nutrition, agriculture, food security and education), social accountability, community systems, gender, equity and social inclusion. As professional service, Ravi is co-chair of the WHO Civil Society Commission and has served on the boards of the Kampala Initiative (aid reform), Health Poverty Action and the Kenya NGO Network. He is active with the African Gender & Development Evaluators Network (AGDEN), the EvalSDGs Guidance Group, the African Evaluation Association, and several civil society coalitions. Ravi holds a PhD from the Johns Hopkins University, an MBA from USIU, an MPH from Boston University, and an AB from Harvard College. |
| Lydiane Mbia Jobin | AGDEN Board Member | As a senior evaluator and board member of the African Gender and Development Evaluators Network, Lydiane designs various qualitative research and systematic reviews at community in Abuja-Nigeria; and has conducted a series of Evaluation Synthesis in humanitarian settings in South Sudan and a Formative evaluation for education for a program in Madagascar. |
| Eddah Kanini | Board member of African Evaluation Association | Holding a Master’s degree in Public health, Monitoring and Evaluation and completed a fellowship program training in Global health leadership awarded by a consortium of Eight Universities from USA and Africa, Eddah is a certified facilitator and trainer of trainers (TOT) in health systems management. As a technical advisor for a USAID funded project, she managed activities in the Eastern and Central region counties. She organized and facilitated capacity building trainings in leadership management and governance skills to county health management teams and health facility managers to strengthen the health interventions. She supported the County and the referral hospitals in developing the Health strategic plans, stakeholder’s validation and consensus forums as well as reviewing the health performance. As an AFYABORA fellow, She was attached at AMREF Kenya and participated in designing and implementing the M&E plan, research tools and conducting baseline surveys. With the M&E manager, she conducted the M&E capacity assessment in AMREF Kenya and facilitated the M&E training for the Kenya Country Office staffs. At the PEPFAR funded HIV ADAM project, she participated in the development and roll out of National training curriculum on HIV M&E. With the program manager as my mentor, she coordinated outreaches and conducted a situation analysis on male involvement in reproductive health. |
| Ruth Ogoi | Speaker | Ruth is a young and emerging evaluator bringing a gender lens to health evaluations, based in Nairobi, Kenya. |
Moderators
| Nome | Título | Biography |
|---|---|---|
| Dr Florence Etta Ebam | Senior Evaluator, past president of AfrEA | Dr Florence Etta is a well recognized senior evaluator, both in Africa and globally. She has served in numerous evaluation groups, including as past president of the African Evaluation Association and current co-chair of EvalSDGs. |