Written by: EnCompass Staff

From October 9 to 14, EnCompass will join monitoring, evaluation, research, and learning (MERL) professionals from around the world at Evaluation 2023, the American Evaluation Association’s (AEA) annual conference, in Indianapolis, Indiana. This year’s conference theme is “The Power of Story,” which aligns strongly with much of our work. We are excited to present workshops and presentations on a range of topics, including:

• Appreciative Evaluation
• How to shape, tell, and receive the “Evaluation Story”
• Using theories of change to tell the story of efforts to measure, navigate, and shape complex systems
• Effectively integrating and synthesizing data to tell a meaningful, evidence-based story
• Using narratives to move evaluation and evaluators forward together
• Tensions at the heart of modern evaluation ethics and how to resolve them
• Equity, social justice, and localization efforts in evaluation

We are thrilled to have many of our staff leading sessions (see below for details) and participating in the conference. EnCompass is an Evaluation 2023 silver-level sponsor this year and will be co-sponsoring the Graduate Evaluation Diversity Initiative (GEDI) reception on Friday night at 6:45 p.m.! Our team will also be at booth #7 in the Exhibitor Hall to share more information about our work, our open positions, and the professional development learning opportunities we offer through the EnCompass Learning Center. Stop by to meet our teams, and enter to win a raffle prize! Continuing our long-standing tradition, we will be contributing EnCompass Learning Center vouchers to the AEA Silent Auction, the proceeds of which support first-time AEA Conference attendees from Lower-Middle Income Countries (LMICs) . Tessie Catsambas, EnCompass’ CEO/CFO, is also donating time to the “Experts’ Corner” of the auction.

If you see Beeta Tahmassebi, our VP of Strategic Initiatives, at the conference, don’t forget to congratulate her on her election to the AEA Board where she will serve as a Member-at-Large.

We’re looking forward to connecting with our MERL colleagues! Follow us at @EnCompass_World for live updates from the conference.

EnCompass’ #Eval23 sessions and workshops:

Introduction to Appreciative Evaluation 2.0
Featuring: Tessie Catsambas, CEO, CFO, EnCompass; and Stewart I. Donaldson, Claremont University

Wednesday, October 11, 2023, 8:00 a.m.–10:45 a.m. EST

Appreciative Evaluation is known for its intentionality in crafting compelling questions about successful experiences, inviting affirming multi-stakeholder engagement, generating insightful stories about lived experience, and grounding an evaluation in a compelling vision of the future. Regardless of the evaluation design and methods selected, Appreciative Evaluation is an excellent addition that will enhance the effectiveness, cultural competence, ethics, and equity of evaluation. A growing body of research in positive psychology helps us understand the impact of embedding Appreciative Evaluation into any design and method. This workshop will cover what Appreciative Evaluation is, the growing evidence behind the methodology, and three ways to apply it in evaluation practice: framing, questions and reporting.

Learning Towards Transformation: Using Theories of Change to Tell the Story of Efforts to Measure, Navigate, and Shape Complex Systems
Featuring: Michael Moses, MEL Advisor; and Amanda Stek, Associate Director, MEL

Wednesday, October 11, 2023, 8:00 a.m.–10:45 a.m. EST

The social sector is aware that achieving sustainable impact is not as simple as implementing a discrete project or program. Rather, impact occurs at the systems level, through iterative, collaborative, and dynamic portfolio-level initiatives that convene diverse sets of actors (funders, practitioners, researchers, and advocates) to strategize, learn, and adapt together over time. Dynamic theories of change, when combined with emerging evaluation approaches, can help changemakers learn to measure, navigate, and shape complex systems to achieve impact. In this interactive workshop, attendees will learn about and practice using different measurement tools and learning approaches to map out and adapt systems-level theories of change. In doing so, they will develop their ability to capture and weave systems change narratives that integrate the perspectives, lessons, and priorities of different partners and allies.

Letting the Data Speak: Effectively Integrating and Synthesizing Your Data to Tell the Story
Featuring: Ghazia Aslam, Senior MEL Advisor; and Jonathan Jones, Director, MEL

Wednesday, October 11, 2023, 11:30 a.m.–2:15 p.m. EST

EnCompass has developed an approach that efficiently and rigorously integrates and interprets data:  the Data Analysis, Integration, and Synthesis (DAIS) process. This participatory approach allows us to integrate complex data streams, often collected and analyzed by multiple evaluators. The DAIS process is interactive and iterative: we put analyzed data visually in front of the evaluation team and work together to integrate them into initial findings. This process can overcome the challenge of moving from analysis to report writing and can lead to a well-structured report with strong findings, a clear narrative reflected in the conclusions, and recommendations grounded in data. In this workshop, we present the DAIS process and invite the participants to practice each step. Throughout, participants will experience how DAIS processes produce traceable, contextualized, and supportable evidence-based findings and user-appropriate conclusions and recommendations, with a confident team standing behind the story the data tell.

Efficiency, Ethics, and Transparency: The Tensions at the Heart of Modern Evaluation Ethics
Featuring: Cecilia Banks Papariello, MEL Advisor, IRB Co-Chair, EnCompass; and Alfred Rizzo, Senior Program Officer, Millennium Challenge Corporation

Thursday, October 12, 2022, 2:30 pm p.m.–3:30 p.m. EST

To share respondents’ stories ethically, evaluators must consider the three foundational principles of human research ethics: respect for persons, beneficence, and justice. Through application of these principles, evaluators can ensure that they identify and sample the range of respondents who can tell the full story of an evaluation without being just a convenience sample (justice), whose benefits will outweigh the risks of sharing (beneficence), and who choose to tell their story (respect for persons). Institutional Review Boards (IRBs) play a critical role in safeguarding respondents. However, two opposing tensions can interfere with IRBs’ ability to play this role: the bureaucracy inherent in an IRB review can slow down or complicate evaluation timelines, but efforts to cut through this bureaucracy can lead to cursory reviews that put respondents at risk. This think-tank session aims to bring together independent evaluators, research ethicists, and donors to co-create strategies for all stakeholders to work together to ensure ethical, efficient evaluations that tell respondents’ stories safely and transparently.

Stories of Professionalization: Using Narratives to Move Evaluation and Evaluators Forward Together
Featuring: Susan Tucker, Evaluation and Development Associates LLC; Tessie Catsambas, CEO, CFO, EnCompass LLC; Veronica Olazabal, BHP; Nick Hart, Data Foundation; and Keiko Kuji-Shikatani, CES Credentialing Board

Thursday, October 12, 2023, 2:30 p.m.–3:30 p.m. EST

AEA’s Board approved a set of evaluator competencies in 2018. AEA continues its commitment to professionalization by creating the Professionalization & Competencies Working Group (P&CWG). P&CWG has been bringing members and other stakeholders together to collaboratively share stories of successful professionalization, as well as suggest ways to integrate these narratives into AEA initiatives. The goal of this Think Tank session is to engage participants in shaping AEA’s next steps regarding: (1) responding to P&CWG collected narratives from evaluators, AEA leadership, commissioners, and representatives from other Voluntary Organizations for Professional Evaluation (VOPEs) about forces they have experienced FOR and AGAINST professionalization; (2) raising questions and policies that should be addressed by AEA to move forward; and (3) validating six strategies emerging from these stories to move both evaluation as a field and individual evaluators forward.

Mapping and Critical Analysis of Localization Efforts, a Shared and Fraught Story Across Continents
Featuring: Karla Giuliano Sarr, Senior MEL Specialist; Kimberly Mahling, Chief of Party, Senegal; and Pragati Godbole, Senior MEL Specialist

Thursday, October 12, 2023, 5:00 p.m.–6:00 p.m. EST

This panel addresses and problematizes localization within evaluation and measurement storytelling to share lessons learned and continue to improve practice. The panel begins by framing the renewed emphasis on localization in evaluation from both historical and theoretical perspectives, emphasizing decolonization. We then apply these lenses to examine current examples from three different evaluation and measurement projects funded by two different U.S. government agencies. The panel concludes with recommendations that apply a pragmatic approach to centering story holders and telling more authentic and representative stories while pushing boundaries of existing paradigms and constraints.

How Evaluation Commissioners Shape, Tell, and Receive the “’Evaluation Story”?
Featuring: Tessie Catsambas, CEO, CFO, EnCompass LLC; Veronica Olazabal, Director of Evaluation at BHP; Huilan Krenn, Director of Evaluation at the Kellogg Foundation; Daniel Kidder, Director of Evaluation, CDC; and E. Jane Davidson, Independent Consultant and Co-Author in the Evaluation Management Book

Friday, October 13, 2023, 10:15 a.m.–11:15 a.m. EST

Using the model presented in the upcoming book Evaluation Management: Commissioning and Conducting Evaluations That Matter, the facilitator will pose questions to panelists that explore the roles and contributions of evaluation commissioners in managing relevant evaluations. How do evaluation managers commission and manage evaluations that tell a useful story for their organizations and the public, a story that highlights accomplishments, challenges assumptions, raises questions, provokes conversations, and helps stakeholders to build a desirable future?

Evaluations in Crisis and Conflict Settings: Operational, Ethical, and Methodological Implications
Featuring: Ghazia Aslam, Senior MEL Advisor; and Lauren Else, Senior MEL Specialist

Friday, October 13, 2023, 2:30 p.m.–3:30 p.m. EST

Fragility, conflict, and violence (FCV) are critical development challenges that have increased dramatically worldwide, with most of the world’s poor living in conflict and crisis contexts. As the aid to these contexts has expanded significantly, so has the need for research, evaluation and learning. Evaluations in FCV contexts have unique needs and challenges, and evaluators must understand these differences and adapt their practices accordingly. In this session, we discuss these challenges and facilitate a discussion on how, as evaluators, we can respond to them.

____________________________________________________________________________________________________________________

In addition to following our EnCompass staff at the conference, we hope you will check out the sessions with our ELC faculty. Many will be presenting at the conference as well!