Evaluation management is an essential competency for evaluators, evaluation team leaders, and commissioners of evaluation, often embedded in other evaluation courses. I’ve been teaching evaluation management as part of courses on developing a theory of change, reframing evaluation through Appreciative Inquiry, gender-responsive evaluation, and other topics. Today, I invite us to think about the intersection of good management and good evaluation, because both are essential to the success of any evaluation.
I can hear your immediate question: What does evaluation success mean? Good question! It’s one I ask participants in evaluation management classes—most recently in the Evaluation Management and Oversight class I teach at The Evaluators’ Institute and the Managing Decentralized Evaluations class I teach for the World Food Programme’s Evaluation Learning Programme. To synthesize the participants’ responses, successful evaluation is relevant to important issues, planned well, conducted with high technical quality by evaluators who retain independence while engaging all stakeholders respectfully, and managed in a collaborative way with an eye to the use of results for learning, decision making, and program improvement.
In other words, successful evaluation is not only about getting the methodology right, but also about negotiating agreements and working with others to plan and organize every step of the evaluation, including its use. Thus, evaluation is “an intervention and a process”1 that requires facilitation and management.
In this post, I share one of my secrets: the three pause points of good evaluation management.
We never begin at the beginning
Evaluators always come in during the middle of the story, and we need to understand what happened before we arrived, the direction things are going, and the different visions of the future that key evaluation participants may have, and then situate our evaluation within that context. Commissioners of evaluation arrive first at the scene, and we must look to them to manage a productive process for coming up with terms of reference that embed values and make the purpose of the evaluation clear. This is helpful to the evaluation team. Still, when the evaluators get hired, they need to revisit the terms of reference, understand what is meant, and make clear agreements around the evaluation purpose, questions, process, timeline, and other factors. Managing this “beginning”—or these multiple beginnings—is an important pause point and worth everyone’s attention.
What did we miss?
A good time to think about this question is early in data collection. That’s when the rubber hits the road and you get useful feedback about what you got right and what you missed. Don’t wait 2 weeks to learn that no one responded to your survey. Check in during the first week. Check in with those conducting data collection in site visits, and with their hosts. Early check-ins help you adjust your questions and scope to include something important that you overlooked, and give you a chance to speak with your client about it in case other adjustments are necessary.
What does it all mean?
The third pause point worth your attention is data analysis. Many evaluators rush this part. There were inevitable delays in data collection, the client pushed for “just three more interviews with really important people,” and here you are: behind schedule and tempted to just start writing. Stop. You put in all this effort, and your data deserve respect. Plus, you promised triangulation in your proposal. So, take the time to see what your data tell you, compare findings from different data sources and methods, step back, and reflect. What does it all mean? What findings do you trust to bring forward into the evaluation report?
Of course, every step in the evaluation is important. But my advice is to prioritize start-up, early points during data collection, and data analysis. And, if you are convinced that evaluation management competencies are a priority for you, we can continue this conversation in my course on project management and oversight for evaluators. I look forward to hearing about your own pause points.
[1] Catsambas, T. T. (2016). Facilitating Evaluation to Lead Meaningful Change. In R. S. Fierro, A. Schwartz, & D. H. Smart (eds.), Evaluation and Facilitation: New Directions for Evaluation 149: 13–23.
Image property of Tessie Tzavaras Catsambas. All rights reserved.
Leave a Reply