Evaluating The Impact Of Multi Intervention Development Projects The Case Of Ethiopias Community Based Integrated Natural Resources Management Project

Author : Dawit Fasil Mengesha
Publisher :
ISBN 13 :
Total Pages : pages
Book Rating : 4.:/5 (917 download)

DOWNLOAD NOW!

Download or read book Meta-evaluation of Two International NGOs' Integrated Development Programs written by Dawit Fasil Mengesha and published by . This book was released on 2014 with total page pages. Available in PDF, EPUB and Kindle. Book excerpt: Evaluation of development interventions play an important role in development cooperation and allow development agencies to learn from past experiences and hold people accountable for what has been accomplished. International development agencies like Save the Children and World Vision learn a lot from evaluation processes and products. One of the products of evaluation is the evaluation report. The information that comes out from evaluation reports is so vital for learning and future programming. Unfortunately, the quality of evaluation reports produced by many of implementing development agencies is often poor (Mayne and Schwartz, 2005). And hence, if the quality evaluation report is compromised, then development agencies may draw wrong conclusions about ongoing and future intervention and lose their steering wheel. Thus, the quality of evaluation reports needs to be evaluated before they are used. Though there is evaluation report review in both Save the Children and World Vision Ethiopia, the culture of meta-evaluation is not in place. As a result of this, it is hardly known as to what extent the evaluation reports quality is up to standard. As a result of this, meta-evaluation becomes inevitably important for such organizations for better informed program intervention decision, learning and accountability.This dissertation assesses the quality of Save the Children and World Vision Ethiopias evaluation reports using techniques of summative meta-evaluation in relation to OECD/DAC evaluation report quality standards, Save the Children and World Vision International evaluation report review tool standard. Specifically, this research aims to analyze as to what extent these organizations evaluation reports are good enough for learning and could be used for future programming. Moreover, the dissertation examines whether there is significant statistical difference between these two international NGOs evaluation reports. And it also seeks to provide recommendations to both organizations on aspects of the end term evaluations that can be improved in order to increase the quality of evaluation products.In order to meet the objectives of this study, each report was assessed against 9 categorical standards which consists of 33 specific standard quality criteria, and the assessment of each one of the criterion was summarized as a rating on a six-point scale ranging from excellent being 5 to very poor, being 0 adopted from Forss et al. (2008) who assessed 34 evaluation reports of Sida. The average score of each category of standards per organization is used to decide whether the reports are good enough in addition to the meta-evaluation mean-average scores of the two organizations. The end term evaluation reports need to score an average of 3 and above for the report to be qualified enough for learning and future programming purpose. Simple descriptive statistics and t-test mean comparison was applied to examine whether there exist significant statistical differences between the two organizations quality of evaluation reports. One major conclusion of this study is that the evaluation reports of both organizations are qualified enough for learning and future programming, with total mean average score of 2.98 which is adequate as per Forss et al. (2008) meta-evaluation rating. But this doesnt mean that both organizations are qualified enough in each and every categorical standard and sub-standard criterion. Both Save the Children and World Vision evaluation reports have significant gaps in meeting category standards clarity of analysis and findings and evaluation question answered. Save the Children evaluation reports are weak in category standards clarity of conclusions and recommendations, explaining acknowledgement of changes and limitations and incorporation of stakeholders comments. On the other hand, World Vision evaluation reports lack adequacy in category standards explanation of context of development intervention, and intervention logic as well as explanation of the methodology used.Moreover, the result of the meta-evaluation mean average comparison test indicates that there is no significant difference in quality of evaluation reports between Save the Children and World Vision Ethiopia. Though there is no significant statistical difference in mean average meta-evaluation score, there were significant differences in category standards explanation of context of the development intervention, and intervention logic explanation of methodology used, evaluation question answered and explanation of acknowledgement of changes and limitations of the evaluation. In light of the identified gaps in meeting evaluation report categories of standards, below are some of the recommendations addressed to both Save the Children and World Vision Ethiopia: Whenever ToRs are developed, they should be realistic enough in showing the evaluation questions vis-à-vis compatible and clear evaluation methodology that can answer the evaluation questions. Moreover, consultants and other stakeholders who are involved in the evaluation process should be involved right from the beginning of ToR development and reach into consensus on what and how the evaluation team answers the agreed evaluation questions. Both organizations should give due attention to a preparation session (planning of evaluation) of the evaluation process before embarking any of the evaluations. There should be a formative meta-evaluation practice to correct some pitfalls during the evaluation process. Though there is a practice of evaluation report review before it gets approved in both organizations, it is not as such enough. Hence, both organizations should begin the practice of peer review using OECD/DAC evaluation standards in combination with their own report review tool before consultants exit.Consultants should be given considerable amount of time to understand the logic of the intervention and the context of the organization prior to preparation of inception papers and implementation of the evaluation.