EOI - Consultant, BHA Ceton Rayuka II (Saving Lives) Final Evaluation at Catholic Relief Services (CRS)

Scope of the Evaluation

• This evaluation will be implemented in three Local Government Areas (Gubio, Kaga and Magumeri) in Borno state. The evaluation will cover all the 3 sectoral foci of the project.
• The project targeted vulnerable community members, water facilities in the communities, and community leaders. Though the entire Household was the focus of the project, women were the primary target during registration.
• The consultant will conduct a preliminary desk review of project documents and monitoring records. With support from program leads, the evaluation will involve identification of the relevant stakeholders and beneficiaries, as well as collection of the required data to answer the evaluation questions.
• Some of the stakeholders engaged during the CRII include the community leaders, youth groups, and Bulamas in the three implementing LGAs, NGOs, BHA, etc.

Evaluation Objectives

The evaluation will focus on achieving the following objectives:

• Assess the relevance, efficiency, effectiveness, coordination, and indicative sustainability of CRII interventions.
• Measure the overall achievement of the project against the targets in the results framework (project goal, sector objectives - outcome and outputs)
• Conduct Nutrition KAP survey to gather information on knowledge, attitudes, beliefs and practices among mothers/caregivers on MIYCN in Gubio, Kaga, and Magumeri LGAs.
• To determine and evaluate the complementarities of CRII sectors in achieving the overall goal.
• To identify lesson learned and actionable recommendations for future programming.

Methodology of Final Evaluation

• Based on the scope and of the evaluation in accordance with CRS MPP policy and BHA guidelines, the consultant will be responsible objectives for developing a detail methodology, tools, and approaches for this evaluation. The evaluation will use a mixed method design, specifically an explanatory sequential design.
• This approach allows for the collection and analysis of both quantitative and qualitative data to gain a more comprehensive understanding of the observed changes.
• The design will follow a sequential process, with the quantitative data being collected and analyzed first to identify any patterns or trends.
• The subsequent qualitative data collection and analysis will then be used to explain the observed changes and provide context to the quantitative results.
• The quantitative component of the evaluation will, in addition to collecting primary HH-level, beneficiary-based cross-sectional endline survey data for Food Assistance and WASH indicators, involve reviewing the baseline data and other available data sources for the endline results of Nutrition indicators.
• The output of the quantitative arm of the study will be a comprehensive presentation of the baseline and endline values for all indicators against the approved targets in the project indicator tracking table across the three sectors. The difference between baseline and end-line values will be tested for statistical significance and discussed where it is possible.
• The qualitative component will use participatory qualitative approaches to gain sufficient in-depth understanding of the contextual factors and the results of the multi-sectoral assistance, as well as the perceptions of the key stakeholders. These two approaches will be complemented by content review and analysis of key project documents, deliverables, and other products.

Data Collection

• The Evaluator will conduct field data collection, CRS will provide the enumerators who will be trained by the consultant who could also make provision for supervisors to coordinating activities during data collection by the enumerators on the field.
• Due to access issues in some of the areas, remote data collection may be considered with strict compliance to CRS remote data collection modalities and BHA guidelines.
• The Evaluator will also use baseline and other surveys datasets and compare with the end-line values for each outcome indicator.
• For the output indicators, the Evaluator will run a trend analysis to assess the key points of change and use qualitative data to triangulate the changes observed.
• The Evaluator will use the BHA Ceton Rayuka M&E plan to guide decisions on specific indicator sampling frame.

Some indicative data collection approaches could include:

• Survey: Data to be collected using a survey questionnaire. The questionnaire must be designed relating but not limited to questions developed during the baseline and should be structured per sector in order to make comparison of the change observed from the baseline. For standardization of question interpretation, the evaluator should extensively train enumerators for them to conceptualize the questions in the local languages and training should incorporate practical demonstration. The consultants shall propose an acceptable sampling methodology (Size, approach and distribution) as part of the technical proposal.
• Desk review: review of strategic and planning documents, baseline report, semi-annual report, monthly report, PDM, pre and post phase assessment reports, sector dashboard, extract reports from red rose and Commcare platform and other related documents.
• Key Informants (KII) and focus group discussions (FGD): the evaluation team is expected to conduct key informant interviews and focus groups with key stakeholders in person or remotely. Key stakeholders will include, but not be limited to project beneficiaries, representatives of community leaders, local authorities, and representatives from project teams directly involved in the coordination of project implementation.
• Direct observation during data collection: Evaluators are expected to apply observation technique on children participated in Nutrition project, WASH facilities, hygiene practices and other factors related to the project. An observation checklist should be developed and agreed with the program managers before field data collection.
• The consultant is expected to review the indicative list of approaches described above and propose alternative or additional data collection approaches in line with BHA Monitoring and Evaluation guideline,

Data Protection

• CRS has ownership to all data collected during this evaluation.
• Data collected using paper-based tools are kept by CRS in a secure location to be accessed only by authorized persons. Electronic data is stored on cloud-based servers which vary depending on the software used for collection: Red Rose Collect data is hosted by Red Rose; Kobo Collect and PowerBi dashboard. Data collected on these platforms can be accessed by authorized MEAL Coordinator and PMs.
• The data will be used primarily for CRS final evaluation and future reference. Only findings from the analysis will be shared with the project partners, donors, and stakeholders. De-identified datasets will be uploaded on CRS Gateway (CRS’ internal knowledge management platform) as well as donor platforms such as USAID Development Experience Clearing House and Development Data Library.
• The consultant is obligated to conduct this evaluation in line with CRS’ Responsible Data Values and Principles and in full compliance with the Nigeria Data Protection Regulation(2019).

Ethical standards:

• The consultant is expected to follow American Evaluation Association’s Guiding Principles for Evaluators 
• Prior to accessing CRS operational areas, the consultant, and his team, including Enumerators will sign CRS Code of Conduct, Safeguarding Policy, and Data Protection Policy.
• The consultant must seek and have a signed informed consent from participants before data collection with beneficiaries.
• The evaluator is expected to keep all data from beneficiary with anonymity and confidentiality, no mentioning of names in questionnaires or reports. and ensure protection of minors.
• The consultant will be expected to use locally acceptable conduct, dressing and norms to ensure the data collection is done in a respectful and dignified manner.
• Collected data can be stored on devices belonging to the CRS, but upon completion of the evaluation, all devices must be submitted to CRS.

Documents:

CRS will make available the following documents to the consultant:

• Project Proposals (inclusive of results framework, M&E Plan, organigram, etc.).
• Project Narrative report – monthly and annual
• Security - monthly reports.
• Previous Baseline and Evaluation reports.
• Post Distribution Monitoring Reports.
• Post-Harvest survey report
• Food Security Cluster reports.

Timeline and Milestones
Date - Milestone:

• July 25th 2023 - Inception report, methodology, workplan and sampling frame
• August 2nd to 4th August, 2023 - Enumerators’ training
• August 7th to 11th August 2023 - Data collection
• August 12th to 22ndAugust 2023 - Data analysis and report writing
• August 23rd to August 25th 2023 - Submission of final evaluation report to CRS
• August 26th to August 29th 2023 - CRS review report and provide feedback
• August 31st 2023 - Final evaluation report and brief submitted and approved

Required Qualifications

Lead Evaluator:

• Post graduate qualification in Development Studies, Monitoring and Evaluation, Statistics, or International Development, PhD will be an added advantage.
• Extensive knowledge of evaluating multisectoral projects in emergencies and good understanding of the emergence response in the Northeast.
• At least 5 years’ experience in conducting evaluations in complex environments, experience evaluating USAID/BHA funded projects. Experience working in Nigeria Northeast states, preferably BAY states will be an added asset.
• Extensive experience in evaluating multi-sectoral programs implemented through multiple partners.
• Excellent oral and written communication skills in English and fluency in Hausa and Kanuri.
• Extensive knowledge of qualitative and quantitative data collection methods and analytical methods and techniques.
• Experience in generating strategic, informative, and action-oriented recommendations to CRS project staff and strategic management.