Final Evaluation of Feed the Future Nigeria Livelihoods Project in the Federal Capital Territory (FCT), Sokoto, Kebbi, Adamawa, Borno and Yobe states. at Catholic Relief Services (CRS)

Details:

Feed the Future Nigeria Livelihoods Project

Terms of Reference
Final Evaluation of Feed the Future Nigeria Livelihoods Project in the Federal Capital Territory (FCT), Sokoto, Kebbi, Adamawa, Borno and Yobe states.
This Terms of Reference (TOR) provides the description and work to be done to conduct a final evaluation of the Feed the Future Nigeria Livelihoods Project. The work required by this TOR includes the design of the final evaluation and implementation. This TOR has the following sections: background; justification, scope; objective; evaluation questions; learning agenda questions; methodology; deliverables; qualifications; roles and responsibility; management arrangement; timeline; payment schedule; and submission guidelines.

Project Background
CRS Nigeria is leading an exciting and innovative project designed to develop sustainable approaches to lift some of the most vulnerable households in Nigeria out of poverty. The project known as Feed the Future Nigeria Livelihoods Project is funded by USAID for a period of five years (from July 2013 to July 2018) for Northern Nigeria states of Sokoto and Kebbi and the Bwari Area Council in the Federal Capital Territory. The project is implemented by a consortium of non-governmental organizations, led by CRS Nigeria. The project is based in rural communities and targets 42,000 vulnerable households. The project consortium works closely with multiple stakeholders within government at state and local levels, and amongst private sector. The project also works with local civil society organizations and incorporates a focus on local capacity building for sustained service delivery in project locations. In 2017 the project was extended to the Northeast states of Adamawa, Borno and Yobe to reach 10,000 vulnerable households.
CRS’ Feed the Future Nigeria Livelihoods Project engages communities in agricultural-led growth interventions, using a multi-sector approach to help the very poor households grow their agriculture production, incomes and improve nutrition. The project aims to improve agricultural practices, promoting post-harvest storage for nutrient rich crops already being produced, and market-oriented approach to diversification of production, ensuring all agricultural activities are adapted to specific agro-ecological and cultural context. The project also assists vulnerable families with income diversification and provision of participatory comprehensive nutrition activities at the community level. To help vulnerable families move along the Pathway to Prosperity, the project utilizes cash transfers to help meet nutritional needs, recover assets and overcome barriers to income-generating activities. For sustainability, the project strengthens the institutional capacity of government systems for implementing poverty reduction programs and reinforce accountability between government and the citizens. The project adopts a cohort approach to household interventions. The project has randomized the households benefiting in the project into three classes: A, B, and C. Although the intervention with each class overlapped with other classes, the project began with Class A, then Class B and later Class C. The project has been working with the whole 10,000 households in North East without disaggregating them into Classes. The North-East intervention is centered on improving agricultural practices with a focus on post-harvest storage for nutrient rich crops already being produced, and promotes a market-oriented approach to diversification of production through ensuring that all agricultural activities are adapted to specific agro-ecological and cultural contexts; as well as SILC (savings and loans) intervention.
Goal: The goal of the project is, “households in targeted states have reduced poverty”. At this goal level are five key impact related indicators on income, prevalence and depth of poverty, prevalence of stunting children and households’ dietary diversity. These goals are being achieved based on the collective outcomes and outputs of the project activities across four sectoral results areas of agriculture, growth in income, nutrition and social safety net and governance.

Intermediate Results (IR)
IR1: Households have increased agriculture production and productivity
This result focuses on increasing the yields of focus commodities of farming households, by adopting improved agriculture practices and diversification of agricultural production.
1.1 Households adopt improved agriculture practices
Local value chain assessed
Sustainable producer groups established and strengthened.
Increased knowledge of production practices
Increased access to extension services
Increased access to production enhancing inputs
Increased access to financial services
Increased access to improved post- harvest and storage techniques
1.2 Households have diversified agricultural production
Increased production (and processing, as appropriate) of dual purpose crops
Increased production of small ruminants
IR 2: Households have increased income
This result focuses on income growth through increase in revenue sources, value of sales, off-farm sales and women taking advantage of market opportunities as well as access to appropriate financial services.
2.1 Households have increased revenue sources
HHs acquire the skills to identify enterprise opportunities
HHs acquire the skills needed to pursue economic opportunities
HHs have access to mentors
2.2 Households have accessed appropriate financial services
Vulnerable HHs have increased their savings
Individuals (particularly women and youth) have increased their financial literacy skills
MFBs develop products adapted to the needs of women and youth
MFBs extend their rural outreach through mobile technology
Individuals have access to financing options
IR3: Households have improved their nutrition status
This result focuses on improving diet and breastfeeding for children, women dietary diversity, safe water and hygiene practices in the households.
3.1 All household members have adequate nutritious diets
HHs have increased knowledge of nutrition and hygiene
Vulnerable HHs have increased production of nutrient-rich foods
3.2 Households have adopted improved hygiene behavior
HHs have increased knowledge of nutrition and hygiene
HHs have increased access to improved sanitation
HHs have increased access to safe drinking water
IR4: Households have stronger social safety net
This result focuses on strengthening the capacities of government institutions and community/village level systems and structures for poverty reduction at the local level and social assistance to extremely vulnerable households.
4.1 Extremely vulnerable households have accessed poverty reduction services
Constituents, CSOs and government agree on CCT parameters
4.2 LGA/AC have increased poverty reduction program implementation capacity
LGAs have strengthened operational and management structures
LGAs have local economic development plans

Justification of the final evaluation
This final evaluation is to be carried out as one of the evaluation procedures in the project activity monitoring and evaluation plan for assessing the relevance, effectiveness, efficiency and sustainability of the project outcomes as well as the strategies employed by the project for poverty reduction. As the project implementation has been completed, it is usual to carry out a detailed review to assess the project goal, determine the efficiency of the program operations in carrying out the agreed activities, assess the relevance of the project design considering implementation and changes in the operational areas and aspirations of the households targeted, and effectiveness of the sustainability measures put in place by the project and to answer some learning agenda questions. Generally, the evaluation study intends to measure programme performance against pre-agreed indicators. Therefore, it is important to evaluate the project to see if it has achieved the set goal.

Scope of the Evaluation
This final evaluation will cover all interventions of the project by the intermediate results areas and cross-cutting, including project management, resource management, monitoring and evaluation, learning agenda questions, partnership and stakeholders’ commitments.
Field Final Evaluation survey will be conducted in
Bwari Area Council in the FCT,
Tangaza, Dange Shuni, Kebbe, Rabah LGAs in Sokoto State
Birnin Kebbi, Dango Wasagu LGAs in Kebbi State
Gombi and Hong LGAs in Adamawa State
Jere, Kaga, Biu and Hawul in Borno State
Bursari and Jakusko LGAs in Yobe State
Objective of the Evaluation
The overall objectives of the evaluation is to evaluate the activities implemented for the Feed the Future Nigeria Livelihoods Project and its partners towards achieving the expected outcomes and to make recommendations on further replication of the project. The recommendations will suggest if deemed necessary – re-orientations and changes in the project design, scope and implementation approach and provide recommendations on management and methodologies to improve performance and delivery of similar or future projects. The evaluation will focus on assessing cost effectiveness of the project, and on the review of structures, processes and systems established during the implementation period.
In addition, the outcome of the evaluation will contribute to the learning documentation of the project. In achieving this, the evaluation will answer key learning agenda questions.
More specifically, the evaluation will:
Assess the relevance of the Feed the Future Nigeria Livelihoods project in terms of the priorities, objectives, implementation plan and beneficiaries’ needs as defined by the project (usefulness, alignment)
Assess the results and progress of the project in terms of effectiveness (achieved outcomes versus planned outcomes) and the efficiency of implementation (output achieved against inputs and budgets used) and
Assess the feasibility and sustainability in terms of design, scope, implementation, partnerships, management and steering of the project.
Identify evidence of programming strengths, weaknesses, emerging opportunities and lessons learned
Identify and Assess the project management and coordination processes and the effects on implementation of the project
Assess the project critical assumptions and triangulate on its effect on the implementation of the project.

Key Evaluation Questions
Impact:
To what extent have results contributed to a reduction in poverty of the rural and vulnerable households?
What is the value of the project in relation to the needs of the rural and vulnerable households, reduction of poverty, increase in agricultural production and productivity, increase in income, improvement in nutrition status and stronger social safety net of rural and vulnerable households?
Relevance:
What is the relevance of the project in relation to the pathway to prosperity model, as well as the caseworker model, women empowerment, youth and adolescents and strengthening local systems?
What does current experience suggest about the appropriateness of the current strategy of the project? Does the strategy successfully address the key issues affecting the rural and vulnerable households that have participated in the project?
Efficiency:
Do the project use resources in the most economical manner to achieve expected results? Are any other economical alternatives feasible?
Effectiveness:
Does the project achieve satisfactory results in relation to stated objectives and expected results of reduced poverty with increased agricultural production and productivity, increased income, improved nutrition status and stronger social safety net of households?
What are the results of the project interventions – intended and unintended, positive and negative – including social, economic, and environmental effects on the rural and vulnerable households?
What proportion of the targeted population has been reached overall? Did the project reach the identified categories of vulnerable households to the same extent that it intended?
What key barriers have hindered reaching all the targeted households?
Sustainability:
Is the project intervention and its impact on households likely to continue when USAID assistance is withdrawn?
1Will the project strategy be more widely replicated or adapted? Is it likely to be scaled-up?

Key Learning Agenda Questions
Does diversified agricultural production improve nutritional status of vulnerable households?
This question seeks to understand the improvements in the nutritional status of vulnerable households once agricultural production has been diversified. This will not only look at the impact of different crops, especially nutrient rich foods, for direct consumption but also generation of income needed to procure the amount and variety of food families needed.
The study was initially intended to be done be done as part of data collection and analysis by the Project M&E team but has now been incorporated into the final evaluation study. The baseline study and annual monitoring surveys will provide additional information to support the study process.
What are the contributing factors and barriers to increased local government investment in poverty reduction programs and services?
This study will be compiled based on project outcomes related to local government assessments, organizational capacity improvements, development and implementation of local economic development plans and some addition focus group surveys at community, local government and state levels.

Key Gender Evaluation Questions
To what extent has women’s engagement in market opportunities influenced their control over resources at the household level?
To what extent did women’s involvement in SILC influence their decision-making power over the use of income in the HH?
To what extent did women’s participation in project activities improve their participation and leadership in community activities?
In what ways did men’s participation in male social forums influence the relations in the HH?

Evaluation Methodology
The evaluation will adopt a mixed method approach. This will include:

• Literature review of various project documents and progress reports at CRS and partner’s offices. The project documents to be reviewed will include project description, work plan, partner coordination meeting reports, baseline report, impact evaluation baseline, annual survey reports, Households Graduation Studies and progress reports to donors.
• Review of materials produced by the project - including caseworker model, household planning diary, education strategy, M&E plan, manuals, communication strategy, accountability strategy, etc.
• Focus group and key informant interviews and discussions with organizations supported by the project, groups and individuals who have benefited directly and indirectly from the project as well as CRS staff and the consortium partners.
• Direct observation by visiting supported groups in the communities.
• Beneficiary survey to measure the project outcomes, using the annual survey design.
• Annual Key Performance Indicators to be measured and disaggregated by state and region
• Yield per unit of land
• Number of members in producer groups that have increased their production of focus commodities
• Number of farmers who have applied new technologies or management practices as a result of USG assistance
• Number of hectares under improvement technologies or management practices as a result of USG assistance
• Average number of different crops produced by participating households
• Average number of revenue sources per household
• Percentage of total income generated from off-farm activities.
• Percent of women who engage in market opportunities.
• Household Dietary Diversity Score
• Women's dietary diversity: Mean number of food groups consumed by women of reproductive age
• Prevalence of children 6-23 months receiving a minimum acceptable diet
• Prevalence of exclusive breastfeeding of children under six months of age
• Percentage of households storing treated water in safe storage containers
• Percent of respondents who know at least 3 out of 5 critical hand washing moments
• Value of household’s sales.
• Other indicators are:
• Number of farmers using PICS bags, Anthropometric status of under-five children, Demographic and socio-economic household profile indicators, Household vulnerability indicators, Conditional cash transfer indicators and Home gardens
• The evaluation will cover the entire range of partners within each result area of the project.
• Sampling will be applied in selection of sites to be visited for meetings and beneficiaries.

Main Deliverables
1) Develop review protocol
Review relevant literature and materials developed and used in the project.
Develop and share the preliminary protocol and tools with CRS project team.
Dialogue with CRS project management team.
Finalize preliminary protocol and tools after having feedback.
2) Conduct and implement the Final Evaluation
Orient and train the evaluation teams to collect data.
Test and modify evaluation tools before starting the evaluation.
Initiate and supervise the data collection.
Data cleaning and processing
Analyze data and share the initial findings with CRS MEAL team.
3) Produce 3-5 publication draft write-up journals/papers on some of the results/outcomes of the evaluation study. Consultant will be recognized as co-author for the papers.
4) Report writing and data dissemination
Write-up data findings and initial draft report.
Develop recommendations for project implementation and operations
Share findings with CRS project team.
Final report. This must at a minimum contain:
Title page
Table of contents
Executive Summary
Introduction
Objectives of the final evaluation
Methodology – process how the evaluation was conducted
Evaluation findings and analysis
Lessons learned
Conclusion and recommendations for the project
Annexes

Required Qualifications

• Proposals will be accepted from consultants/firms with verified previous experience in conducting evaluation of projects in the field of agriculture and livelihoods including nutrition in rural areas. The consultants should possess the following combination of skills and expertise:
• Post graduate degree in agriculture, agricultural economics, social science, development studies, etc., and with formal research skills.
• At least 5 years of advanced experience in conducting evaluations of complex program design with a clear understanding of various social research methodologies.
• Demonstrated research and analytical report writing skills with a sound experience in participatory review and evaluation methodologies
• Experience in multi-sector and multi partnership approach to poverty reduction and livelihoods programming.
• Multiple experience in interventions programs in agriculture production, income generation, nutrition, social safety net and governance with poor households and local institutions.
• Ability to conduct interview and discussions in English and local language desirable.
• Ability to analyze and synthesize data from different sources relating to the scope of the evaluation.
• Ability to clean, analyze and synthesize data using survey data management packages such as CSPro and Excel as well as SPSS, R, STATA, SAS, and Epi-Info or other appropriate software to analyze and report large datasets.
• Good eye for detail, adherence to logic, and capacity for inductive reasoning
• Strong presentation, facilitating, communication and team working skills.
• Strong computer skills and knowledge of ICT4D.
• Good interpersonal skills, including the ability to conduct discussions with a diversity of people ranging from senior management to project participants.
• Willingness and ability to travel to the different project's sites in the country.

Roles and Responsibilities
Project Team
The project team will be responsible for providing the existing documents related to the project and logistics support to the consultant during the final evaluation field data collection and presentation to CRS.
Evaluation Consultant
The role of the evaluation consultant is to work closely with the project management team to develop the evaluation design and implementation – outlining the methodology, key evaluation questions, identify appropriate evaluation tools, develop the data collection instruments, carry out data collection, data analysis and writing the evaluation report. The consultant will present evaluation plan and findings to the project team. The consultant should use the key principles of gender analysis and participatory approaches when working with communities and project partners. S/he also should adhere to USAID evaluation principles and standards for conducting project evaluation.

Supervision
The MEAL team will be involved in all the phases of the final evaluation survey and will take supervisory role in ensuring quality and data integrity. Specifically, the team will be responsible for the recruiting of the survey team for data collection and other supervisory responsibilities.

Payment Schedule
Payment will be made to the consultant on submission of the final evaluation report. The consultant is expected to make a formal presentation of the report to the project team.
Proposal Submission Guidelines
Cover letter (maximum one page)
Technical Proposal (maximum seven pages, to be elaborated into a work plan in the inception phase):

• i. Relevant experience of the consultants/consultancy firm
• ii. Understanding of the assignment/objectives of the evaluation
• iii. Proposal for the methodology, including sampling and approaches to be used
• iv. Proposed timeline for completing the evaluation
• v. A plan for analysis and sense making of the data
• vi. A proposed outline of the report
• vii. Team composition and roles, including profiles/resumes of the personnel to be involved in the assignment (in annex)

Financial Proposal (maximum one-page): breakdown of cost estimates for services rendered. This should include: daily consultancy fees, accommodation, travel and other logistics. Note that, CRS will pay directly for enumerators, and printing of any materials to be used.
Signed statement of independence from CRS Feed the Future Nigeria Livelihoods Project.
Sample copies of three previous related work completed.
Deliverables and Timeline
The survey is expected to take place within a period of not more than 53 days, commencing
April 9th to May 31st, 2018.
Deliverable
Description of Deliverable
Timeframe
Number of days
(1). Inception phase report which must include final research instrument and research methodology.
Inception phase (including review of research instruments and finalisation of methodology)
1st – 6th April 2018..............6 days
(2). Final training manual, field activity report including sampling frame.
Primary research - Field work, including pilot and Training
7th-26th April 2018................20 days
(3). Data list and preliminary results
Data entry, Cleaning and analysis
27th April-May 11th 2018........10 days
(4). Final Evaluation Survey first draft Report.
Development of Draft Report for review by CRS
12th - 19th May 2018...........8 days
(5). Final Evaluation Survey second draft report
Presentation of draft report
25th-28th May 2018.........4 days

6) Final Evaluation Survey Final Report
Final report (preparation and submission)
30th -31st May 2018.........2 days
Total days.....50 days