Consultancy (Financial Inclusion Outcome Evaluation) at Mercy Corps

Project Details

• Catholic Relief Services (CRS), in partnership with Mercy Corps (MC), the Catholic Caritas Foundation of Nigeria (CCFN), the Federation of Muslim Women Association of Nigeria (FOMWAN) and eight (8) Civil Society Organizations (CSOs) is implementing the Feed the Future Nigeria Livelihoods Project (FFNLP), with support from the United States Agency for International Development (USAID). The project employs a multi-sector approach that will help 42,000 vulnerable households (HHs) in rural communities in northern Nigeria’s Sokoto and Kebbi states, and in the Federal Capital Territory (FCT) grow their agricultural and non-agricultural incomes.

Job Description
The Evaluation Consultant will conduct an in-depth and accurate evaluation of the performance outcomes of the various IR2 sub-interventions on beneficiaries’ abilities to successfully manage and grow businesses for which they received technical and financial assistance. Along with predefined indicators developed below, the evaluator will use the following elements in the performance of this task to determine:

• Relevance: Has the intervention(s) and overall approach been adequate and appropriate to help beneficiaries manage and grow their business enterprise?
• Efficacy: To what extent i.e. how effective are the financial interventions provided (conditional cash grant vis-a-vis micro-franchising/loan product) vis-à-vis technical assistance and gender in enabling beneficiaries start and improve their business
• Sustainability: Are the project beneficiaries likely to continue to manage and grow their enterprises long after the intervention is over?
• Lessons Learned: What key lessons can stakeholders learn based on findings gathered from the outcome performance evaluation study?

Key Evaluation Questions
The following questions should guide the evaluator and be addressed in the assessment:

• Were the project strategies, interventions and areas appropriately targeted?
  • Did the intervention activities properly target the needs of beneficiaries already in business, and those new to business: providing them with the adequate skills, and finance to start and manage a business, providing them with the markets to help their businesses increase their revenue and in turn income.?
  • Was the program’s aim to enable the beneficiaries above to start and improve on their businesses, and access finance options in order to diversify revenue sources while increasing income levels an appropriate response to the needs in target communities?
  • Did the program target and reach the intended beneficiaries?
  • Were there unintended consequences, both positive and negative, that arose from the project strategies, interventions and targeting?
• To what extent have the grants, product loans, business and TVET trainings, as well as SILC groups formed individually and collectively enabled targeted beneficiaries access finance and appropriate trainings to start and manage a business, identify and link up with new markets in order to help them businesses increase their revenue and in turn income assisting vulnerable households escape the poverty trap?
  • What evidence is there to determine whether the project achieved its objectives?
  • Which elements of the 5 intervention activities above individually and collectively saw the greatest impact since the beginning of the program?
• What lessons can be learned regarding the approaches used by the project?
  • Which delivery mechanisms were most effective? Conditional grants or loans or none at all, the case worker model etc.
  • What were the major challenges looking across the various project stakeholders?
  • What did the program do to address these challenges?
  • What recommendations can be made towards similar projects in Nigeria?
• To what extent are results achieved likely to be sustainable?
  • Will vulnerable households be able to maintain economic stability based on their current business, savings and investment status after the close of the grant period?
  • Did Mercy Corps sufficiently engage with local partners, government, community stakeholders and beneficiaries themselves to ensure knowledge gained in program implementation continues to reach vulnerable populations after program closure?
  • How likely is it for the beneficiaries to continue to access affordable credit (cash and products) beyond the life of the project?
• Did Mercy Corps effectively coordinate with the following groups in the implementation of its project activities?
  • Beneficiaries;
  • Implementing CSOs;
  • Local community heads;
  • State government; and
  • Implementation and donor communities?
  • How has such coordination helped or hindered the program in the accomplishment of its objectives?

Impact Measurement:
In addition to the above questions, the assessment will measure impact along the following dimensions and indicators. There is no baseline data using these indicators and a methodology to assess impact must be included in the work plan/methodology submitted in response to this Scope of Work.
Dimension:

• Entrepreneurship

Indicators:

• Self-employment rate
• Number of jobs created (direct employment and voluntary skills transfer beyond FtF)
• %age own initial capital invested
• Number of new businesses created post intervention
• Self-employment rate

Dimension:

• Access to finance

Indicator:

• Access to loans (bank, SILC, family and friends other sources) per microenterprise

Dimension:

• Access to Market

Indicators:

• Microenterprise with new markets that they did not reach before
• Existing market: size and value per microenterprise
• Total Market: size and value per microenterprise

Dimensions

• Skill and Innovation

Indicators

• SME introducing new product, and/ service, marketing or organizational innovations

Dimension

• Financial Performance

Indicators

• Gross margin
• Net revenue

Specific Tasks to be Accomplished and Evaluation Methodology
The evaluation will be conducted by an external consultant recruited by Mercy Corps. Although the methodological approach of the evaluation depends on the strategic choice of the consultant, it is important that the choice be participatory. Therefore, the consultant must present a detailed methodology on the content, approach, proposed evaluation tools, and the logistical planning of the mission (including proposed travel dates) in her/his bid. Although the development of this methodology will be left to the consultant, it should include, at minimum:

• A detailed and comprehensive outcome performance evaluation plan with appropriate methodology to guide the implementation of the evaluation study.
• Specific indicators and areas of evaluation using the evaluation indicators already developed and approved by Mercy Corps and indicators proposed by the consultant.
• A review and analysis of documents, reports, and other materials beneficial to the impact evaluation, including quarterly and annual review reports from CSOs and MCN
• Extensive field visits to project sites where relevant Feed the Future interventions have been implemented.
• Proposed design of data collection instruments [surveys, focus group discussion and key informant interview] to capture all variables of interest.
• Methodology behind proposed data analysis with qualitative and quantitative evidence of impact, lessons learned and recommendations from the analyzed results.
• Meetings with stakeholders and target groups including local government heads, state governments, CSOs and the project beneficiaries.

Key Deliverables
The consultant is responsible for the following services:

• Detailed outcome performance evaluation plan.
• Data collection instruments and indicator sheet.
• Comprehensive outcome evaluation report with lessons learned.
• A draft and final report submitted to Mercy Corps that includes:
  • Cover page;
  • Table of contents;
  • Executive summary;
  • Background;
  • Assessment Methodology and Limitations;
  • Findings:
    • Findings should assess outcomes and impact on vulnerable households;
    • Findings should be presented as analyzed facts, evidence and data and not be based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and support by strong qualitative or quantitative evidence;
  • Lessons learned;
  • Recommendations;
  • Annexes/Appendices.
• At least 2 publishable articles in a reputable journal within 6 months post evaluation.
• Weekly project update to the project management office and debrief country staff prior to close of the project on the initial and final findings from the evaluation study.

Place of Performance, Duration and Planning
The assessment will be conducted in the following areas:

• Federal Capital Territory Abuja (2 communities from 1 LGAs FCT: Bwari municipal council)
• Kebbi State (8 communities from 2 LGAs Birni Kebbi and Dankowasago)
• Sokoto State (16 communities from 4 LGAs Dangeshuni, Tangaza, Rabbah, Kebbe)

The consultant is responsible for providing a work plan to Mercy Corps. The entire assessment should not take longer than 35 days. An illustrative work plan is:

• Week 1: Preliminary meetings, document review, preparation of the evaluation plan and finalization of tools and data collection methods.
• Weeks 2 and 3: Field work (collection of primary data in the field).
• Week 4: Data analysis, interim report, and partial sharing of findings. The final report will be delivered within three days of the sharing of findings. Presentation to the Embassy of Sweden in Liberia and Mercy Corps.

Resources
Prior to and during the evaluation, the Mercy Corps FFNLP staff will provide the consultant/evaluator with the following resources:

• Relevant documents;
• Access to project communities;
• Logistical support.

Evaluator Response

• The Evaluator will respond to this scope of work providing a detailed work plan, methodology, and detailed work schedule as well as resume of the lead assessor and his/her project team (if proposed).
• The Evaluator will also provide a detailed cost estimate (professional fee, operational cost of carrying out the evaluation).

Required/ Preferred Skills and Experience
The consultant should have the following qualifications:

• At least a Master's degree in relevant area (Economics, Statistics, Impact Evaluation, Public Policy etc.).
• At least 7 years of experience in research or practice on both experimental and quasi-experimental impact evaluation designs.
• Working experience in Africa, and preferably in Nigeria is desirable.
• Excellent command of English.
• Excellent communication, organisation and teamwork skills.
• Working experience with international non-governmental organisations and civil society organizations.
• Prior assessment experience.