Consultant - Endline Evaluation of the Health Workers Capacity Building Project at Save The Children

Project Background, Objectives and Beneficiaries

• Since June 2015, Save the Children with funding from Glaxo SmithKline (GSK) has been implementing a Health Workers Capacity Building (HCB) project (hereafter referred to as the project) in Gombe, Kaduna and Lagos states.

Aim

• The project aims to contribute towards an overall goal of reducing maternal, newborn and child mortality (MNCH) rates in Lagos, Gombe and Kaduna states by training 5000 frontline health workers (Doctors, Nurses/Midwives and Community Health Extension Workers) to deliver quality MNCH services to women, families and communities in these states.

Objectives

• The project has two main objectives
• To ensure that health workers are correctly applying improved skills and knowledge in the provision of MNCH services in the three focal states by 2018
• To enable a policy environment which supports the delivery of quality MNCH services in the three focal states by 2018
  

Beneficiaries
Direct Beneficiaries:

• Five thousand frontline health workers will have their capacities built in areas with the highest potential to positively impact maternal and child health outcomes.

Indirect Beneficiaries:

• Approximately 404,000 children under the age of five and their mothers will benefit indirectly from this project, as a result of improved healthcare services delivered by the trained health workers.
• 2,250 additional health workers who will be trained by the government as a result of advocacy efforts
• Our advocacy efforts will result in government training 2,250 additional health workers (Nurses and CHEWs) as part of implementing the Task-shifting and Task-sharing policy.

Status and description of the HCB Project
Presently, the project is in its 3rd and final year of implementation and is scheduled to close out at the end of November 2018. Thus far, a total of 4736 health workers have been trained across the 3 focal states using nationally approved training manuals/guidelines. Furthermore, the project is estimated to have reached 489,751 beneficiaries indirectly (women and children under five).

Key training areas are Integrated Management of Childhood Illnesses (IMCI), Integrated Community Case Management of Common Childhood illnesses (ICCM), Essential Newborn Care Course (ENCC), Active Management of the third stage of labour (AMTSL), New Vaccine Administration (NVA), Essential Management of Vaccine services, Adverse Events following Immunization (AEFI), Routine Immunization Service delivery/Reaching Every Ward (RI/REW) and Basic guide on routine Immunization (BGRISP).

Beneficiaries of each of these training were selected through a series of training needs assessment that were jointly conducted by the project technical officers and relevant officials of the State Primary Health Care Development Agency (SPHCDA)/State Ministry of Health (SMOH) in each of the aforementioned states.

Additionally, in each of its focal states, the project is contributing to the creation of an enabling policy environment for the delivery of quality MNCH services by supporting the rollout of relevant national HRH policies and strategic plans; raising awareness amongst policy makers and stakeholders for improvements in health workforce planning, management and development; and leading multi-sectoral advocacy efforts for the allocation and timely release of adequate financial resources to the health system required to attract and retain qualified health workers.

Key interventions conducted in this regard include domestication/adaptation of National HRH policies and strategic plans, domestication/adaptation of national Task Shifting/Task Sharing (TSTS) policy, development of a training of trainers curriculum to facilitate the implementation of TSTS policy in each of the focal states, support for the establishment of functional HRH desks and development of advocacy tools such as policy, briefs, fact sheets, IEC materials and position papers.

Scope of the Endline Evaluation

• In line with international best practice, a rigorous endline evaluation of the project will be carried out to determine its effectiveness, successes and challenges.
• The primary aim of this endline study is to evaluate the model of working as well to compare health worker knowledge in comparison to baseline.
• The primary input into the endline evaluation is high-quality, reliable, and consistent data collected from a number of sources across the areas of implementation.
• These data will be collected from the following group of persons who are either direct or indirect beneficiaries of the project: Doctors, Nurses/Midwives, CHEWs, In-charges of primary health facilities, Community Oriented Resource Persons (CORPs), Focal Persons for Health Programs at LGA, SMOH and SPHCDA levels and Pregnant Women and their Households.
• Respondents will also include relevant government officials at state and LGA levels. Sample sizes will be determined in consultation with the project’s technical leads. 

The objectives of the evaluation are to:

• Assess the level of efficiency of the project compared to the inputs (Competencies of the project staff, funds management practices etc.)
• To assess the effectiveness of the project in improving the skills and knowledge of health workers delivering MNCH services in each of the focal states
• To assess the level of achievement of the project towards contributing to improved MNCH services in each of the focal states.
• To assess the quality of healthcare provided by trained health workers as well as CORPs.
• To identify best practices and formulate recommendations for actions to improve the design and implementation of a follow-on to the HCB project.
• To assess the likelihood that the benefits from the project will be maintained for a reasonably long period of time when the project ends.

Specifically, the endline evaluation will seek to determine:

• What improvements in the knowledge of the health workers can be observed as a result of training delivered under the project? 
• Are health workers who benefited from the training correctly applying the improved skills and knowledge acquired in the provision of MNCH services? 
• Have there been significant improvements in immunization related services in facilities with health workers that have been exposed to the project’s immunization related services?
• To what extent can changes that have occurred as a result of the project’s intervention be identified and measured?
• Has the ICCM component of the training been effective in reaching underserved and hard to reach communities?
• Were adequate measures taken during planning and implementation stages of the project to ensure that resources were efficiently used?
• Were the project outputs delivered in a timely manner?
• Could project activities and outputs have been delivered with fewer resources without compromise in quality and quantity?
• Have there been adequate follow-up, monitoring and supervision of trained health workers by the appropriate government officials/agencies/bodies?
• Are clients accessing services in facilities with health workers exposed to one or more training areas satisfied with the quality of care received in these facilities?
• In each of the project states, have there been any significant improvements in health planning, management and development as a result of the advocacy initiatives of the project?
• How could collaborating with the Ministry better have enhanced our impact? {in planning and design}
• Have the HRH training conducted for the HRH TWG had an impact on how the managers carry out their HR activities in Gombe and Kaduna? Has their planning changed? How do they now address some of the key issues – health workers in rural areas, influencing posting?
• Did the project’s training approach offer appropriate value for money?

Overall, attention must be given to:

• What worked well on the project and what were the enabling factors?
• What did not work well on the project and were where the militating factors? 

Tasks

• The successful Consultant will deliver high-quality data according to the expectations and within a timeframe agreed to with Save the Children.
• The selected Consultant will work closely with the Project technical leads in each of the focal states and in particular with the project’s M&E officer during all phases of the evaluation. SCI may be open to considering electronic data collection using tablets or androids phones for this evaluation, and interested consultants should consider this when preparing their proposals. However, the list of tasks below assumes paper-based data collection.

The selected consultant(s) will:

• Review all relevant project related documents and primary interviews with project staff and thereafter develop an inception report. The inception report would include a study protocol, timeline and detailed budget, final report structure, as well as all data collection tools.
• Recruit, train and contract a sufficient quantity of
• Experienced interviewers who are able to ensure quality and integrity of all data;
• Data entry personnel;
• Any other support staff necessary to successfully complete the assignment to the full satisfaction of SCI.
• Train interviewers and supervisors in the proper administration of study questionnaires.
• Train data entry operators in data entry protocols and use of the data entry program
• Carry out piloting of all tools and review/amend the tools accordingly (including any retraining of interviewers/enumerators as may be necessary)
• Carry out all data management, cleaning and quality control on data; including at data collection and analysis stage. Oversee data collection by interviewers/enumerators. 
• Oversee data entry
• Oversee any data translation/retranslation, as required
• Ensure safe handling, confidentiality and security of all data in both hard and soft formats.
• Enforce strict anonymity of data, to protect study respondents and comply with ethical standards.
• Report Writing. This will include:
• Providing a clear structure
• Transmit full data sets to the SCI.Weekly field report during data collection detailing area covered, problems encountered and proposed solutions, and including summary statistics of entered data
• Final field report, including summary statistics of all entered data. 

Evaluation Design and Methodology
Interested consultants are expected to send in a detailed proposal that clearly articulates the following:

• Detailed methodology
• Consultant’s profile(s) and capacities
• Statement of experience
• Work-plan and delivery timelines
• Detailed budget.

The methodology will be expected to use an appropriate combination of primary and secondary data and an adequate mix of quantitative and qualitative research methods.

In particular the following research methods are expected to be considered by the consultant:

• Comprehensive desk review
• Secondary data analysis
• Qualitative research methods (incl. key informant interviews and focus group discussions with beneficiaries and stakeholders)
• An assessment of health workers competencies.

Deliverables

• Inception report including a detailed research protocol to be approved by Save the Children technical team.
• At least two interim progress report.
• Draft and final evaluation reports.
• A final presentation to the project team.

All work must be completed by the end of November 2018.

Minimum Qualifications

• Eligibility to perform the aforementioned work in Nigeria.
• At least 5 years’ experience in areas of public health, MNCH, health/social science
• At least 5 years’ experience implementing surveys to collect individual, household, community and stakeholder level data.
• Strong capacity and experience in planning and organizing survey field work
• Good network of experience enumerators, supervisors, and data entry clerks and ability to mobilize and retain them.
• Proven experience with data quality assurance mechanisms (field work management, data entry programming, etc.).
• Strong knowledge of data collection and analysis software such as SPSS and Epi-info.
• Familiarity/experience with both field and central-based data entry methods desired.
• Ability to communicate in English and Hausa (desired) for quality data collection.
• Familiarity with the health sector in Nigeria desired.

Terms and Conditions
Structure of Contract:

• The consultant(s) will be under contract with Save the Children Nigeria. Selection will be made on the basis of proposal quality and cost. Prior to signature of the final contract, all terms and conditions in this TOR are subject to modification.

Structure of Proposal:

• Consultant(s) are requested to structure their proposals so that activity and cost is clearly delineated.  Failure to do so will result in disqualification of the proposal.

Data release:

• The Consultant(s) will be responsible for delivering high-quality data according to the expectation and protocols of Save the Children Nigeria.  The Consultant(s) will deliver data within the agreed upon timeline. The consultant(s) is required to submit all documentation related to the evaluation, including analytic files related to sampling, data files related to sampling, and documentation related to successful and high quality data collection (such as final data collection instruments, field manuals, training materials, and data entry program).

Future use of the data:

• All data and information collected will belong to SCI The consultant(s) may not use the data for his/her own purposes, nor license the data to be used by others, without the written consent of Save the Children.

Application Closing Date
16th August, 2018.