Consultant Analyst - Data Analysis, QA & Capacity Building at Evidence Action

Background

• Evidence Action’s Monitoring, Learning and Evaluation (MLE) function in the West and Central Africa (WCA) region delivers the Comprehensive Facility Survey (CFS) analysis annually for each country program. In 2025, responsibility for the CFS analysis transitioned from the ESA team to the WCA Data Analytics and Learning (DAL) unit. The first two rounds — Cameroon (December 2025) and Liberia (March 2026) — revealed significant gaps in the quality and completeness of the analysis outputs, which were documented through a formal Root Cause Analysis.
• The RCA identified four root causes: an abrupt transition without adequate knowledge transfer, the absence of a documented analysis protocol and quality assurance process, insufficient pre-analysis pipeline review, and a supervisory review process that was not yet functioning as an effective quality gate. A structured improvement plan has been developed to address these gaps ahead of the next CFS round (Cameroon, October 2026).
• This consultancy is being engaged to accelerate the improvement plan, provide technical coaching to the DAL team, and serve as a quality gate on analysis outputs during the engagement period.

Objective

• The primary objective of this consultancy is to strengthen the WCA DAL team’s capacity to independently deliver high-quality CFS analysis by October 2026. The consultant will do this by providing hands-on technical coaching, co-developing structured analysis protocols and SOPs, training the team on their application, and functioning as the primary quality assurance reviewer on all analysis outputs produced during the engagement period.
• A secondary objective is to contribute to the broader analytical capacity of the DAL unit by supporting the team on data analysis workflows, results generation, and evidence-based reporting practices that extend beyond the CFS package.

Scope of Work
CFS Analysis Quality Assurance and Capacity Building (Primary — approx. 70% of effort)
Technical Coaching:

• Work alongside the primary analyst (Associate level) to build familiarity with the CFS analytical logic, including KPI construction from raw survey data, compound conditions (AND/OR operators, nested skip-logic), denominator specification, and rebinning procedures.
• Lead or co-lead structured learning sessions on the CFS package covering KPI definitions and rationale, survey design and skip-logic patterns, codebook conventions, and how indicators are derived from raw data. Sessions should be practical and tied to the team’s actual analysis work, not theoretical.
• Coach the DAL team lead (Manager level) on CFS-specific review techniques, including the ability to detect common error patterns such as wrong variable selection, denominator mismatches, implausible output values, and code-to-output discrepancies.
• Stress-test the team lead’s reviewer capacity through KPI exercises and planted-error review practice sets, providing feedback on review accuracy and identifying areas requiring further development.

Quality Assurance:

• Serve as the primary quality assurance reviewer on analysis outputs, reviewing at each stage of the two-stage review protocol: (1) independently reviewing rebinning and frequency outputs alongside the team lead before KPI calculations proceed, and (2) reviewing the codes used to develop KPIs and a KPI sample against guidance as a second pair of eyes, using a sense-check checklist.
• During the October analysis round (if engagement is extended), serve as peer reviewer on the output — checking an independent sample of KPIs after the team lead’s Stage 2 review, and participating in the final sense-check protocol before submission.
• Serve as the quality gate on all CFS-related outputs during the engagement period, ensuring that errors are identified and resolved before any draft is submitted to the Global MLE Strategy team.

Protocol and SOP Development and Team Training:

• Co-develop with the DAL team a structured pre-analysis protocol including: codebook-vs-data reconciliation, indicator computability mapping, and a documented pipeline walkthrough.
• Support the co-development of a CFS analysis SOP with the Global MLE Strategy team, covering indicator definitions, quality checks, and review points. If Strategy is unable to engage within the timeline, lead WCA’s independent development of the SOP.
• Co-develop a structured two-stage review protocol with documented checklists for the team to use in future analysis cycles.
• For each protocol, SOP, or review process developed, conduct formal onboarding and training sessions with the relevant team members. Training should be hands-on and practical. The consultant must document evidence that learning has taken place — through structured assessments, walk-through exercises, or supervised application of the protocol on actual data — confirming that team members are competent in applying each protocol or review stage independently.

Blind Re-Analysis:

• Provide coaching support during the blind re-analysis of Cameroon or Liberia (August–September 2026), where the team works from raw data without referencing Strategy’s completed outputs.
• Review the final blind re-analysis output and provide an independent written assessment of the team’s accuracy and readiness. This assessment feeds into the formal readiness assessment conducted jointly by the Global MLE Strategy team and the consultant by mid-September 2026.

Broader Data Analytics Support (Secondary — approx. 30% of effort):

• Develop a standardised output validation protocol for non-CFS analyses — a checklist the team applies to any results generation across programs (MMS, SQLNS, DTW, etc.) that codifies sense-checking habits. Train the team on its application with documented evidence of competency.
• Conduct an applied quality review of one non-CFS program analysis cycle (MMS, SQLNS, or DTW, depending on what is active) end-to-end: from the analysis code and output validation through to the analytical inputs that feed into the donor report. Produce a written review with findings, corrections, and recommendations.
• Where relevant, provide guidance on data visualization approaches and the translation of analytical outputs into accessible formats for program teams and senior management.

Deliverables

• Complete onboarding activities, including review of the Root Cause Analysis (RCA), improvement plan, and Phase 1 diagnostic outputs by the end of Week 2.
• Review, finalize, and implement a structured pre-analysis protocol, including training the DAL team and documenting evidence of competency by early July 2026.
• Co-develop a Comprehensive Facility Survey (CFS) analysis Standard Operating Procedure (SOP) with the Global MLE Strategy team, or independently if required, and train the team on its application by early July 2026.
• Develop and document a two-stage review protocol with quality assurance checklists, and assess team competency on each review stage by the end of July 2026.
• Deliver structured learning sessions covering KPI definitions, survey design, skip-logic patterns, and codebook conventions by the end of August 2026.
• Design and facilitate reviewer stress-test exercises for the DAL team lead, including documented assessments of review accuracy by mid-August 2026.
• Provide coaching support during blind re-analysis exercises and complete an independent review of analysis outputs by the end of August 2026.
• Contribute written findings and recommendations to the formal readiness assessment conducted jointly with the Global MLE Strategy team by mid-September 2026.
• Develop and implement a standardized validation protocol for non-CFS analyses, including team training and documented evidence of competency by the end of August 2026.
• Conduct an end-to-end quality review of one non-CFS analysis cycle and produce a written review covering analytical findings, corrections, and recommendations by the end of August 2026.

Requirements
Required Qualifications and Experience:
Essential:

• Master’s Degree in Statistics, Economics, Public Health, Epidemiology, or a related quantitative field.
• A minimum of 5 years of experience in data analysis for public health, development, or social sector programs, with at least 2 years in a supervisory or mentoring capacity.
• Strong proficiency in Stata for data management and analysis, with demonstrated experience writing and reviewing code for survey data cleaning, indicator construction, and results generation.
• Direct experience with large facility-level survey analysis with a vast number of variables (CFS, SPA, SDI, or comparable instruments) — including familiarity with KPI construction, skip-logic handling, rebinning, and weighted estimation.
• Experience developing or co-developing analysis SOPs, quality assurance protocols, or structured review checklists for survey data analysis.
• Demonstrated ability to coach and build capacity in analytical teams — not just review outputs, but explain why an output is wrong and how to fix it.
• Strong written communication skills, including the ability to produce clear, concise analytical documentation and progress reports.

Desirable:

• Experience with Evidence Action programs or similar large-scale public health delivery programs.
• Familiarity with M&E frameworks, indicator tracking systems, and donor reporting requirements in the development sector.
• Experience with data visualization tools (Power BI, Tableau) and automated reporting workflows.
• Familiarity with additional statistical software (R, Python) and data management platforms (DHIS2, SurveyCTO).

Duration and Timeline:

• The consultancy is for an initial period of three (3) months, starting 15 June 2026 through end of September 2026, with the possibility of extension through December 2026 to cover the blind re-analysis, readiness assessment, and the October Cameroon analysis round. Any extension will be subject to evidence of strong performance during the initial period and will be agreed in writing. (An earlier start will be considered)
• This role requires up to 80% face-to-face interaction with the DAL team in Abuja. The hands-on coaching, training, and quality assurance functions are most effective when delivered in person alongside the team.

Reporting and Working Arrangements:

• The consultant reports directly to the Senior Manager, MLE WCA.
• Day-to-day coordination will be with the DAL team lead (Manager, Data Analytics and Learning).
• The consultant will provide weekly written updates to the Senior Manager, each including evidence of tangible results achieved during the reporting period — such as completed training assessments, reviewed outputs, protocol drafts, or documented team progress. A formal written mid-point progress update is due by the end of July 2026.
• Up to 80% of the consultant’s time should be spent in face-to-face engagement with the team in Abuja. Remote working may be agreed for specific tasks that do not require in-person interaction.