Data Engineer at HRD Solutions

Job Summary

• To design, build, and maintain robust, scalable, and high-performance ETL/ELT data pipelines for reporting, business intelligence, and machine learning initiatives
• The role is critical for ensuring the quality, lineage, and governance of all critical data assets.

Responsibilities

• Build and optimize data pipelines using tools like Airflow/Prefect to ingest data from core banking, payment, and third-party sources.
• Design and implement dimensional and denormalized data models within the Data Warehouse (e.g., Postgres/Oracle/BigQuery).
• Utilize streaming technologies like Kafka and transformation tools like DBT to process data in real-time or near real-time.
• Implement data quality checks and maintain data lineage documentation for governance.
• Leverage Python and SQL extensively for scripting, data manipulation, and pipeline development.

Key Performance Indicators
KPI Area:

• Data Pipeline Reliability
• Data Freshness & Delivery
• Efficiency

Measure:

• Pipeline SLA Adherence (Uptime) & Percentage of automated tests (e.g., dbt tests).
• Mean Latency for critical reports/data sets
• Data Warehouse query run time (p95).

Target:

• >99.9% Uptime, 90% of Critical Data Models Tested.
• <1 hour for batch, <5 seconds for streaming data
• Reduced by 20% QoQ.

Requirements

• Education: Bachelor’s degree or Higher National Diploma from any Approved University or Polytechnic.
• Relevant certifications for DBA are an added advantage
• Experience: 4 - 6 years of hands-on experience in full-stack development.
• Specific Experience: Prior experience in banking, fintech, or other financial services environments is highly desirable.

Knowledge:

• Warehousing principles, Data Governance frameworks, Dimensional Modelling, Kafka/Stream processing, Data Quality

Skills / Competencies:

• Python, Advanced SQL, ETL/ELT tools (Airflow/Prefect), Postgres/Oracle/BigQuery, DBT, APIs, Git, CI/CD

Salary
Open to discussion.