Lead Data Engineer at Andela

About the Role

• The Lead Data Engineer role will work closely with data, engineering, and product leadership to deliver best-in-class data platforms and data solutions to further Andela’s product strategy.
• The ideal candidate should have extensive experience developing complex data solutions at scale, using SaaS and PaaS frameworks in a cloud environment.
• As an engineering lead, the candidate will take ownership of feature work to create innovative solutions for category-defining data platforms, by leading and mentoring other specialists on the team in finding solutions and adhering to engineering discipline and rigor.

Responsibilities

• Understand Andela’s platform, value proposition, and roadmap by working with Data Engineering leadership.
• Take ownership of sprint work to help create innovative solutions for a category-defining platform
• Lead the scrum team’s efforts during sprints, and mentor engineers on problem-solving and software engineering fields, through mechanisms including joint design, test-driven development, code reviews, and continuous delivery.
• Work with data engineering leadership to define software architecture and processes for Andela’s data platform, in the specific areas of data platform technical architecture, MLOps pipeline frameworks, and end-to-end orchestration of data life cycles for batch and streaming modalities
• Own the excitement of entropy! Our platform, products, and offerings are constantly changing and rapidly growing. We encourage and enable each individual at Andela to drive a commercial idea when they identify one, which builds a constantly evolving data landscape.

Requirements

• 10+ years of experience working in data-intensive environments and platforms, building data solutions in a hands-on manner. At least 2+ years of experience should be in a lead role, leading and mentoring engineering team members in pods to tackle technically complicated problems, in an agile and collaborative team environment.
• Extensive level of experience working with modern SaaS and PaaS solutions in the cloud on modern data processing frameworks, including at least two of Apache Spark, Hive, BigQuery, and Snowflake, using languages including Scala, Python, and SQL.
• High level of experience in ETL content data at big data scale and in batch and streaming modalities, for building data lakes in the lake house pattern, including the processing of unstructured, semi-structured, and structured data, MDM enrichments, and sophisticated business logic.
• High level of experience working in cloud platforms AWS, Azure, GCP (GCP is a plus).
• High level of experience in data modeling, data structure & algorithms, and data platform architecture.
• High level of experience with data orchestration frameworks, including at least one of Airflow, Dagster, AWS Step Functions, Azure LogicApps, Apache Nifi.
• Experience in MLOps, as well as supporting operationalization of data science models and pipelines, including Pytorch, Spark.ML, Tensorflow, MLFlow, and other popular frameworks.
• Experience applying standard best practice software engineering practices, including TDD, CI/CD, IaC.

Benefits

• Fully Remote work culture.
• A fair and competitive salary.
• Bring your own device stipend - buy your own laptop with funds from Andela.
• Quarterly work-from-home stipends.
• Generous paid time off.
• Additional paid holidays.
• Flexible working hours.
• Health insurance (country-specific).
• Equity.
• 401k (US only).
• Andela Affinity Groups.
• And more!