Data Engineer at Autochek

Responsibilities

• We seek to engage a highly skilled and experienced Data Engineer who will work within the Business Intelligence team to orchestrate data extraction from multiple data sources.
• Our data landscape is a largely diverse enterprise with data sources ranging from traditional relational database management systems to 3rd Party cloud solution providers and no-code/low-code platforms
• As such, a lot of data extraction jobs are largely API driven.
• You would be required to build systems that perfectly execute data extraction and transformation to support the data analytics teams and drive speedy decision-making across the enterprise.

Requirements

• B.Sc. Degree or its equivalent in Statistics, Mathematics, Engineering or a related field.
• 3+ Years building and orchestrating data pipelines
• 5+ years writing clean, usable and well-documented Python code
• Experience working with a startup will be considered an advantage
• Proficiency in ETL development using Python and SQL.
• You have experience working with ELT platforms like StitchData, and Fivetran.
• Hands-on experience with Python data processing frameworks such as PySpark and Pandas.
• Good Knowledge of Software Development Standards and best practices such as Test- Driven Development (TDD), KISS etc
• Experience with using CI/CD tooling to analyse, build, test and deploy your code
• Knowledge of Data Warehousing Concepts and Data Warehouse Modelling.
• Experience working with at least one Cloud Data Warehouse Solution (Google BigQuery Preferred).
• Good understanding of design choices for data storage and data processing, with a particular focus on cloud data services.
• Experience with data flow orchestration tools such as Apache Airflow or Airbyte is an advantage.
• Experience with distributed computing and container orchestration (Kubernetes) is an advantage.
• Experience with Microservices and Event-Driven architecture is an advantage