Data Engineer at Techduce Africa

Description

• Responsible for designing and implementing data pipelines and architectures to support data processing and analysis.

Responsibilities

• Designing and building data pipelines to extract, transform, and load data from various sources into a data storage solution
• Developing and maintaining data storage solutions, such as data warehouses and data lakes, to ensure data quality, reliability, and accessibility
• Creating and implementing data security and privacy policies to ensure data privacy and compliance
• Designing and implementing data processing systems, such as batch processing and stream processing, to support data analysis and decision making
• Collaborating with Data Scientists and other stakeholders to understand their data needs and design data solutions that meet those needs
• Optimizing data processing performance and scalability to ensure data systems can handle large and growing data volumes
• Monitoring and maintaining the health and availability of data systems, and responding to and resolving issues as needed
• Participating in the development of data governance and management policies to ensure data accuracy and consistency
• Staying up-to-date with new data technologies and incorporating them into work as appropriate.

Qualifications

• A bachelor’s or master’s degree in computer science, software engineering, or a related field
• Strong programming skills in languages such as Python, Java, or Scala
• Strong understanding of data storage and processing technologies, such as Hadoop, Spark, or SQL
• Experience with cloud computing platforms such as AWS, Google Cloud, or Azure
• Excellent problem-solving and analytical skills
• Ability to effectively communicate with technical and non-technical stakeholders
• Ability to independently manage and prioritize multiple projects
• Strong attention to detail and accuracy
• Experience with data security and privacy technologies and best practices
• Familiarity with data analysis tools and techniques
• Experience with version control systems, such as Git.