Data Engineer at Palladium Group

Primary Duties and Responsibilities:

• Responsible for conceptualizing, building, and maintaining optimal data pipeline architectures for the project.
• Work with the development team to conceptualize, design, and build the infrastructure required for optimal extraction, transformation, and loading of data from various data sources using SQL and other data technologies.
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability.
• Work with the data analytics team to build analytics tools that utilize the data pipeline to provide actionable insights into program performance and provision of information for decision-making.
• Responsible for writing testable, scalable, and efficient queries
• Provide technical support and resolution to data-related technical issues and data infrastructure needs for system users and stakeholders.
• Responsible for data governance of the Central repositories. Work with the information security and quality assurance teams to ensure data security and privacy.
• Contribute to the planning and documentation of technical specifications of database design and solution features. 
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• Engage in sprint planning and scrum meetings as part of the agile team.
• Implement Palladium’s standardized software development approach, including accompanying tools and standard operating procedures (SOPs) as part of the informatics unit.
• Provide technical leadership in adoption and compliance to global best practices including the principles of digital development, applicable coding conventions, open standards, and security standards.

Required Qualifications:

• Bachelor’s degree in computer science, Informatics, Information Systems or any other relevant field.
• Minimum five years of experience developing and implementing large-scale digital tools specifically relating to technology and data solutions in a data-driven environment.
• Advanced working SQL knowledge and experience working with relational databases, including Postgres, query authoring (SQL), and working familiarity with various databases.
• Strong understanding of database systems, distributed computing systems, and enterprise-level data stream-processing technologies such as Kafka. 
• Familiarity with cloud platforms such as Azure, AWS, Google Cloud
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• Experience with object-oriented/object function scripting languages e.g., Java.
• Experience building and optimizing data pipelines, architectures and data sets at large scale.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets and data exchange formats (e.g., Json)
• Experience in manipulating, processing and extracting value from large, disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable data stores.
• Exceptional time management skills and solid interpersonal skills
• Solid technical background with understanding or hands on experience in development and implementation of Information Systems that support health and social protection programs will be an added advantage.
• Experience with Global Goods for digital health such as DHIS, OpenMRS and OpenHIE will be an added advantage.