Cloud Data Engineer at Datamellon

Responsibilities:

• Design, develop, and maintain cloud-based data pipelines and data processing systems.
• Collaborate with stakeholders to gather requirements and define data engineering solutions.
• Implement data integration and ETL processes to extract, transform, and load data from various sources.
• Ensure data quality, integrity, and consistency across different data sets.
• Optimise and fine-tune data processing and data storage systems for performance and scalability.
• Monitor and troubleshoot data pipelines and resolve any issues that arise.
• Collaborate with cross-functional teams to implement data governance and security measures.

Requirements:

• Minimum of two years of experience in Cloud Data engineering, Machine Learning and Visualisation.
• Proficiency in AWS Boto3 Python SDK
• Experience with big data processing frameworks such as Apache Spark or Hadoop.
• Familiarity with data modelling, data warehousing, and database design principles.
• Knowledge of Cloud Warehousing Solution such as Amazon Redshift and Google BigQuery
• Experience with ETL tools such as Amazon Glue
• Knowledge of SQL and NoSQL databases.
• Understanding of data governance, data security, and privacy best practices.
• Knowledge of data visualisation tools (e.g., Data Studio, QuickSight, Tableau, Power BI).
• Relevant Solution Architecture, Data engineering and Machine learning Certification,
• Experience with containerisation technologies (e.g., Docker, Kubernetes).