Team Lead, Data Engineer at Interswitch

Department: Purepay - Transaction Switching & Payment Processing

Job Description

• Design, implement and maintain scalable and highly available data pipelines on large volume data sets, that will enable impactful insights & strategy for our product and services.

Responsibilities

• Data Engineering.
• Work closely with data analyst, product team, business lines, engineering, data science, and other data analytics teams to understand requirements/goals and provide viable and innovative data solutions
• Translate business requirements by business team into data engineering specifications.
• Ensure all company data is available and accessible in a central location.
• Build scalable data marts from the available raw data that can be used to derive business metrics/insights as specified by stakeholders • Work hand in hand with the DevOps team and develop monitoring and alerting scripts on various data pipelines and jobs.
• Process, clean and validate the integrity of data to be used for analytics purposes.
• Work with engineering and product team to ensure that products are designed to properly capture required data fields.
• Expand and optimize data and data pipeline architecture.
• Collaborate with data analyst teams to incorporate 3rd party data assets to augment internal data.
• Effectively plan tasks across multiple projects simultaneously with different requirements, needs, and priorities.
• Provide Support and trainings for data usage.
• Mentor team member and provide leadership.

Education and Experience
Academic Qualification(s):

• Good First Degree in Computer Science / Statistics / related field.

Experience (Number of relevant years):

• Minimum of 5 years in a data engineering role with at least one of those years managing a team.

Key Skills Required Technical:

• Big Data Concepts.
• Big Data Technology.
• Data Modeling
• ETL & ELT tools (SSIS, Spark, Sqoop, Apache Airflow).
• Programing Language (SQL, Python/Scala).
• Database Management (MS SQL, Mongo, Casandra, MaprDBMaprFS)
• Machine Learning (Data Preprocessing).
• Realtime Data processing (Kafka).
• Cloud Services (AWS / Azure).
• Problem Analysis

Requirement Analysis Soft Skills:

• People management
• Innovation & creativity
• Customer Relationship Management
• Work under pressure
• Communication (Verbal & Written)
• Attention to details