Data Engineer - Big Data at Propel

Description

• Our client is looking for Talent to join the Business Intelligence & Analytics team, you will be in charge of the development distributed computing tasks using Big Data technologies such as Hadoop, NoSQL and other distributed environment technologies based on our needs.
• You will also analyze, designing, programming, debugging and modifying software enhancements and new products used in distributed, large-scale analytics solutions.

What You’ll Do

• Design scalable, end -to-end process to consume and integrate large volume, complex data from sources such as Hive, Flume, Kafka or Storm;
• Provide Data Engineering expertise to multiple teams across our organization.
• Provide guidance to software engineers with industry and internal data best practices;
• Research and implement new data technologies; Work with other teams to understand needs and provide solutions;
• Work with the Business Intelligence development team on migration and improve existing SQL Serverbased ETLs to Map Reduce and Hive (Cloud) technology to achieve scale and performance; Help develop new processes on the data warehouse platform and work with Data Scientists to transform big data into model-- ready forms to support projects.

Requirements

• Experienced in working with large datasets (both structured and unstructured) using technologies such as MapReduce, Hadoop, HBase, Hive, Spark and NoSQL technologies;
• Experienced in programming languages (Java, C++, or Python);
• Knowledgeable in distributed systems; Experienced with real-time analysis of sensor and other data from Internet of Things (IoTs) or other connected devices;
• An excellent in grasping of algorithmic concepts in computer science (e.g., sorting, data structures);
• Experienced in the development and release of enterprise scale applications;
• Experienced with version control.