Data Engineer (Energy & IoT Systems) at PowerLabs Inc

Role Overview:

• As a Data Engineer, you will design, implement, and maintain data ingestion, storage, optimization processing (pre/post), and analytics pipelines for time-series IoT and energy data. You will work closely with data scientists/analysts, software engineers, and optimization experts to ensure seamless integration of real-time and batch data workflows.

Key Responsibilities:

• Design and implement ETL/ELT pipelines for ingesting data from IoT sensors, third-party APIs, and industrial protocols (MODBUS, OPC-UA, MQTT).
• Develop real-time and batch data processing systems using Kafka, RabbitMQ, Apache Flink, and Spark/PySpark.
• Optimize storage and retrieval of high-frequency time-series data using TimescaleDB, InfluxDB, or DuckDB.
• Design and maintain data lake and warehouse solutions (S3 + Apache Iceberg, Delta Lake, or BigQuery).
• Build and maintain pre-processing and post-processing pipelines for optimization workflows, ensuring seamless integration with forecasting and dispatch models.
• Design, develop, and maintain robust APIs to expose processed data and insights for applications, dashboards, and external integrations.
• Work closely with backend and DevOps teams to ensure seamless API deployment, scalability, and security.
• Work with cloud/DevOps engineers to implement monitoring and alerting systems to ensure high availability and fault tolerance.
• Work with machine learning and optimization teams to provide clean and structured datasets for forecasting and energy dispatch models.
• Ensure data security and compliance with industry standards.
• Automate and orchestrate workflows using Apache Airflow, Terraform, Lambda etc.

Required Skills & Qualifications:

• 3+ years of experience in data engineering with a focus on IoT, energy, or industrial data.
• Proficiency in Python and SQL; experience with Scala is a plus.
• Experience building scalable ETL/ELT pipelines using Apache Airflow, NiFi, Prefect etc.
• Strong knowledge of distributed data processing frameworks (Dask, Spark, Flink, or Kafka Streams).
• Hands-on experience with both relational and NoSQL databases, including PostgreSQL, MySQL, Cassandra, DynamoDB, or MongoDB.
• Hands-on experience with time-series databases (TimescaleDB, InfluxDB, ClickHouse, or DuckDB).
• Familiarity with data lake architectures and formats (Apache Iceberg, Delta Lake, Parquet).
• Experience with cloud data services (AWS: S3, Glue, Lambda, Athena; GCP: BigQuery, Dataflow).
• Ability to optimize SQL queries and database performance.
• Experience in MLOps and data pipelines for ML workflows.
• Knowledge of energy systems, SCADA protocols, or grid optimization is a plus.
• Strong problem-solving skills and ability to work in cross-functional teams.

Nice-to-Have:

• Experience with real-time stream processing (Flink, Kafka Streams).
• Familiarity with Pyomo, Gurobi, or other optimization frameworks for LPs and MIPs.
• Previous experience in industrial automation or energy sector projects.