Data Engineer

Quick Facts

• Role: Data Engineer (project-based)

• Focus: Building and scaling an IoT data platform for the insurance domain

Description

You will design, develop, and maintain high-performance data pipelines (ETL/ELT) using Spark, Python, and AWS services. The project includes building a modern data platform for multiple products, integrating external sources (IoT, OEM, partner systems), and orchestrating workflows with CI/CD automation via AWS Step Functions. You will also support ML teams by preparing and delivering data for SageMaker-based AI/ML solutions, while ensuring data quality and optimizing performance and cloud infrastructure costs.

Responsibilities

• Design, develop, and maintain efficient ETL/ELT data pipelines using Spark, Python, and AWS

• Build and evolve a modern data platform supporting multiple products and business areas

• Implement workflow orchestration and automate processes with CI/CD using AWS Step Functions

• Integrate and process data from external sources (IoT, OEM, partner systems)

• Support ML teams in preparing and delivering data for SageMaker-based solutions

• Optimize data processing performance and cloud infrastructure costs

• Ensure data quality through validation, monitoring, and observability

Requirements

• Minimum 4 years of commercial experience as a Data Engineer or similar

• Commercial experience with AWS and technologies: Spark, Python, SQL

• Practical knowledge of Data Mesh architecture and solutions for large-scale data processing

• Hands-on experience building scalable, high-performance data engineering solutions in a cloud-native environment

• Practical knowledge of Amazon SageMaker and ability to prepare environments and data for AI/ML solutions

• Experience with orchestration and automation tools, particularly AWS Step Functions

• Knowledge of AWS Glue, including creating and maintaining Glue Jobs and managing data in Glue Catalog

Benefits

• Project collaboration model with stage-based billing and acceptance protocols

• Administrative and billing support from Connectis

• Occasional coordination meetings in Warsaw (approximately once per quarter)

• Flexible B2B cooperation form

• Fast and remote process

• Participation in industry events and tech meetups to share knowledge and experience