Data Engineer (Spark)

Quick Facts

• Role: Data Engineer (Spark)

• Focus: High-performance data processing platforms (streaming + batch)

Description

You will develop and maintain large-scale data processing platforms using modern Big Data technologies. The role includes designing and optimizing data pipelines, working with data lake technologies, monitoring and troubleshooting for high availability and accuracy, and leveraging cloud infrastructure for scalable ingestion and processing.

Responsibilities

• Develop and maintain a high-performance platform for automotive data processing with scalability and reliability

• Design and implement data pipelines for large volumes in both streaming and batch modes

• Optimize ingestion, processing, and storage workflows using Spark, Cloudera, and Airflow

• Work with data lake technologies such as Iceberg

• Collaborate with cross-functional teams to integrate diverse data sources

• Monitor and troubleshoot the platform to ensure availability, performance, and accuracy

• Use cloud services (AWS) to manage infrastructure and scale workloads

• Write and maintain high-quality Python (or Java/Scala) code for data processing and automation

Requirements

• At least 3 years of commercial experience implementing, developing, or maintaining Big Data systems and related governance/management processes

• Strong programming skills in Python (or Java/Scala), including clean code and OOP design

• Hands-on with Big Data technologies such as Spark, Cloudera, Airflow, and the data platform ecosystem (e.g., Kafka, NiFi, Docker, Iceberg)

• Strong understanding of dimensional data and data modeling

• Experience implementing and deploying solutions in cloud environments

• Consulting experience with excellent communication and client management; ability to work directly with clients

• Ability to work independently and take ownership of deliverables

• English proficiency (C1)

• Bachelor’s degree in technical or mathematical studies

Benefits

• Supportive AI & Big Data team and international projects

• Flexible work (remote or modern offices/coworking)

• Career growth via knowledge-sharing, sponsored training/conferences (including Databricks and Anthropic training/certifications)

• Team-building events and integration budget

• Medical and sports packages, eye care, and well-being support including psychotherapy and coaching

• Full work equipment (laptop and necessary devices)

• Opportunities to build your personal brand (conferences, blog, meetups)

• Smooth onboarding with a dedicated buddy and an autonomous culture