Senior Data Engineer (Spark)

Quick Facts

• Role: Senior Data Engineer (Spark)

• Focus: Building scalable data processing platforms for streaming and batch workloads

• Tech scope: Spark/Cloudera/Airflow/Iceberg plus cloud and reporting stacks

Description

You will develop and maintain high-performance data processing platforms, designing pipelines for large volumes of data in both streaming and batch modes. The role includes optimizing ingestion, processing, and storage workflows, managing data lake technologies, and ensuring high availability and accuracy. You will also write production-grade code in Python (or Java/Scala) and work with cloud infrastructure to scale workloads.

Responsibilities

• Develop and maintain a scalable automotive data processing platform.

• Design and implement streaming and batch data pipelines.

• Optimize data workflows for efficient ingestion, processing, and storage.

• Work with data lake technologies such as Iceberg.

• Collaborate with cross-functional teams to integrate data sources.

• Monitor, troubleshoot, and ensure performance, availability, and accuracy.

• Use cloud services (AWS) to manage infrastructure and scale workloads.

• Write and maintain high-quality Python (or Java/Scala) code for data processing and automation.

Requirements

• At least 3 years of commercial experience implementing, developing, or maintaining Big Data systems and data governance/management processes.

• Strong programming skills in Python (or Java/Scala), including clean code and OOP design.

• Hands-on experience with Big Data technologies such as Spark, Cloudera, Airflow, Kafka, NiFi, Docker, and Iceberg.

• Excellent understanding of dimensional data and data modeling techniques.

• Experience implementing and deploying solutions in cloud environments.

• Consulting experience with strong communication and client management skills.

• Ability to work independently and take ownership of deliverables.

• Fluent English (C1 or higher).

• Bachelor’s degree in technical or mathematical studies.

Benefits

• Supportive team and international projects with global enterprises and startups.

• Flexible work arrangements (remote or modern offices/coworking spaces).

• Career growth through knowledge-sharing, language classes, and sponsored training/conferences (including Databricks and Anthropic training/certifications).

• Team-building events and integration budget; celebrations for milestones.

• Medical and sports packages, eye care, and well-being support (including psychotherapy and coaching).

• Full work equipment (laptop and necessary devices).

• Opportunities to build your personal brand via conferences, blogging, and meetups.

• Smooth onboarding with a dedicated buddy and a friendly, autonomous culture.