Senior Data Engineer (Spark)

Quick Facts

• Role: Senior Data Engineer (Spark)

Description

You will work on challenging, enterprise-grade data engineering projects across industries, including automotive, aerospace, and telecommunications. The focus is on building and maintaining scalable data processing platforms supporting both streaming and batch workloads. Responsibilities include designing pipelines, optimizing ingestion/processing/storage, monitoring platform health, and writing production-quality code.

Responsibilities

• Develop and maintain high-performance data processing for automotive data with scalability and reliability

• Design and implement streaming and batch data pipelines for large data volumes

• Optimize data workflows for efficient ingestion, processing, and storage using Spark, Cloudera, and Airflow

• Use data lake technologies (e.g., Iceberg) to manage structured and unstructured data

• Collaborate with cross-functional teams to understand data requirements and integrate sources

• Monitor and troubleshoot the platform to ensure availability, performance, and processing accuracy

• Use cloud services (AWS) to manage infrastructure and scale workloads

• Write and maintain high-quality Python (or Java/Scala) code for data processing and automation

Requirements

• At least 3 years of commercial experience implementing, developing, or maintaining Big Data systems and data governance/management processes

• Strong programming skills in Python (or Java/Scala), including clean code and OOP design

• Hands-on experience with Big Data technologies such as Spark, Cloudera, Airflow, Kafka, NiFi, Docker, and Iceberg

• Strong understanding of dimensional data and data modeling

• Experience implementing and deploying cloud-based solutions

• Consulting experience with strong communication and client management skills (client-facing preferred)

• Ability to work independently and take ownership of deliverables

• English proficiency at C1 level or higher

• Bachelor’s degree in technical or mathematical studies

Benefits

• Supportive team working on AI & Big Data projects

• Opportunity to work with global enterprises and startups on international projects

• Flexible work arrangements (remote or modern offices/coworking)

• Career growth with knowledge sharing, language classes, and sponsored training/conferences (Databricks and Anthropic partnership)

• Team-building events and integration budget

• Medical and sports packages, eye care, and wellbeing support (including psychotherapy/coaching)

• Full work equipment (e.g., laptop and necessary devices)

• Opportunities to build a personal brand via conferences, blogging, and meetups

• Smooth onboarding with a dedicated buddy and an autonomous culture