Senior Data Engineer / Tech Lead (Spark)

Quick Facts

• Role: Senior Data Engineer / Tech Lead (Spark)

Description

You will design, develop, and maintain scalable data pipelines and modern data processing solutions for production-grade lakehouse systems. The role combines hands-on engineering with ~30% technical leadership, including technical ownership and contributions to architectural decisions. You will also collaborate with stakeholders to translate business and technical requirements into reliable pipelines.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Python, SQL, Spark, and Airflow

• Build and improve data processing solutions using Apache Iceberg and modern lakehouse architectures

• Work with PyArrow, PyIceberg, PostgreSQL, and Trino

• Take technical ownership of selected areas and help drive technical/architectural decisions

• Combine development with leadership, knowledge sharing, and engineering best practices

• Build and maintain containerized workloads with Docker and Kubernetes

• Ensure pipelines are scalable, reliable, maintainable, and production-ready

• Collaborate to translate requirements into effective solutions

• Contribute to code quality, testing, CI/CD, and automation

• Monitor, troubleshoot, and continuously improve pipelines and platform components

• Apply modern development practices (including AI-assisted development) to improve engineering efficiency

Requirements

• At least 4 years of commercial experience in Data Engineering or Big Data

• Strong programming skills in Python

• Very good knowledge of SQL

• Strong hands-on experience with Apache Spark and distributed data processing

• Practical experience with workflow orchestration tools (preferably Apache Airflow)

• Experience with Apache Iceberg and/or modern lakehouse architectures

• Experience working with Docker and Kubernetes

• Good understanding of data modeling, data pipelines, and data processing architectures

• Experience with relational databases and/or query engines such as PostgreSQL or Trino

• General understanding of cloud environments and cloud-based data solutions

• Ability to take ownership of technical solutions and contribute to architectural/technical decisions

• Experience supporting other engineers via guidance or mentoring

• Ability to work independently and own deliverables

• Strong communication and collaboration skills

• Fluent English (at least C1)

• Bachelor’s degree in technical or mathematical studies

Benefits

• Supportive AI & Big Data-focused team and international projects

• Flexible work arrangements (remote or modern offices/coworking spaces)

• Career growth via learning paths, language classes, and sponsored training/conferences (including Databricks and Anthropic training/certifications)

• Team-building events and integration budget

• Medical and sports packages, eye care, well-being support (including psychotherapy and coaching)

• Full work equipment (laptop and required devices)

• Opportunities to build a personal brand by speaking, writing, and attending meetups

• Smooth onboarding with a dedicated buddy and an autonomous culture