Spark / Data Lakehouse Developer

Quick Facts

• Role: Spark / Data Lakehouse Developer

• Team focus: Data Platform / Data Engineering (Lakehouse)

Description

We are looking for an experienced Spark / Data Lakehouse Developer to build and evolve data engineering solutions for data reconciliation and a Data Lakehouse architecture. The role covers Spark-based pipelines, MongoDB integration, and Apache Iceberg, including creation of Lakehouse layers (raw/bronze/silver/gold). You will also work on Spark performance optimization and data quality control for large datasets.

Responsibilities

• Develop Data Engineering solutions for a Data Lakehouse platform
• Design and implement Lakehouse layers: raw/bronze/silver/gold
• Integrate MongoDB and support reconciliation and data quality checks
• Optimize Spark jobs using techniques such as partitioning and caching
• Work with CI/CD and monitoring tools (Jenkins, Prometheus, Grafana)

Requirements

• Minimum 4 years of experience with Apache Spark
• Very good knowledge of PySpark and Spark SQL
• Experience with MongoDB and MongoDB Spark Connector
• Practical knowledge of Apache Iceberg
• Experience building Data Lakehouse layers (raw/bronze/silver/gold)
• Knowledge of CI/CD and Jenkins
• Knowledge of Prometheus and Grafana
• Experience with Scrum/Kanban and working with Jira/Confluence
• Ability to optimize Spark processes
• Experience in data reconciliation and data quality control
• Knowledge of partitioning, caching, broadcast joins, and AQE
• Experience working with large datasets
• Independence and analytical approach to problem-solving
• Ability to collaborate with Data, QA, and DevOps teams

Benefits

• Work with a modern Data Lakehouse platform
• Participation in interesting projects involving large datasets
• Work with a modern stack: Spark, MongoDB, Iceberg, Kubernetes, Prometheus, Grafana
• Opportunity to grow and influence technological solutions
• Collaborate with an experienced team