Senior Big Data Engineer

Quick Facts

• Role: Senior Big Data Engineer

Description

Build and optimize scalable reporting pipelines to process massive datasets. Design, code, and verify distributed data jobs in Scala and Apache Spark, ensuring accurate statistical algorithms for stakeholders. You will also own streaming, workflow orchestration, and data modeling across multiple database engines while integrating Kubernetes and DevOps/CI-CD into the data lifecycle.

Responsibilities

• Design, develop, and optimize scalable big data processing pipelines using Scala and Apache Spark
• Implement and verify complex data jobs to ensure reliable algorithmic metrics
• Use Kubernetes to manage and optimize how Spark runs on cluster infrastructure
• Orchestrate scheduling and complex data workflows with Apache Airflow
• Build and maintain data streaming architectures using Spark Streaming and Apache Kafka
• Collaborate with an international distributed team on big data engineering
• Design and query complex data models across Oracle, Postgres, Teradata, and Cassandra
• Integrate DevOps and CI/CD practices into the data lifecycle

Requirements

• Strong software development experience with Scala and deep proficiency in Apache Spark (DataFrame/SQL API)
• Practical experience with Spark Streaming for real-time processing
• Kubernetes knowledge, specifically running Spark on Kube clusters
• Hands-on experience with distributed storage and data lakehouse formats such as Hive and Iceberg
• Proven experience with Apache Airflow workflow orchestration
• Strong SQL with advanced SparkSQL knowledge (window functions, complex group-by)
• Solid understanding of distributed computing patterns and hands-on experience with Hadoop

Benefits

• Competitive salary
• Flexible schedule
• Medical insurance and sports benefits
• Opportunity to work on cutting-edge projects
• Professional development opportunities
• Well-equipped office
• Corporate social events