Senior PySpark Data Engineer – Big Data & Cloud Integration

As a Senior PySpark Data Engineer, you will convert cutting-edge experimental code into robust, production-ready data pipelines, enabling innovative solutions across diverse projects. You will help accelerate data-driven decision-making and technological advancement.

Quick Facts

• Hybrid work model

Responsibilities

• Convert experimental code into scalable, efficient workflows and modules.
• Develop and optimize data pipelines using PySpark and Spark on YARN.
• Collaborate with data scientists and engineers to productionize analytical solutions.
• Ensure code quality, perform performance tuning, and implement data engineering best practices.
• Maintain and improve the performance of data workflows in a large-scale environment.
• Integrate data pipelines within cloud ecosystems, primarily Google Cloud Platform (GCP).
• Support continuous improvement through performance monitoring and optimization.

Requirements

• 5+ years of experience in data engineering and PySpark/Spark development.
• Strong SQL skills and experience with Spark optimization techniques.
• Hands-on experience with the Hadoop ecosystem, especially Spark on YARN.
• Communicative English proficiency.
• Existing legal right to work in the European Union.

Preferred Qualifications

• Knowledge of cloud platforms, preferably GCP.
• Certifications or additional experience in cloud computing or big data solutions.

Application

• Include a CV, preferably in English, and a statement confirming consent to the processing and storage of personal data.