Data Engineer (Python, Apache Spark, Hadoop)

Quick Facts

Hybrid work (Kraków) — 2 days per week in the client office.

Description

Data Engineer role for an international banking project. You will build scalable cloud ETL pipelines on GCP and design data validation and data quality frameworks for business-critical data, ensuring schemas, transformations, and business rules are verified before production.

Responsibilities

• Develop data processing pipelines in the cloud (GCP) and Big Data environments
• Design and optimize Spark/PySpark jobs running on GCP Dataproc clusters and within the Hadoop ecosystem
• Build validation frameworks in PySpark/Python to verify schemas, transformations, and business rules before release to production
• Implement automated tests and validation rules (Data Quality) using Python/PySpark
• Ingest and transfer data from sources into a cloud data warehouse (BigQuery)
• Automate deployments and schedule tasks using Airflow and Jenkins

Requirements

• Minimum 4 years of experience in data engineering in distributed environments
• Advanced Python with a focus on OOP (design patterns, clean modular code)
• Practical experience with PySpark & Apache Spark (job optimization, partitioning, and cluster monitoring)
• Experience with the Hadoop ecosystem and working with large data volumes
• Practical knowledge of Dataproc, BigQuery, and Cloud Storage
• Ability to build automated tests at the Spark/Python boundary; experience with Airflow and CI/CD using Jenkins
• English proficiency at least B2 for daily communication
• Nice to have: Experience in the banking or finance domain

Benefits

• Job stability: long-term projects and opportunities to develop across different IT areas
• Flexible cooperation models: B2B, employment contract, or assignment
• Benefits: Medicover (extended with dentistry), FitProfit, and English lessons
• F45 gym training sessions at the office with professional coaches
• Internal referral program
• Funding for IT certifications
• Depending on the project: hybrid, remote, or onsite work
• Company events and integration trips; support for charitable initiatives