Data Engineer (Azure)

We’re seeking a self-motivated Data Engineer with strong Python/PySpark skills to join the Data Engineering Team and help build the Azure Data Analytics Platform. The ideal candidate is an independent, collaborative team player who leads projects, identifies process gaps, and continuously develops expertise in Human Capital technology.

The role involves developing reusable, metadata-driven data pipelines, automating platform processes, building data integrations, extending ETL libraries, writing unit tests, creating Databricks monitoring solutions, proactively resolving ETL issues, collaborating on cloud resources, updating documentation, conducting code reviews, and enhancing platform architecture.

Quick Facts

• Stack: Python/PySpark, SQL, Databricks Spark, knowledge of Azure cloud-native solutions
• Salary: 130–150 PLN net/h on B2B
• Working model: 100% remote

Recruitment Process

• Call with a recruiter (30 min)
• Online interview with a technical case (1.5h)

Responsibilities

• Build reusable, metadata-driven data pipelines.
• Automate and optimize data platform processes.
• Develop integrations with data sources and consumers.
• Extend shared ETL libraries with transformation methods.
• Write unit tests.
• Create monitoring solutions for the Databricks platform.
• Proactively address ETL performance and quality issues.
• Collaborate with infrastructure teams on cloud resources.
• Update data platform wiki and documentation.
• Conduct code reviews to ensure quality.
• Initiate and implement architecture improvements.

Requirements

• Strong experience with Python/PySpark and SQL.
• Hands-on experience building robust data pipelines using Databricks Spark.
• Experience processing large-scale datasets in production environments.
• Strong knowledge of Kafka or other streaming platforms such as Azure Event Hubs or Amazon Kinesis.
• Experience building streaming data integrations and event-driven data pipelines.
• Good understanding of Spark Structured Streaming.
• Familiarity with CDC (Change Data Capture) concepts; experience with Debezium is a plus.
• Strong knowledge of Databricks Delta optimization, including partitioning, Z-ordering, and compaction.
• Experience developing reusable Python libraries and packages.
• Hands-on experience with CI/CD pipelines.
• Good understanding of networking fundamentals.
• Familiarity with Agile/Scrum methodologies.

What We Offer

• 100% remote work model.
• Superior co-working and personal development experience in an international setting.