Senior Azure Data Engineer

Quick Facts

• Role: Senior Azure Data Engineer

• Focus: Scalable Azure-based data platform for predictive maintenance (aircraft engines)

• Work mode: Fully remote

Description

You will help build and maintain a scalable data platform on Microsoft Azure to support predictive maintenance for aircraft engines. The work involves designing and optimizing data pipelines for large-scale operational and technical datasets using Azure Databricks, Apache Spark, Python, and SQL, along with CI/CD and data quality controls. The role emphasizes performance, stability, and clear technical documentation.

Responsibilities

• Design, develop, and maintain scalable data pipelines in Azure Databricks using Apache Spark and Python with SQL

• Process and transform large volumes of operational and technical data

• Develop and optimize ETL/ELT processes

• Work with Azure services for data processing and data storage

• Optimize performance of Spark-based workloads

• Implement data validation mechanisms and quality controls

• Collaborate with Data Engineers, Tech Leads, and other stakeholders to translate business/technical requirements into solutions

• Troubleshoot data processing issues and improve pipeline stability

• Create and update technical documentation

• Set up and run CI/CD for the Databricks environment using Azure tools and Terraform/Databricks Asset Bundles

• Maintain and evolve the existing data platform

Requirements

• Minimum 7 years of IT experience, including 5 years as an Azure Data Engineer

• Commercial experience in Data Engineering

• Very strong Databricks experience

• Experience with Apache Spark/PySpark

• Strong Python knowledge

• Strong SQL knowledge

• Experience designing and developing ETL/ELT processes

• Experience with Azure data processing and storage services

• Experience implementing CI/CD for Databricks applications

• Knowledge of Terraform and/or Databricks Asset Bundles

• Knowledge of Big Data processing standards and tools

• English B2–C1 for comfortable communication in an international environment

• Availability ASAP; start within up to 30 calendar days after the decision

Benefits

• Private medical care

• Multisport card

• Ongoing remote work

• Modern equipment provided (hardware, software, configuration)

• Long-term engagement; compensation 130–160 PLN net/hour + VAT

• Support for setting up and running a business (for candidates without existing business experience)