Senior Data Engineer

Quick Facts

• Role: Senior Data Engineer

• Focus: Data engineering on Microsoft Azure (Databricks, Synapse, Lakehouse)

Description

You will develop and maintain modern data engineering solutions in Azure, including Databricks, Synapse Analytics, and a Data Lakehouse architecture. The role includes designing and optimizing ETL/ELT flows, implementing data processing with Python/SQL/Spark, and evolving platforms using Delta Live Tables and Unity Catalog. You will also support CI/CD automation, monitoring/logging, production reliability, performance optimization, and data governance.

Responsibilities

• Develop and maintain data engineering solutions on Azure (Databricks, Synapse Analytics, Lakehouse)

• Design, implement, and optimize ETL/ELT pipelines for integrating and processing data from multiple sources

• Implement data processing using Python, SQL, and Apache Spark

• Build and maintain Databricks data pipelines/workflows and solutions based on Delta Live Tables

• Participate in platform modernization, technology migrations, and Unity Catalog implementation

• Create metadata-driven processing frameworks and solutions supporting process orchestration

• Collaborate with data architects, platform teams, and analysts to deliver business-aligned solutions within architectural standards

• Ensure data quality, security, consistency, availability, and overall platform reliability

• Analyze and resolve production issues, identify root causes, and implement improvements

• Develop CI/CD processes and automate deployments, infrastructure, and operational tasks

• Configure and evolve monitoring, logging, and control mechanisms for data processes

• Optimize processing performance and resource/cost usage

• Participate in technical reviews, develop engineering standards, and implement data governance best practices

• Support the team via knowledge sharing and technical mentoring

• Deliver scalable data solutions supporting reporting, analytics, and AI/ML initiatives

Requirements

• Practical knowledge of Apache Spark, Python, and SQL

• Experience with Microsoft Azure, especially Azure Databricks, Azure Data Factory, and Azure Data Lake

• Knowledge of Azure Synapse Analytics and Azure Data Explorer

• Experience designing, developing, and optimizing ETL/ELT processes

• Knowledge of Data Lakehouse architecture and experience with Databricks solutions

• Practical knowledge of Delta Live Tables (DLT) and Unity Catalog

• Experience implementing CI/CD with Azure DevOps

• Knowledge of infrastructure automation, monitoring, and logging

• Ability to optimize performance, ensure data quality, and troubleshoot issues in production environments

• Ability to collaborate with technical and business teams on data projects

Benefits

Not specified in the provided posting.