Senior Data Engineer

Quick Facts

Role: Senior Data Engineer responsible for designing and developing a scalable Lakehouse-based Data Platform.

Description

Build and evolve a Lakehouse Data Platform with a strong focus on data architecture standards, ELT pipelines, and analytical layers for reporting, analytics, and AI/ML. The role combines advanced data engineering with technical leadership to define the platform direction and ensure quality, security, and compliance.

Responsibilities

• Design and develop the Data Platform in a Lakehouse architecture.
• Contribute to architectural decisions about the Data Platform and data integration.
• Design and develop ELT processes for reporting, analytics, and AI/ML.
• Create data models and analytical layers.
• Optimize solutions for performance, cost, and scalability.
• Define Data Engineering standards, automation, and best practices.
• Ensure data quality, security, and data compliance.
• Collaborate with business, architects, and IT teams during solution design.
• Act as the technical leader in Data Platform initiatives.

Requirements

• At least 3 years of experience in Data Engineering.
• Experience designing and maintaining production data pipelines.
• Strong knowledge of Python, PySpark, SQL, Terraform, YAML, and Databricks Asset Bundles.
• Experience with Apache Spark, Databricks, Delta Lake, and large-scale data volumes.
• Strong knowledge of Microsoft Azure, especially: Azure Data Factory, ADLS, Azure Databricks, Unity Catalog, Azure DevOps, Azure Event Hub.
• Experience with Apache Airflow, CI/CD (Azure DevOps Repos & Pipelines), and Lakehouse/Medallion Architecture.
• Knowledge of Data Quality, Data Governance, data modeling, and access control in Unity Catalog.
• Ability to translate business requirements into technical solutions.
• Experience with code review, testing, deployment automation, and documentation.
• Willingness to serve as a technical leader and support the team with standards and best practices.

Benefits

Not specified in the provided posting.