Senior Azure Databricks Data Engineer

Overview of the Role

As a Senior Azure Databricks Data Engineer, you will design, develop, and optimize large-scale data platforms and modern data pipelines on Microsoft Azure. The role supports enterprise data analytics, reporting, and business intelligence initiatives by delivering scalable, high-performance data solutions.

Detailed Responsibilities

• Design, develop, and maintain robust data pipelines using Azure Databricks and PySpark for complex enterprise data requirements.
• Build scalable ETL/ELT solutions for large-scale, heterogeneous data ecosystems across multiple business domains.
• Analyze business requirements and translate them into sustainable data engineering solutions.
• Develop and optimize data transformation frameworks and workflows for performance and reliability.
• Perform performance tuning, troubleshooting, and optimization of Databricks workloads and SQL queries.
• Architect and implement data ingestion, transformation, and integration processes across cloud and hybrid environments.
• Collaborate with solution architects, business analysts, and stakeholders to deliver end-to-end data platform solutions.
• Implement best practices for data quality, governance, security, and operational excellence.
• Support CI/CD, DevOps, and cloud deployment activities for automated data delivery.
• Mentor junior engineers and contribute to technical leadership.

Skill Requirements

• 8–12 years of experience in Data Engineering and ETL Development.
• At least 7 years of hands-on experience with Azure Databricks.
• Strong proficiency in Python and PySpark frameworks and libraries.
• At least 7 years of advanced SQL experience, including query optimization and performance tuning.
• At least 7 years designing and implementing complex ETL processes and data workflows.
• Deep understanding of data transformation frameworks and industry-standard patterns.
• Proven expertise in enterprise data ecosystems, including data storage, processing, visualization, and integration architectures.
• Experience processing large-scale structured and unstructured datasets.
• Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related discipline.

Other Requirements

• Azure Data Factory expertise is a plus.
• Ability to read and analyze database execution plans.
• Experience with Azure DevOps and CI/CD cloud implementations.
• Familiarity with Configuration Management, Data Lake, Delta Lake, Synapse Analytics, and modern cloud data architectures.
• Azure certifications are highly preferred.
• Strong analytical, problem-solving, and communication skills.
• Experience in global delivery and customer-facing environments.
• Demonstrated ability to mentor junior engineers and drive technical excellence.
• Ownership mindset with a focus on quality and continuous improvement.

Career Development Opportunities

• Global learning and development programs focused on next-generation technologies and leadership skills.
• Opportunities for promotion to Lead Data Engineer, Data Architect, or Solution Architect roles.
• Participation in industry conferences, certification programs, and specialized technical training.
• Involvement in high-impact projects with Fortune 500 clients and exposure to cutting-edge solutions.
• A culture that values innovation, continuous improvement, and high performance.

Benefits

• Life insurance
• Private medical care
• MultiSport Card
• Subsidy for glasses
• Subsidy to language courses
• Christmas and holiday bonuses