Senior Data Engineer

Quick Facts

• Role: Senior Data Engineer

• Focus: Azure and Databricks data platforms, pipelines, and AI-enabled data discovery

Description

As a Senior Data Engineer, you will design, develop, and optimize complex data solutions and pipelines on Azure and Databricks, taking end-to-end technical ownership. You will write production-grade code in Python, PySpark, and SQL, contribute to Databricks-based platform migration and modernization, and help build AI-enabled discovery experiences using RAG and conversational access to enterprise data. This is a hands-on, senior role solving non-standard problems with strong architectural awareness.

Responsibilities

• Build and optimize complex data pipelines and solutions on Azure and Databricks

• Own engineering work end-to-end: problem understanding, design, implementation, testing, and production deployment

• Implement AI-enabled capabilities for data discovery (e.g., Databricks AI, RAG, conversational interfaces)

• Migrate and modernize existing data platforms into Databricks architectures

• Contribute to governance and scalable data discovery approaches using Unity Catalog

• Improve architecture, performance, scalability, reliability, and maintainability; challenge existing approaches

• Collaborate with Data Engineers, Architects, Product Owners, DataOps, QA, and DevOps

• Move between strategic initiatives as priorities evolve

Requirements

• 8+ years of professional experience in Data Engineering, Software Engineering, or closely related work

• Strong hands-on commercial experience with Azure Databricks

• Very good knowledge of the Azure cloud ecosystem and cloud-based data architectures

• Excellent Python programming skills

• Advanced PySpark and large-scale distributed data processing experience

• Advanced SQL (complex transformations, optimization, analytical/window functions)

• Strong data engineering fundamentals (ETL/ELT, pipelines, distributed processing, production data platforms)

• Ability to independently design and deliver solutions with limited supervision

• Strong troubleshooting and problem-solving skills

• Production-grade practices: version control, testing, CI/CD, monitoring, and deployment

• Clean, maintainable, well-structured code practices

• High ownership, independence, and proactivity

• Excellent English communication (written and spoken)

Benefits

• High-impact, strategic data initiatives instead of repetitive business-as-usual work

• Significant technical ownership and freedom in how problems are solved

• Hands-on work with Azure, Databricks, Python, PySpark, SQL, and emerging AI capabilities

• Exposure to modern AI and RAG approaches for enterprise data management and discovery

• Opportunity to influence technical architecture and engineering practices

• Collaboration with experienced engineers and architects across an international technology organization