Senior Data Engineer

Quick Facts

• Work mode: Hybrid from Krakow (3 days/week)

Description

We are looking for an experienced Senior Data Engineer (7+ years) with strong hands-on expertise in Databricks, PySpark, SQL, and Delta Lake. You will also develop and maintain FastAPI-based services, implement real-time streaming with Databricks Structured Streaming, and lead Unity Catalog migration and data governance.

Responsibilities

• Design, develop, enhance, and maintain FastAPI-based services, APIs, and integrations
• Build and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, SQL, and Delta Lake
• Lead and contribute to Unity Catalog migration and broader data modernization initiatives
• Design and implement real-time data processing and streaming solutions using Databricks Structured Streaming
• Perform Spark and Databricks performance tuning to improve scalability, reliability, and processing efficiency
• Integrate and work with enterprise data sources including Oracle, SQL Server, and Informatica
• Ensure solutions follow enterprise standards for data quality, security, governance, scalability, and reliability
• Collaborate closely with business stakeholders, data architects, engineering teams, and vendors
• Provide technical guidance, contribute to solution architecture, and support delivery of complex data engineering initiatives
• Troubleshoot production issues and drive continuous improvement of data platforms and services

Requirements

• 7+ years of experience in data engineering and Python development
• Hands-on expertise in FastAPI, including API development, integration, enhancement, and maintenance
• Extensive hands-on experience with Databricks, PySpark, SQL, and Delta Lake
• Proven experience with Unity Catalog migration and Databricks data governance
• Strong hands-on experience with Databricks Structured Streaming and real-time data processing
• Experience working with Oracle, SQL Server, and Informatica
• Ability to independently design, develop, and implement scalable and high-performance data engineering solutions
• Good understanding of the Banking, Financial Services, and Insurance (BFSI) domain

Preferred / Additional Skills

• Experience with Microsoft Azure Cloud and Databricks administration
• Hands-on experience with CI/CD, Git, Azure DevOps, and/or GitLab
• Strong understanding of Apache Spark performance tuning and optimization
• Experience providing technical leadership, mentoring engineers, and managing stakeholder communication
• Databricks certifications