Senior Data Engineer

Quick Facts

Role: Senior Data Engineer (Databricks, PySpark & Python)
Location: Kraków, Poland (Hybrid: 3 days/week in office)
Experience: 7+ years
Domain: Banking, Financial Services & Insurance (BFSI)

Description

We are looking for a hands-on Senior Data Engineer with strong Python development experience and expertise in Databricks, PySpark, SQL, and modern data engineering practices. You will develop FastAPI-based services, build scalable ETL/ELT pipelines, implement real-time streaming solutions, and lead Databricks Unity Catalog migration initiatives. The role requires independent design, development, optimization, and maintenance of enterprise-grade data solutions in the regulated BFSI domain.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, SQL, and Delta Lake.
• Develop and maintain FastAPI-based services, REST APIs, and integrations for enterprise data platforms.
• Implement robust data processing solutions using Databricks, Delta Lake, and lakehouse architecture principles.
• Lead Unity Catalog migration initiatives: data governance, access controls, metadata management, and migration validation.
• Develop and optimize Databricks Structured Streaming for real-time and near-real-time ingestion and processing.
• Integrate and modernize data workflows involving Oracle, Microsoft SQL Server, and Informatica.
• Identify and resolve performance bottlenecks across Spark jobs, SQL queries, streaming workloads, and data pipelines.
• Support automated deployments, CI/CD pipelines, version control, and release management using Azure DevOps, GitLab, and Git.
• Translate business and technical requirements into scalable, secure, reliable, and maintainable solutions.
• Collaborate with architects, business analysts, and stakeholders to deliver high-quality solutions aligned with enterprise standards.
• Ensure compliance with data governance, security, auditability, and applicable banking/financial-services regulatory requirements.

Requirements

• 7+ years of professional experience in data engineering and software development with strong hands-on Python expertise.
• Strong proficiency in Databricks, PySpark, SQL, and Delta Lake.
• Hands-on experience developing and integrating FastAPI-based APIs and REST services.
• Proven experience with Databricks Unity Catalog migrations.
• Strong experience with Databricks Structured Streaming and real-time/near-real-time data processing.
• Practical integration experience with Oracle, Microsoft SQL Server, and Informatica.
• Ability to independently design and implement scalable, high-performance data pipelines and processing solutions.
• Good understanding of data modeling, data integration, ETL/ELT patterns, and data quality practices.
• Experience working in or supporting the Banking, Financial Services, and Insurance (BFSI) domain.
• Strong analytical, troubleshooting, communication, and problem-solving skills.

Benefits

Not specified in the provided posting.