Senior Data Engineer

Quick Facts

• Senior Data Engineer focused on enterprise big data, lakehouse, and scalable data pipelines.
• Requires 10+ years of software/data engineering experience and strong technical leadership.
• Strong hands-on Databricks expertise (Python, Apache Spark, Delta Lake, Delta Live Tables).

Description

Lead data engineering initiatives, architecting and building high-throughput, fault-tolerant batch and real-time streaming pipelines. Emphasize low-latency processing, high data availability, and horizontal scalability while working across data warehouse, data lake, and lakehouse architectures. Support production delivery through CI/CD, automated testing, and modern engineering practices.

Responsibilities

• Architect and implement enterprise big data solutions across Data Warehouses, Data Lakes, and Lakehouse architectures.
• Build and optimize high-throughput, fault-tolerant data pipelines for massive datasets (batch and streaming).
• Deliver system designs with non-functional requirements such as low latency, high availability, and scalability.
• Apply cloud and automation practices, including CI/CD and automated test suites.

Requirements

• Master’s (or equivalent) degree in Computer Science, Engineering, or related technical discipline.
• 10+ years of progressive software/data engineering experience with technical leadership.
• 5+ years hands-on Databricks ecosystem expertise using Python, Apache Spark, Delta Lake, and Delta Live Tables.
• Proven track record architecting enterprise Big Data solutions across data warehouse, data lake, and lakehouse.
• Extensive experience with high-throughput, fault-tolerant pipelines for batch and real-time streaming.
• Strong architectural background for low-latency processing, high data availability, and horizontal scalability.
• Solid database engineering skills: SQL, PostgreSQL, Azure SQL, PL/SQL.
• Deep familiarity with cloud platforms (preferably Azure), Linux/Unix, and shell scripting.
• Production experience establishing robust CI/CD workflows and automated test suites using GitLab.