Senior Databricks Engineer

Quick Facts

• Role: (Senior) Databricks Engineer

• Focus: End-to-end lakehouse platform engineering on Databricks (platform, pipelines, data products, operations)

Description

You will build Databricks lakehouse platforms for clients end to end—from workspace/platform setup and ingestion to pipelines, data models, analytics and machine learning data products, and ongoing operations. You own pipelines, data products, and parts of the platform within a workstream; at senior level you lead the technical design, review other engineers’ work, and act as the point of contact for client architects. You’ll also support migration of workloads from legacy warehouses, Hadoop, or notebooks to Databricks, and ensure performance and cost optimization.

Responsibilities

• Set up and configure Databricks platforms (workspaces, compute policies, Unity Catalog basics, networking and identity); at senior level own platform design and cost model

• Build ingestion and pipelines using Lakeflow Connect/Auto Loader; batch and streaming Lakeflow pipelines; Spark and SQL transformations; Delta Lake medallion model

• Model data for consumption: data products, Databricks SQL, AI/BI dashboards, feature tables, and model serving with MLflow; at senior level define modeling approach

• Migrate and refactor legacy workloads into Databricks

• Engineer with software practices: Git, CI/CD with Databricks Asset Bundles, infrastructure as code with Terraform, automated tests, code review; at senior level run reviews and own release process

• Run and optimize: monitoring, job orchestration, performance tuning, cost control (cluster sizing, serverless, Photon, query tuning)

• Support governance layer when included: data quality expectations, lineage, access requests in Unity Catalog

• At senior level: estimate and plan workstreams, support bids, coach engineers, and keep certifications current

Requirements

• Engineer: 3+ years in data engineering with at least 1 year hands-on Databricks in production

• Senior Engineer: 5+ years in data engineering with at least 3 years on Databricks; hands-on experience across at least three of: platform setup, batch/streaming pipelines, data modeling, migration, CI/CD & infrastructure as code, performance/cost tuning

• Demonstrated ownership/leadership of technical design for a workstream

• Strong Spark (PySpark or Scala) and SQL; Python; Delta Lake

• Lakeflow pipelines and jobs; Databricks SQL

• Cloud knowledge of Azure (preferred) or AWS/GCP covering storage, identity, and networking for Databricks

• Git and CI/CD; experience with Databricks Asset Bundles and/or Terraform

• Databricks Certified Data Engineer Associate held or in progress (Professional for senior)

• Unity Catalog basics: catalogs, permissions, lineage; interest in data governance preferred

• Clear communication in English; ability (senior) to present technical design to client architects

Benefits

• Yearly financial bonus

• Private medical care with additional packages (e.g., dental, senior care, oncology)

• Life insurance and access to NAIS benefit platform

• Access to 70+ training tracks and certification opportunities (NEXT platform), plus free Education First language platform, TED Talks, and Udemy Business

• Hybrid working model and home office package (laptop, monitor, chair)