Senior Databricks Engineer (Governance)

Quick Facts

• Role: Senior Databricks Engineer (Governance)

• Focus: Unity Catalog governance, access control, audit/lineage evidence, data quality, and governed AI assets

Description

You will own the governance layer of Databricks platforms built for clients. You design and implement Unity Catalog (metastore/catalog structure, access models, lineage and audit evidence), build data quality capabilities in pipelines, and govern AI assets using Unity Catalog, MLflow approvals, and AI Gateway policies. You also implement sovereignty/residency controls, migrate from Hive metastore and legacy permission models, and create reusable governance assets as code.

Responsibilities

• Design and implement Unity Catalog: metastore/catalog structure, workspace binding, external locations/storage credentials, identity federation (SCIM, service principals), ownership model

• Build and document access control: groups/privileges, row filters, column masks, tag-based attribute policies, data classification

• Produce governance evidence from the platform: lineage, audit logs, billing from system tables, access reviews, dashboards; integrate with Purview/Collibra when used

• Implement data quality in pipelines: pipeline expectations, Lakehouse Monitoring, quality metrics published to the catalog and governance team

• Govern AI assets: register models/functions in Unity Catalog, MLflow approval workflows, AI Gateway policies for LLM endpoints, traceability logging

• Implement sovereignty and residency controls: region placement, customer-managed keys, private connectivity, Delta Sharing and Lakehouse Federation configuration

• Migrate clients from Hive metastore/legacy permissions to Unity Catalog and clean up what governance cannot see

• Build reusable governance-by-design assets as code (Terraform, Databricks Asset Bundles), access policy patterns, evidence dashboards

Requirements

• 5+ years of hands-on Databricks engineering in production

• 2+ years designing and administering Unity Catalog at enterprise scale (multiple workspaces, many teams)

• Deep knowledge of Unity Catalog: privilege model, row/column security, tags/attribute-based policies, system tables, lineage, Delta Sharing, Lakehouse Federation; including Hive metastore migration

• Strong Spark, SQL, and Python; Delta Lake; Lakeflow pipelines and jobs

• Infrastructure as code with Terraform and Databricks Asset Bundles; CI/CD experience

• Working knowledge of Azure or AWS security for Databricks (identity, storage access, private networking, key management)

• Databricks Certified Data Engineer Professional or Associate (Platform Administrator accreditation is a plus)

• Interest in data governance and regulation; experience with Purview/Collibra or GDPR/EU AI Act/sector-regulated data is a plus

• Ability to explain access models or lineage to non-technical stakeholders; fluent English

Benefits

• Yearly financial bonus

• Private medical care (Medicover) with additional packages (dental, senior care, oncology)

• Life insurance and access to the NAIS benefit platform

• Over 70 training tracks with certification opportunities (NEXT platform) and free access to Education First languages, TED Talks, and Udemy Business materials

• Hybrid working model after onboarding; home office package (laptop, monitor, chair)

• Access to cutting-edge IT innovation and training with large enterprise partners