AI Architect (Generative AI & Databricks)

Quick Facts

• Role: AI Architect focused on Generative AI and Databricks Lakehouse architecture

• Work mode: 100% remote (Poland), with flexibility to work from anywhere in Poland/Romania/Greece

Description

Architect and implement an enterprise GenAI platform on Databricks that automates ingestion, triage, and structured extraction of incoming customer commercial pricing requests. Build an anti-hallucination and zero-inference verification engine with strict schemas and mandatory citations, producing ERP-ready tabular data. Deploy a Human-in-the-Loop review cockpit using Databricks Apps, bridging LLM orchestration with governed Lakehouse patterns.

Responsibilities

• Design and oversee end-to-end GenAI Lakehouse architecture on Databricks

• Transform unstructured email threads and attachments into verified, ERP-ready tabular data

• Build secure, incremental ingestion pipelines using Microsoft Graph API and PySpark parsing

• Implement deterministic extraction workflows using Databricks Model Serving with strict Pydantic schemas

• Create an anti-hallucination verification engine to prevent hallucinated data from reaching production

• Develop a Human-in-the-Loop review app on Databricks Apps (Streamlit/Dash) with reconciliation and manual overrides

• Stage pricing data in Gold-layer tables and automate batch feeds to downstream ERP via Databricks SQL Warehouses and REST APIs

• Establish Unity Catalog governance, data lineage, error-quarantine routing, and system security best practices

• Maintain architectural documentation, data models, and interface specifications; mentor client teams

Requirements

• Bachelor’s or Master’s degree in Computer Science, Artificial Intelligence, Data Engineering, or a related field

• 7+ years of experience in data engineering, data architecture, or applied machine learning systems

• 4+ years hands-on experience designing and deploying solutions on the Databricks Lakehouse Platform (Unity Catalog, Delta Lake, Databricks Model Serving, Databricks SQL)

• Experience as an Applied GenAI/LLMOps engineer/architect building structured extraction pipelines using function calling, Pydantic validation, and guardrail/citation-verification systems

• Hands-on document parsing experience (Python/PySpark) using PyMuPDF, pdfplumber, OpenPyXL, BeautifulSoup4

• Experience developing and deploying interactive apps with Streamlit or Dash, including containerized apps in Databricks Apps

• Strong enterprise identity and API integration knowledge: Microsoft Graph API, OAuth 2.0 client credential flows, Microsoft Entra ID (Azure AD), Databricks Secret Scopes

• Strong data modeling, governance, and auditability in Unity Catalog (Volumes, Delta tables, ACLs, row/column security)

• Experience integrating or working with downstream enterprise ERP patterns

• Ability to design resilient production systems that prevent hallucinated data from reaching systems of record

• Excellent communication and consulting skills; ability to explain complex GenAI concepts to technical and executive stakeholders

Benefits

• 100% remote cooperation model (Poland) with flexibility to work from anywhere in Poland/Romania/Greece

• Apple MacBook Air M4 15" provided

• Referral bonus

• Databricks certification support

• Support to reach highest Databricks expertise tiers (e.g., Champion program)

• Opportunities for conference presentations and technical thought leadership

• Mentorship via collaboration with Databricks MVPs and core product teams