Data Engineer (Databricks & Agentic Automation)

Quick Facts

Role: Data Engineer focused on Databricks and agentic automation. Builds scalable ETL/ELT pipelines and productionizes agentic workflows for data engineering automation.

Description

Develop scalable ETL/ELT pipelines using Databricks with PySpark and SQL. Build and operationalize agentic workflows to automate data validation, issue identification, troubleshooting, and workflow execution across Databricks, GCP, BigQuery, and Delta Lake. Ensure production-grade reliability through data quality checks, monitoring, validation, exception handling, and performance optimization.

Responsibilities

• Design, develop, and optimize scalable ETL/ELT data pipelines using Databricks, PySpark, and SQL.
• Build and operationalize agentic workflows to automate data engineering and operational processes.
• Integrate agentic capabilities with Databricks, GCP, BigQuery, and Delta Lake environments.
• Develop data pipelines and processing solutions supporting new business requirements and datasets.
• Create reusable frameworks and components for multiple use cases.
• Implement data quality checks, monitoring, validation, exception handling, and production controls.
• Optimize PySpark and SQL workloads for performance, reliability, and scalability.
• Support testing, deployment, productionization, and ongoing enhancements.
• Troubleshoot complex data and production issues and implement sustainable solutions.
• Collaborate with business, data engineering, and platform teams to identify further automation opportunities.

Requirements

• 2+ years hands-on experience with Databricks and PySpark.
• Advanced SQL and data-processing skills.
• Hands-on experience with GCP, particularly BigQuery.
• Experience with Delta Lake and modern data lake/lakehouse architectures.
• Strong understanding of ETL/ELT, pipeline design, performance optimization, and data quality.
• Experience building reliable, scalable, production-grade data solutions.
• Strong analytical and troubleshooting skills.
• Understanding of software engineering practices (testing, version control, deployment, monitoring, production support).
• Upper-intermediate English level.

Benefits

• Growth: mentorship, internal TechTalks, challenging projects, and an annual learning budget.
• Competitive compensation with regular performance and compensation reviews.
• Flexibility: 100% remote work with flexible hours.
• Meaningful, modern projects with modern technologies and global teams.
• Collaborative culture with zero micromanagement.
• Well-being & support via local programs and people-focused assistance.