Databricks Specialist / Data Engineer

Quick Facts

• Role: Databricks Specialist / Data Engineer

Description

You will design and develop scalable data platform solutions built on Databricks for analytics, reporting, and other data use cases. You will create scalable ETL/ELT pipelines using PySpark and Spark SQL and integrate data from ERP, CRM, APIs, databases, and streaming sources. You will also support Lakehouse/Data Warehouse implementation, ensure data quality and operational stability, optimize performance and costs, and set up CI/CD and deployment automation.

Responsibilities

• Design and develop modern data platforms on Databricks
• Build scalable ETL/ELT pipelines
• Perform data transformations using PySpark and Spark SQL
• Integrate data from ERP, CRM, APIs, databases, and streaming sources
• Participate in Lakehouse and Data Warehouse architecture implementation
• Collaborate with architects, analysts, BI teams, and business stakeholders
• Ensure data quality, monitoring, and operational stability
• Optimize performance and data environment costs
• Implement CI/CD processes and deployment automation
• Contribute to technical analysis, estimations, and solution design
• Prepare technical documentation and handover materials

Requirements

• Practical experience with Databricks and Apache Spark
• Understanding of distributed data processing
• Experience with PySpark and Spark SQL
• Experience with Azure, AWS, or GCP
• Understanding of Lakehouse, Data Warehouse, or Medallion architecture
• Ability to orchestrate using Airflow, Azure Data Factory, or Databricks Workflows
• Proficiency in SQL and data modeling
• Experience with Git and CI/CD pipelines
• Experience with REST APIs and data integrations
• Ability to communicate with both technical and business teams
• Independent working style with strong teamwork

Benefits

• Challenging, high-impact projects and technological growth
• Room to grow and learn new skills
• Opportunity to shape the solution
• Flexible work arrangements (e.g., remote or hybrid)
• Supportive, fair collaboration with experienced teammates