Data Engineer

Quick Facts

• Role: Data Engineer

Description

Design, develop, and maintain scalable data pipelines and ETL solutions using Databricks and big data/streaming technologies. Implement ingestion frameworks for structured and unstructured data and build data quality, validation, standardization, and governance processes. Support analytics, reporting, machine learning, and business intelligence initiatives while partnering with Data Scientists and BI teams.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL solutions

• Build and optimize data ingestion frameworks for structured and unstructured data

• Implement data quality, validation, standardization, and governance processes

• Develop data infrastructure integrating multiple heterogeneous data sources

• Support analytics, reporting, machine learning, and business intelligence initiatives

• Prepare and curate data for analytics with Data Scientists and BI teams

• Develop scalable solutions for Big Data processing and streaming workloads

• Analyze data and provide actionable insights to business stakeholders

• Gather requirements and translate them into technical solutions

• Lead and mentor junior data engineers

• Collaborate across projects as the Data Engineering SME

• Drive adoption of Data Engineering best practices and standards

Requirements

• Databricks (Notebooks, Delta Lake, Delta Live Tables, Unity Catalog)

• Python and PySpark

• Spark SQL and Apache Spark

• ETL/data pipeline development

• Big data technologies: Hadoop, Spark, Kafka, Hive, HDFS, Sqoop

• Strong SQL querying and data modeling

• Data ingestion, transformation, cleansing, and validation

• Cloud platforms (AWS/GCP)

• Data warehouse and large-scale data processing experience

• Streaming data experience with Kafka and Spark Streaming

• Agile methodologies (Scrum, Kanban, SAFe)

Benefits

• Discretionary annual incentive

• Comprehensive medical coverage (medical & health, dental & vision, disability planning & insurance, pet insurance plans)

• Maternal and parental leaves

• Auto & home insurance and identity theft protection

• Commuter benefits and certification & training reimbursement

• Vacation, time off, sick leave, and holidays

• Legal assistance, 401K plan, performance bonus, college fund, and student loan refinancing