Senior Data Engineer with DevOps

About the role

Join the Data Operations team to modernize the Analytics Platform and build services, tooling, and infrastructure supporting data ingestion, schema management, and reliable delivery of analytics data into Databricks.

As a core contributor, you will build dependable data solutions capable of processing petabytes of information. Work spans backend services, schema and metadata tooling, cloud infrastructure, CI/CD, and production operations. You will help product teams and internal platform users adopt new data standards, migrate safely from legacy systems, and operate services with improved reliability, observability, and efficiency.

Use your experience with large-scale data systems and production infrastructure to design and operate a modern data platform that is easier to evolve, safer to change, and aligned with future data engineering needs.

Responsibilities

• Design, build, and operate services and pipelines that power analytics data ingestion, schema management, and delivery into Databricks.
• Modernize legacy platform components by migrating services, libraries, and deployment workflows to modern Java and application standards.
• Build automation, CI/CD flows, and scenario/integration test coverage to make platform changes safe and repeatable.
• Design and maintain production infrastructure in AWS using Infrastructure as Code tooling such as Terraform.
• Improve data quality through stronger schema validation, metadata capture, lineage tracking, and operational guardrails.
• Build tooling and paved paths that help internal customers migrate to new ingestion and schema standards.
• Collaborate with cross-functional teams to prepare design documents, rollout plans, and implementation strategies for platform changes.
• Monitor live systems, investigate incidents, and participate in the on-call team providing 3rd-line support for live analytics services.
• Support performance, reliability, and cost optimization across ingestion and processing flows.

Required qualifications

• Minimum 5 years of commercial work experience in software engineering, data engineering, platform engineering, or a related field.
• Bachelor's degree or higher in Computer Science, Software Engineering, or a related field.
• Commercial coding experience in Java, Python, or Golang.
• Knowledge of Infrastructure as Code tooling, such as Terraform.
• Experience with CI/CD tooling, such as GitHub Actions, Jenkins, and Docker.
• Experience with streaming and schema-driven systems, such as Kafka, Protobuf, JSON Schema, or schema registries.
• Commercial experience with Airflow and dbt.
• Commercial experience with Databricks.
• Commercial experience with Spark/PySpark.
• Effective communication and teamwork skills.

Nice to have

• Experience migrating legacy services or frameworks to modern application stacks.
• Experience with Databricks Delta Live Tables and Unity Catalog.
• Experience with AWS, especially Kafka/MSK-based data flows, S3, IAM, and VPC/networking.
• Experience with Buf, Protobuf tooling, or JSON-to-Protobuf migration work.
• Familiarity with Golang for internal tooling or automation.
• Experience with infrastructure monitoring and on-call practices using tools such as Datadog and PagerDuty.
• Experience working with cross-discipline organizations that build data products.
• Experience in the gaming industry, particularly with online multiplayer games.
• Proficiency in large-scale data manipulation across various data types.
• Demonstrated ability to troubleshoot and optimize complex ETL pipelines.