Azure Data Engineer

Quick Facts

• Role: Data Engineer (Azure)

• Team: Cyber Science and Analytics (CS&A) under the Principal Engineer

• Focus: Data engineering for cyber security enablement and data-driven business decisions

Description

You will design, improve, and implement data pipelines and data buses that collect data from disparate and external sources, transport it, and deliver it to data warehouses and data lakes. You will use advanced data manipulation across the pipeline to ensure data is available and in the right form for each system, service, and customer. You will also apply regulatory, security, compliance, and governance controls such as lineage, metadata, data quality rules, access controls, and retention.

Responsibilities

• Design, improve, and implement data pipelines and data buses across enterprise and external data sources

• Transport data and deliver it to data warehouses and data lakes

• Programmatically manipulate data throughout data flows to meet downstream system needs

• Ensure data quality, data security, and data compliance requirements are met

• Provide ongoing support to customers inside and outside the science and analytics environment

• Apply data governance controls (lineage, metadata, quality rules, access controls, retention)

Requirements

• Strong background in data engineering with experience deploying complex cloud data and big data systems

• Understanding of data engineering methods applied to the cyber security domain

• Ability to communicate complex and technical topics clearly and authoritatively to diverse audiences

• Experience designing, building, and maintaining data pipelines and ETL workflows across disparate datasets

• Experience with Azure data engineering tooling (including Azure Data Factory, Azure Databricks, Azure Functions, Azure Logic Apps, Azure Monitor/Log Analytics, Azure Data Lake Store, Synapse Analytics and/or PowerBI) and/or S3

• Experience with Azure DevOps; scripting (bash/PowerShell, Azure CLI); code (Python, Spark); query (SQL, Kusto query language)

• Experience with version control (e.g., Git) and CI/CD systems

• Advanced understanding of data transport, data pipelining, data cleaning, and data quality methods

• Ability to analyze datasets and understand/derive schemas for existing and new/unseen data

• Understanding of model lifecycle controls and experience working within data governance frameworks in a regulated environment

Benefits

• Full-time employment contract

• Stable and long-term cooperation

• Well-defined career path

• Opportunity to relocate and work in different branches

• Potential to engage in a subsequent project after project completion

• Dedicated Competences Manager

• Medicover medical care; dental care; Benefits platform; Sport card

• Referral program

• Group life insurance; pension scheme

• Access to professional certifications with technology partners

• Access to relevant industry events and a Knowledge Sharing Program

• Access to referral program opportunities