Data Engineer

Description

We are seeking a talented Data Engineer (Senior Consultant) to enhance and manage Azure data pipelines that support the CMO's single source of truth (SSOT). The ideal candidate will ensure reliable data ingestion, consolidation, and continuity across all cost domains, while facilitating effective data delivery through Palantir Foundry and discovery via Sinequa where applicable.

Main Responsibilities

• Deliver robust ELT/ETL pipelines that integrate multi-source cost data and maintain the integrity of SSOT datasets.
• Automate refresh, incremental loads, and, when relevant, streaming ingestion for improved timeliness and efficiency.
• Produce curated, consumption-ready datasets in Palantir Foundry to support analytics and reporting at scale.
• Embed data quality checks and reconciliation controls to minimize incidents and enhance resolution times.
• Monitor pipeline health and troubleshoot failures to swiftly restore service and protect SSOT availability.
• Collaborate with business teams to verify cost outputs and drive improvements in data accuracy.
• Maintain comprehensive documentation, catalog metadata, and lineage to enhance governance adoption.
• Enable Sinequa discovery by providing indexed datasets and enforcing security protocols.

Key Requirements

• Platform engineering (Azure): Proficiency in ADLS Gen2, networking fundamentals, Key Vault/secrets management, managed identities, and Azure Monitor/Log.
• Palantir Foundry: Prior experience using this platform.
• Sinequa: Familiarity with the Sinequa platform.
• Experience with Azure data services, particularly ADLS Gen2 and Azure Monitor.
• Familiarity with ETL/ELT pipeline development.
• Ability to handle multi-source data integration.
• Strong troubleshooting skills for addressing data pipeline issues.

Nice to Have

• Experience utilizing Palantir Foundry.
• Knowledge of the Sinequa platform.
• Past experience in cost data domains.

Other Details

This position offers an opportunity to work in a dynamic and supportive environment that prioritizes data excellence. Remote work options are available to enhance work-life balance.