Lead Engineer, Alternative Data & Web Extraction

Quick Facts

• Role: Lead Engineer, Alternative Data & Web Extraction

• Focus: high-throughput, low-latency ingestion of alternative data (web, APIs, multimedia) and delivery of data products

• Leadership: hands-on architecture, mentorship, and technical leadership for an engineering squad

Description

You will lead architecture and engineering execution for automated web extraction and large-scale alternative data ingestion systems. You’ll drive agentic AI and LLM-assisted workflows (e.g., autonomous subagents and MCP-based tool use) to collect, parse, and normalize complex market information. You’ll act as the technical anchor for web extraction while collaborating closely with data researchers and data science stakeholders.

Responsibilities

• Provide technical guidance and mentorship; establish coding standards and run high-quality code reviews

• Architect resilient, scalable, low-latency extraction frameworks for unstructured web, API, and multimedia data

• Design and deploy agent-driven pipelines leveraging LLMs, MCPs, subagents, and automated workflows

• Build and maintain fault-tolerant streaming and orchestration using queueing systems and workflow engines (Kafka, Airflow, relational/NoSQL datastores)

• Partner with data researchers and quantitative/domain experts to deliver production-grade data products

• Coordinate human-in-the-loop validation and quality assurance workflows

• Track and adopt emerging techniques for anti-bot mitigation, browser automation, and LLM-powered extraction

Requirements

• 7–10+ years of commercial software engineering/data engineering experience, including leading/managing engineers

• Bachelor’s or Master’s degree in a STEM discipline

• Expert-level Python proficiency plus data manipulation ecosystem (Pandas, NumPy) and advanced extraction/scraping toolkits

• Strong SQL and database design (relational and/or NoSQL), plus networking and web standards (HTML, DOM, REST/GraphQL) and anti-scraping mitigation

• Practical familiarity with agentic workflows (e.g., Claude Code, MCPs, tool-use, subagents, LLM-assisted parsing/extraction)

• Practical experience with streaming and orchestration (e.g., Apache Airflow, Kafka)

• Excellent English communication (C1/C2) and stakeholder management

Benefits

• High-impact technical leadership with visibility into mission-critical systems

• Freedom to innovate with access to modern enterprise LLM tooling and cloud infrastructure

• Competitive compensation with attractive bonus schemes

• Premium Warsaw office space and flexible hybrid setup; international career trajectory

• Comprehensive benefits: private healthcare, life insurance, and professional development allowance