Spark Software Engineer

Quick Facts

• Role: Spark Software Engineer (hybrid)
• Domain: Risk & Compliance (financial crime, market abuse, identity threats, regulatory risks)
• Focus: Modern data platforms and big data solutions for large-scale surveillance and risk management

Description

You will work with a global Agile team building modern data platforms and big data solutions for risk and compliance use cases. The role involves designing and implementing scalable ETL pipelines, developing data processing components, and collaborating with engineers, architects, analysts, and data scientists. You will also support productionization of analytical models and solutions.

Responsibilities

• Collaborate with Product Owners and Business Analysts to understand business and stakeholder requirements
• Design, develop, and maintain data processing components and storage solutions
• Onboard new data sources and build scalable ETL/data processing pipelines
• Work closely with Data Scientists to productionize analytical models and solutions
• Develop software following CI/CD and Test-Driven Development (TDD) practices
• Partner with Solution Architects to validate and implement technical designs
• Perform unit testing and ensure high code quality standards
• Contribute to continuous improvement of development processes and engineering practices

Requirements

• Must have: Strong commercial experience with Apache Spark
• Must have: Strong programming experience in Java or Scala
• Must have: Experience building large-scale data processing solutions
• Must have: Understanding of software engineering best practices and Agile methodologies
• Must have: Strong communication skills and ability to work effectively in an international team
• Must have: English proficiency at B2 level or higher
• Nice to have: Apache Hadoop (YARN, HDFS)
• Nice to have: Spring Framework
• Nice to have: Apache Airflow
• Nice to have: SQL databases and Elasticsearch
• Nice to have: Python
• Nice to have: Deploying services on Google Cloud Platform (GCP)
• Nice to have: Maven and Git

Benefits

• Private healthcare (LuxMed) and sports package (Multisport)
• Opportunity to work on global-scale Big Data solutions
• Collaboration with international teams and industry experts
• Modern technology stack and challenging engineering environment