Big Data Developer

Quick Facts

• Rate: up to 130 PLN/h net on B2B + VAT
• Work model: remote
• Benefits: private medical care and sport card

Description

You’ll build and optimize Big Data solutions using Python and Apache Spark/PySpark. The work is performed in a Kubernetes environment with Spark Operator, Jupyter, and Airflow, including technical support for unit, integration, and UAT testing and documentation of code and data flows.

Responsibilities

• Create and optimize Python-based solutions
• Work in Kubernetes using Spark Operator, Jupyter, and Airflow
• Develop Big Data solutions with Apache Spark / PySpark
• Provide technical support for unit, integration, and UAT testing
• Document code and data pipelines/flow

Requirements

• Minimum 5 years of practical programming experience in Scala, Python, or Java
• Commercial experience implementing projects involving distributed systems
• Minimum 3 years of commercial experience with Big Data solutions, especially Apache Spark and Apache Iceberg
• Commercial experience optimizing processing of large data sets
• Experience integrating systems at large scale
• Experience with relational databases (RDBMS) and very strong SQL knowledge
• Practical knowledge of the Linux environment and scripting languages
• Experience with containerization, including modifying a Dockerfile
• Basic knowledge of Kubernetes