April 8, 2026

Fullstack Senior Data Scientist

Senior • Remote

105 - 170 PLN/hr

Warsaw, Poland

ABOUT THE COMPANY

Our client is an end-to-end data services partner to global enterprises, founded in 2008 and headquartered in Warsaw. Our teams work with over 75 leading consumer packaged goods brands across more than 30 countries, helping them unlock the full value of their data — from strategy and development through to operations and adoption.

Our work spans supply chain analytics, customer analytics, AI and machine learning, data platforms, and digital commerce. We are recognised as a Strong Performer in the Gartner Peer Insights Voice of the Customer report for data and analytics, and hold Great Place to Work certification in multiple countries.

ABOUT THE ROLE

We're looking for an experienced Senior Data Scientist with deep expertise in Generative AI to lead projects building and implementing advanced LLM-based systems — including chatbots, AI agents, and RAG pipelines. This is a technical leadership role: you'll make architectural decisions, select technologies, set best practices, and mentor other engineers, while remaining hands-on across the full solution lifecycle.

You'll work closely with data engineers, product owners, and full-stack developers to deliver scalable GenAI applications for large enterprise clients, primarily in CPG, retail, and manufacturing.

WHAT YOU'LL WORK ON

Solution design and discovery

Lead discovery and solution design for GenAI use cases — translating business problems into concrete architectures covering LLM selection, RAG, fine-tuning, agents, and guardrails.

End-to-end GenAI applications

Build complete GenAI solutions covering data ingestion, retrieval layers, orchestration (LangChain, LlamaIndex, LangGraph), API and backend, and lightweight UI where needed.

RAG pipeline design

Design and implement RAG pipelines with vector databases, hybrid search, rerankers, query transformation, and evaluation frameworks for relevance and robustness.

Model selection, prompting, and fine-tuning

Own prompting strategies, model selection, and fine-tuning (LoRA, QLoRA, SFT) for text, code, and multimodal models, including evaluation and A/B testing.

Safety and governance

Implement safety, compliance, and governance controls including input/output filters, PII handling, audit logs, and human-in-the-loop review where required.

Requirements and estimation

Gather technical requirements from stakeholders and produce reliable estimates for planned work.

Mentorship and knowledge sharing

Mentor other data scientists and engineers in GenAI patterns, code quality, and best practices. Contribute to internal libraries, templates, and reusable components.

Staying current

Track the GenAI landscape — new open and hosted models, agentic frameworks, evaluation techniques — and run targeted PoCs to validate emerging approaches.

WHAT WE LOOK FOR

6+ years in Data Science or AI engineering

Broad experience across the data science and AI stack, with a track record of delivering production systems.

4+ years of production Python for AI

Fluent in writing production-ready Python for AI and ML workloads — clean, maintainable, and deployable.

2+ years of production LLM development

Hands-on experience building and shipping LLM-based systems in production environments, not just research or prototypes.

Strong analytical and problem-solving skills

Able to break down ambiguous problems, make sound architectural decisions under uncertainty, and defend those decisions clearly.

Excellent English communication

Comfortable working directly with international clients and cross-functional teams. Able to translate technical complexity for non-technical stakeholders.

THE TEAM

You'll join a specialist Data Science and AI practice working alongside experienced consultants, ML engineers, and data engineers. The team delivers solutions for large international clients across CPG, retail, and manufacturing. There is a strong knowledge-sharing culture, with internal communities, competency centres, and structured learning programmes built into how the team operates.

COMPENSATION & BENEFITS

Rate

105 – 170 PLN per hour on a B2B contract, depending on experience.

Work model

Fully remote or office-based — your choice. Flexibility on working hours and contract form.

Workation policy

Option to work remotely from other locations for defined periods.

Onboarding

Comprehensive online onboarding programme with a dedicated buddy from day one.

Learning and development

Unlimited access to the Udemy learning platform from day one. Certificate training programmes, upskilling support, capability development programmes, competency centres, knowledge sharing sessions, community webinars, and over 110 training opportunities per year.

Career growth

Internal promotion pathways — 76% of managers were promoted internally. Cooperation with top-tier engineers and domain experts across the organisation.

Referral bonuses

Financial rewards for successful employee referrals.

Wellbeing

Activities to support health and wellbeing, with opportunities to contribute to charitable causes and environmental initiatives.

Equipment

Modern office equipment provided.

Employer recognition

Great Place to Work certified employer.

Similar jobs you might like

Technology

Capgemini Polska

Fullstack Gen AI Developer

Mid

Hybrid

Warsaw, Poland

🏢 Summary: The role involves designing, building, and deploying fullstack Generative AI applications that integrate advanced LLMs and multimodal models into scalable products. You will develop frontend and backend systems, implement AI-driven workflows such as RAG pipelines, and integrate foundation model APIs into production-ready solutions. The position focuses on delivering secure, scalable AI-powered web applications using modern cloud and MLOps practices. 🗂️ Requirements: Minimum 2 years of experience in frontend and backend software development, Strong programming skills in Python, Strong programming skills in JavaScript or TypeScript, Experience with FastAPI, Flask, Node.js, React or Next.js, Hands-on experience building AI-powered web applications, Knowledge of LLMs, prompt engineering and fine-tuning, Experience implementing RAG pipelines, Experience integrating GenAI APIs, Experience with vector databases, Familiarity with AWS, Azure or GCP, Experience with CI/CD pipelines, Understanding of embeddings and model-serving infrastructure, Professional proficiency in English 📃 Skills: Python, JavaScript, TypeScript, FastAPI, Flask, Node.js, React, Next.js, LLM, RAG, FAISS, Pinecone, Weaviate, OpenAI, Anthropic, HuggingFace, Stability, AWS, Azure, GCP, CI/CD, LangChain, LangGraph 🏢 Description: At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow. Informed and validated by science and data. Superpowered by creativity and design. All underpinned by technology created with purpose. YOUR TASKS Design, build, and deploy fullstack Generative AI applications, integrating advanced LLMs, image, and multimodal models into scalable products. Develop and maintain both frontend interfaces (React, Next.js, etc.) and backend systems (Python, Node.js, or similar) that power GenAI features. Implement AI-driven workflows — including prompt orchestration, RAG pipelines, and real-time model interactions. Integrate foundation models and APIs (OpenAI, Anthropic, Hugging Face, Stability, etc.) into robust backend services and interactive user experiences. Collaborate closely with data scientists and product teams to translate business requirements into AI-enabled digital experiences. Ensure system scalability, performance, and security, leveraging cloud infrastructure and MLOps best practices. Stay at the forefront of GenAI innovations, experimenting with new models, frameworks, and architectures. YOUR PROFILE Passion for Generative AI and fullstack development, with hands-on experience delivering AI-powered web applications. 2+ years of experience in software development, covering both frontend and backend environments. Strong programming skills in Python and JavaScript/TypeScript, with experience in frameworks like FastAPI, Flask, Node.js, React, Next.js. Working knowledge of LLMs, prompt engineering, fine-tuning, and retrieval-augmented generation (RAG). Experience connecting to GenAI APIs and managing vector databases (FAISS, Pinecone, Weaviate, etc.). Familiarity with cloud platforms (AWS, Azure, GCP) and CI/CD pipelines for deploying AI-driven applications. Understanding of data pipelines, embeddings, and model-serving infrastructure. Strong communication skills in English (German or Polish are a plus), and the ability to collaborate in cross-functional, international teams. Nice to have: Experience with LangChain / LangGraph , OpenAI Function Calling , or custom agent frameworks . Familiarity with Vercel AI SDK , Next.js Server Actions, or other GenAI-focused web stacks. Background in UX for AI applications , multimodal systems, or real-time AI interfaces (e.g., chat, co-creation tools). Experience building end-to-end prototypes or AI SaaS products from concept to production. WHAT YOU’LL LOVE ABOUT WORKING HERE Practical benefits: yearly financial bonus, private medical care with Medicover with additional packages (e.g., dental, senior care, oncology) available on preferential terms, life insurance and access to NAIS benefit platform. Award-winning development programs to support your career at every stage. Connected Manager - our leadership development program has already helped over 300 employees accelerate their growth! Access to over 70 training tracks with certification opportunities (e.g., GenAI, Excel, Business Analysis, Project Management) on our NEXT platform. Dive into a world of knowledge with free access to Education First languages platform, Pluralsight, TED Talks, Coursera and Udemy Business materials and trainings. Cutting-Edge Technology: Position yourself at the forefront of IT innovation, working with the latest technologies and platforms. Capgemini partners with top global enterprises, including 145 Fortune 500 companies. Enjoy hybrid working model that fits your life - after completing onboarding, connect work from a modern office with ergonomic work from home, thanks to home office package (including laptop, monitor, and chair). Ask your recruiter about the details. GET TO KNOW US Capgemini is committed to diversity and inclusion, ensuring fairness in all employment practices. We evaluate individuals based on qualifications and performance, not personal characteristics, striving to create a workplace where everyone can succeed and feel valued. Do you want to get to know us better? Check our Instagram — @capgeminipl or visit our Facebook profile — Capgemini Polska . You can also find us on YouTube . ABOUT CAPGEMINI Capgemini is a global leader in partnering with companies to transform and manage their business by harnessing the power of technology. The Group is guided everyday by its purpose of unleashing human energy through technology for an inclusive and sustainable future. It is a responsible and diverse organization of over 360,000 team members globally in more than 50 countries. With its strong 55-year heritage and deep industry expertise, Capgemini is trusted by its clients to address the entire breadth of their business needs, from strategy and design to operations, fueled by the fast evolving and innovative world of cloud, data, AI, connectivity, software, digital engineering and platforms. Apply now!

Technology

Capgemini Polska

Fullstack Gen AI Developer

Mid

Hybrid

Warsaw, Poland

🏢 Summary: The role focuses on designing, building, and deploying fullstack Generative AI applications that integrate advanced LLMs and multimodal models into scalable products. It involves developing AI-driven web solutions, implementing RAG pipelines, and integrating foundation model APIs into secure, cloud-based environments. The position combines frontend and backend development with cutting-edge GenAI technologies to deliver production-ready AI-powered applications. 🗂️ Requirements: Minimum 2 years of experience in software development (frontend and backend), Strong programming skills in Python, Strong programming skills in JavaScript or TypeScript, Experience with frontend frameworks such as React or Next.js, Experience with backend frameworks such as FastAPI, Flask, or Node.js, Working knowledge of LLMs and prompt engineering, Experience with retrieval-augmented generation (RAG), Experience integrating GenAI APIs, Experience with vector databases, Familiarity with cloud platforms (AWS, Azure, or GCP), Experience with CI/CD pipelines, Understanding of data pipelines and model-serving infrastructure, Fluent English communication skills 📃 Skills: Python, JavaScript, TypeScript, React, Next.js, FastAPI, Flask, Node.js, LLM, RAG, FAISS, Pinecone, Weaviate, OpenAI, Anthropic, HuggingFace, Stability, AWS, Azure, GCP, CI/CD, LangChain, LangGraph 🏢 Description: At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow. Informed and validated by science and data. Superpowered by creativity and design. All underpinned by technology created with purpose. YOUR TASKS Design, build, and deploy fullstack Generative AI applications, integrating advanced LLMs, image, and multimodal models into scalable products. Develop and maintain both frontend interfaces (React, Next.js, etc.) and backend systems (Python, Node.js, or similar) that power GenAI features. Implement AI-driven workflows — including prompt orchestration, RAG pipelines, and real-time model interactions. Integrate foundation models and APIs (OpenAI, Anthropic, Hugging Face, Stability, etc.) into robust backend services and interactive user experiences. Collaborate closely with data scientists and product teams to translate business requirements into AI-enabled digital experiences. Ensure system scalability, performance, and security, leveraging cloud infrastructure and MLOps best practices. Stay at the forefront of GenAI innovations, experimenting with new models, frameworks, and architectures. YOUR PROFILE Passion for Generative AI and fullstack development, with hands-on experience delivering AI-powered web applications. 2+ years of experience in software development, covering both frontend and backend environments. Strong programming skills in Python and JavaScript/TypeScript, with experience in frameworks like FastAPI, Flask, Node.js, React, Next.js. Working knowledge of LLMs, prompt engineering, fine-tuning, and retrieval-augmented generation (RAG). Experience connecting to GenAI APIs and managing vector databases (FAISS, Pinecone, Weaviate, etc.). Familiarity with cloud platforms (AWS, Azure, GCP) and CI/CD pipelines for deploying AI-driven applications. Understanding of data pipelines, embeddings, and model-serving infrastructure. Strong communication skills in English (German or Polish are a plus), and the ability to collaborate in cross-functional, international teams. Nice to have: Experience with LangChain / LangGraph , OpenAI Function Calling , or custom agent frameworks . Familiarity with Vercel AI SDK , Next.js Server Actions, or other GenAI-focused web stacks. Background in UX for AI applications , multimodal systems, or real-time AI interfaces (e.g., chat, co-creation tools). Experience building end-to-end prototypes or AI SaaS products from concept to production. WHAT YOU’LL LOVE ABOUT WORKING HERE Practical benefits: yearly financial bonus, private medical care with Medicover with additional packages (e.g., dental, senior care, oncology) available on preferential terms, life insurance and access to NAIS benefit platform. Award-winning development programs to support your career at every stage. Connected Manager - our leadership development program has already helped over 300 employees accelerate their growth! Access to over 70 training tracks with certification opportunities (e.g., GenAI, Excel, Business Analysis, Project Management) on our NEXT platform. Dive into a world of knowledge with free access to Education First languages platform, Pluralsight, TED Talks, Coursera and Udemy Business materials and trainings. Cutting-Edge Technology: Position yourself at the forefront of IT innovation, working with the latest technologies and platforms. Capgemini partners with top global enterprises, including 145 Fortune 500 companies. Enjoy hybrid working model that fits your life - after completing onboarding, connect work from a modern office with ergonomic work from home, thanks to home office package (including laptop, monitor, and chair). Ask your recruiter about the details. GET TO KNOW US Capgemini is committed to diversity and inclusion, ensuring fairness in all employment practices. We evaluate individuals based on qualifications and performance, not personal characteristics, striving to create a workplace where everyone can succeed and feel valued. Do you want to get to know us better? Check our Instagram — @capgeminipl or visit our Facebook profile — Capgemini Polska . You can also find us on YouTube . ABOUT CAPGEMINI Capgemini is a global leader in partnering with companies to transform and manage their business by harnessing the power of technology. The Group is guided everyday by its purpose of unleashing human energy through technology for an inclusive and sustainable future. It is a responsible and diverse organization of over 360,000 team members globally in more than 50 countries. With its strong 55-year heritage and deep industry expertise, Capgemini is trusted by its clients to address the entire breadth of their business needs, from strategy and design to operations, fueled by the fast evolving and innovative world of cloud, data, AI, connectivity, software, digital engineering and platforms. Apply now!

Technology

Sii

Senior AI Engineer – finance industry (f/m/x)

Senior

Hybrid

Krakow, Poland

20,000 - 26,000 PLN

🏢 Summary: Opportunity to join a newly formed AI team in the banking sector to build a modern AI-native platform from scratch. The role focuses on designing and delivering production-grade solutions using LLMs, RAG, and agent-based architectures, while shaping AI strategy and architectural decisions. You will develop secure, transparent, and scalable AI systems that drive real business value in a regulated financial environment. 🗂️ Requirements: At least 3 years of experience in AI/ML, Minimum 5 years of experience in product development, Strong Python skills, Hands-on experience with LLMs, RAG, and agent-based systems in production, Experience with AI frameworks such as Agno, LangChain, LlamaIndex or similar, Knowledge of prompt engineering and AI model evaluation, Experience with Azure or AWS, Understanding of vector databases and embeddings, Experience with workflow orchestration tools (Temporal, Prefect, Airflow), Ability to design scalable and reliable AI solutions, Experience in model monitoring and MLOps practices, Ability to make architectural decisions and assess business value of AI, Fluent English 📃 Skills: Python, LLM, RAG, LangChain, LlamaIndex, Agno, Azure, AWS, Temporal, Prefect, Airflow, MLOps, Embeddings, VectorDB 🏢 Description: We are looking for an experienced Senior AI Engineer to join a newly formed AI team delivering a strategic initiative in the banking sector. This is a unique opportunity to co-create a modern AI-native platform from scratch and influence the direction of AI within the organisation. In this role, you will design and develop solutions based on LLMs, RAG systems, and agent-based architectures to support data integration and transformation processes. You will contribute to key decisions around AI strategy, model selection, architecture, and quality standards. We are seeking a candidate with strong technical skills and hands-on experience in building production-grade AI solutions—someone who understands where AI brings real business value and can deliver high-quality, transparent, and secure solutions in a financial environment. Your tasks Design and develop solutions based on LLMs, RAG, and agent-based architectures Co-create and execute an AI strategy for a modern banking platform Select models, frameworks, and approaches for production-grade AI solutions Build systems for data integration, transformation, and validation automation Define prompt engineering, model evaluation, and AI quality standards Develop monitoring, evaluation, and continuous improvement mechanisms Collaborate with engineering and product teams on new features Ensure high quality, transparency, and auditability of AI outputs Contribute to key architectural decisions and AI direction Mentor and share knowledge within the AI team Requirements At least 3 years in AI/ML and a minimum of 5 years in product development Strong Python skills Hands-on experience with LLMs, RAG, and agent-based systems in production Familiarity with frameworks such as Agno, LangChain, LlamaIndex, or similar Knowledge of prompt engineering and AI model evaluation Experience with Azure or AWS Understanding of vector databases and embeddings Experience with workflow orchestration tools (e.g., Temporal, Prefect, Airflow) Ability to design scalable, reliable AI solutions Experience in model monitoring and MLOps practices Capability to make architectural decisions and assess the business value of AI Strong collaboration skills with technical and business stakeholders Fluent English Nice-to-have requirements Previous work in financial services, fintech, or other regulated industries Knowledge of compliance, auditability, and AI security Familiarity with MLOps and large-scale ML/LLM deployments Experience fine-tuning open-source models for specific domains Proven track record in developing AI solutions for data platforms, analytics, or SaaS products Background in AI R&D projects Publications, conference speaking, or open-source contributions Experience building and scaling technical teams What we offer Great Place to Work since 2015 - it’s thanks to feedback from our workers that we get this special title and constantly implement new ideas Employment stability - revenue of PLN 2.1BN, no debts, since 2006 on the market We share the profit with Workers - over PLN 76M has already been allocated for this aim since 2022 Attractive benefits package - private healthcare, benefits cafeteria platform, car discounts and more Comfortable workplace – class A offices or remote work Dozens of fascinating projects for prestigious brands from all over the world PLN 1 000 000 per year for your ideas - with this amount, we support the passions and voluntary actions of our workers Investment in your growth – meetups, webinars, training platform and technology blog – you choose Fantastic atmosphere created by all Sii Power People If you want to work on systems with high operational significance — apply now!

Technology

emagine Polska

AI Engineer - Gen AI & LLM & RAG

Senior

Remote

Warsaw, Poland

🏢 Summary: Full-time remote Senior AI Engineer role focused on designing and operating production-grade GenAI solutions, including agentic workflows and RAG pipelines integrated into enterprise systems. The position emphasizes software engineering excellence, cloud-based AI services, and orchestration of LLM capabilities rather than model research. You will build reliable, secure, and observable AI services integrated with internal platforms and APIs. 🗂️ Requirements: 5+ years of professional software development experience, Strong background in designing and operating production systems, Proficiency in Python or another backend language for AI systems, Experience building and integrating RESTful APIs, Understanding of distributed systems fundamentals, Experience with API integrations and event-driven architectures, Solid SQL and database design knowledge, Experience with vector search, Hands-on experience building end-to-end LLM applications, Experience implementing RAG pipelines, Strong knowledge of Azure cloud services, Experience with unit and integration testing, Ability to build AI evaluation and regression testing frameworks 📃 Skills: Python, REST, GraphQL, SQL, Azure, RAG, LLM, LangChain, LangGraph, ASP.NET, APIs, Embeddings, VectorSearch, AzureFunctions, AzureStorage, KeyVault, AppConfiguration, ApplicationInsights 🏢 Description: Workload: full-time Work model: 100% Remote We are seeking a Senior AI Engineer who combines strong software engineering fundamentals with hands-on experience building production GenAI solutions, including agentic workflows and Retrieval-Augmented Generation (RAG). This is an engineering and orchestration role focused on integrating LLM capabilities into enterprise systems – not a traditional model-training/ML research role. 5+ years of professional software development experience (ideally 7+ years across backend/API/integration and cloud platforms). Proven ability to ship production-grade LLM applications ( RAG , tool/function calling, agent orchestration) with reliability, security, and observability. Strong ownership mindset and passion for AI engineering – curiosity, experimentation, and a drive to continuously improve the product and the team. Excellent communication and collaboration skills; ability to guide, mentor, and unblock other engineers as we build out an AI engineering capability. Main Responsibilities: Design, build, and operate agentic AI services that orchestrate tools, workflows, and integrations across cloud systems and enterprise data sources. Implement and continuously improve RAG pipelines for tax artifacts and internal knowledge, including ingestion, retrieval tuning, and evaluation. Integrate AI workflows with existing internal platforms (e.g., assistant frameworks) and back-end services through robust APIs. Define and maintain tool/function schemas and orchestration patterns; implement streaming updates, interrupts, and human-in-the-loop steps as needed. Partner with other engineers to set direction, mentor, and unblock the team — helping establish strong foundations for the AI initiative. Build in quality from day one: automated tests, evaluation checks, monitoring/telemetry, and performance optimization for network-bound workloads. Participate in Agile ceremonies (daily scrums, refinement/grooming, planning) and collaborate through peer review, pair programming, and strong documentation. Apply best practices, design principles, and security standards throughout the SDLC, with a focus on reliability and responsible AI. Key Requirements: Strong software engineering background (not a research-only data science profile): designing, building, and operating production systems. Proficiency in at least one backend language used for AI systems (Python preferred). Hands-on experience building and integrating RESTful APIs ; GraphQL experience is a plus. Strong understanding of distributed systems fundamentals : concurrency, async I/O, resiliency/retries, rate limits, caching, and performance optimization. Experience integrating with external services and internal platforms via APIs and event-driven patterns. Solid database fundamentals (SQL design, performance, migrations); experience with vector search is required, and hybrid search stores are a plus. Hands-on experience building LLM-powered applications end-to-end : prompt design, tool/function interfaces, structured outputs, and streaming user experiences. Experience with RAG systems : document ingestion pipelines, chunking/metadata, embeddings, retrieval strategies, grounding, and evaluation. Cloud services expertise : Strong knowledge of Azure cloud services used for enterprise AI solutions (e.g., Functions, Storage, Key Vault, App Configuration, Application Insights). Development practices experience : Strong background in unit and integration testing; ability to build and maintain AI evaluation harnesses (golden sets, regression tests, automated checks). Nice to Have: Direct experience with LangGraph and/or LangChain for multi-agent workflows. Familiarity with emerging agentic ecosystem concepts/protocols (e.g., MCP, A2A, ADK or similar). Experience integrating AI services into .NET (ASP.NET Core) applications or building AI microservices that serve enterprise applications. Experience with event-driven architectures (service bus, event hubs) and real-time updates/streaming to UI. Experience working with tax/enterprise document corpora and governance constraints (PII, retention, access control). Other Details: This position is designed for remote work and has a flexible duration, allowing for innovation in the AI space within an agile environment.

Technology

emagine Polska

AI Engineer - Gen AI & LLM & RAG

Senior

Remote

Warsaw, Poland

🏢 Summary: Full-time remote Senior AI Engineer role focused on designing and operating production-grade GenAI systems, including agentic workflows and RAG pipelines integrated into enterprise platforms. The position emphasizes backend engineering, API integrations, and orchestration of LLM capabilities in secure, scalable cloud environments rather than ML research. The role also includes mentoring engineers and establishing best practices for reliable, observable AI services. 🗂️ Requirements: 5+ years professional software development experience, Proficiency in Python, Experience building production LLM applications, Hands-on experience with RAG systems, Experience designing and integrating RESTful APIs, Strong understanding of distributed systems fundamentals, Experience with SQL databases and performance optimization, Experience with vector search, Experience integrating external services via APIs, Knowledge of Azure cloud services, Experience with unit and integration testing, Ability to build AI evaluation and testing frameworks 📃 Skills: Python, REST, GraphQL, SQL, Azure, RAG, LLM, LangChain, LangGraph, ASP.NET, APIs, Microservices, DistributedSystems, Concurrency, AsyncIO, VectorSearch, Embeddings, Caching, AzureFunctions, Storage, KeyVault, AppConfiguration, ApplicationInsights, ServiceBus, EventHubs 🏢 Description: Workload: full-time Work model: 100% Remote We are seeking a Senior AI Engineer who combines strong software engineering fundamentals with hands-on experience building production GenAI solutions, including agentic workflows and Retrieval-Augmented Generation (RAG). This is an engineering and orchestration role focused on integrating LLM capabilities into enterprise systems – not a traditional model-training/ML research role. 5+ years of professional software development experience (ideally 7+ years across backend/API/integration and cloud platforms). Proven ability to ship production-grade LLM applications ( RAG , tool/function calling, agent orchestration) with reliability, security, and observability. Strong ownership mindset and passion for AI engineering – curiosity, experimentation, and a drive to continuously improve the product and the team. Excellent communication and collaboration skills; ability to guide, mentor, and unblock other engineers as we build out an AI engineering capability. Main Responsibilities: Design, build, and operate agentic AI services that orchestrate tools, workflows, and integrations across cloud systems and enterprise data sources. Implement and continuously improve RAG pipelines for tax artifacts and internal knowledge, including ingestion, retrieval tuning, and evaluation. Integrate AI workflows with existing internal platforms (e.g., assistant frameworks) and back-end services through robust APIs. Define and maintain tool/function schemas and orchestration patterns; implement streaming updates, interrupts, and human-in-the-loop steps as needed. Partner with other engineers to set direction, mentor, and unblock the team — helping establish strong foundations for the AI initiative. Build in quality from day one: automated tests, evaluation checks, monitoring/telemetry, and performance optimization for network-bound workloads. Participate in Agile ceremonies (daily scrums, refinement/grooming, planning) and collaborate through peer review, pair programming, and strong documentation. Apply best practices, design principles, and security standards throughout the SDLC, with a focus on reliability and responsible AI. Key Requirements: Strong software engineering background (not a research-only data science profile): designing, building, and operating production systems. Proficiency in at least one backend language used for AI systems (Python preferred). Hands-on experience building and integrating RESTful APIs ; GraphQL experience is a plus. Strong understanding of distributed systems fundamentals : concurrency, async I/O, resiliency/retries, rate limits, caching, and performance optimization. Experience integrating with external services and internal platforms via APIs and event-driven patterns. Solid database fundamentals (SQL design, performance, migrations); experience with vector search is required, and hybrid search stores are a plus. Hands-on experience building LLM-powered applications end-to-end : prompt design, tool/function interfaces, structured outputs, and streaming user experiences. Experience with RAG systems : document ingestion pipelines, chunking/metadata, embeddings, retrieval strategies, grounding, and evaluation. Cloud services expertise : Strong knowledge of Azure cloud services used for enterprise AI solutions (e.g., Functions, Storage, Key Vault, App Configuration, Application Insights). Development practices experience : Strong background in unit and integration testing; ability to build and maintain AI evaluation harnesses (golden sets, regression tests, automated checks). Nice to Have: Direct experience with LangGraph and/or LangChain for multi-agent workflows. Familiarity with emerging agentic ecosystem concepts/protocols (e.g., MCP, A2A, ADK or similar). Experience integrating AI services into .NET (ASP.NET Core) applications or building AI microservices that serve enterprise applications. Experience with event-driven architectures (service bus, event hubs) and real-time updates/streaming to UI. Experience working with tax/enterprise document corpora and governance constraints (PII, retention, access control). Other Details: This position is designed for remote work and has a flexible duration, allowing for innovation in the AI space within an agile environment.

Technology

EPAM Systems

Lead AI Testing Engineer

Senior

Remote

Gdansk, Poland

🏢 Summary: Lead AI Testing Engineer role focused on building and driving QA strategy for AI-driven, knowledge graph, ontology, and RAG-based LLM systems. The position involves automating evaluation frameworks, validating graph and ontology layers, integrating testing into CI/CD pipelines, and ensuring end-to-end quality across backend, frontend, and AI components. The offer includes flexible remote work within Poland, professional development programs, certifications, and comprehensive benefits. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Python test automation with pytest or equivalent, Experience with AI/ML library integration, Knowledge Graph or Data Ontology testing experience, OWL2, RDF, SPARQL, SHACL validation, GraphDB experience, SQL data validation, Cypher query language, REST API testing, Frontend test automation with Playwright or equivalent, Experience testing RAG-based Gen AI / LLM applications, Knowledge of Gen AI / LLM evaluation metrics, Automation framework development experience, CI/CD pipeline integration, Agile delivery experience, Git, Azure DevOps or JIRA, English proficiency at B2 level or higher, Hands-on experience with coding agents and agentic development 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, Jenkins, Postman, Bruno, LLM, RAG, GenAI, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Responsibilities Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate Build test harnesses to scale evaluation beyond human-in-the-loop processes Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions Establish quality gates, acceptance criteria and test coverage models Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation Integrate test suites into CI/CD pipelines and define release quality gates Champion automation-first and shift-left quality practices Leverage agentic AI and Gen AI tooling in testing and framework development Requirements 8+ years of experience in software development, testing automation and DevOps Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems Background in backend and API testing (REST) covering data validation, integration and E2E testing Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework Hands-on experience using coding agents and agentic development daily Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale Experience in Agile delivery using Git and Azure DevOps or JIRA English proficiency at B2 level or higher Nice to have Skills in semantic search testing and evaluation Familiarity with Vector Database integration and retrieval validation Background in data science covering ML concepts, data pipelines or data engineering collaboration Proficiency in API tooling such as Postman and Bruno Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer We gather like-minded people: Top tech minds driving innovation in AI, cloud and digital platform modernization Supportive team and agile, startup-like culture Hybrid by design mode and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Career development programs Thought leadership, mentoring, soft skills and well-being programs Certification (Anthropic, Gemini, GCP, Azure, AWS) English classes We cover it all: Stable pay Participation in the Employee Stock Purchase Plan with a 15% discount Benefits package (health insurance, multisport, shopping vouchers) Referral bonuses up to $2,000 Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and more Corporate, social and well-being events Please, note: Benefits listed above are available to employees only We are open for working with Contractors. Terms of B2B cooperation agreements are agreed individually We will reach out to selected candidates exclusively EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Technology

EPAM Systems

Lead AI Testing Engineer

Senior

Remote

Katowice, Poland

🏢 Summary: Lead AI Testing Engineer role focused on defining and automating QA strategy for AI-driven, knowledge graph, ontology, and RAG-based LLM systems. The position involves building scalable test frameworks, validating graph and ontology data, and integrating automated quality gates into CI/CD pipelines across backend, frontend, and AI layers. Candidates should have strong expertise in Python automation, semantic technologies, GenAI evaluation, and end-to-end testing for complex data-heavy systems. 🗂️ Requirements: 8+ years in software development, test automation and DevOps, Proficiency in Python test automation, Experience with pytest or equivalent frameworks, Expertise in OWL2, RDF and SPARQL, Experience testing Knowledge Graph or Data Ontology solutions, Skills in SHACL validation, Experience with GraphDB or graph databases, Ability to define QA strategy for complex data-heavy systems, Experience building automation frameworks and quality gates, Skills in SQL, SPARQL and Cypher, Experience with REST API testing, Experience with frontend automation using Playwright or equivalent, Hands-on experience with coding agents and agentic development, Experience testing RAG-based GenAI and LLM applications, Knowledge of GenAI/LLM evaluation metrics, Ability to automate GenAI/RAG evaluation at scale, Experience with Git and Azure DevOps or JIRA, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, JIRA, Azure, Jenkins, Postman, Bruno, LLM, RAG, GenAI, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Responsibilities Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate Build test harnesses to scale evaluation beyond human-in-the-loop processes Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions Establish quality gates, acceptance criteria and test coverage models Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation Integrate test suites into CI/CD pipelines and define release quality gates Champion automation-first and shift-left quality practices Leverage agentic AI and Gen AI tooling in testing and framework development Requirements 8+ years of experience in software development, testing automation and DevOps Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems Background in backend and API testing (REST) covering data validation, integration and E2E testing Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework Hands-on experience using coding agents and agentic development daily Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale Experience in Agile delivery using Git and Azure DevOps or JIRA English proficiency at B2 level or higher Nice to have Skills in semantic search testing and evaluation Familiarity with Vector Database integration and retrieval validation Background in data science covering ML concepts, data pipelines or data engineering collaboration Proficiency in API tooling such as Postman and Bruno Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Technology

EPAM Systems

Lead AI Testing Engineer

Senior

Remote

Lodz, Poland

🏢 Summary: Lead AI Testing Engineer role focused on defining and automating QA strategies for AI-driven systems, knowledge graphs, ontologies and RAG-based LLM applications. The position involves building scalable test frameworks, validating data-heavy systems, integrating quality gates into CI/CD pipelines and ensuring end-to-end quality across backend, frontend and AI layers. Candidates will work with graph technologies, GenAI evaluation frameworks and automation-first testing practices. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Python test automation with pytest or equivalent, Experience with AI/ML library integration, Knowledge Graph or Data Ontology testing experience, OWL2, RDF, SPARQL, SHACL validation, GraphDB experience, QA strategy design for complex data-heavy systems, SQL and graph query language validation, Cypher, Backend and REST API testing, Frontend test automation with Playwright or equivalent, Experience with coding agents and agentic development, RAG-based Gen AI / LLM testing and evaluation, Gen AI / LLM evaluation frameworks and metrics, Automation of Gen AI/RAG evaluation at scale, Agile delivery experience, Git, Azure DevOps or JIRA, English proficiency B2 or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, RAG, LLM, Git, Azure, JIRA, Postman, Bruno, Jenkins, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Responsibilities Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate Build test harnesses to scale evaluation beyond human-in-the-loop processes Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions Establish quality gates, acceptance criteria and test coverage models Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation Integrate test suites into CI/CD pipelines and define release quality gates Champion automation-first and shift-left quality practices Leverage agentic AI and Gen AI tooling in testing and framework development Requirements 8+ years of experience in software development, testing automation and DevOps Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems Background in backend and API testing (REST) covering data validation, integration and E2E testing Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework Hands-on experience using coding agents and agentic development daily Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale Experience in Agile delivery using Git and Azure DevOps or JIRA English proficiency at B2 level or higher Nice to have Skills in semantic search testing and evaluation Familiarity with Vector Database integration and retrieval validation Background in data science covering ML concepts, data pipelines or data engineering collaboration Proficiency in API tooling such as Postman and Bruno Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Technology

EPAM Systems

Lead AI Testing Engineer

Senior

Remote

Wroclaw, Poland

🏢 Summary: Lead AI Testing Engineer role focused on building and driving QA strategy for AI-driven, knowledge graph, ontology and RAG-based LLM systems. The position involves developing scalable automation frameworks, validating complex data and rule engines, and integrating AI evaluation into CI/CD pipelines. The role also includes end-to-end testing across backend, frontend, graph databases and GenAI applications. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Python test automation with pytest or equivalent, Experience with AI/ML library integration, Knowledge Graph or Data Ontology testing experience, OWL2 knowledge, RDF knowledge, SPARQL expertise, SHACL validation skills, GraphDB experience, QA strategy development for complex data-heavy systems, SQL data validation, Cypher query language knowledge, Backend and REST API testing, Frontend test automation with Playwright or equivalent, Experience with coding agents and agentic development, RAG-based GenAI/LLM testing experience, GenAI/LLM evaluation metrics and frameworks knowledge, Automation of GenAI/RAG evaluation at scale, Agile delivery experience, Git experience, Azure DevOps or JIRA experience, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, Jenkins, Postman, Bruno, RAG, LLM, GenAI, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Responsibilities Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate Build test harnesses to scale evaluation beyond human-in-the-loop processes Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions Establish quality gates, acceptance criteria and test coverage models Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation Integrate test suites into CI/CD pipelines and define release quality gates Champion automation-first and shift-left quality practices Leverage agentic AI and Gen AI tooling in testing and framework development Requirements 8+ years of experience in software development, testing automation and DevOps Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems Background in backend and API testing (REST) covering data validation, integration and E2E testing Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework Hands-on experience using coding agents and agentic development daily Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale Experience in Agile delivery using Git and Azure DevOps or JIRA English proficiency at B2 level or higher Nice to have Skills in semantic search testing and evaluation Familiarity with Vector Database integration and retrieval validation Background in data science covering ML concepts, data pipelines or data engineering collaboration Proficiency in API tooling such as Postman and Bruno Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer We gather like-minded people: Top tech minds driving innovation in AI, cloud and digital platform modernization Supportive team and agile, startup-like culture Hybrid by design mode and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Career development programs Thought leadership, mentoring, soft skills and well-being programs Certification (Anthropic, Gemini, GCP, Azure, AWS) English classes We cover it all: Stable pay Participation in the Employee Stock Purchase Plan with a 15% discount Benefits package (health insurance, multisport, shopping vouchers) Referral bonuses up to $2,000 Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and more Corporate, social and well-being events Please, note: Benefits listed above are available to employees only We are open for working with Contractors. Terms of B2B cooperation agreements are agreed individually We will reach out to selected candidates exclusively EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.

Technology

EPAM Systems

Lead AI Testing Engineer

Senior

Remote

Warsaw, Poland

🏢 Summary: Lead AI Testing Engineer role focused on defining QA strategy and building automation frameworks for AI-driven systems, knowledge graphs, ontologies and RAG-based LLM features. The position covers end-to-end testing across data ingestion, graph layers, rule engines, backend APIs and frontend applications with integration into CI/CD pipelines. The offer includes remote work flexibility, professional development programs, certifications and comprehensive benefits. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Python test automation with pytest or equivalent, Experience with AI/ML library integration, Knowledge Graph or Data Ontology testing experience, Expertise in OWL2, Expertise in RDF, Expertise in SPARQL, SHACL validation skills, Experience with GraphDB or similar graph databases, QA strategy definition for complex data-heavy systems, SQL data validation skills, SPARQL and Cypher query language skills, Backend and REST API testing experience, Frontend test automation with Playwright or equivalent, Hands-on experience with coding agents and agentic development, Experience testing RAG-based Gen AI / LLM applications, Knowledge of Gen AI / LLM evaluation frameworks and metrics, Experience automating Gen AI/RAG evaluation at scale, Agile delivery experience, Experience with Git, Experience with Azure DevOps or JIRA, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, Jenkins, Postman, Bruno, LLM, RAG, GenAI 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Responsibilities Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate Build test harnesses to scale evaluation beyond human-in-the-loop processes Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions Establish quality gates, acceptance criteria and test coverage models Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation Integrate test suites into CI/CD pipelines and define release quality gates Champion automation-first and shift-left quality practices Leverage agentic AI and Gen AI tooling in testing and framework development Requirements 8+ years of experience in software development, testing automation and DevOps Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems Background in backend and API testing (REST) covering data validation, integration and E2E testing Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework Hands-on experience using coding agents and agentic development daily Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale Experience in Agile delivery using Git and Azure DevOps or JIRA English proficiency at B2 level or higher Nice to have Skills in semantic search testing and evaluation Familiarity with Vector Database integration and retrieval validation Background in data science covering ML concepts, data pipelines or data engineering collaboration Proficiency in API tooling such as Postman and Bruno Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.