June 19, 2026

Senior AI Solutions Architect with LLM and RAG

Senior • Remote

30,000 - 39,500 PLN

Wroclaw, Poland

Project overview

This project focuses on building a scalable AI platform that transforms expert knowledge into structured graph based intelligence and connects it with enterprise grade language models. The solution emphasizes semantic search, agent workflows, and robust evaluation frameworks to ensure reliable outputs.

Position overview

We are looking for a Senior AI Solutions Architect to lead the design and implementation of advanced LLM driven solutions with a strong focus on retrieval augmented generation and knowledge graph integration. You will take ownership of the full orchestration layer, shaping how structured and unstructured data is transformed into high quality, context aware AI responses.

Technology stack

Python, SQL, vector databases, graph databases, Google Cloud Platform, Cloud Spanner, Vertex AI, Gemini, LLM frameworks, embedding models, observability tools, IAM, encryption

Responsibilities

  • Design and manage end to end LLM orchestration and retrieval pipelines

  • Define embedding model selection and chunking strategies, including context window management and trade offs affecting retrieval quality and cost

  • Own the entity extraction pipeline to convert unstructured content into graph nodes and relationships

  • Implement entity resolution, relationship normalization, and deduplication processes

  • Design and refine semantic search strategies and retrieval logic across graph and vector layers

  • Develop prompt engineering approaches and agentic workflows for advanced use cases

  • Integrate graph based outputs with enterprise AI platforms such as Gemini

  • Design and maintain evaluation frameworks including ground truth dataset creation

  • Measure and improve retrieval quality using metrics such as recall, precision at K, faithfulness, and answer relevance

  • Establish systematic regression testing practices for AI pipelines

  • Optimize LLM usage costs across the full retrieval and generation lifecycle

  • Implement observability, logging, and tracing to monitor performance and reliability

Requirements

  • Experience designing and implementing LLM based systems in production environments

  • Hands on experience with retrieval augmented generation and semantic search

  • Strong understanding of embeddings, vector search, and chunking strategies

  • Experience building entity extraction pipelines and working with knowledge graphs

  • Proficiency in Python and data processing workflows

  • Understanding of prompt engineering and agent workflow design

  • Experience defining evaluation frameworks and quality metrics for AI systems

  • Familiarity with distributed systems and scalable data architectures

  • Experience implementing observability, logging, and tracing in data intensive environments

Nice to have

  • Experience with Google Cloud Platform services including Cloud Spanner and Vertex AI

  • Familiarity with enterprise AI platforms such as Gemini

  • Knowledge of cost optimization techniques for large scale LLM systems

  • Experience with graph data models and hybrid architectures combining graph, relational, and vector data

  • Exposure to advanced evaluation techniques for generative AI and ranking systems

What We Offer:

  • Vacation days: Up to 26 business days per year.

  • 10 illness/special days off per year (fully paid, no medical papers needed) for all contract types

  • Health and life insurance (Luxmed)

  • MyBenefit platform with Multisport option

  • Internal psychological support service

  • English language classes from the first working day

  • Access to external learning platforms: O’Reilly, LinkedIn Learning, Udemy, and a wide catalog of diverse internal training

  • Flexible workplace: work from the office, from home, or choose a hybrid option

  • Tech Skills Mentoring Program

  • Opportunities to develop as a public speaker, mentor, or technical interviewer

  • Fully paid idle (bench) when not involved in a project

  • Certification reimbursement (AWS, GCP, Microsoft, etc.)

 

Similar jobs you might like

Technology

emagine Polska

AI Engineer - Gen AI & LLM & RAG

Senior

Remote

Warsaw, Poland

🏢 Summary: Full-time remote Senior AI Engineer role focused on designing and operating production-grade GenAI solutions, including agentic workflows and RAG pipelines integrated into enterprise systems. The position emphasizes software engineering excellence, cloud-based AI services, and orchestration of LLM capabilities rather than model research. You will build reliable, secure, and observable AI services integrated with internal platforms and APIs. 🗂️ Requirements: 5+ years of professional software development experience, Strong background in designing and operating production systems, Proficiency in Python or another backend language for AI systems, Experience building and integrating RESTful APIs, Understanding of distributed systems fundamentals, Experience with API integrations and event-driven architectures, Solid SQL and database design knowledge, Experience with vector search, Hands-on experience building end-to-end LLM applications, Experience implementing RAG pipelines, Strong knowledge of Azure cloud services, Experience with unit and integration testing, Ability to build AI evaluation and regression testing frameworks 📃 Skills: Python, REST, GraphQL, SQL, Azure, RAG, LLM, LangChain, LangGraph, ASP.NET, APIs, Embeddings, VectorSearch, AzureFunctions, AzureStorage, KeyVault, AppConfiguration, ApplicationInsights 🏢 Description: Workload: full-time Work model: 100% Remote We are seeking a Senior AI Engineer who combines strong software engineering fundamentals with hands-on experience building production GenAI solutions, including agentic workflows and Retrieval-Augmented Generation (RAG). This is an engineering and orchestration role focused on integrating LLM capabilities into enterprise systems – not a traditional model-training/ML research role. 5+ years of professional software development experience (ideally 7+ years across backend/API/integration and cloud platforms). Proven ability to ship production-grade LLM applications ( RAG , tool/function calling, agent orchestration) with reliability, security, and observability. Strong ownership mindset and passion for AI engineering – curiosity, experimentation, and a drive to continuously improve the product and the team. Excellent communication and collaboration skills; ability to guide, mentor, and unblock other engineers as we build out an AI engineering capability. Main Responsibilities: Design, build, and operate agentic AI services that orchestrate tools, workflows, and integrations across cloud systems and enterprise data sources. Implement and continuously improve RAG pipelines for tax artifacts and internal knowledge, including ingestion, retrieval tuning, and evaluation. Integrate AI workflows with existing internal platforms (e.g., assistant frameworks) and back-end services through robust APIs. Define and maintain tool/function schemas and orchestration patterns; implement streaming updates, interrupts, and human-in-the-loop steps as needed. Partner with other engineers to set direction, mentor, and unblock the team — helping establish strong foundations for the AI initiative. Build in quality from day one: automated tests, evaluation checks, monitoring/telemetry, and performance optimization for network-bound workloads. Participate in Agile ceremonies (daily scrums, refinement/grooming, planning) and collaborate through peer review, pair programming, and strong documentation. Apply best practices, design principles, and security standards throughout the SDLC, with a focus on reliability and responsible AI. Key Requirements: Strong software engineering background (not a research-only data science profile): designing, building, and operating production systems. Proficiency in at least one backend language used for AI systems (Python preferred). Hands-on experience building and integrating RESTful APIs ; GraphQL experience is a plus. Strong understanding of distributed systems fundamentals : concurrency, async I/O, resiliency/retries, rate limits, caching, and performance optimization. Experience integrating with external services and internal platforms via APIs and event-driven patterns. Solid database fundamentals (SQL design, performance, migrations); experience with vector search is required, and hybrid search stores are a plus. Hands-on experience building LLM-powered applications end-to-end : prompt design, tool/function interfaces, structured outputs, and streaming user experiences. Experience with RAG systems : document ingestion pipelines, chunking/metadata, embeddings, retrieval strategies, grounding, and evaluation. Cloud services expertise : Strong knowledge of Azure cloud services used for enterprise AI solutions (e.g., Functions, Storage, Key Vault, App Configuration, Application Insights). Development practices experience : Strong background in unit and integration testing; ability to build and maintain AI evaluation harnesses (golden sets, regression tests, automated checks). Nice to Have: Direct experience with LangGraph and/or LangChain for multi-agent workflows. Familiarity with emerging agentic ecosystem concepts/protocols (e.g., MCP, A2A, ADK or similar). Experience integrating AI services into .NET (ASP.NET Core) applications or building AI microservices that serve enterprise applications. Experience with event-driven architectures (service bus, event hubs) and real-time updates/streaming to UI. Experience working with tax/enterprise document corpora and governance constraints (PII, retention, access control). Other Details: This position is designed for remote work and has a flexible duration, allowing for innovation in the AI space within an agile environment.

Technology

emagine Polska

AI Engineer - Gen AI & LLM & RAG

Senior

Remote

Warsaw, Poland

🏢 Summary: Full-time remote Senior AI Engineer role focused on designing and operating production-grade GenAI systems, including agentic workflows and RAG pipelines integrated into enterprise platforms. The position emphasizes backend engineering, API integrations, and orchestration of LLM capabilities in secure, scalable cloud environments rather than ML research. The role also includes mentoring engineers and establishing best practices for reliable, observable AI services. 🗂️ Requirements: 5+ years professional software development experience, Proficiency in Python, Experience building production LLM applications, Hands-on experience with RAG systems, Experience designing and integrating RESTful APIs, Strong understanding of distributed systems fundamentals, Experience with SQL databases and performance optimization, Experience with vector search, Experience integrating external services via APIs, Knowledge of Azure cloud services, Experience with unit and integration testing, Ability to build AI evaluation and testing frameworks 📃 Skills: Python, REST, GraphQL, SQL, Azure, RAG, LLM, LangChain, LangGraph, ASP.NET, APIs, Microservices, DistributedSystems, Concurrency, AsyncIO, VectorSearch, Embeddings, Caching, AzureFunctions, Storage, KeyVault, AppConfiguration, ApplicationInsights, ServiceBus, EventHubs 🏢 Description: Workload: full-time Work model: 100% Remote We are seeking a Senior AI Engineer who combines strong software engineering fundamentals with hands-on experience building production GenAI solutions, including agentic workflows and Retrieval-Augmented Generation (RAG). This is an engineering and orchestration role focused on integrating LLM capabilities into enterprise systems – not a traditional model-training/ML research role. 5+ years of professional software development experience (ideally 7+ years across backend/API/integration and cloud platforms). Proven ability to ship production-grade LLM applications ( RAG , tool/function calling, agent orchestration) with reliability, security, and observability. Strong ownership mindset and passion for AI engineering – curiosity, experimentation, and a drive to continuously improve the product and the team. Excellent communication and collaboration skills; ability to guide, mentor, and unblock other engineers as we build out an AI engineering capability. Main Responsibilities: Design, build, and operate agentic AI services that orchestrate tools, workflows, and integrations across cloud systems and enterprise data sources. Implement and continuously improve RAG pipelines for tax artifacts and internal knowledge, including ingestion, retrieval tuning, and evaluation. Integrate AI workflows with existing internal platforms (e.g., assistant frameworks) and back-end services through robust APIs. Define and maintain tool/function schemas and orchestration patterns; implement streaming updates, interrupts, and human-in-the-loop steps as needed. Partner with other engineers to set direction, mentor, and unblock the team — helping establish strong foundations for the AI initiative. Build in quality from day one: automated tests, evaluation checks, monitoring/telemetry, and performance optimization for network-bound workloads. Participate in Agile ceremonies (daily scrums, refinement/grooming, planning) and collaborate through peer review, pair programming, and strong documentation. Apply best practices, design principles, and security standards throughout the SDLC, with a focus on reliability and responsible AI. Key Requirements: Strong software engineering background (not a research-only data science profile): designing, building, and operating production systems. Proficiency in at least one backend language used for AI systems (Python preferred). Hands-on experience building and integrating RESTful APIs ; GraphQL experience is a plus. Strong understanding of distributed systems fundamentals : concurrency, async I/O, resiliency/retries, rate limits, caching, and performance optimization. Experience integrating with external services and internal platforms via APIs and event-driven patterns. Solid database fundamentals (SQL design, performance, migrations); experience with vector search is required, and hybrid search stores are a plus. Hands-on experience building LLM-powered applications end-to-end : prompt design, tool/function interfaces, structured outputs, and streaming user experiences. Experience with RAG systems : document ingestion pipelines, chunking/metadata, embeddings, retrieval strategies, grounding, and evaluation. Cloud services expertise : Strong knowledge of Azure cloud services used for enterprise AI solutions (e.g., Functions, Storage, Key Vault, App Configuration, Application Insights). Development practices experience : Strong background in unit and integration testing; ability to build and maintain AI evaluation harnesses (golden sets, regression tests, automated checks). Nice to Have: Direct experience with LangGraph and/or LangChain for multi-agent workflows. Familiarity with emerging agentic ecosystem concepts/protocols (e.g., MCP, A2A, ADK or similar). Experience integrating AI services into .NET (ASP.NET Core) applications or building AI microservices that serve enterprise applications. Experience with event-driven architectures (service bus, event hubs) and real-time updates/streaming to UI. Experience working with tax/enterprise document corpora and governance constraints (PII, retention, access control). Other Details: This position is designed for remote work and has a flexible duration, allowing for innovation in the AI space within an agile environment.

Technology

EPAM Systems

Lead AI Testing Engineer

Senior

Remote

Gdansk, Poland

🏢 Summary: Lead AI Testing Engineer role focused on building and driving QA strategy for AI-driven, knowledge graph, ontology, and RAG-based LLM systems. The position involves automating evaluation frameworks, validating graph and ontology layers, integrating testing into CI/CD pipelines, and ensuring end-to-end quality across backend, frontend, and AI components. The offer includes flexible remote work within Poland, professional development programs, certifications, and comprehensive benefits. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Python test automation with pytest or equivalent, Experience with AI/ML library integration, Knowledge Graph or Data Ontology testing experience, OWL2, RDF, SPARQL, SHACL validation, GraphDB experience, SQL data validation, Cypher query language, REST API testing, Frontend test automation with Playwright or equivalent, Experience testing RAG-based Gen AI / LLM applications, Knowledge of Gen AI / LLM evaluation metrics, Automation framework development experience, CI/CD pipeline integration, Agile delivery experience, Git, Azure DevOps or JIRA, English proficiency at B2 level or higher, Hands-on experience with coding agents and agentic development 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, Jenkins, Postman, Bruno, LLM, RAG, GenAI, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Responsibilities Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate Build test harnesses to scale evaluation beyond human-in-the-loop processes Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions Establish quality gates, acceptance criteria and test coverage models Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation Integrate test suites into CI/CD pipelines and define release quality gates Champion automation-first and shift-left quality practices Leverage agentic AI and Gen AI tooling in testing and framework development Requirements 8+ years of experience in software development, testing automation and DevOps Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems Background in backend and API testing (REST) covering data validation, integration and E2E testing Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework Hands-on experience using coding agents and agentic development daily Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale Experience in Agile delivery using Git and Azure DevOps or JIRA English proficiency at B2 level or higher Nice to have Skills in semantic search testing and evaluation Familiarity with Vector Database integration and retrieval validation Background in data science covering ML concepts, data pipelines or data engineering collaboration Proficiency in API tooling such as Postman and Bruno Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer We gather like-minded people: Top tech minds driving innovation in AI, cloud and digital platform modernization Supportive team and agile, startup-like culture Hybrid by design mode and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Career development programs Thought leadership, mentoring, soft skills and well-being programs Certification (Anthropic, Gemini, GCP, Azure, AWS) English classes We cover it all: Stable pay Participation in the Employee Stock Purchase Plan with a 15% discount Benefits package (health insurance, multisport, shopping vouchers) Referral bonuses up to $2,000 Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and more Corporate, social and well-being events Please, note: Benefits listed above are available to employees only We are open for working with Contractors. Terms of B2B cooperation agreements are agreed individually We will reach out to selected candidates exclusively EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Technology

EPAM Systems

Lead AI Testing Engineer

Senior

Remote

Katowice, Poland

🏢 Summary: Lead AI Testing Engineer role focused on defining and automating QA strategy for AI-driven, knowledge graph, ontology, and RAG-based LLM systems. The position involves building scalable test frameworks, validating graph and ontology data, and integrating automated quality gates into CI/CD pipelines across backend, frontend, and AI layers. Candidates should have strong expertise in Python automation, semantic technologies, GenAI evaluation, and end-to-end testing for complex data-heavy systems. 🗂️ Requirements: 8+ years in software development, test automation and DevOps, Proficiency in Python test automation, Experience with pytest or equivalent frameworks, Expertise in OWL2, RDF and SPARQL, Experience testing Knowledge Graph or Data Ontology solutions, Skills in SHACL validation, Experience with GraphDB or graph databases, Ability to define QA strategy for complex data-heavy systems, Experience building automation frameworks and quality gates, Skills in SQL, SPARQL and Cypher, Experience with REST API testing, Experience with frontend automation using Playwright or equivalent, Hands-on experience with coding agents and agentic development, Experience testing RAG-based GenAI and LLM applications, Knowledge of GenAI/LLM evaluation metrics, Ability to automate GenAI/RAG evaluation at scale, Experience with Git and Azure DevOps or JIRA, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, JIRA, Azure, Jenkins, Postman, Bruno, LLM, RAG, GenAI, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Responsibilities Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate Build test harnesses to scale evaluation beyond human-in-the-loop processes Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions Establish quality gates, acceptance criteria and test coverage models Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation Integrate test suites into CI/CD pipelines and define release quality gates Champion automation-first and shift-left quality practices Leverage agentic AI and Gen AI tooling in testing and framework development Requirements 8+ years of experience in software development, testing automation and DevOps Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems Background in backend and API testing (REST) covering data validation, integration and E2E testing Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework Hands-on experience using coding agents and agentic development daily Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale Experience in Agile delivery using Git and Azure DevOps or JIRA English proficiency at B2 level or higher Nice to have Skills in semantic search testing and evaluation Familiarity with Vector Database integration and retrieval validation Background in data science covering ML concepts, data pipelines or data engineering collaboration Proficiency in API tooling such as Postman and Bruno Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Technology

EPAM Systems

Lead AI Testing Engineer

Senior

Remote

Lodz, Poland

🏢 Summary: Lead AI Testing Engineer role focused on defining and automating QA strategies for AI-driven systems, knowledge graphs, ontologies and RAG-based LLM applications. The position involves building scalable test frameworks, validating data-heavy systems, integrating quality gates into CI/CD pipelines and ensuring end-to-end quality across backend, frontend and AI layers. Candidates will work with graph technologies, GenAI evaluation frameworks and automation-first testing practices. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Python test automation with pytest or equivalent, Experience with AI/ML library integration, Knowledge Graph or Data Ontology testing experience, OWL2, RDF, SPARQL, SHACL validation, GraphDB experience, QA strategy design for complex data-heavy systems, SQL and graph query language validation, Cypher, Backend and REST API testing, Frontend test automation with Playwright or equivalent, Experience with coding agents and agentic development, RAG-based Gen AI / LLM testing and evaluation, Gen AI / LLM evaluation frameworks and metrics, Automation of Gen AI/RAG evaluation at scale, Agile delivery experience, Git, Azure DevOps or JIRA, English proficiency B2 or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, RAG, LLM, Git, Azure, JIRA, Postman, Bruno, Jenkins, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Responsibilities Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate Build test harnesses to scale evaluation beyond human-in-the-loop processes Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions Establish quality gates, acceptance criteria and test coverage models Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation Integrate test suites into CI/CD pipelines and define release quality gates Champion automation-first and shift-left quality practices Leverage agentic AI and Gen AI tooling in testing and framework development Requirements 8+ years of experience in software development, testing automation and DevOps Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems Background in backend and API testing (REST) covering data validation, integration and E2E testing Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework Hands-on experience using coding agents and agentic development daily Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale Experience in Agile delivery using Git and Azure DevOps or JIRA English proficiency at B2 level or higher Nice to have Skills in semantic search testing and evaluation Familiarity with Vector Database integration and retrieval validation Background in data science covering ML concepts, data pipelines or data engineering collaboration Proficiency in API tooling such as Postman and Bruno Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Technology

EPAM Systems

Lead AI Testing Engineer

Senior

Remote

Wroclaw, Poland

🏢 Summary: Lead AI Testing Engineer role focused on building and driving QA strategy for AI-driven, knowledge graph, ontology and RAG-based LLM systems. The position involves developing scalable automation frameworks, validating complex data and rule engines, and integrating AI evaluation into CI/CD pipelines. The role also includes end-to-end testing across backend, frontend, graph databases and GenAI applications. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Python test automation with pytest or equivalent, Experience with AI/ML library integration, Knowledge Graph or Data Ontology testing experience, OWL2 knowledge, RDF knowledge, SPARQL expertise, SHACL validation skills, GraphDB experience, QA strategy development for complex data-heavy systems, SQL data validation, Cypher query language knowledge, Backend and REST API testing, Frontend test automation with Playwright or equivalent, Experience with coding agents and agentic development, RAG-based GenAI/LLM testing experience, GenAI/LLM evaluation metrics and frameworks knowledge, Automation of GenAI/RAG evaluation at scale, Agile delivery experience, Git experience, Azure DevOps or JIRA experience, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, Jenkins, Postman, Bruno, RAG, LLM, GenAI, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Responsibilities Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate Build test harnesses to scale evaluation beyond human-in-the-loop processes Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions Establish quality gates, acceptance criteria and test coverage models Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation Integrate test suites into CI/CD pipelines and define release quality gates Champion automation-first and shift-left quality practices Leverage agentic AI and Gen AI tooling in testing and framework development Requirements 8+ years of experience in software development, testing automation and DevOps Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems Background in backend and API testing (REST) covering data validation, integration and E2E testing Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework Hands-on experience using coding agents and agentic development daily Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale Experience in Agile delivery using Git and Azure DevOps or JIRA English proficiency at B2 level or higher Nice to have Skills in semantic search testing and evaluation Familiarity with Vector Database integration and retrieval validation Background in data science covering ML concepts, data pipelines or data engineering collaboration Proficiency in API tooling such as Postman and Bruno Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer We gather like-minded people: Top tech minds driving innovation in AI, cloud and digital platform modernization Supportive team and agile, startup-like culture Hybrid by design mode and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Career development programs Thought leadership, mentoring, soft skills and well-being programs Certification (Anthropic, Gemini, GCP, Azure, AWS) English classes We cover it all: Stable pay Participation in the Employee Stock Purchase Plan with a 15% discount Benefits package (health insurance, multisport, shopping vouchers) Referral bonuses up to $2,000 Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and more Corporate, social and well-being events Please, note: Benefits listed above are available to employees only We are open for working with Contractors. Terms of B2B cooperation agreements are agreed individually We will reach out to selected candidates exclusively EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.

Technology

EPAM Systems

Lead AI Testing Engineer

Senior

Remote

Warsaw, Poland

🏢 Summary: Lead AI Testing Engineer role focused on defining QA strategy and building automation frameworks for AI-driven systems, knowledge graphs, ontologies and RAG-based LLM features. The position covers end-to-end testing across data ingestion, graph layers, rule engines, backend APIs and frontend applications with integration into CI/CD pipelines. The offer includes remote work flexibility, professional development programs, certifications and comprehensive benefits. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Python test automation with pytest or equivalent, Experience with AI/ML library integration, Knowledge Graph or Data Ontology testing experience, Expertise in OWL2, Expertise in RDF, Expertise in SPARQL, SHACL validation skills, Experience with GraphDB or similar graph databases, QA strategy definition for complex data-heavy systems, SQL data validation skills, SPARQL and Cypher query language skills, Backend and REST API testing experience, Frontend test automation with Playwright or equivalent, Hands-on experience with coding agents and agentic development, Experience testing RAG-based Gen AI / LLM applications, Knowledge of Gen AI / LLM evaluation frameworks and metrics, Experience automating Gen AI/RAG evaluation at scale, Agile delivery experience, Experience with Git, Experience with Azure DevOps or JIRA, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, Jenkins, Postman, Bruno, LLM, RAG, GenAI 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Responsibilities Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate Build test harnesses to scale evaluation beyond human-in-the-loop processes Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions Establish quality gates, acceptance criteria and test coverage models Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation Integrate test suites into CI/CD pipelines and define release quality gates Champion automation-first and shift-left quality practices Leverage agentic AI and Gen AI tooling in testing and framework development Requirements 8+ years of experience in software development, testing automation and DevOps Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems Background in backend and API testing (REST) covering data validation, integration and E2E testing Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework Hands-on experience using coding agents and agentic development daily Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale Experience in Agile delivery using Git and Azure DevOps or JIRA English proficiency at B2 level or higher Nice to have Skills in semantic search testing and evaluation Familiarity with Vector Database integration and retrieval validation Background in data science covering ML concepts, data pipelines or data engineering collaboration Proficiency in API tooling such as Postman and Bruno Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Technology

EPAM Systems

Lead AI Testing Engineer

Senior

Remote

Krakow, Poland

🏢 Summary: Lead AI Testing Engineer role focused on defining and automating QA strategy for AI-driven, RAG-based LLM, knowledge graph, and data ontology systems. The position involves building scalable automation frameworks, validating complex data and rule-engine workflows, and integrating quality processes into CI/CD pipelines. Candidates will work across backend, frontend, graph databases, and AI evaluation frameworks in large-scale data-heavy environments. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Proficiency in Python test automation, Experience with pytest or equivalent frameworks, Expertise in OWL2, RDF and SPARQL, Experience testing Knowledge Graph or Data Ontology solutions, Skills in SHACL validation, Experience with GraphDB or similar graph databases, Ability to define and drive QA strategy independently, Experience building automation frameworks and quality gates, Skills in SQL, SPARQL and Cypher, Experience with REST API testing, Experience with frontend automation using Playwright or equivalent, Hands-on experience with coding agents and agentic development, Experience testing RAG-based Gen AI and LLM applications, Knowledge of Gen AI and LLM evaluation metrics, Experience automating Gen AI/RAG evaluation at scale, Experience with Git and Azure DevOps or JIRA, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, LLM, RAG, GenAI, CI/CD, Jenkins, Postman, Bruno 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Responsibilities Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate Build test harnesses to scale evaluation beyond human-in-the-loop processes Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions Establish quality gates, acceptance criteria and test coverage models Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation Integrate test suites into CI/CD pipelines and define release quality gates Champion automation-first and shift-left quality practices Leverage agentic AI and Gen AI tooling in testing and framework development Requirements 8+ years of experience in software development, testing automation and DevOps Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems Background in backend and API testing (REST) covering data validation, integration and E2E testing Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework Hands-on experience using coding agents and agentic development daily Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale Experience in Agile delivery using Git and Azure DevOps or JIRA English proficiency at B2 level or higher Nice to have Skills in semantic search testing and evaluation Familiarity with Vector Database integration and retrieval validation Background in data science covering ML concepts, data pipelines or data engineering collaboration Proficiency in API tooling such as Postman and Bruno Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer We gather like-minded people: Top tech minds driving innovation in AI, cloud and digital platform modernization Supportive team and agile, startup-like culture Hybrid by design mode and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Career development programs Thought leadership, mentoring, soft skills and well-being programs Certification (Anthropic, Gemini, GCP, Azure, AWS) English classes We cover it all: Stable pay Participation in the Employee Stock Purchase Plan with a 15% discount Benefits package (health insurance, multisport, shopping vouchers) Referral bonuses up to $2,000 Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and more Corporate, social and well-being events Please, note: Benefits listed above are available to employees only We are open for working with Contractors. Terms of B2B cooperation agreements are agreed individually We will reach out to selected candidates exclusively EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups. With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.

Technology

EPAM Systems

Lead AI Testing Engineer

Senior

Remote

Poznan, Poland

🏢 Summary: Lead AI Testing Engineer role focused on defining QA strategy and automation for AI-driven systems, knowledge graphs, ontologies and RAG-based LLM applications. The position covers end-to-end testing across data ingestion, graph layers, rule engines, backend/frontend services and CI/CD pipelines using automation-first practices. Candidates will work on scalable evaluation frameworks for GenAI features, data validation and complex enterprise integrations. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Proficiency in Python test automation, Experience with pytest or equivalent framework, Expertise in Knowledge Graph or Data Ontology testing, Knowledge of OWL2, Knowledge of RDF, Knowledge of SPARQL, Experience with SHACL validation, Experience with GraphDB or similar graph databases, Ability to define and drive QA strategy independently, Experience building automation frameworks and quality gates, Skills in SQL and graph query languages, Understanding of deterministic and probabilistic systems, Experience in backend and REST API testing, Experience in frontend test automation using Playwright or equivalent, Hands-on experience with coding agents and agentic development, Experience testing RAG-based GenAI and LLM applications, Knowledge of GenAI/LLM evaluation frameworks and metrics, Experience automating GenAI/RAG evaluation at scale, Experience with Agile delivery, Knowledge of Git, Experience with Azure DevOps or JIRA, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, Jenkins, Postman, Bruno, RAG, LLM, GenAI, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Responsibilities Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate Build test harnesses to scale evaluation beyond human-in-the-loop processes Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions Establish quality gates, acceptance criteria and test coverage models Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation Integrate test suites into CI/CD pipelines and define release quality gates Champion automation-first and shift-left quality practices Leverage agentic AI and Gen AI tooling in testing and framework development Requirements 8+ years of experience in software development, testing automation and DevOps Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems Background in backend and API testing (REST) covering data validation, integration and E2E testing Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework Hands-on experience using coding agents and agentic development daily Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale Experience in Agile delivery using Git and Azure DevOps or JIRA English proficiency at B2 level or higher Nice to have Skills in semantic search testing and evaluation Familiarity with Vector Database integration and retrieval validation Background in data science covering ML concepts, data pipelines or data engineering collaboration Proficiency in API tooling such as Postman and Bruno Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer We gather like-minded people: Top tech minds driving innovation in AI, cloud and digital platform modernization Supportive team and agile, startup-like culture Hybrid by design mode and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Career development programs Thought leadership, mentoring, soft skills and well-being programs Certification (Anthropic, Gemini, GCP, Azure, AWS) English classes We cover it all: Stable pay Participation in the Employee Stock Purchase Plan with a 15% discount Benefits package (health insurance, multisport, shopping vouchers) Referral bonuses up to $2,000 Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee and more Corporate, social and well-being events Please, note: Benefits listed above are available to employees only We are open for working with Contractors. Terms of B2B cooperation agreements are agreed individually We will reach out to selected candidates exclusively EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Technology

Procter & Gamble

Senior Full-Stack Engineer

Senior

Hybrid

Warsaw, Poland

🏢 Summary: Senior Engineer role focused on architecting and building an AI-powered web platform that enables conversational interaction with enterprise data using LLMs. The position covers end-to-end development from React/TypeScript frontend to backend services and LLM orchestration, ensuring scalable, production-grade solutions. The role includes technical leadership, mentoring, and implementation of best engineering practices. 🗂️ Requirements: Bachelor’s or Master’s degree in Computer Science or related field, Experience building production web applications with React and TypeScript, Experience with Power BI, semantic models, DAX, Power BI Embedded or REST APIs, Hands-on experience with LLM integration, prompt engineering, and multi-turn conversation orchestration, Proficiency with LLM frameworks such as Vercel AI SDK, LangChain, Azure OpenAI or OpenAI APIs, Strong understanding of context management and preventing context degradation, Strong backend development skills in Node.js and/or Python, Experience building APIs, async workflows, and service integrations, Experience with state management and server-state patterns (TanStack Query, SWR or similar), Ability to lead and mentor junior engineers, Experience with CI/CD, Git-based workflows, testing, and secure development practices 📃 Skills: React, TypeScript, TanStack, PowerBI, DAX, REST, AzureOpenAI, OpenAI, LangChain, VercelAI, Node.js, Python, LLM, CI/CD, Git, Databricks, Spark, Next.js, Remix, Pinecone, Weaviate, Azure, SQL 🏢 Description: We are building an AI-powered data platform at P&G that transforms how individuals/teams interact with their data by moving beyond static dashboards to a conversational, action-oriented experience. The platform enables users to "chat with their data" against Power BI semantic models, Databricks, and other enterprise data sources. We are looking for a Senior Engineer to own the technical architecture and development of this platform from the React/TypeScript frontend to the LLM integration layer. You will also mentor junior developers and help grow the team's capabilities in modern web development and applied AI. Job Responsibilities: Architect and build a production-grade web application using React, TypeScript, and TanStack (Router, Query) as the unified interface for conversational data exploration and actionable workflows, Design and implement "chat with your data" features using LLMs (Azure OpenAI), streaming chat interfaces (e.g., Vercel AI SDK), and structured context management strategies to maintain accuracy across multi-turn conversations, Build LLM orchestration layers that manage context injection, prevent context degradation, and ensure high-quality responses as conversation complexity grows, Develop API services and server-side logic (TypeScript/Node.js, Python a plus) that orchestrate between the frontend, LLM providers, and enterprise data sources, Implement agentic AI patterns , function calling, tool use, and multi-step reasoning, to enable intelligent data exploration beyond simple Q&A, Translate business requirements into technical solutions by collaborating with analytics teams, data engineers, and business stakeholders, Lead and mentor junior developers, establish coding standards, code review practices, and a culture of continuous learning, Champion engineering best practices: CI/CD, Git-based workflows, testing, and secure development — ensuring the platform is reliable, scalable, and maintainable. Qualifications Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field, Experience with building production web applications with React and TypeScript , Experience with Power BI , semantic models, DAX, Power BI Embedded, or REST APIs, Hands-on experience with LLM integration , prompt engineering, context window management, multi-turn conversation orchestration, and streaming chat interfaces, Proficiency with LLM frameworks and tools : Vercel AI SDK, LangChain, Azure OpenAI / OpenAI APIs, or similar, Strong understanding of context management strategies , preventing context degradation over extended conversations through structured prompting, summarization, and selective context injection, Strong backend skills in Node.js and/or Python , building APIs, async workflows, and service integrations, Experience with state management and server-state patterns (TanStack Query, SWR, or similar), Demonstrated ability to lead and mentor junior engineers, Strong problem-solving skills and ability to work autonomously in ambiguous, fast-moving environments, Excellent communication skills, able to explain technical concepts to non-technical stakeholders. Nice to Have Experience with Databricks, Spark, or similar data platforms, Experience with TanStack Start or similar full-stack React meta-frameworks (Next.js, Remix), Knowledge of vector databases (Pinecone, Weaviate, Azure AI Search) for retrieval-augmented workflows, Experience with Azure cloud services (Azure Containerized Apps, Azure App Service, Azure AI), Background in data engineering, pipelines, ETL, SQL. What we offer Responsibilities as of day 1. You will have project ownership and autonomy to deliver change and results from the beginning. Dynamic and encouraging work environment. At P&G our employees are at the core, we value every individual and encourage initiatives, promoting agility and work/life balance. Continuous mentoring, you will work with hardworking people and receive ongoing coaching and mentoring from your line manager and other colleagues. Corporate and functional training will enable you to succeed and develop from day one. Industry Certifications (ITIL, DevOps, MS portfolio etc), full additional benefit program like private health care, P&G Dynamic Living programs like sport cards, in-office fitness center, PG stock options, saving plans, lunch subsidy, regular salary increases and possible promotions, flexible work arrangements, mentoring programs & trainings. Big Picture understanding of P&G IT and Product Supply organization and its Services in global multi-functional teams with several locations across continents. At P&G #weseeequal We are an equal opportunity employer and value diversity at our company. At P&G we strive to build a culture where everyone feels welcome, included, and able to bring their full selves to work. We ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process. Please click here if you require an accommodation during the application process. Please make sure to wait to hear back from us regarding your accommodation before proceeding with the online assessment, we thank you in advance for your patience. Kindly be advised that at P&G, employment is exclusively extended on the basis of "Umowa o Pracę" (Full-time Employment Contract). Apply only if you agree to these conditions.