Lead AI Testing Engineer

🏢 Summary: Lead AI Testing Engineer role focused on defining and automating QA strategy for AI-driven, knowledge graph, ontology, and RAG-based LLM systems. The position involves building scalable test frameworks, validating graph and ontology data, and integrating automated quality gates into CI/CD pipelines across backend, frontend, and AI layers. Candidates should have strong expertise in Python automation, semantic technologies, GenAI evaluation, and end-to-end testing for complex data-heavy systems. 🗂️ Requirements: 8+ years in software development, test automation and DevOps, Proficiency in Python test automation, Experience with pytest or equivalent frameworks, Expertise in OWL2, RDF and SPARQL, Experience testing Knowledge Graph or Data Ontology solutions, Skills in SHACL validation, Experience with GraphDB or graph databases, Ability to define QA strategy for complex data-heavy systems, Experience building automation frameworks and quality gates, Skills in SQL, SPARQL and Cypher, Experience with REST API testing, Experience with frontend automation using Playwright or equivalent, Hands-on experience with coding agents and agentic development, Experience testing RAG-based GenAI and LLM applications, Knowledge of GenAI/LLM evaluation metrics, Ability to automate GenAI/RAG evaluation at scale, Experience with Git and Azure DevOps or JIRA, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, JIRA, Azure, Jenkins, Postman, Bruno, LLM, RAG, GenAI, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. Responsibilities - Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate - Build test harnesses to scale evaluation beyond human-in-the-loop processes - Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios - Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation - Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions - Establish quality gates, acceptance criteria and test coverage models - Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers - Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation - Integrate test suites into CI/CD pipelines and define release quality gates - Champion automation-first and shift-left quality practices - Leverage agentic AI and Gen AI tooling in testing and framework development Requirements - 8+ years of experience in software development, testing automation and DevOps - Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration - Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL - Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness - Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates - Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems - Background in backend and API testing (REST) covering data validation, integration and E2E testing - Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework - Hands-on experience using coding agents and agentic development daily - Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks - Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale - Experience in Agile delivery using Git and Azure DevOps or JIRA - English proficiency at B2 level or higher Nice to have - Skills in semantic search testing and evaluation - Familiarity with Vector Database integration and retrieval validation - Background in data science covering ML concepts, data pipelines or data engineering collaboration - Proficiency in API tooling such as Postman and Bruno - Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer - Engineering community of industry professionals - Friendly team and enjoyable working environment - Flexible schedule and opportunity to work remotely within Poland - Chance to work abroad for up to 60 days annually - Business-driven relocation opportunities - Outstanding career roadmap - Leadership development, career advising, soft skills and well-being programs - Certification opportunities (GCP, Azure, AWS) - Unlimited access to LinkedIn Learning, Get Abstract and Cloud Guru - English classes - Stable income (Employment Contract or B2B) - Participation in the Employee Stock Purchase Plan - Benefits package including health insurance, multisport and shopping vouchers - Offices with entertainment and relaxation zones, free snacks and coffee - Referral bonuses - Corporate, social and well-being events Please note: The set of bonuses might vary based on the role.

🏢 Summary: Lead AI Testing Engineer role focused on building and driving QA strategy for AI-driven, knowledge graph, ontology, and RAG-based LLM systems. The position involves automating evaluation frameworks, validating graph and ontology layers, integrating testing into CI/CD pipelines, and ensuring end-to-end quality across backend, frontend, and AI components. The offer includes flexible remote work within Poland, professional development programs, certifications, and comprehensive benefits. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Python test automation with pytest or equivalent, Experience with AI/ML library integration, Knowledge Graph or Data Ontology testing experience, OWL2, RDF, SPARQL, SHACL validation, GraphDB experience, SQL data validation, Cypher query language, REST API testing, Frontend test automation with Playwright or equivalent, Experience testing RAG-based Gen AI / LLM applications, Knowledge of Gen AI / LLM evaluation metrics, Automation framework development experience, CI/CD pipeline integration, Agile delivery experience, Git, Azure DevOps or JIRA, English proficiency at B2 level or higher, Hands-on experience with coding agents and agentic development 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, Jenkins, Postman, Bruno, LLM, RAG, GenAI, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Quick Facts - Flexible schedule and opportunity to work remotely within Poland - Chance to work abroad for up to 60 days annually - Employment Contract or B2B - Certification opportunities including GCP, Azure and AWS Responsibilities - Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate - Build test harnesses to scale evaluation beyond human-in-the-loop processes - Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios - Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation - Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions - Establish quality gates, acceptance criteria and test coverage models - Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers - Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation - Integrate test suites into CI/CD pipelines and define release quality gates - Champion automation-first and shift-left quality practices - Leverage agentic AI and Gen AI tooling in testing and framework development Requirements - 8+ years of experience in software development, testing automation and DevOps - Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration - Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL - Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness - Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates - Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems - Background in backend and API testing (REST) covering data validation, integration and E2E testing - Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework - Hands-on experience using coding agents and agentic development daily - Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks - Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale - Experience in Agile delivery using Git and Azure DevOps or JIRA - English proficiency at B2 level or higher Nice to Have - Skills in semantic search testing and evaluation - Familiarity with Vector Database integration and retrieval validation - Background in data science covering ML concepts, data pipelines or data engineering collaboration - Proficiency in API tooling such as Postman and Bruno - Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We Offer - Engineering community of industry professionals - Friendly team and enjoyable working environment - Flexible schedule and remote work opportunities within Poland - Business-driven relocation opportunities - Outstanding career roadmap - Leadership development, career advising, soft skills and well-being programs - Unlimited access to LinkedIn Learning, Get Abstract and Cloud Guru - English classes - Stable income options - Participation in the Employee Stock Purchase Plan - Benefits package including health insurance, multisport and shopping vouchers - Modern office spaces with entertainment and relaxation zones - Referral bonuses - Corporate, social and well-being events Please note that the set of bonuses may vary depending on the role.

🏢 Summary: Lead AI Testing Engineer role focused on defining and automating QA strategy for AI-driven, RAG-based LLM, knowledge graph, and data ontology systems. The position involves building scalable automation frameworks, validating complex data and rule-engine workflows, and integrating quality processes into CI/CD pipelines. Candidates will work across backend, frontend, graph databases, and AI evaluation frameworks in large-scale data-heavy environments. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Proficiency in Python test automation, Experience with pytest or equivalent frameworks, Expertise in OWL2, RDF and SPARQL, Experience testing Knowledge Graph or Data Ontology solutions, Skills in SHACL validation, Experience with GraphDB or similar graph databases, Ability to define and drive QA strategy independently, Experience building automation frameworks and quality gates, Skills in SQL, SPARQL and Cypher, Experience with REST API testing, Experience with frontend automation using Playwright or equivalent, Hands-on experience with coding agents and agentic development, Experience testing RAG-based Gen AI and LLM applications, Knowledge of Gen AI and LLM evaluation metrics, Experience automating Gen AI/RAG evaluation at scale, Experience with Git and Azure DevOps or JIRA, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, LLM, RAG, GenAI, CI/CD, Jenkins, Postman, Bruno 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. Responsibilities - Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate - Build test harnesses to scale evaluation beyond human-in-the-loop processes - Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios - Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation - Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions - Establish quality gates, acceptance criteria and test coverage models - Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers - Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation - Integrate test suites into CI/CD pipelines and define release quality gates - Champion automation-first and shift-left quality practices - Leverage agentic AI and Gen AI tooling in testing and framework development Requirements - 8+ years of experience in software development, testing automation and DevOps - Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration - Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL - Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness - Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates - Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems - Background in backend and API testing (REST) covering data validation, integration and E2E testing - Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework - Hands-on experience using coding agents and agentic development daily - Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks - Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale - Experience in Agile delivery using Git and Azure DevOps or JIRA - English proficiency at B2 level or higher Nice to have - Skills in semantic search testing and evaluation - Familiarity with Vector Database integration and retrieval validation - Background in data science covering ML concepts, data pipelines or data engineering collaboration - Proficiency in API tooling such as Postman and Bruno - Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer - Engineering community of industry professionals - Friendly team and enjoyable working environment - Flexible schedule and opportunity to work remotely within Poland - Chance to work abroad for up to 60 days annually - Business-driven relocation opportunities - Outstanding career roadmap - Leadership development, career advising, soft skills and well-being programs - Certification opportunities (GCP, Azure, AWS) - Unlimited access to LinkedIn Learning, Get Abstract and Cloud Guru - English classes - Stable income (Employment Contract or B2B) - Participation in the Employee Stock Purchase Plan - Benefits package including health insurance, multisport and shopping vouchers - Modern offices with entertainment and relaxation zones, free snacks and coffee - Referral bonuses - Corporate, social and well-being events Please note that the set of bonuses may vary depending on the role.

🏢 Summary: Lead AI Testing Engineer role focused on defining QA strategy and automation for AI-driven systems, knowledge graphs, ontologies and RAG-based LLM applications. The position covers end-to-end testing across data ingestion, graph layers, rule engines, backend/frontend services and CI/CD pipelines using automation-first practices. Candidates will work on scalable evaluation frameworks for GenAI features, data validation and complex enterprise integrations. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Proficiency in Python test automation, Experience with pytest or equivalent framework, Expertise in Knowledge Graph or Data Ontology testing, Knowledge of OWL2, Knowledge of RDF, Knowledge of SPARQL, Experience with SHACL validation, Experience with GraphDB or similar graph databases, Ability to define and drive QA strategy independently, Experience building automation frameworks and quality gates, Skills in SQL and graph query languages, Understanding of deterministic and probabilistic systems, Experience in backend and REST API testing, Experience in frontend test automation using Playwright or equivalent, Hands-on experience with coding agents and agentic development, Experience testing RAG-based GenAI and LLM applications, Knowledge of GenAI/LLM evaluation frameworks and metrics, Experience automating GenAI/RAG evaluation at scale, Experience with Agile delivery, Knowledge of Git, Experience with Azure DevOps or JIRA, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, Jenkins, Postman, Bruno, RAG, LLM, GenAI, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. You will define and execute QA practices across data ingestion, graph layers, rule evaluation engines and AI/LLM components while building automation frameworks that scale evaluation across complex, data-heavy systems. Quick Facts - Flexible schedule and opportunity to work remotely within Poland - Chance to work abroad for up to 60 days annually - Employment Contract or B2B Responsibilities - Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate - Build test harnesses to scale evaluation beyond human-in-the-loop processes - Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios - Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation - Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions - Establish quality gates, acceptance criteria and test coverage models - Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers - Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation - Integrate test suites into CI/CD pipelines and define release quality gates - Champion automation-first and shift-left quality practices - Leverage agentic AI and Gen AI tooling in testing and framework development Requirements - 8+ years of experience in software development, testing automation and DevOps - Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration - Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL - Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness - Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates - Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems - Background in backend and API testing (REST) covering data validation, integration and E2E testing - Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework - Hands-on experience using coding agents and agentic development daily - Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks - Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale - Experience in Agile delivery using Git and Azure DevOps or JIRA - English proficiency at B2 level or higher Nice to Have - Skills in semantic search testing and evaluation - Familiarity with Vector Database integration and retrieval validation - Background in data science covering ML concepts, data pipelines or data engineering collaboration - Proficiency in API tooling such as Postman and Bruno - Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We Offer - Engineering community of industry professionals - Friendly team and enjoyable working environment - Flexible schedule and remote work opportunities within Poland - Business-driven relocation opportunities - Outstanding career roadmap - Leadership development, career advising, soft skills and well-being programs - Certification opportunities (GCP, Azure, AWS) - Unlimited access to LinkedIn Learning, Get Abstract and Cloud Guru - English classes - Stable income (Employment Contract or B2B) - Participation in the Employee Stock Purchase Plan - Benefits package including health insurance, multisport and shopping vouchers - Offices with entertainment and relaxation zones, free snacks and coffee - Referral bonuses - Corporate, social and well-being events Please note that the set of bonuses may vary depending on the role.

🏢 Summary: Lead AI Testing Engineer role focused on defining QA strategy and building automation frameworks for AI-driven systems, knowledge graphs, ontologies and RAG-based LLM features. The position covers end-to-end testing across data ingestion, graph layers, rule engines, backend APIs and frontend applications with integration into CI/CD pipelines. The offer includes remote work flexibility, professional development programs, certifications and comprehensive benefits. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Python test automation with pytest or equivalent, Experience with AI/ML library integration, Knowledge Graph or Data Ontology testing experience, Expertise in OWL2, Expertise in RDF, Expertise in SPARQL, SHACL validation skills, Experience with GraphDB or similar graph databases, QA strategy definition for complex data-heavy systems, SQL data validation skills, SPARQL and Cypher query language skills, Backend and REST API testing experience, Frontend test automation with Playwright or equivalent, Hands-on experience with coding agents and agentic development, Experience testing RAG-based Gen AI / LLM applications, Knowledge of Gen AI / LLM evaluation frameworks and metrics, Experience automating Gen AI/RAG evaluation at scale, Agile delivery experience, Experience with Git, Experience with Azure DevOps or JIRA, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, Jenkins, Postman, Bruno, LLM, RAG, GenAI 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. Quick Facts - Flexible schedule and opportunity to work remotely within Poland - Chance to work abroad for up to 60 days annually - Employment Contract or B2B Responsibilities - Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate - Build test harnesses to scale evaluation beyond human-in-the-loop processes - Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios - Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation - Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions - Establish quality gates, acceptance criteria and test coverage models - Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers - Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation - Integrate test suites into CI/CD pipelines and define release quality gates - Champion automation-first and shift-left quality practices - Leverage agentic AI and Gen AI tooling in testing and framework development Requirements - 8+ years of experience in software development, testing automation and DevOps - Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration - Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL - Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness - Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates - Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems - Background in backend and API testing (REST) covering data validation, integration and E2E testing - Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework - Hands-on experience using coding agents and agentic development daily - Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks - Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale - Experience in Agile delivery using Git and Azure DevOps or JIRA - English proficiency at B2 level or higher Nice to Have - Skills in semantic search testing and evaluation - Familiarity with Vector Database integration and retrieval validation - Background in data science covering ML concepts, data pipelines or data engineering collaboration - Proficiency in API tooling such as Postman and Bruno - Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We Offer - Engineering community of industry professionals - Friendly team and enjoyable working environment - Flexible schedule and remote work opportunities within Poland - Business-driven relocation opportunities - Outstanding career roadmap - Leadership development, career advising, soft skills and well-being programs - Certification opportunities (GCP, Azure, AWS) - Unlimited access to LinkedIn Learning, Get Abstract and Cloud Guru - English classes - Stable income (Employment Contract or B2B) - Participation in the Employee Stock Purchase Plan - Benefits package including health insurance, multisport and shopping vouchers - Modern office spaces with entertainment and relaxation zones - Referral bonuses - Corporate, social and well-being events Please note: The set of bonuses might vary based on the role applied for. Only selected candidates will be contacted.

🏢 Summary: Lead AI Testing Engineer role focused on building and driving QA strategy for AI-driven, knowledge graph, ontology and RAG-based LLM systems. The position involves developing scalable automation frameworks, validating complex data and rule engines, and integrating AI evaluation into CI/CD pipelines. The role also includes end-to-end testing across backend, frontend, graph databases and GenAI applications. 🗂️ Requirements: 8+ years of experience in software development, test automation and DevOps, Python test automation with pytest or equivalent, Experience with AI/ML library integration, Knowledge Graph or Data Ontology testing experience, OWL2 knowledge, RDF knowledge, SPARQL expertise, SHACL validation skills, GraphDB experience, QA strategy development for complex data-heavy systems, SQL data validation, Cypher query language knowledge, Backend and REST API testing, Frontend test automation with Playwright or equivalent, Experience with coding agents and agentic development, RAG-based GenAI/LLM testing experience, GenAI/LLM evaluation metrics and frameworks knowledge, Automation of GenAI/RAG evaluation at scale, Agile delivery experience, Git experience, Azure DevOps or JIRA experience, English proficiency at B2 level or higher 📃 Skills: Python, pytest, OWL2, RDF, SPARQL, SHACL, GraphDB, SQL, Cypher, REST, Playwright, Git, Azure, JIRA, Jenkins, Postman, Bruno, RAG, LLM, GenAI, CI/CD 🏢 Description: We are looking for a Lead AI Testing Engineer to drive the quality strategy for AI-driven and data migration initiatives spanning knowledge graphs, ontologies and RAG-based LLM features. Responsibilities - Design and automate evaluation of RAG-based and LLM-driven features including grounding, answer accuracy, determinism/reproducibility, precision, recall and hallucination rate - Build test harnesses to scale evaluation beyond human-in-the-loop processes - Create data-driven test suites for underwriting rules and validate rule execution via SPARQL evaluator, arithmetic/threshold logic and multi-condition scenarios - Verify ontology schema correctness and instance accuracy against source data, perform reasoner consistency checks and SHACL validation - Define and drive end-to-end QA strategy covering data ingestion, ontology/graph layer, rule evaluation engine, AI/LLM layer and integration with underwriting solutions - Establish quality gates, acceptance criteria and test coverage models - Maintain automation test suites across knowledge graph, data ontology layer, backend services and frontend layers - Validate end-to-end flows across ontology, rule engine and decision output, perform contract testing and cross-system data consistency validation - Integrate test suites into CI/CD pipelines and define release quality gates - Champion automation-first and shift-left quality practices - Leverage agentic AI and Gen AI tooling in testing and framework development Requirements - 8+ years of experience in software development, testing automation and DevOps - Proficiency in Python test automation using pytest or equivalent, scripting and AI/ML library integration - Expertise in testing Knowledge Graph or Data Ontology solutions using OWL2, RDF and SPARQL - Skills in SHACL validation and graph databases such as GraphDB, including ontology validation, entity/relationship integrity, reasoner consistency checks and graph query correctness - Proven ability to define and drive QA strategy independently for complex data-heavy systems, including building automation frameworks and implementing quality gates - Skills in data validation using SQL and graph query languages such as SPARQL and Cypher, along with understanding of deterministic vs probabilistic systems - Background in backend and API testing (REST) covering data validation, integration and E2E testing - Proficiency in frontend web application test automation using Playwright or equivalent Python-compatible framework - Hands-on experience using coding agents and agentic development daily - Demonstrated experience testing and evaluating RAG-based Gen AI / LLM applications including grounding, answer accuracy and hallucination/determinism checks - Applied knowledge of Gen AI / LLM evaluation frameworks and metrics such as precision, recall, criteria recall and efficiency, with proven ability to automate Gen AI/RAG evaluation at scale - Experience in Agile delivery using Git and Azure DevOps or JIRA - English proficiency at B2 level or higher Nice to have - Skills in semantic search testing and evaluation - Familiarity with Vector Database integration and retrieval validation - Background in data science covering ML concepts, data pipelines or data engineering collaboration - Proficiency in API tooling such as Postman and Bruno - Experience with quality-gate implementation in CI/CD pipelines such as Azure DevOps or Jenkins We offer - Engineering community of industry professionals - Friendly team and enjoyable working environment - Flexible schedule and opportunity to work remotely within Poland - Chance to work abroad for up to 60 days annually - Business-driven relocation opportunities - Outstanding career roadmap - Leadership development, career advising, soft skills and well-being programs - Certification opportunities (GCP, Azure, AWS) - Unlimited access to LinkedIn Learning, Get Abstract and Cloud Guru - English classes - Stable income (Employment Contract or B2B) - Participation in the Employee Stock Purchase Plan - Benefits package including health insurance, multisport and shopping vouchers - Offices with entertainment and relaxation zones, free snacks and coffee - Referral bonuses - Corporate, social and well-being events Please note: The set of bonuses might vary based on the role.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based applications through both manual and automated approaches. The position centers on building scalable LLM test harnesses, designing Gen AI evaluation frameworks, and ensuring reliability, accuracy, and performance of AI-driven systems. The role requires strong Python-based automation skills and experience with AI evaluation metrics, cloud environments, and CI/CD integration. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Experience with manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of evaluation metrics: precision, recall, criteria recall, efficiency, Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git, Copilot 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based systems through both manual and automated approaches. The position emphasizes building scalable automation frameworks, LLM test harnesses, and evaluating GenAI features using advanced metrics. The role requires strong Python automation skills, experience with AI evaluation frameworks, and collaboration across product and engineering teams. 🗂️ Requirements: 5+ years of software QA experience, At least 1 year testing AI agents or LLM-based systems, Experience with manual and automated testing of AI agents, Strong Python programming for test automation, Experience with pytest or equivalent framework, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry, or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Proficiency in at least one programming language for test automation (Python, Java, JavaScript, TypeScript, C#) Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and automating evaluation of Gen AI and LLM-based applications, including AI agents and integrated systems. The position combines manual and automated testing, development of LLM test harnesses, and implementation of AI evaluation frameworks to ensure reliability, accuracy, and performance. Work is conducted in collaboration with cross-functional teams, with working hours aligned to 13:00–21:00 Polish time. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI / LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of Gen AI / LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems and CI/CD integration, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based applications through both manual and automated approaches. The position emphasizes building scalable LLM test harnesses, designing GenAI evaluation frameworks, and ensuring quality across AI-driven systems, APIs, and databases. Requires strong Python automation skills and hands-on experience with AI evaluation metrics and cloud environments. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming for test automation, Experience with pytest or equivalent framework, Expertise in AI agent frameworks and prompt engineering, Experience evaluating GenAI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, Git, CI/CD 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Proficiency in at least one programming language for test automation (Python, Java, JavaScript, TypeScript, C#) Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.