Senior Software Test Engineer - AI Agent

🏢 Summary: Senior Software Engineer role focused on building scalable backend systems and customer-facing APIs for AI-driven logistics solutions. The position centers on designing core service layers, workflow engines, and integrations with HERE and third-party APIs to enable intelligent delivery optimization at scale. It emphasizes hands-on backend development, system reliability, and production-quality engineering. 🗂️ Requirements: Strong backend development experience, Proficiency in Python and/or TypeScript, Experience building APIs and backend systems, Experience designing scalable and distributed systems, Understanding of API design and data modeling, Experience with cloud platforms (AWS, GCP, or Azure), Experience with CI/CD practices, Experience integrating external APIs, Ability to debug and troubleshoot production systems 📃 Skills: Python, TypeScript, API, REST, DistributedSystems, AWS, GCP, Azure, CI/CD, LLM, RAG, Integration, DataModeling, Cloud, Git 🏢 Description: What's the role? We are looking for a Senior Software Engineer to build core backend systems powering next-generation agentic AI solutions for the logistics industry. The systems developed by this team optimize deliveries at scale improving driver efficiency, increasing reliability, reducing operational costs, and lowering environmental impact. In this role, you will focus on building the core service layer, customer-facing APIs, workflow engines, and integrations with HERE APIs and third-party systems that enable intelligent logistics workflows. You will work closely with senior engineers and technical leaders to design and implement scalable, reliable systems that support AI-driven planning and optimization use cases. This is a hands-on engineering role with a strong focus on backend development, system reliability, and production-quality delivery . What You’ll Do: Design, build, and maintain core backend services, APIs, and workflow orchestration systems for logistics applications Implement integrations with HERE APIs and external third-party systems to support real-world logistics workflows Translate product and technical requirements into scalable and maintainable system components Write clean, efficient, and well-tested code in Python and/or TypeScript Contribute to system design discussions and implement solutions aligned with the overall architecture Build and maintain customer-facing APIs with a strong focus on usability, performance, and reliability Ensure high-quality delivery through testing, monitoring, and debugging in production environments Collaborate with engineers, product managers, and domain experts to deliver end-to-end features Continuously improve system performance, scalability, and operational efficiency Leverage AI-assisted development tools to improve development speed and code quality Who are you? What We’re Looking For: Software engineering experience with strong backend development skills Proficiency in Python and/or TypeScript, with experience building APIs and backend systems Experience designing and building scalable services and working with distributed systems Strong understanding of API design, data modeling, and system integration patterns Experience working with cloud platforms (AWS, GCP, or Azure) and modern CI/CD practices Familiarity with building or integrating AI/ML systems (e.g., working with LLM APIs, RAG-based systems, or similar) is a plus Experience integrating with external APIs and handling real-world data and system constraints Solid debugging and problem-solving skills in production environments Ability to work independently on well-defined problems and collaborate effectively within a team Good communication skills and a pragmatic approach to engineering challenges What We Offer: The opportunity to tackle meaningful and challenging problems A chance to continuously learn and stay on top of the latest technology trends Work that has real-world impact, shaping the future of mobility and technology Regular feedback to help you grow and succeed in your role A collaborative and supportive team environment where your contributions are valued Competitive salary plus bonus Flexible working hours and a hybrid working environment Medical coverage for you and your family This role is eligible for Creative Tax Incentive scheme in Poland” or KUP (Autorskie Koszty Uzyskania Przychodu) Option to work on a B2B contract (please note: benefits, bonus and KUP do not apply in this case) Change is HERE. Apply Now! #LI-AK8   #LI-HYBRID Life at HERE in Poland comes with a competitive total rewards package designed to support your health, wellbeing, and performance. This includes a base salary, a Short-Term Incentive (STI) bonus (percentage based on role), a creative tax advantage for eligible positions, private medical care (including dental), life insurance, a meal allowance, vision reimbursement, a remote work allowance (if applicable), access to MyBenefit and Multisport programs, and various wellbeing initiatives. Paid time off, sick leave, and parental leave are provided in accordance with the Polish Labor Code. As part of HERE Technologies employment process, candidates will be required to successfully complete a pre-employment screening process. This offer and any related claims are subject to the successful completion of a pre-employment screening. This will involve employment, education, and criminal verification if applicable. HERE is an equal opportunity employer. We evaluate qualified applicants without regard to race, color, age, gender identity, sexual orientation, marital status, parental status, religion, sex, national origin, disability, veteran status, and other legally protected characteristics. Who are we? HERE Technologies is a location data and technology platform company. We empower our customers to achieve better outcomes – from helping a city manage its infrastructure or a business optimize its assets to guiding drivers to their destination safely. At HERE we take it upon ourselves to be the change we wish to see. We create solutions that fuel innovation, provide opportunity and foster inclusion to improve people’s lives. If you are inspired by an open world and driven to create positive change, join us. Learn more about us on our YouTube Channel. Apply for this job online Email this job to a friend Share on your newsfeed Connect With Us! Not ready to apply? Connect with us to receive industry updates and job alerts related to your interests!

🏢 Summary: Full-Stack Senior Software Engineer role focused on building user-facing web applications and conversational interfaces for AI-driven logistics systems. The position involves designing intuitive human–AI interaction flows and integrating frontend applications with backend services and agent-based AI systems. It combines strong frontend development with backend integration to deliver scalable, high-quality end-to-end solutions. 🗂️ Requirements: Strong full-stack software engineering experience, Proficiency in JavaScript and TypeScript, Experience with React or similar frontend frameworks, Solid understanding of backend systems and APIs, Experience integrating frontend with backend services and APIs, Experience building user-facing applications with focus on usability and performance, Understanding of UX principles and interaction design, Experience working with asynchronous workflows and real-world data, Familiarity with AI-powered applications or conversational interfaces, Experience with cloud platforms (AWS, GCP, or Azure), Strong debugging skills across frontend and backend 📃 Skills: JavaScript, TypeScript, React, Next.js, APIs, REST, UX, LLM, AWS, GCP, Azure, HTML, CSS 🏢 Description: What's the role? We are looking for a strong Full-Stack Senior Software Engineer to build intuitive, high-quality user experiences for next-generation agentic AI solutions in the logistics domain. The systems developed by this team optimize deliveries at scale improving driver efficiency, increasing reliability, reducing operational costs, and lowering environmental impact. In this role, you will develop user-facing web applications and conversational interfaces that enable users to interact seamlessly with AI-powered systems. You will focus on creating clear, efficient, and reliable human–AI interaction flows, while integrating tightly with backend services, agent systems, and APIs. This is a hands-on role that requires both strong frontend expertise and solid backend understanding , along with a sharp sense of product and user experience. What You’ll Do: Design and build user-facing web applications and conversational interfaces for AI-driven logistics workflows Develop intuitive human AI interaction flows that make complex system behavior understandable and usable Implement frontend applications using modern frameworks (e.g., React, Next.js) with a strong focus on performance and usability Integrate UI layers with backend services, agent systems, and APIs to deliver end-to-end functionality Collaborate with backend engineers and AI engineers to ensure smooth data flow and system interactions Translate product requirements and user needs into clean, usable, and scalable UI solutions Contribute to backend components where needed (e.g., API integration, lightweight services, orchestration logic) Ensure high-quality delivery through testing, monitoring, and continuous iteration based on user feedback Improve overall user experience through better workflows, interaction patterns, and system responsiveness Leverage AI-assisted development tools to improve development speed and code quality Who are you? What We’re Looking For: Software engineering experience with strong full-stack development skills Strong proficiency in frontend technologies (JavaScript/TypeScript, React or similar frameworks) Solid understanding of backend systems, APIs, and service integration patterns Experience building user-facing applications with a focus on usability, performance, and maintainability Good understanding of UX principles and ability to design flows that simplify complex systems Experience integrating with APIs and working with real-world data and asynchronous workflows Familiarity with AI-powered applications (e.g., conversational interfaces, LLM integrations, agent-driven systems) is a strong plus Experience with cloud platforms (AWS, GCP, or Azure) and modern development practices Strong debugging and problem-solving skills across frontend and backend layers Ability to work independently on well-defined problems and collaborate effectively in cross-functional teams Good communication skills and a strong sense of ownership What We Offer: The opportunity to tackle meaningful and challenging problems A chance to continuously learn and stay on top of the latest technology trends Work that has real-world impact, shaping the future of mobility and technology Regular feedback to help you grow and succeed in your role A collaborative and supportive team environment where your contributions are valued Competitive salary plus bonus Flexible working hours and a hybrid working environment Medical coverage for you and your family This role is eligible for Creative Tax Incentive scheme in Poland” or KUP (Autorskie Koszty Uzyskania Przychodu) Option to work on a B2B contract (please note: benefits, bonus and KUP do not apply in this case) Change is HERE. Apply Now! #LI-AK8   #LI-HYBRID Life at HERE in Poland comes with a competitive total rewards package designed to support your health, wellbeing, and performance. This includes a base salary, a Short-Term Incentive (STI) bonus (percentage based on role), a creative tax advantage for eligible positions, private medical care (including dental), life insurance, a meal allowance, vision reimbursement, a remote work allowance (if applicable), access to MyBenefit and Multisport programs, and various wellbeing initiatives. Paid time off, sick leave, and parental leave are provided in accordance with the Polish Labor Code. As part of HERE Technologies employment process, candidates will be required to successfully complete a pre-employment screening process. This offer and any related claims are subject to the successful completion of a pre-employment screening. This will involve employment, education, and criminal verification if applicable. HERE is an equal opportunity employer. We evaluate qualified applicants without regard to race, color, age, gender identity, sexual orientation, marital status, parental status, religion, sex, national origin, disability, veteran status, and other legally protected characteristics. Who are we? HERE Technologies is a location data and technology platform company. We empower our customers to achieve better outcomes – from helping a city manage its infrastructure or a business optimize its assets to guiding drivers to their destination safely. At HERE we take it upon ourselves to be the change we wish to see. We create solutions that fuel innovation, provide opportunity and foster inclusion to improve people’s lives. If you are inspired by an open world and driven to create positive change, join us. Learn more about us on our YouTube Channel. Apply for this job online Email this job to a friend Share on your newsfeed Connect With Us! Not ready to apply? Connect with us to receive industry updates and job alerts related to your interests!

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based applications through both manual and automated approaches. The position centers on building scalable LLM test harnesses, designing Gen AI evaluation frameworks, and ensuring reliability, accuracy, and performance of AI-driven systems. The role requires strong Python-based automation skills and experience with AI evaluation metrics, cloud environments, and CI/CD integration. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Experience with manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of evaluation metrics: precision, recall, criteria recall, efficiency, Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git, Copilot 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based systems through both manual and automated approaches. The position emphasizes building scalable automation frameworks, LLM test harnesses, and evaluating GenAI features using advanced metrics. The role requires strong Python automation skills, experience with AI evaluation frameworks, and collaboration across product and engineering teams. 🗂️ Requirements: 5+ years of software QA experience, At least 1 year testing AI agents or LLM-based systems, Experience with manual and automated testing of AI agents, Strong Python programming for test automation, Experience with pytest or equivalent framework, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry, or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and automating evaluation of Gen AI and LLM-based applications, including AI agents and integrated systems. The position combines manual and automated testing, development of LLM test harnesses, and implementation of AI evaluation frameworks to ensure reliability, accuracy, and performance. Work is conducted in collaboration with cross-functional teams, with working hours aligned to 13:00–21:00 Polish time. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI / LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of Gen AI / LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems and CI/CD integration, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based applications through both manual and automated approaches. The position emphasizes building scalable LLM test harnesses, designing GenAI evaluation frameworks, and ensuring quality across AI-driven systems, APIs, and databases. Requires strong Python automation skills and hands-on experience with AI evaluation metrics and cloud environments. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming for test automation, Experience with pytest or equivalent framework, Expertise in AI agent frameworks and prompt engineering, Experience evaluating GenAI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, Git, CI/CD 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and evaluating AI agents and LLM-based systems using both manual and automated approaches. The position emphasizes building scalable LLM test harnesses, designing GenAI evaluation frameworks, and ensuring reliability, accuracy, and performance of AI-driven applications. Work is conducted in collaboration with cross-functional teams with required availability from 13:00 to 21:00 Polish time. 🗂️ Requirements: 5+ years of experience in software QA, Minimum 1 year of experience testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent automation frameworks, Experience integrating AI/ML libraries in test automation, Expertise in AI agent frameworks and prompt engineering, Experience evaluating GenAI/LLM systems including grounding and hallucination checks, Knowledge of evaluation metrics: precision, recall, criteria recall, determinism, Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, Git, CI/CD 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based systems through both manual and automated approaches. The position emphasizes building scalable automation frameworks, LLM test harnesses, and robust evaluation processes for Gen AI features. You will collaborate cross-functionally to ensure reliability, accuracy, and quality of AI-driven applications. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents or LLM-based systems, Experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent framework, Expertise in prompt engineering and LLM evaluation metrics, Experience evaluating Gen AI/LLM applications (grounding, hallucination, determinism), Knowledge of precision, recall, criteria recall metrics, Experience with Jira, QMetry or TestRail, Experience with CI/CD integration and version control systems, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, Jira, QMetry, TestRail, AWS, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and automating quality assurance for Gen AI and LLM-based applications, including AI agents and integrated systems. The position combines manual and automated testing, development of LLM test harnesses, and evaluation of model quality metrics such as grounding, precision, and recall. Work is conducted in collaboration with cross-functional teams, with working hours aligned to 13:00–21:00 Polish time. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent framework, Experience integrating AI/ML libraries in automated tests, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of Gen AI/LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems and CI/CD integration, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Principal AI/ML Engineer role focused on defining architecture and delivering a scalable Agentic AI Framework, including runtime, orchestration, and reusable agentic components. The position combines hands-on development with technical leadership to enable product teams to build reliable, production-grade agent-based systems. It emphasizes extensible frameworks, LLM engineering, MLOps, and distributed system design. 🗂️ Requirements: Extensive experience delivering production-grade AI/ML systems, Experience architecting extensible software frameworks for autonomous agents or distributed systems, Strong hands-on experience with modern ML/LLM engineering, Experience building and operating agentic systems with evaluation and safety mechanisms, Solid knowledge of system design and distributed systems, Experience with API design and observability practices, Proven MLOps experience including CI/CD and model versioning, Hands-on use of AI tools in daily engineering workflows, Ability to design modular, reusable architectures with standardized APIs 📃 Skills: Python, LLM, RAG, MLOps, CI/CD, APIs, SDKs, DistributedSystems, Microservices, Orchestration, Prompting, FineTuning, Monitoring, Versioning, Caching, Optimization 🏢 Description: What's the role? We are looking for a Principal AI/ML Engineer to drive the architecture and hands-on delivery of HERE’s Agentic AI Framework. In this role, you will define technical direction, build and evolve core agentic capabilities, and help product teams deliver reliable, scalable, customer-facing agent-based systems that combine HERE Location & Navigation intelligence with modern Agentic AI techniques. You will operate with a broad scope, influencing multiple teams through technical leadership, mentoring, and cross-functional alignment. Key Responsibilities: Own the reference architecture for HERE Agentic AI Framework (runtime, orchestration, and integration patterns) Design, implement, and productionize agentic components (planning, tool-use, memory/state, evaluation, and safety/guardrails) that can be re-used at scale Translate ambiguous problem statements into end-to-end technical solutions Drive framework adoption by providing high-quality SDKs/APIs, documentation, reference implementations, and support for product teams Lead technical reviews and decision-making forums; identify and retire architectural risks and tech debt across multiple initiatives Mentor engineers and act as a force multiplier through coaching, design guidance, and pragmatic engineering standards Who are you? Extensive experience as a senior/principal engineer delivering production-grade AI/ML systems (including ownership of design, implementation, rollout, and operations) Proficiency in architecting extensible software frameworks (specifically designed for autonomous agents or complex distributed systems). Be an adept at designing modular architectures that leverage reusable software components, abstraction layers, and standardized APIs to enable rapid agent deployment (by implementing scalable, decoupled logic that integrates diverse toolsets, memory modules, and LLM-orchestration patterns) Strong hands-on skills in modern ML/LLM engineering (prompting and orchestration patterns, RAG, tool/function calling, fine-tuning/adaptation where appropriate) Experience building and operating agentic systems, including evaluation strategies, reliability patterns, and safety/guardrails Solid software engineering fundamentals: system design, distributed systems, API design, observability, and performance optimization Proven MLOps experience (data/model versioning, CI/CD, automated testing, online/offline evaluation, monitoring, and incident response) Ability to lead through influence: drive alignment across teams, communicate trade-offs, and make high-quality technical decisions Excellent communication skills with the ability to explain complex topics to engineering, product, and executive stakeholders Mandatory hands-on experience using AI tools in day-to-day engineering activities to increase productivity and effectiveness Ability to embed AI-assisted workflows into engineering practices (coding, design reviews, testing, documentation, and operations) Preferred Qualifications Experience building customer-facing AI platforms/frameworks used by multiple teams, with clear patterns for reuse and governance Experience with cost-aware LLM serving, caching, and latency optimization at scale Familiarity with location-based services, navigation, or geospatial data products Contributions to open-source, publications, patents, or recognized technical leadership in the AI/ML community What We Offer: The opportunity to tackle meaningful and challenging problems A chance to continuously learn and stay on top of the latest technology trends Work that has real-world impact, shaping the future of mobility and technology Regular feedback to help you grow and succeed in your role A collaborative and supportive team environment where your contributions are valued Competitive salary plus bonus Flexible working hours and a hybrid working environment Medical coverage for you and your family This role is eligible for Creative Tax Incentive scheme in Poland” or KUP (Autorskie Koszty Uzyskania Przychodu) Option to work on a B2B contract (please note: benefits, bonus and KUP do not apply in this case) Change is HERE. Apply Now! #LI-AK8   #LI-HYBRID Life at HERE in Poland comes with a competitive total rewards package designed to support your health, wellbeing, and performance. This includes a base salary, a Short-Term Incentive (STI) bonus (percentage based on role), a creative tax advantage for eligible positions, private medical care (including dental), life insurance, a meal allowance, vision reimbursement, a remote work allowance (if applicable), access to MyBenefit and Multisport programs, and various wellbeing initiatives. Paid time off, sick leave, and parental leave are provided in accordance with the Polish Labor Code. As part of HERE Technologies employment process, candidates will be required to successfully complete a pre-employment screening process. This offer and any related claims are subject to the successful completion of a pre-employment screening. This will involve employment, education, and criminal verification if applicable. HERE is an equal opportunity employer. We evaluate qualified applicants without regard to race, color, age, gender identity, sexual orientation, marital status, parental status, religion, sex, national origin, disability, veteran status, and other legally protected characteristics. Who are we? HERE Technologies is a location data and technology platform company. We empower our customers to achieve better outcomes – from helping a city manage its infrastructure or a business optimize its assets to guiding drivers to their destination safely. At HERE we take it upon ourselves to be the change we wish to see. We create solutions that fuel innovation, provide opportunity and foster inclusion to improve people’s lives. If you are inspired by an open world and driven to create positive change, join us. Learn more about us on our YouTube Channel. Apply for this job online Email this job to a friend Share on your newsfeed Connect With Us! Not ready to apply? Connect with us to receive industry updates and job alerts related to your interests!