Senior QA Automation & Performance Test Engineer

🏢 Summary: Senior QA Automation Engineer role focused on building and maintaining Java-based automated testing frameworks for a scalable corporate action infrastructure in a global financial environment. The position involves BDD-driven automation, CI/CD integration, performance testing, and collaboration with development and DevOps teams in an agile setup. Hybrid work model with strong opportunities for professional growth and exposure to advanced technologies such as Agentic Applications. 🗂️ Requirements: 5+ years of Test Automation experience using Java, Knowledge of Java SE 8+, Experience with Maven, Experience with JUnit, Experience with AssertJ, Proficiency in Selenium, Proficiency in Cucumber, Expertise in BDD, Familiarity with Jacoco, Ability to work in multi-disciplinary teams, Willingness to work on legacy and new codebases, Upper-Intermediate English (B2+) 📃 Skills: Java, Maven, JUnit, AssertJ, Selenium, Cucumber, BDD, Jacoco, GitLab, CI/CD, Azure, CosmosDB, BlobStorage, RAG, Agentic, DevOps 🏢 Description: We are seeking a proactive and creative Senior QA Automation Engineer to join our Operations stream (DigiOps) at a leading global financial institution where operational excellence, cutting-edge technology, and quality engineering come together to drive meaningful change. In this role, you will help deliver a scalable, compliant corporate action infrastructure by designing and maintaining robust automated test frameworks. Working in a dynamic, fast-paced agile environment, you will collaborate across diverse teams to build quality into our products from the ground up. This role offers a strong mix of hands-on Java and BDD-driven automation, CI/CD integration, and exposure to advanced tech like Agentic Applications. The position follows a hybrid work model, with 3 days per week in the office. Responsibilities Create, execute and maintain automated test code and test cases using BDD methodologies to validate software functionality Collaboration with DevOps and development teams to integrate automated tests into GitLab CI/CD pipelines for rapid, reliable releases Identify, document and track software defects, collaborating with developers to resolve issues Conduct performance and load testing to ensure applications meet benchmarks and can handle enterprise-level traffic Participation in all lifecycle phases from requirements gathering to deployment, providing QA feedback during design reviews to ensure early testability Define quality metrics and generate reports to communicate testing progress, risks and quality status to stakeholders Requirements 5+ years of experience in Test Automation using Java Knowledge of Java SE (minimum v8) along with experience in Maven, JUnit and AssertJ Proficiency in Selenium and Cucumber Expertise in BDD (Behavior-Driven Development) and familiarity with Jacoco for code coverage Willingness to contribute to multi-disciplinary teams and work on older code bases as well as new initiatives Upper-Intermediate English language proficiency (B2+) Nice to have Familiarity with Azure Cosmos DB SDK and Azure Blob Storage SDK Experience with GitLab CI/CD Skills in testing Agentic Application Understanding of RAG Architecture Competency in Intelligent Document Processing testing We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Senior QA Automation Engineer role focused on building and maintaining automated testing frameworks for a scalable corporate action infrastructure in a global financial environment. The position involves Java-based BDD automation, Selenium testing, CI/CD integration with GitLab, and collaboration with DevOps and development teams in an agile setup. Hybrid work model with opportunities to work on advanced technologies such as Agentic Applications and cloud-based services. 🗂️ Requirements: 5+ years of Test Automation experience with Java, Knowledge of Java SE 8+, Experience with Maven, Experience with JUnit, Experience with AssertJ, Proficiency in Selenium, Proficiency in Cucumber, Expertise in BDD, Familiarity with Jacoco, Experience integrating tests with GitLab CI/CD, Ability to work in multi-disciplinary teams, Willingness to work on legacy and new codebases, Upper-Intermediate English (B2+) 📃 Skills: Java, Maven, JUnit, AssertJ, Selenium, Cucumber, BDD, Jacoco, GitLab, CI/CD, Azure, CosmosDB, BlobStorage, RAG, Agentic, DevOps 🏢 Description: We are seeking a proactive and creative Senior QA Automation Engineer to join our Operations stream (DigiOps) at a leading global financial institution where operational excellence, cutting-edge technology, and quality engineering come together to drive meaningful change. In this role, you will help deliver a scalable, compliant corporate action infrastructure by designing and maintaining robust automated test frameworks. Working in a dynamic, fast-paced agile environment, you will collaborate across diverse teams to build quality into our products from the ground up. This role offers a strong mix of hands-on Java and BDD-driven automation, CI/CD integration, and exposure to advanced tech like Agentic Applications. The position follows a hybrid work model, with 3 days per week in the office. Responsibilities Create, execute and maintain automated test code and test cases using BDD methodologies to validate software functionality Collaboration with DevOps and development teams to integrate automated tests into GitLab CI/CD pipelines for rapid, reliable releases Identify, document and track software defects, collaborating with developers to resolve issues Conduct performance and load testing to ensure applications meet benchmarks and can handle enterprise-level traffic Participation in all lifecycle phases from requirements gathering to deployment, providing QA feedback during design reviews to ensure early testability Define quality metrics and generate reports to communicate testing progress, risks and quality status to stakeholders Requirements 5+ years of experience in Test Automation using Java Knowledge of Java SE (minimum v8) along with experience in Maven, JUnit and AssertJ Proficiency in Selenium and Cucumber Expertise in BDD (Behavior-Driven Development) and familiarity with Jacoco for code coverage Willingness to contribute to multi-disciplinary teams and work on older code bases as well as new initiatives Upper-Intermediate English language proficiency (B2+) Nice to have Familiarity with Azure Cosmos DB SDK and Azure Blob Storage SDK Experience with GitLab CI/CD Skills in testing Agentic Application Understanding of RAG Architecture Competency in Intelligent Document Processing testing We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: The offer is for a Test Automation Engineer with strong Python expertise to design, develop, and maintain automated testing solutions for web and API applications. The role focuses on building Python-based automation frameworks, integrating tests into CI/CD pipelines, and ensuring high software quality through robust automated processes. You will collaborate with development and DevOps teams to deliver efficient and scalable testing practices. 🗂️ Requirements: 2+ years in QA Automation, Strong Python programming, Experience with PyTest or similar frameworks, Experience with Selenium for web automation, API testing with Postman or Python Requests, Knowledge of HTML, CSS, JavaScript, Experience with Git version control, Experience with Jenkins or GitLab CI, Experience with Microsoft Azure Cloud, English level B2+ 📃 Skills: Python, PyTest, Selenium, Postman, Requests, Git, Jenkins, GitLab, Azure, HTML, CSS, JavaScript, CI/CD 🏢 Description: We are looking for a skilled Test Automation Engineer with Python expertise to join our team and help ensure the delivery of high-quality software. This role involves leveraging Python-based automation frameworks, performing API testing, and executing web automation. You will work closely with teams across development and QA to ensure robust and efficient testing processes. Responsibilities Develop, maintain, and execute automated test scripts using Python and frameworks such as PyTest or similar Perform API testing using tools like Postman or the Python Requests library Execute browser automation and web validation using Selenium and knowledge of HTML, CSS, and JavaScript Collaborate with development and DevOps teams to integrate automated tests into CI/CD pipelines (e.g., Jenkins, GitLab CI) Document test results, track defects, and support the resolution of issues Assist in test planning and contribute to QA best practices Optional: Participate in Big Data testing and leverage AI-assisted testing tools where applicable Requirements 2+ years of experience in QA Automation roles Strong programming skills in Python Familiarity with version control (Git) and CI/CD tools such as Jenkins or GitLab CI Knowledge of automated testing tools and frameworks such as PyTest, Selenium, or similar Understanding of web technologies, including HTML, CSS, and JavaScript Capability to perform API testing using tools like Postman or Python Requests library Experience with Microsoft Azure Cloud English proficiency at a B2+ level Nice to have Experience with Big Data testing Knowledge of AI-assisted test automation techniques Exposure to AWS Cloud We offer/Benefits We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based applications through both manual and automated approaches. The position centers on building scalable LLM test harnesses, designing Gen AI evaluation frameworks, and ensuring reliability, accuracy, and performance of AI-driven systems. The role requires strong Python-based automation skills and experience with AI evaluation metrics, cloud environments, and CI/CD integration. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Experience with manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of evaluation metrics: precision, recall, criteria recall, efficiency, Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git, Copilot 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and automating evaluation of Gen AI and LLM-based applications, including AI agents and integrated systems. The position combines manual and automated testing, development of LLM test harnesses, and implementation of AI evaluation frameworks to ensure reliability, accuracy, and performance. Work is conducted in collaboration with cross-functional teams, with working hours aligned to 13:00–21:00 Polish time. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI / LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of Gen AI / LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems and CI/CD integration, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and evaluating AI agents and LLM-based systems using both manual and automated approaches. The position emphasizes building scalable LLM test harnesses, designing GenAI evaluation frameworks, and ensuring reliability, accuracy, and performance of AI-driven applications. Work is conducted in collaboration with cross-functional teams with required availability from 13:00 to 21:00 Polish time. 🗂️ Requirements: 5+ years of experience in software QA, Minimum 1 year of experience testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent automation frameworks, Experience integrating AI/ML libraries in test automation, Expertise in AI agent frameworks and prompt engineering, Experience evaluating GenAI/LLM systems including grounding and hallucination checks, Knowledge of evaluation metrics: precision, recall, criteria recall, determinism, Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, Git, CI/CD 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based systems through both manual and automated approaches. The position emphasizes building scalable automation frameworks, LLM test harnesses, and evaluating GenAI features using advanced metrics. The role requires strong Python automation skills, experience with AI evaluation frameworks, and collaboration across product and engineering teams. 🗂️ Requirements: 5+ years of software QA experience, At least 1 year testing AI agents or LLM-based systems, Experience with manual and automated testing of AI agents, Strong Python programming for test automation, Experience with pytest or equivalent framework, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry, or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Proficiency in at least one programming language for test automation (Python, Java, JavaScript, TypeScript, C#) Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based systems through both manual and automated approaches. The position emphasizes building scalable automation frameworks, LLM test harnesses, and robust evaluation processes for Gen AI features. You will collaborate cross-functionally to ensure reliability, accuracy, and quality of AI-driven applications. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents or LLM-based systems, Experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent framework, Expertise in prompt engineering and LLM evaluation metrics, Experience evaluating Gen AI/LLM applications (grounding, hallucination, determinism), Knowledge of precision, recall, criteria recall metrics, Experience with Jira, QMetry or TestRail, Experience with CI/CD integration and version control systems, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, Jira, QMetry, TestRail, AWS, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Proficiency in at least one programming language for test automation (Python, Java, JavaScript, TypeScript, C#) Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and automating quality assurance for Gen AI and LLM-based applications, including AI agents and integrated systems. The position combines manual and automated testing, development of LLM test harnesses, and evaluation of model quality metrics such as grounding, precision, and recall. Work is conducted in collaboration with cross-functional teams, with working hours aligned to 13:00–21:00 Polish time. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent framework, Experience integrating AI/ML libraries in automated tests, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of Gen AI/LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems and CI/CD integration, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Proficiency in at least one programming language for test automation (Python, Java, JavaScript, TypeScript, C#) Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based applications through both manual and automated approaches. The position emphasizes building scalable LLM test harnesses, designing GenAI evaluation frameworks, and ensuring quality across AI-driven systems, APIs, and databases. Requires strong Python automation skills and hands-on experience with AI evaluation metrics and cloud environments. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming for test automation, Experience with pytest or equivalent framework, Expertise in AI agent frameworks and prompt engineering, Experience evaluating GenAI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, Git, CI/CD 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Proficiency in at least one programming language for test automation (Python, Java, JavaScript, TypeScript, C#) Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.