Fullstack QA Engineer

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based systems through both manual and automated approaches. The position emphasizes building scalable automation frameworks, LLM test harnesses, and robust evaluation processes for Gen AI features. You will collaborate cross-functionally to ensure reliability, accuracy, and quality of AI-driven applications. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents or LLM-based systems, Experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent framework, Expertise in prompt engineering and LLM evaluation metrics, Experience evaluating Gen AI/LLM applications (grounding, hallucination, determinism), Knowledge of precision, recall, criteria recall metrics, Experience with Jira, QMetry or TestRail, Experience with CI/CD integration and version control systems, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, Jira, QMetry, TestRail, AWS, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Proficiency in at least one programming language for test automation (Python, Java, JavaScript, TypeScript, C#) Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and automating quality assurance for Gen AI and LLM-based applications, including AI agents and integrated systems. The position combines manual and automated testing, development of LLM test harnesses, and evaluation of model quality metrics such as grounding, precision, and recall. Work is conducted in collaboration with cross-functional teams, with working hours aligned to 13:00–21:00 Polish time. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent framework, Experience integrating AI/ML libraries in automated tests, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of Gen AI/LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems and CI/CD integration, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Proficiency in at least one programming language for test automation (Python, Java, JavaScript, TypeScript, C#) Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based applications through both manual and automated approaches. The position centers on building scalable LLM test harnesses, designing Gen AI evaluation frameworks, and ensuring reliability, accuracy, and performance of AI-driven systems. The role requires strong Python-based automation skills and experience with AI evaluation metrics, cloud environments, and CI/CD integration. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Experience with manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of evaluation metrics: precision, recall, criteria recall, efficiency, Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git, Copilot 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based systems through both manual and automated approaches. The position emphasizes building scalable automation frameworks, LLM test harnesses, and evaluating GenAI features using advanced metrics. The role requires strong Python automation skills, experience with AI evaluation frameworks, and collaboration across product and engineering teams. 🗂️ Requirements: 5+ years of software QA experience, At least 1 year testing AI agents or LLM-based systems, Experience with manual and automated testing of AI agents, Strong Python programming for test automation, Experience with pytest or equivalent framework, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry, or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Proficiency in at least one programming language for test automation (Python, Java, JavaScript, TypeScript, C#) Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and automating evaluation of Gen AI and LLM-based applications, including AI agents and integrated systems. The position combines manual and automated testing, development of LLM test harnesses, and implementation of AI evaluation frameworks to ensure reliability, accuracy, and performance. Work is conducted in collaboration with cross-functional teams, with working hours aligned to 13:00–21:00 Polish time. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI / LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of Gen AI / LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems and CI/CD integration, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based applications through both manual and automated approaches. The position emphasizes building scalable LLM test harnesses, designing GenAI evaluation frameworks, and ensuring quality across AI-driven systems, APIs, and databases. Requires strong Python automation skills and hands-on experience with AI evaluation metrics and cloud environments. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming for test automation, Experience with pytest or equivalent framework, Expertise in AI agent frameworks and prompt engineering, Experience evaluating GenAI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, Git, CI/CD 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Proficiency in at least one programming language for test automation (Python, Java, JavaScript, TypeScript, C#) Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and evaluating AI agents and LLM-based systems using both manual and automated approaches. The position emphasizes building scalable LLM test harnesses, designing GenAI evaluation frameworks, and ensuring reliability, accuracy, and performance of AI-driven applications. Work is conducted in collaboration with cross-functional teams with required availability from 13:00 to 21:00 Polish time. 🗂️ Requirements: 5+ years of experience in software QA, Minimum 1 year of experience testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent automation frameworks, Experience integrating AI/ML libraries in test automation, Expertise in AI agent frameworks and prompt engineering, Experience evaluating GenAI/LLM systems including grounding and hallucination checks, Knowledge of evaluation metrics: precision, recall, criteria recall, determinism, Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, Git, CI/CD 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: The offer is for a QA Engineer to support the full Software Development Lifecycle in a fast-paced product environment, leveraging AI-assisted tools to design, generate, and maintain test assets. The role focuses on embedding quality across planning, development, and release stages while collaborating closely with developers and product teams. It combines traditional QA practices with modern AI-driven and automation-oriented approaches in a remote setup. 🗂️ Requirements: Basic to intermediate experience in software QA, testing, or software development, Knowledge of at least one programming or scripting language, Ability to use AI-assisted development tools in QA processes, Understanding of SDLC processes, Ability to collaborate within cross-functional technical teams, Critical thinking in identifying and mitigating quality risks 📃 Skills: QA, SDLC, JavaScript, Python, Java, AI, Automation, CI/CD, Agile, Scrum 🏢 Description: About the Role We are looking for a proactive and adaptable QA Engineer to support the Software Development Lifecycle (SDLC) for a fast-moving product track. This role is ideal for someone who is technically curious, AI-savvy and eager to collaborate closely with developers, product teams and QA leadership. Rather than relying solely on traditional manual QA approaches, this role will leverage AI-assisted development tools to help design, generate, and maintain test assets and quality processes. Key Responsibilities Support the Software Development Lifecycle (SDLC) by contributing to quality processes across planning, development, and release stages, Use AI-assisted development tools to help design, generate, and maintain test assets, Collaborate closely with developers, product teams, and QA leadership to define and uphold quality standards, Communicate findings clearly and adapt processes in response to a changing product environment, Demonstrate critical thinking in identifying quality risks and propose practical, pragmatic solutions. Required Qualifications Basic to intermediate experience in software QA, testing, or software development, Knowledge of at least one programming or scripting language (JavaScript, Python, Java or equivalent), Interest in AI-assisted development and modern engineering practices, Strong communication and collaboration skills, Proactive and curious approach - willing to learn and grow, Ability to think critically and adapt quickly in a changing environment. Nice to Have Exposure to test automation tools or CI/CD processes, Familiarity with Agile/Scrum environments, Experience using AI coding assistants or productivity tools. All our positions are Remote and are available as Full Time Employment or B2B Contracts

🏢 Summary: The offer is for a QA Engineer to join an Agile cross-functional team, focusing on manual and automated testing of web and mobile applications with strong emphasis on Playwright/Python automation and API testing. The role involves building and maintaining automated test suites, supporting CI/CD processes, and leveraging AI-powered QA tools to enhance test creation and defect analysis. You will collaborate closely with developers and product teams to ensure high-quality software delivery. 🗂️ Requirements: 4+ years of commercial experience in software testing, Hands-on experience in manual and automated testing, Good knowledge of Playwright, Good knowledge of Python, Solid API testing experience with Postman, Experience with CI/CD pipelines, Experience with test case management tools, Ability to design and maintain automated test scripts, Experience working in Agile/Scrum environment, Fluent English (spoken and written) 📃 Skills: Playwright, Python, Postman, GitLab, CI/CD, Jira, Confluence, Docker, Kubernetes, MySQL, MariaDB, PHP, Symfony, Laravel, JavaScript, TypeScript, Go, Flutter, Dart, RabbitMQ, UNIX 🏢 Description: About the Role We are looking for a QA Engineer to join our cross-functional Agile team and ensure the delivery of high-quality software across web platforms. In this role, you’ll work closely with developers, product owners, and other QAs to execute manual and automated tests, create test cases, and contribute to test automation development (Playwright/Python). You’ll also get hands-on experience with modern AI-powered QA tools, helping to optimize test case generation, defect analysis, and regression suite maintenance. Your Responsibilities Testing web and mobile applications with all available methods (including automated regression suites, API validation, and exploratory testing of the app) Perform risk-based exploratory testing to identify critical defects beyond the scope of scripted checks, ensuring testing efforts are always prioritized where they matter most. Design, maintain, and improve automated test scripts in Playwright/Python. Analyze the product and its business/technical requirements to design high-value test scenarios and scripts that serve as a foundation for robust automation. Perform API testing (Postman), including automated API test execution. Collaborate with developers and product teams in an Agile/Scrum environment. Support CI/CD testing processes (GitLab CI). Report and track defects using Jira and ensure proper documentation in Confluence. Participate in the introduction and daily use of AI testing tools (e.g., for test creation, defect analysis). Contribute to improving QA processes and sharing knowledge with the team. Requirements Must have: +4 years of experience in software testing in commercial projects. High level of ownership and proactive mindset Exceptional critical thinking skills and the ability to explain your testing logic. Hands-on experience in both manual and automated testing. Good knowledge of Playwright and Python for test automation. Solid API testing experience with Postman (manual & automated). Familiarity with CI/CD pipelines (GitLab CI preferred). Experience with test case management tools. Strong analytical and problem-solving skills. Good time management and ability to work independently on assigned tasks. Fluent communication skills in English (spoken & written). Nice to have: Practical experience with AI tools (Claude Code, Junie etc.). Experience testing mobile applications (real devices or device farms). Knowledge of Agile/Scrum methodologies. ISTQB Foundation or Advanced certification (or equivalent practical experience (we value your actual testing skills over certificates).). Experience with performance and/or security testing. Exposure to AI/ML testing tools (e.g., intelligent test creation). Our Tech & QA Stack Languages/Frameworks: PHP 8+/Symfony 6+/Laravel 9+, JS/TypeScript, Go, PHPUnit, Codeception, Flutter/Dart Databases: MySQL, MariaDB Infrastructure: UNIX, Kubernetes, RabbitMQ Testing: Playwright, Python, Dart, Postman CI/CD: GitLab CI, Code Magic Containerization: Docker AI Tools: Claude Cod, Cursor, Jetbrains AI, Gemini for Google Workspace Tools: Jira, Requirements & Test Management for Jira, Confluence, Slack, IDE (freedom of choice),  G Suite, Sentry Engineering Culture Agile Scrum + XP elements OOP, TDD, DDD, SOLID, Clean Code Code reviews, pull requests, automation mindset QA as part of cross-functional team Benefits and Perks Private healthcare Cafeteria system Training budget €800 per year Integration meetings (internal training, hackathon, integration trips and meetings) Referral Bonus Startup atmosphere No dress code Flexible hours Flexible working format - remote or hybrid Modern office, Chillout zone, Standing desk, parking for bicycles, and a cloakroom with showers Regular office lunches and integration meetings Some international travels included

🏢 Summary: Manual QA Engineer role focused on testing web applications, ensuring software quality, and supporting Agile delivery teams through test design, execution, and defect tracking. The position involves functional, integration, regression, and user acceptance testing with collaboration across the development process. Remote B2B cooperation is offered with access to modern technologies and long-term projects. 🗂️ Requirements: 2–3 years of manual testing experience, Manual testing of web applications, Knowledge of software testing methodologies, Test design techniques, Defect management processes, Writing test cases, Experience with Xray, Experience with Confluence, Experience with Jira, Testing responsive web applications, Cross-browser testing, Agile methodology, Kanban, Polish communication skills, English communication skills 📃 Skills: QA, Testing, Xray, Confluence, Jira, Agile, Kanban, REST, Microservices, API, MFA, 2FA 🏢 Description: QA Engineer Our team is seeking a QA Engineer who will focus on manual testing. The ideal candidate should have strong testing knowledge, attention to detail, and a proactive approach to ensuring software quality. Responsibilities - How you will get the job done: Actively contribute to the project delivery team, ensuring the quality of delivered IT solutions. Review requirements, specifications, and technical design documents to provide timely and meaningful feedback. Design, execute, and document manual test scenarios and test cases. Estimate, prioritize, and plan test activities. Identify, record, and track defects, maintaining clear communication within the team. Execute functional, integration, regression and user acceptance testing activities. Keep track of new testing trends and tools to suggest improvements to the project's QA process. Effectively manage time to deliver results within established deadlines. Requirements: Strong communication skills in both Polish and English. Min. 2–3 years of experience in manual testing of web applications. Strong understanding of software testing methodologies, test design techniques, and defect management processes. Hands-on experience in writing clear and comprehensive test cases. Familiarity with tools supporting test activities, including Xray, Confluence, and Jira. Experience testing responsive web applications across different browsers and devices. Experience in software development within the Agile/Kanban process. Great at working with others and sharing knowledge, but also perfectly fine handling tasks on their own. Nice to have Knowledge of API testing (e.g., REST, Microservices). Experience testing web applications and customer-facing systems. Experience testing authentication and authorisation features (login, registration, password reset, MFA/2FA). Experience analysing logs and supporting defect investigation. Basic knowledge of browser developer tools. What we offer: The opportunity to participate in a variety of projects Remote work from any location B2B cooperation with a stable, long-term project environment Competitive daily rate aligned with your experience and market standards Access to modern technologies and challenging, impactful assignments Supportive, collaborative team culture focused on knowledge exchange and innovation