Senior QA Automation Engineer (Python)

🏢 Summary: Senior QA Automation Engineer role focused on building a scalable, AI-enabled test automation platform for web services and APIs. The position develops reusable Python-based automation, integrates CI/CD testing, and delivers intelligent test generation, execution, analysis, and reporting across projects. 🗂️ Requirements: 5+ years of QA automation engineering experience, Python test automation proficiency, Selenium expertise, SQL knowledge for validation and querying, Test automation framework development and maintenance experience, Web Service/API testing experience, AI Agent development familiarity, English proficiency at B2 level or higher 📃 Skills: Python, Selenium, SQL, API, CI/CD, AI 🏢 Description: We are looking for a Senior QA Automation Engineer to join an AI-enabled test automation platform initiative for the client. In this role, you will design and develop a scalable test automation platform for web services and APIs, build reusable automation solutions, integrate CI/CD pipelines, and implement AI-powered capabilities to automate test generation, execution, analysis, and reporting, improving software quality and delivery speed. You will collaborate with cross-functional teams to create intelligent automation workflows and enable efficient, data-driven quality engineering across multiple projects. Responsibilities • Design and develop an enterprise test automation platform using Python, Selenium, SQL, and API testing frameworks • Build reusable automation libraries and integrate automated testing into CI/CD pipelines • Develop AI-powered automation capabilities for intelligent test generation, execution, failure analysis, and reporting • Automate Web Service/API validation and improve test reliability through scalable framework architecture • Collaborate with development and QA teams to define automation standards and accelerate software delivery • Optimize automation infrastructure Requirements • 5+ years of experience in QA automation engineering • Proficiency in Python for test automation development • Expertise in Selenium for automated testing • Knowledge of SQL for data validation and querying • Experience building and maintaining test automation frameworks • Experience in Web Service / API testing • Familiarity with AI Agent development • English proficiency at B2 level or higher We offer • Hybrid-by-design work mode and opportunity to work remotely within Poland • Chance to work abroad for up to 60 days annually • Business-driven relocation opportunities • Career development programs • Thought leadership, mentoring, soft skills, and well-being programs • Certification opportunities: Anthropic, Gemini, GCP, Azure, AWS • English classes • Stable pay • Participation in the Employee Stock Purchase Plan with a 15% discount • Benefits package, including health insurance, multisport, and shopping vouchers • Referral bonuses up to $2,000 • Offices with entertainment and relaxation zones, table tennis, football, free snacks, and coffee • Corporate, social, and well-being events Benefits listed above are available to employees only. Contractor engagement is also available. Terms of B2B cooperation agreements are agreed individually. Only selected candidates will be contacted.

🏢 Summary: Senior QA Automation Engineer role focused on building a scalable, AI-enabled test automation platform for web services and APIs. The position involves developing reusable Python-based automation, integrating CI/CD testing, and using AI to improve test generation, execution, analysis, and reporting. 🗂️ Requirements: 5+ years of QA automation engineering experience, Python proficiency for test automation, Selenium expertise, SQL knowledge for validation and querying, Test automation framework development and maintenance experience, Web service and API testing experience, AI agent development familiarity, English at B2 level or higher 📃 Skills: Python, Selenium, SQL, API, CI/CD, AI 🏢 Description: We are looking for a Senior QA Automation Engineer to join an AI-enabled test automation platform initiative for the client. In this role, you will design and develop a scalable test automation platform for web services and APIs, build reusable automation solutions, integrate CI/CD pipelines, and implement AI-powered capabilities to automate test generation, execution, analysis, and reporting, improving software quality and delivery speed. You will collaborate with cross-functional teams to create intelligent automation workflows and enable efficient, data-driven quality engineering across multiple projects. Responsibilities - Design and develop an enterprise test automation platform using Python, Selenium, SQL, and API testing frameworks - Build reusable automation libraries and integrate automated testing into CI/CD pipelines - Develop AI-powered automation capabilities for intelligent test generation, execution, failure analysis, and reporting - Automate Web Service/API validation and improve test reliability through scalable framework architecture - Collaborate with development and QA teams to define automation standards and accelerate software delivery Requirements - 5+ years of experience in QA automation engineering - Proficiency in Python for test automation development - Expertise in Selenium for automated testing - Knowledge of SQL for data validation and querying - Experience building and maintaining test automation frameworks - Experience in Web Service / API testing - Familiarity with AI Agent development - English proficiency at B2 level or higher We offer - Hybrid-by-design work mode and opportunity to work remotely within Poland - Chance to work abroad for up to 60 days annually - Business-driven relocation opportunities - Career development programs - Thought leadership, mentoring, soft skills, and well-being programs - Certification opportunities: Anthropic, Gemini, GCP, Azure, AWS - English classes - Stable pay - Participation in the Employee Stock Purchase Plan with a 15% discount - Benefits package: health insurance, multisport, shopping vouchers - Referral bonuses up to $2,000 - Offices with entertainment and relaxation zones, table tennis and football, free snacks, coffee, and more - Corporate, social, and well-being events Benefits listed above are available to employees only. Contractor engagement is also available; terms of B2B cooperation agreements are agreed individually. Only selected candidates will be contacted.

🏢 Summary: Senior QA Automation Engineer role focused on building a scalable, AI-enabled test automation platform for web services and APIs. The position covers reusable Python-based automation, CI/CD integration, and intelligent test generation, execution, analysis, and reporting to improve software quality and delivery speed. 🗂️ Requirements: 5+ years of QA automation engineering experience, Python proficiency, Selenium expertise, SQL knowledge, Test automation framework experience, Web service and API testing experience, AI agent development familiarity, English proficiency at B2 level or higher 📃 Skills: Python, Selenium, SQL, API, CI/CD, AI 🏢 Description: We are looking for a Senior QA Automation Engineer to join an AI-enabled test automation platform initiative for the client. In this role, you will design and develop a scalable test automation platform for web services and APIs, build reusable automation solutions, integrate CI/CD pipelines, and implement AI-powered capabilities to automate test generation, execution, analysis, and reporting, improving software quality and delivery speed. You will collaborate with cross-functional teams to create intelligent automation workflows and enable efficient, data-driven quality engineering across multiple projects. Responsibilities • Design and develop an enterprise test automation platform using Python, Selenium, SQL, and API testing frameworks • Build reusable automation libraries and integrate automated testing into CI/CD pipelines • Develop AI-powered automation capabilities for intelligent test generation, execution, failure analysis, and reporting • Automate Web Service/API validation and improve test reliability through scalable framework architecture • Collaborate with development and QA teams to define automation standards and accelerate software delivery Requirements • 5+ years of experience in QA automation engineering • Proficiency in Python for test automation development • Expertise in Selenium for automated testing • Knowledge of SQL for data validation and querying • Experience building and maintaining test automation frameworks • Experience in Web Service / API testing • Familiarity with AI Agent development • English proficiency at B2 level or higher We offer • Supportive team and agile, startup-like culture • Hybrid-by-design mode and opportunity to work remotely within Poland • Chance to work abroad for up to 60 days annually • Business-driven relocation opportunities Growth opportunities • Career development programs • Thought leadership, mentoring, soft skills, and well-being programs • Certification opportunities: Anthropic, Gemini, GCP, Azure, AWS • English classes Benefits • Stable pay • Participation in the Employee Stock Purchase Plan with a 15% discount • Benefits package: health insurance, multisport, shopping vouchers • Referral bonuses up to $2,000 • Offices featuring entertainment and relaxation zones, table tennis and football, free snacks, coffee, and more • Corporate, social, and well-being events Benefits listed above are available to employees only. Contractor cooperation is available. Terms of B2B cooperation agreements are agreed individually. Only selected candidates will be contacted.

🏢 Summary: Senior QA Automation Engineer role focused on building a scalable, AI-enabled test automation platform for web services and APIs. The position covers reusable Python-based automation, CI/CD integration, and intelligent test generation, execution, analysis, and reporting. 🗂️ Requirements: 5+ years of QA automation engineering experience, Python proficiency for test automation development, Selenium expertise, SQL knowledge for data validation and querying, Experience building and maintaining test automation frameworks, Web Service/API testing experience, AI Agent development familiarity, English proficiency at B2 level or higher 📃 Skills: Python, Selenium, SQL, API, CI/CD, AI, Agents 🏢 Description: We are looking for a Senior QA Automation Engineer to join an AI-enabled test automation platform initiative for the client. In this role, you will design and develop a scalable test automation platform for web services and APIs, build reusable automation solutions, integrate CI/CD pipelines, and implement AI-powered capabilities to automate test generation, execution, analysis, and reporting, improving software quality and delivery speed. You will collaborate with cross-functional teams to create intelligent automation workflows and enable efficient, data-driven quality engineering across multiple projects. Responsibilities • Design and develop an enterprise test automation platform using Python, Selenium, SQL, and API testing frameworks • Build reusable automation libraries and integrate automated testing into CI/CD pipelines • Develop AI-powered automation capabilities for intelligent test generation, execution, failure analysis, and reporting • Automate Web Service/API validation and improve test reliability through scalable framework architecture • Collaborate with development and QA teams to define automation standards and accelerate software delivery • Optimize automation infrastructure Requirements • 5+ years of experience in QA automation engineering • Proficiency in Python for test automation development • Expertise in Selenium for automated testing • Knowledge of SQL for data validation and querying • Experience building and maintaining test automation frameworks • Experience in Web Service / API testing • Familiarity with AI Agent development • English proficiency at B2 level or higher We offer • Hybrid-by-design work mode and opportunity to work remotely within Poland • Chance to work abroad for up to 60 days annually • Business-driven relocation opportunities • Career development programs • Thought leadership, mentoring, soft skills, and well-being programs • Certification opportunities: Anthropic, Gemini, GCP, Azure, AWS • English classes • Stable pay • Participation in the Employee Stock Purchase Plan with a 15% discount • Benefits package including health insurance, multisport, and shopping vouchers • Referral bonuses up to $2,000 • Offices with entertainment and relaxation zones, table tennis and football, free snacks, coffee, and more • Corporate, social, and well-being events Benefits listed above are available to employees only. Contractor engagement is also available; terms of B2B cooperation agreements are agreed individually. Only selected candidates will be contacted.

🏢 Summary: Senior QA Automation Engineer role focused on building a scalable, AI-enabled automation platform for web services and APIs. The offer involves developing reusable Python-based test frameworks, integrating CI/CD, and using AI for test generation, execution, failure analysis, and reporting. 🗂️ Requirements: 5+ years of QA automation engineering experience, Python proficiency for test automation, Selenium expertise, SQL knowledge for data validation and querying, Experience building and maintaining test automation frameworks, Web Service/API testing experience, AI Agent development familiarity, English at B2 level or higher 📃 Skills: Python, Selenium, SQL, API, AI, CI/CD 🏢 Description: We are looking for a Senior QA Automation Engineer to join an AI-enabled test automation platform initiative for the client. In this role, you will design and develop a scalable test automation platform for web services and APIs, build reusable automation solutions, integrate CI/CD pipelines, and implement AI-powered capabilities to automate test generation, execution, analysis, and reporting, improving software quality and delivery speed. You will collaborate with cross-functional teams to create intelligent automation workflows and enable efficient, data-driven quality engineering across multiple projects. Responsibilities • Design and develop an enterprise test automation platform using Python, Selenium, SQL, and API testing frameworks • Build reusable automation libraries and integrate automated testing into CI/CD pipelines • Develop AI-powered automation capabilities for intelligent test generation, execution, failure analysis, and reporting • Automate Web Service/API validation and improve test reliability through scalable framework architecture • Collaborate with development and QA teams to define automation standards and accelerate software delivery • Optimize automation infrastructure to s Requirements • 5+ years of experience in QA automation engineering • Proficiency in Python for test automation development • Expertise in Selenium for automated testing • Knowledge of SQL for data validation and querying • Experience building and maintaining test automation frameworks • Experience in Web Service / API testing • Familiarity with AI Agent development • English proficiency at B2 level or higher We offer • Hybrid-by-design work mode and the opportunity to work remotely within Poland • Chance to work abroad for up to 60 days annually • Business-driven relocation opportunities • Career development programs • Thought leadership, mentoring, soft-skills, and well-being programs • Certification opportunities in Anthropic, Gemini, GCP, Azure, and AWS • English classes • Stable pay • Participation in the Employee Stock Purchase Plan with a 15% discount • Benefits package, including health insurance, Multisport, and shopping vouchers • Referral bonuses up to $2,000 • Offices with entertainment and relaxation zones, table tennis and football, free snacks, coffee, and more • Corporate, social, and well-being events Benefits listed above are available to employees only. Contractor cooperation is available; B2B agreement terms are agreed individually. Only selected candidates will be contacted.

🏢 Summary: Senior QA Automation Engineer role focused on building a scalable, AI-enabled test automation platform for web services and APIs. The position covers reusable Python-based automation, CI/CD integration, and intelligent testing capabilities that improve quality and delivery speed. 🗂️ Requirements: 5+ years of QA automation engineering experience, Python proficiency, Selenium expertise, SQL knowledge, Test automation framework experience, Web Service/API testing experience, AI Agent development familiarity, English proficiency at B2 level or higher 📃 Skills: Python, Selenium, SQL, API, CI/CD, AI 🏢 Description: We are looking for a Senior QA Automation Engineer to join an AI-enabled test automation platform initiative for the client. In this role, you will design and develop a scalable test automation platform for web services and APIs, build reusable automation solutions, integrate CI/CD pipelines, and implement AI-powered capabilities to automate test generation, execution, analysis, and reporting, improving software quality and delivery speed. You will collaborate with cross-functional teams to create intelligent automation workflows and enable efficient, data-driven quality engineering across multiple projects. Responsibilities • Design and develop an enterprise test automation platform using Python, Selenium, SQL, and API testing frameworks • Build reusable automation libraries and integrate automated testing into CI/CD pipelines • Develop AI-powered automation capabilities for intelligent test generation, execution, failure analysis, and reporting • Automate Web Service/API validation and improve test reliability through scalable framework architecture • Collaborate with development and QA teams to define automation standards and accelerate software delivery Requirements • 5+ years of experience in QA automation engineering • Proficiency in Python for test automation development • Expertise in Selenium for automated testing • Knowledge of SQL for data validation and querying • Experience building and maintaining test automation frameworks • Experience in Web Service / API testing • Familiarity with AI Agent development • English proficiency at B2 level or higher We offer • Hybrid-by-design work mode and an opportunity to work remotely within Poland • Chance to work abroad for up to 60 days annually • Business-driven relocation opportunities • Career development programs • Thought leadership, mentoring, soft-skills, and well-being programs • Certification opportunities in Anthropic, Gemini, GCP, Azure, and AWS • English classes • Stable pay • Participation in the Employee Stock Purchase Plan with a 15% discount • Benefits package, including health insurance, multisport, and shopping vouchers • Referral bonuses up to $2,000 • Offices with entertainment and relaxation zones, table tennis and football, and free snacks and coffee • Corporate, social, and well-being events Benefits listed above are available to employees only. Contractor arrangements are available. Terms of B2B cooperation agreements are agreed individually. Only selected candidates will be contacted.

🏢 Summary: Senior QA Automation Engineer role focused on building and maintaining Java-based automated testing frameworks for a scalable corporate action infrastructure in a global financial environment. The position involves BDD-driven automation, CI/CD integration, performance testing, and collaboration with development and DevOps teams in an agile setup. Hybrid work model with strong opportunities for professional growth and exposure to advanced technologies such as Agentic Applications. 🗂️ Requirements: 5+ years of Test Automation experience using Java, Knowledge of Java SE 8+, Experience with Maven, Experience with JUnit, Experience with AssertJ, Proficiency in Selenium, Proficiency in Cucumber, Expertise in BDD, Familiarity with Jacoco, Ability to work in multi-disciplinary teams, Willingness to work on legacy and new codebases, Upper-Intermediate English (B2+) 📃 Skills: Java, Maven, JUnit, AssertJ, Selenium, Cucumber, BDD, Jacoco, GitLab, CI/CD, Azure, CosmosDB, BlobStorage, RAG, Agentic, DevOps 🏢 Description: We are seeking a proactive and creative Senior QA Automation Engineer to join our Operations stream (DigiOps) at a leading global financial institution where operational excellence, cutting-edge technology, and quality engineering come together to drive meaningful change. In this role, you will help deliver a scalable, compliant corporate action infrastructure by designing and maintaining robust automated test frameworks. Working in a dynamic, fast-paced agile environment, you will collaborate across diverse teams to build quality into our products from the ground up. This role offers a strong mix of hands-on Java and BDD-driven automation, CI/CD integration, and exposure to advanced tech like Agentic Applications. The position follows a hybrid work model, with 3 days per week in the office. Responsibilities Create, execute and maintain automated test code and test cases using BDD methodologies to validate software functionality Collaboration with DevOps and development teams to integrate automated tests into GitLab CI/CD pipelines for rapid, reliable releases Identify, document and track software defects, collaborating with developers to resolve issues Conduct performance and load testing to ensure applications meet benchmarks and can handle enterprise-level traffic Participation in all lifecycle phases from requirements gathering to deployment, providing QA feedback during design reviews to ensure early testability Define quality metrics and generate reports to communicate testing progress, risks and quality status to stakeholders Requirements 5+ years of experience in Test Automation using Java Knowledge of Java SE (minimum v8) along with experience in Maven, JUnit and AssertJ Proficiency in Selenium and Cucumber Expertise in BDD (Behavior-Driven Development) and familiarity with Jacoco for code coverage Willingness to contribute to multi-disciplinary teams and work on older code bases as well as new initiatives Upper-Intermediate English language proficiency (B2+) Nice to have Familiarity with Azure Cosmos DB SDK and Azure Blob Storage SDK Experience with GitLab CI/CD Skills in testing Agentic Application Understanding of RAG Architecture Competency in Intelligent Document Processing testing We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Senior QA Automation Engineer role focused on building and maintaining automated testing frameworks for a scalable corporate action infrastructure in a global financial environment. The position involves Java-based BDD automation, Selenium testing, CI/CD integration with GitLab, and collaboration with DevOps and development teams in an agile setup. Hybrid work model with opportunities to work on advanced technologies such as Agentic Applications and cloud-based services. 🗂️ Requirements: 5+ years of Test Automation experience with Java, Knowledge of Java SE 8+, Experience with Maven, Experience with JUnit, Experience with AssertJ, Proficiency in Selenium, Proficiency in Cucumber, Expertise in BDD, Familiarity with Jacoco, Experience integrating tests with GitLab CI/CD, Ability to work in multi-disciplinary teams, Willingness to work on legacy and new codebases, Upper-Intermediate English (B2+) 📃 Skills: Java, Maven, JUnit, AssertJ, Selenium, Cucumber, BDD, Jacoco, GitLab, CI/CD, Azure, CosmosDB, BlobStorage, RAG, Agentic, DevOps 🏢 Description: We are seeking a proactive and creative Senior QA Automation Engineer to join our Operations stream (DigiOps) at a leading global financial institution where operational excellence, cutting-edge technology, and quality engineering come together to drive meaningful change. In this role, you will help deliver a scalable, compliant corporate action infrastructure by designing and maintaining robust automated test frameworks. Working in a dynamic, fast-paced agile environment, you will collaborate across diverse teams to build quality into our products from the ground up. This role offers a strong mix of hands-on Java and BDD-driven automation, CI/CD integration, and exposure to advanced tech like Agentic Applications. The position follows a hybrid work model, with 3 days per week in the office. Responsibilities Create, execute and maintain automated test code and test cases using BDD methodologies to validate software functionality Collaboration with DevOps and development teams to integrate automated tests into GitLab CI/CD pipelines for rapid, reliable releases Identify, document and track software defects, collaborating with developers to resolve issues Conduct performance and load testing to ensure applications meet benchmarks and can handle enterprise-level traffic Participation in all lifecycle phases from requirements gathering to deployment, providing QA feedback during design reviews to ensure early testability Define quality metrics and generate reports to communicate testing progress, risks and quality status to stakeholders Requirements 5+ years of experience in Test Automation using Java Knowledge of Java SE (minimum v8) along with experience in Maven, JUnit and AssertJ Proficiency in Selenium and Cucumber Expertise in BDD (Behavior-Driven Development) and familiarity with Jacoco for code coverage Willingness to contribute to multi-disciplinary teams and work on older code bases as well as new initiatives Upper-Intermediate English language proficiency (B2+) Nice to have Familiarity with Azure Cosmos DB SDK and Azure Blob Storage SDK Experience with GitLab CI/CD Skills in testing Agentic Application Understanding of RAG Architecture Competency in Intelligent Document Processing testing We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based applications through both manual and automated approaches. The position centers on building scalable LLM test harnesses, designing Gen AI evaluation frameworks, and ensuring reliability, accuracy, and performance of AI-driven systems. The role requires strong Python-based automation skills and experience with AI evaluation metrics, cloud environments, and CI/CD integration. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Experience with manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of evaluation metrics: precision, recall, criteria recall, efficiency, Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git, Copilot 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and automating evaluation of Gen AI and LLM-based applications, including AI agents and integrated systems. The position combines manual and automated testing, development of LLM test harnesses, and implementation of AI evaluation frameworks to ensure reliability, accuracy, and performance. Work is conducted in collaboration with cross-functional teams, with working hours aligned to 13:00–21:00 Polish time. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI / LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of Gen AI / LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems and CI/CD integration, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.