QA Engineer

🏢 Summary: Senior QA Engineer role in the insurance/financial services industry, focused on ensuring high-quality deliverables through manual and automated testing, including projects involving generative AI. The position emphasizes building and scaling test automation practices in a 100% remote, long-term B2B engagement. The role involves close collaboration with developers and active contribution to automation strategy and quality processes. 🗂️ Requirements: Fluency in English, Fluency in Polish, 5+ years of experience in test automation, 3+ years of experience with Python, Experience in manual testing, Experience in automated testing, Knowledge of AWS or Azure, ISTQB certification, Willingness to work with AI-related solutions 📃 Skills: Python, AWS, Azure, ISTQB, Locust, Xray, Zephyr, Jira, AzureDevOps, SITACH, UAT, QAT, Pentesting 🏢 Description: GENERAL INFORMATION: Industry: Insurance / Financial Services Location: 100% remote Rate: up to 135 PLN/hour net + VAT Contract type: B2B – long-term cooperation Summary: The role of the Senior QA Engineer focuses on ensuring high-quality deliverables through both manual and automated testing, particularly in advanced technology projects, including generative AI initiatives. The primary goal is to enhance the testing processes and foster a culture of test automation within the team. Responsibilities: Designing, creating, and executing manual and automated test cases according to agreed standards. Testing new functionalities and performing regression testing . Reporting defects identified during testing and collaborating with developers for resolution and closure. Participating in test planning activities , including estimation and task prioritization. Designing, building, testing, and deploying test automation solutions , including test concepts and automation strategies. Collaborating with developers on automation scripts from the early stages of the development process. Coordinating execution of automated tests within assigned areas or projects. Participating in the development and expansion of test automation practices within the organization. Consulting and coaching the testing team in the area of test automation. Performing testing activities related to SITACH, UAT, QAT, and security testing (pentests) . Must Haves: Fluency in English and Polish . 5+ years of experience in test automation. 3+ years of experience with Python . Experience with manual and automated testing . Knowledge of cloud solutions (AWS or Azure) . ISTQB certification . Willingness to learn and work with new AI-related solutions . Nice to Haves: Openness to changing technology stacks. Experience in performance testing (preferably using Locust). Experience with Xray, Zephyr , or other test management tools. Knowledge of Jira or Azure DevOps . Other Details: Team Structure: The QA team consists of 1 QA Manager and 3 testers. Onboarding Process: Described as very supportive and smooth. Organizational Focus: Moving strongly toward test automation, with support for skill development.

🏢 Summary: The offer is for an experienced QA Automation Engineer responsible for designing, implementing, and maintaining automated test suites for capital market systems. The role involves frontend and backend test automation, integration and performance testing, and close collaboration with development teams in a hybrid work model. It provides stable employment, competitive salary, and professional growth opportunities. 🗂️ Requirements: Minimum 3 years of experience in functional test automation (frontend and backend), Minimum 3 years of experience in integration, acceptance, and performance testing, Strong knowledge of Cypress, Puppeteer, SoapUI, JMeter, Postman, Experience with GUI testing, REST, SOAP, Kafka, Proficiency in TypeScript and JavaScript, Understanding of Docker and WebSockets, Experience with GIT version control 📃 Skills: Cypress, Puppeteer, SoapUI, JMeter, Postman, REST, SOAP, Kafka, TypeScript, JavaScript, Docker, WebSockets, GIT, GUI, TCP, UDP, Linux 🏢 Description: QA Automation Engineer Location: Warsaw (Hybrid: 1–2 days a week in the office) Salary: 14,000 – 17,000 PLN Gross We are looking for an experienced QA Automation Engineer to join our team. In this role, you will play a key part in ensuring the quality of our capital market systems by designing, implementing, and maintaining automated test suites. Key Responsibilities Test Automation: Actively participate in the test automation process, creating and maintaining automated test scripts and tools. Test Design: Design and implement comprehensive test cases based on technical requirements. Bug Tracking: Identify, document, and report software defects in a clear and timely manner. Collaboration: Work closely with Developers, Architects, and fellow QA Engineers to ensure the highest software quality and find optimal solutions to technical issues. Specification & Analysis: Collaborate on preparing technical specifications and requirements in the testing domain. Documentation: Assist in creating and updating technical documentation for developed tools and test scripts within the transactional system. Requirements Experience: Minimum 3 years of hands-on experience in functional test automation (both frontend and backend). Testing Diversity: At least 3 years of experience in integration, acceptance, and performance testing. Tech Stack & Tools: Strong knowledge of testing tools and protocols: Cypress, Puppeteer, SoapUI, JMeter, Postman, as well as GUI testing, REST, SOAP, and Kafka. Programming Languages: Proficiency in TypeScript and JavaScript. DevOps & Tools: Solid understanding of Docker, WebSockets, and version control using GIT. Nice to Have Experience in testing software built on low-level programming languages. Knowledge of network protocols (UDP, TCP) and Linux environments. What We Offer Stability: Employment contract (Umowa o pracę). Growth: A challenging and rewarding role in the dynamic capital markets industry with clear professional development opportunities. Compensation: Attractive remuneration system matching your skills. Learning: Access to professional training, courses, and industry conferences. Benefits: A rich benefits package, including a cafeteria system, private medical care, life insurance, and PPE (Employee Pension Plan). Flexibility: Hybrid work model (1–2 days a week in our Warsaw office). This role is not perfectly suited for you, but you have a friend who would fit? Recommend your friend and get up to 5000 zł! Referral Program: Talent from your network

🏢 Summary: The offer is for a QA Automation Engineer responsible for ensuring high-quality software delivery through designing and maintaining automated tests in close collaboration with development teams. The role focuses on Python-based test automation, API, security, and performance testing, and driving a quality-first mindset within Agile environments. It includes remote work options and opportunities for professional growth in a stable technology company. 🗂️ Requirements: 4+ years of experience in software testing, Minimum 2+ years of experience in test automation, Proficiency in Python, Experience with pytest, Knowledge of automation frameworks (Playwright, Selenium or Robot Framework), Familiarity with IaC and CI/CD practices, Experience working in Agile environments, Knowledge of test and project management tools (JIRA, Xray), Experience with AI tools for QA automation, Polish language proficiency C1+, English language proficiency B2+ 📃 Skills: Python, pytest, Playwright, Selenium, Robot, CI/CD, IaC, JIRA, Xray, Gatling, Locust, AI 🏢 Description: As a QA Automation Engineer you will help us to release quality software products for our clients. You will be working closely with a development team to build and maintain modern software solutions. You’ll play a key role in assuring best quality by designing and delivering automated tests speeding up the product delivery cycle and compliance requirements. Your responsibilities: Assuring quality of created products, exploratory testing, security testing, API testing and performance testing, Design and development of automated tests using Python , Document issues found, Close cooperation with developers and business stakeholders in product design, Active engagement in growth of QA best practices and tools, driving Quality-first mindset within the team and contributing to quality culture within the organisation. Our requirements: 4+ years of experience in tests, including 2+ years in test automation, Proficiency in Python and pytest, working knowledge of automation frameworks (e.g. Playwright, Selenium, Robot Framework), Familiarity IaaC and CI/CD approaches followed by common testing principles, Experience in Agile environments, Knowledge of test and project management tools (eg. JIRA, xray), Experience in using AI tools to automate QA tasks, Self-organisation, adaptability, curious mindset and high communication skills, Ability to communicate in Polish (C1+) and English (B2). Nice to have: Experience with testing Cloud solutions / data platforms , ISTQB Foundation or Advanced level certificate, Security testing or mobile testing experience, Working knowledge of performance testing tools (e.g. Gatling, Locust). What we offer: Work in a stable, growing technology company listed on the stock exchange, B2B or employment contract , Comfortable work setup and equipment tailored to your needs (laptop, accessories, etc.), Benefits (e.g. Multisport card, private medical care, group insurance, employee referral program, team events), Opportunity to work on interesting projects, grow, and further develop your skills, 100% remote work or from our office in Bydgoszcz/Warsaw. If this environment sounds like a fit — apply now!

🏢 Summary: Senior QA Engineer role focused on automation testing for the Credit Assessment and Decision team, with responsibility for API and regression test automation, release support, and quality assurance throughout the delivery lifecycle. The position emphasizes an automation-first approach, collaboration with developers and analysts, and maintaining high-quality testing practices in enterprise environments. Fully remote work is available with a B2B contract. 🗂️ Requirements: 5+ years of senior-level QA experience, Automation testing experience, Robot Framework, Python, API testing, System testing, Automated test suite maintenance, CI/CD pipelines, English 📃 Skills: Robot, Python, API, CI/CD, Jenkins, qTest, Pega, BizDevOps 🏢 Description: Industry: Retail Rate: up to 120pln/h net+VAT Location: Warsaw, Tricity, Łódź (Fully remote option available) Type of contract: B2B Summary: The Senior QA Engineer role is designed to enhance the quality assurance process within the Credit Assessment and Decision team by focusing on automation testing and ensuring project quality from early stages of delivery. Responsibilities: Design, create, and maintain automated tests for API-based systems and regression testing. Support release activities by preparing, executing, and following up on quality assurance tasks. Perform selective exploratory and release-focused manual testing where automation is not the most effective. Manage and maintain test assets with proper traceability and reporting in enterprise test management tools. Investigate failed test runs to identify issues related to product defects, test defects, data problems, or environment instability. Collaborate closely with developers and business analysts to ensure quality is addressed early in the delivery process. Act as a senior QA voice within the project by promoting best testing practices and a strong automation-first approach. Contribute to knowledge sharing and participate in cross-project tester ceremonies. Requirements: 5+ years of senior-level QA experience with a focus on automation testing. Hands-on experience with Robot Framework and Python for test automation. Strong experience in API and system testing. Proficiency in maintaining automated test suites. Experience with CI/CD pipelines to support automated test execution. Good spoken and written English. Nice to have: Familiarity with qTest as a test management tool. Experience in financial services or regulated environments. Exposure to Pega-based solutions or comparable enterprise platforms. Experience in BizDevOps environments. Familiarity with Jenkins and similar CI/CD tools.

🏢 Summary: QA Automation Engineer role focused on automating end-to-end testing across Web, API, and Mobile platforms in a hybrid work model. The position involves designing and implementing automated tests, analyzing system logs, and ensuring quality within Agile teams. Ideal for mid to senior engineers with strong experience in Playwright or Selenium and cross-platform testing. 🗂️ Requirements: Minimum 4 years of experience in test automation, Experience with Playwright (Java or TypeScript) or Selenium WebDriver (Java) and Appium, Experience testing web applications, APIs, and mobile applications, Knowledge of testing pyramid concept, Ability to analyze and interpret system logs, Basic SQL knowledge, Experience working in Agile/Scrum teams 📃 Skills: Playwright, Selenium, WebDriver, Appium, Java, TypeScript, SQL, API, Agile, Scrum 🏢 Description: QA Automation Engineer (Web, API & Mobile) Rate: Up to 119 PLN/h net + VAT (B2B contract) Location: Warsaw Work model: Hybrid (currently 2 days in the office, 3 days remote) Experience level: Mid / Senior (min. 4 years of experience) We are looking for an experienced QA Automation Engineer to take ownership of ensuring the quality of end-to-end processes in our projects. If the testing pyramid concept is the foundation of your work, you can read system logs like an open book, and you can smoothly bridge the worlds of Web, API, and Mobile – we want to meet you! What will you be responsible for? Automating tests for End-to-End processes (both Frontend + Backend), Designing and automating API tests, Analyzing system logs, identifying bugs, and conducting root-cause analysis, Reviewing project documentation and creating high-quality test scenarios based on it, Supporting and handling User Acceptance Testing ( UAT ), Collaborating closely with the development team and business stakeholders in an Agile environment. What do we expect from you? (Must-have): Minimum 4 years of hands-on experience in test automation, Practical knowledge of Playwright (Java or TypeScript) OR Selenium Webdriver (Java) along with Appium , Proven experience in testing all three areas: web applications, APIs, and mobile apps , Strong understanding and ability to test according to the testing pyramid concept, Proficiency in system log analysis and interpretation, Basic knowledge of SQL , Experience working in agile development teams ( Agile/Scrum ). This role is not perfectly suited for you, but you have a friend who would fit? Recommend your friend and get up to 5000 zł! Referral Program: Talent from your network

🏢 Summary: Senior QA Engineer role focused on combining manual, functional, API, UI, and automated testing within a cross-functional product team. The position emphasizes shift-left quality practices, product-oriented test automation, and collaboration in Agile environments. Hybrid work model based in Warsaw with a balanced mix of manual and automation responsibilities. 🗂️ Requirements: Experience writing product-centric test scenarios, Hands-on experience with Java, Hands-on experience with Cucumber, Experience in manual testing, Experience in functional testing, Experience in API testing, Experience in UI testing, Experience with CI/CD pipelines, Understanding of Agile methodologies, Ability to work in large-scale complex projects, Ability to switch between manual and automated testing 📃 Skills: Java, Cucumber, BDD, API, UI, CI/CD, DevSecOps, Agile, ISTQB, TMAP 🏢 Description: QA Engineer (Manual & Automation) – Hybrid | Warsaw Location: Warsaw, Poland Work Model: Hybrid – 50% from the office (Warsaw, Wołoska) About the Role We are looking for an experienced Senior QA Engineer to join a cross-functional Product Team. In this role, you will be the guardian of product quality, combining manual testing, functional testing, and automation with a strong product and system-thinking mindset. You will promote a shift-left quality approach, ensuring issues are identified as early as possible and that every delivered feature is valuable, reliable, and resilient. This role requires a balanced QA profile — someone comfortable with both hands-on manual testing and automation, rather than a purely manual tester or a full-time automation specialist. Key Responsibilities Write and execute manual test scenarios using Gherkin (BDD) Identify, analyze, prioritize, and report defects Deliver clear and actionable quality reports to stakeholders Design, implement, and improve regression test scenarios Participate in and facilitate quality ceremonies (Bug Triage, 3 Amigos, Quality Chapter) Design, develop, and maintain product-oriented automated tests (end-user focused) Contribute to the evolution of the global Quality Strategy Share QA best practices within the squad and across teams Skills & Experience Hard Skills Proven experience writing product-centric test scenarios Hands-on experience with Java & Cucumber (automation and BDD) Solid background in manual, functional, API, and UI testing Ability to work effectively in complex, large-scale projects Experience with CI/CD pipelines and a DevSecOps-oriented quality approach Strong understanding of Agile methodologies ISTQB, TMAP, or Agile certification is a plus Soft Skills Strong critical thinking and results-driven mindset System thinking with a continuous improvement orientation Proactive and solution-focused approach Open to feedback and effective in collaborative, complex environments Additional Information The ideal candidate has seniority and versatility, combining manual testing, functional testing, and automation. Automation is not a full-time responsibility at present, but the team is moving toward increased automation. Due to the current team setup (few testers, many developers), we are not able to hire a full-time automation-only specialist. Candidates should be comfortable switching between manual and automated testing, depending on project needs.

🏢 Summary: Senior QA Automation Engineer role for a high-impact global project, focused on building and leading automated testing in complex systems. The position involves end-to-end testing, strong API validation, and supporting a migration from monolith to microservices. Fully remote with initial onboarding in Warsaw. 🗂️ Requirements: 8+ years of QA experience, Strong automation testing experience, Expert-level Playwright knowledge, Strong API testing experience, Experience with mocking and stubbing, Experience supporting monolith to microservices migration, Solid testing fundamentals 📃 Skills: Playwright, API, Automation, Testing, Mocking, Stubbing, Microservices, Monolith, QA 🏢 Description: We are looking for an experienced and highly skilled Senior QA Automation Engineer to join a demanding, high-impact project for a global client. This role is fully remote, with a one-day onboarding in Warsaw at the start. If you thrive in technically challenging environments and have a proactive mindset, this opportunity is for you. Key Responsibilities Build and maintain automated test frameworks and suites Lead end-to-end testing across complex systems Perform API testing (functional, integration, regression) Create tests using mocking and stubbing techniques Support testing strategy during monolith to microservices migration Collaborate with cross-functional teams to ensure quality Improve QA processes and best practices Must Have 8+ years in QA with strong automation experience Solid testing fundamentals and technical expertise Expert in Playwright Strong experience in API testing Experience with mocking/stubbing Experience with monolith → microservices transition Proactive and self-driven mindset Nice to Have Degree in Computer Science or related field Knowledge of distributed systems Experience in financial systems or payments

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based applications through both manual and automated approaches. The position centers on building scalable LLM test harnesses, designing Gen AI evaluation frameworks, and ensuring reliability, accuracy, and performance of AI-driven systems. The role requires strong Python-based automation skills and experience with AI evaluation metrics, cloud environments, and CI/CD integration. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Experience with manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of evaluation metrics: precision, recall, criteria recall, efficiency, Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git, Copilot 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and automating evaluation of Gen AI and LLM-based applications, including AI agents and integrated systems. The position combines manual and automated testing, development of LLM test harnesses, and implementation of AI evaluation frameworks to ensure reliability, accuracy, and performance. Work is conducted in collaboration with cross-functional teams, with working hours aligned to 13:00–21:00 Polish time. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI / LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of Gen AI / LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems and CI/CD integration, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and evaluating AI agents and LLM-based systems using both manual and automated approaches. The position emphasizes building scalable LLM test harnesses, designing GenAI evaluation frameworks, and ensuring reliability, accuracy, and performance of AI-driven applications. Work is conducted in collaboration with cross-functional teams with required availability from 13:00 to 21:00 Polish time. 🗂️ Requirements: 5+ years of experience in software QA, Minimum 1 year of experience testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent automation frameworks, Experience integrating AI/ML libraries in test automation, Expertise in AI agent frameworks and prompt engineering, Experience evaluating GenAI/LLM systems including grounding and hallucination checks, Knowledge of evaluation metrics: precision, recall, criteria recall, determinism, Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, Git, CI/CD 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.