Senior QA Engineer

🏢 Summary: Senior Software Engineer (Python) role in a fully remote Martech/e-commerce project focused on building an AI-powered strategic component, including an API gateway and orchestration layer. The position involves designing and developing data-driven web applications using Python and AWS, with strong emphasis on clean architecture and quality. You will also mentor junior engineers and contribute to product discovery and technical decision-making. 🗂️ Requirements: 7+ years commercial experience with Python, Experience building data-driven web applications, Knowledge of SOLID principles, Experience with Domain-Driven Design, Minimum 2 years experience in Agile (Scrum/Kanban) environments, Experience designing and supporting SQL databases (MySQL, MariaDB, Postgres), Basic knowledge of machine learning algorithms, Ability to work in cross-functional teams, B2 level English proficiency 📃 Skills: Python, AWS, FastAPI, FastMCP, SQL, MySQL, MariaDB, Postgres, LLM, NLP, MachineLearning, Scrum, Kanban, Agile, DDD, SOLID 🏢 Description: Join the e-commerce team as a Senior Software Engineer (Python) and create innovative solutions powered by Artificial Intelligence and Large Language Models! Let’s talk about the role of: 🤓 Senior Software Engineer (Python) Ref. no: SSEP/04/2026/P 🎯 Project: You will take part in building a strategic component in the field of artificial intelligence (API gateway, orchestration layer) for a Martech/e-commerce solutions ecosystem. 99% of new functionalities. 💻STACK: Python, AWS, FastAPI, FastMCP 🏣Industry: Martech/E-commerce 📍 Fully remote, PL. What will you be doing on a daily basis? 🚀Design/architecture, develop, implement, and test web-based applications 🚀Collaborate with Product Design and Product Management to develop applications according to the defined specifications. 🚀Implement new features and maintain the existing codebase. 🚀Follow coding standards and best practices. 🚀Ensure quality, clean, tested code is produced (unit, integration, and other QA tests). 🚀Assist the Support and Engineering team in debugging and troubleshooting software defects 🚀Coach and mentor junior software engineers in the team. 🚀 Help create team and development processes, and selection of tools to assist in these efforts 🚀 Be proactive and contribute to the research and specification phases with Product Managers when exploring new product opportunities and features. What will help you get the job done? 🎯 7+ years of practical experience in the development of commercial-grade web-based software applications using Python. 🎯 Demonstrated proficiency with data-driven web applications. 🎯 Practical use of SOLID programming principles and Domain-Driven Design programming. 🎯 Nice to have: Experience working with Large Language Models (LLMs) and other NLP technologies in a multi-agent system. 🎯 2 or more years of competitive web development experience and working in a Scrum/Kanban/Agile environment. 🎯 Basic knowledge of machine learning methods and algorithms. 🎯 Demonstrated ability to work well with business analysts, programmers, and end users in a cross-functional team. 🎯 Experience with data model design, the implementation, and support of SQL databases (MySQL, MariaDB, Postgres). 🎯 Highly developed communications skills and impeccable written communication skills, a team player with an entrepreneurial spirit. 🎯 At least B2 English proficiency level. We offer you: 🎖️Vacation days, private medical packages (Luxmed or Medicover), and Benefits System (multi-sport). 🎖️ Supportive company culture – flat structure, strong teamwork, and openness to new ideas. 🎖️ Remote-first setup. Do you prefer slippers over shoes, or are you expecting an Amazon delivery? No worries, we’ve got you covered. Hello! I’m Ann, and I am responsible for this recruitment process. If you have any questions, please feel free to reach out. I will be happy to meet you! Contact details: 👩 💻 Anna Drzał 📧 e-mail:anna.drzal@hrcontact.co ☎️ +48 503 922 611 hrcontact.co CAZ: 11828

🏢 Summary: Senior Software Engineer (Python) role focused on building a strategic AI component (API gateway and orchestration layer) for a Martech/e-commerce ecosystem. The position involves designing and developing data-driven web applications and integrating AI/LLM-based solutions in a largely greenfield environment. Fully remote role within Poland. 🗂️ Requirements: 7+ years experience in commercial web application development with Python, Proficiency in building data-driven web applications, Practical application of SOLID principles, Experience with Domain-Driven Design, Minimum 2 years experience in Agile (Scrum/Kanban) environments, Experience in SQL database design and implementation, Knowledge of machine learning methods and algorithms, B2 level English proficiency 📃 Skills: Python, AWS, FastAPI, FastMCP, SQL, MySQL, MariaDB, Postgres, LLM, NLP, MachineLearning, SOLID, DDD, Scrum, Kanban, Agile 🏢 Description: Join the e-commerce team as a Senior Software Engineer (Python) and create innovative solutions powered by Artificial Intelligence and Large Language Models! Let’s talk about the role of: 🤓 Senior Software Engineer (Python) Ref. no: SSEP/04/2026/P 🎯 Project: You will take part in building a strategic component in the field of artificial intelligence (API gateway, orchestration layer) for a Martech/e-commerce solutions ecosystem. 99% of new functionalities. 💻STACK: Python, AWS, FastAPI, FastMCP 🏣Industry: Martech/E-commerce 📍 Fully remote, PL. What will you be doing on a daily basis? 🚀Design/architecture, develop, implement, and test web-based applications 🚀Collaborate with Product Design and Product Management to develop applications according to the defined specifications. 🚀Implement new features and maintain the existing codebase. 🚀Follow coding standards and best practices. 🚀Ensure quality, clean, tested code is produced (unit, integration, and other QA tests). 🚀Assist the Support and Engineering team in debugging and troubleshooting software defects 🚀Coach and mentor junior software engineers in the team. 🚀 Help create team and development processes, and selection of tools to assist in these efforts 🚀 Be proactive and contribute to the research and specification phases with Product Managers when exploring new product opportunities and features. What will help you get the job done? 🎯 7+ years of practical experience in the development of commercial-grade web-based software applications using Python. 🎯 Demonstrated proficiency with data-driven web applications. 🎯 Practical use of SOLID programming principles and Domain-Driven Design programming. 🎯 Nice to have: Experience working with Large Language Models (LLMs) and other NLP technologies in a multi-agent system. 🎯 2 or more years of competitive web development experience and working in a Scrum/Kanban/Agile environment. 🎯 Basic knowledge of machine learning methods and algorithms. 🎯 Demonstrated ability to work well with business analysts, programmers, and end users in a cross-functional team. 🎯 Experience with data model design, the implementation, and support of SQL databases (MySQL, MariaDB, Postgres). 🎯 Highly developed communications skills and impeccable written communication skills, a team player with an entrepreneurial spirit. 🎯 At least B2 English proficiency level. We offer you: 🎖️Vacation days, private medical packages (Luxmed or Medicover), and Benefits System (multi-sport). 🎖️ Supportive company culture – flat structure, strong teamwork, and openness to new ideas. 🎖️ Remote-first setup. Do you prefer slippers over shoes, or are you expecting an Amazon delivery? No worries, we’ve got you covered. Hello! I’m Ann, and I am responsible for this recruitment process. If you have any questions, please feel free to reach out. I will be happy to meet you! Contact details: 👩 💻 Anna Drzał 📧 e-mail:anna.drzal@hrcontact.co ☎️ +48 503 922 611 hrcontact.co CAZ: 11828

🏢 Summary: Senior Software Engineer (Python) role focused on building AI-driven components (API gateway and orchestration layer) for a Martech/e-commerce ecosystem. The position involves designing and developing data-driven web applications using Python and AWS, with strong emphasis on clean architecture and testing. Fully remote role within a product-oriented, Agile environment. 🗂️ Requirements: 7+ years commercial experience with Python web applications, Experience building data-driven web applications, Strong knowledge of SOLID principles, Experience with Domain-Driven Design, Experience with SQL databases (MySQL, MariaDB, Postgres), Minimum 2 years experience in Agile (Scrum or Kanban), Experience designing and implementing data models, Knowledge of machine learning methods and algorithms, Ability to write unit and integration tests, English proficiency at B2 level or higher 📃 Skills: Python, AWS, FastAPI, FastMCP, SQL, MySQL, MariaDB, Postgres, SOLID, DDD, LLMs, NLP, MachineLearning, Scrum, Kanban, Agile, DataModeling, Testing, APIs, Orchestration 🏢 Description: Join the e-commerce team as a Senior Software Engineer (Python) and create innovative solutions powered by Artificial Intelligence and Large Language Models! Let’s talk about the role of: 🤓 Senior Software Engineer (Python) Ref. no: SSEP/04/2026/P 🎯 Project: You will take part in building a strategic component in the field of artificial intelligence (API gateway, orchestration layer) for a Martech/e-commerce solutions ecosystem. 99% of new functionalities. 💻STACK: Python, AWS, FastAPI, FastMCP 🏣Industry: Martech/E-commerce 📍 Fully remote, PL. What will you be doing on a daily basis? 🚀Design/architecture, develop, implement, and test web-based applications 🚀Collaborate with Product Design and Product Management to develop applications according to the defined specifications. 🚀Implement new features and maintain the existing codebase. 🚀Follow coding standards and best practices. 🚀Ensure quality, clean, tested code is produced (unit, integration, and other QA tests). 🚀Assist the Support and Engineering team in debugging and troubleshooting software defects 🚀Coach and mentor junior software engineers in the team. 🚀 Help create team and development processes, and selection of tools to assist in these efforts 🚀 Be proactive and contribute to the research and specification phases with Product Managers when exploring new product opportunities and features. What will help you get the job done? 🎯 7+ years of practical experience in the development of commercial-grade web-based software applications using Python. 🎯 Demonstrated proficiency with data-driven web applications. 🎯 Practical use of SOLID programming principles and Domain-Driven Design programming. 🎯 Nice to have: Experience working with Large Language Models (LLMs) and other NLP technologies in a multi-agent system. 🎯 2 or more years of competitive web development experience and working in a Scrum/Kanban/Agile environment. 🎯 Basic knowledge of machine learning methods and algorithms. 🎯 Demonstrated ability to work well with business analysts, programmers, and end users in a cross-functional team. 🎯 Experience with data model design, the implementation, and support of SQL databases (MySQL, MariaDB, Postgres). 🎯 Highly developed communications skills and impeccable written communication skills, a team player with an entrepreneurial spirit. 🎯 At least B2 English proficiency level. We offer you: 🎖️Vacation days, private medical packages (Luxmed or Medicover), and Benefits System (multi-sport). 🎖️ Supportive company culture – flat structure, strong teamwork, and openness to new ideas. 🎖️ Remote-first setup. Do you prefer slippers over shoes, or are you expecting an Amazon delivery? No worries, we’ve got you covered. Hello! I’m Ann, and I am responsible for this recruitment process. If you have any questions, please feel free to reach out. I will be happy to meet you! Contact details: 👩 💻 Anna Drzał 📧 e-mail:anna.drzal@hrcontact.co ☎️ +48 503 922 611 hrcontact.co CAZ: 11828

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based applications through both manual and automated approaches. The position centers on building scalable LLM test harnesses, designing Gen AI evaluation frameworks, and ensuring reliability, accuracy, and performance of AI-driven systems. The role requires strong Python-based automation skills and experience with AI evaluation metrics, cloud environments, and CI/CD integration. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Experience with manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of evaluation metrics: precision, recall, criteria recall, efficiency, Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git, Copilot 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based systems through both manual and automated approaches. The position emphasizes building scalable automation frameworks, LLM test harnesses, and evaluating GenAI features using advanced metrics. The role requires strong Python automation skills, experience with AI evaluation frameworks, and collaboration across product and engineering teams. 🗂️ Requirements: 5+ years of software QA experience, At least 1 year testing AI agents or LLM-based systems, Experience with manual and automated testing of AI agents, Strong Python programming for test automation, Experience with pytest or equivalent framework, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry, or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and automating evaluation of Gen AI and LLM-based applications, including AI agents and integrated systems. The position combines manual and automated testing, development of LLM test harnesses, and implementation of AI evaluation frameworks to ensure reliability, accuracy, and performance. Work is conducted in collaboration with cross-functional teams, with working hours aligned to 13:00–21:00 Polish time. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent testing frameworks, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI / LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of Gen AI / LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems and CI/CD integration, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based applications through both manual and automated approaches. The position emphasizes building scalable LLM test harnesses, designing GenAI evaluation frameworks, and ensuring quality across AI-driven systems, APIs, and databases. Requires strong Python automation skills and hands-on experience with AI evaluation metrics and cloud environments. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming for test automation, Experience with pytest or equivalent framework, Expertise in AI agent frameworks and prompt engineering, Experience evaluating GenAI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Ability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, Git, CI/CD 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and evaluating AI agents and LLM-based systems using both manual and automated approaches. The position emphasizes building scalable LLM test harnesses, designing GenAI evaluation frameworks, and ensuring reliability, accuracy, and performance of AI-driven applications. Work is conducted in collaboration with cross-functional teams with required availability from 13:00 to 21:00 Polish time. 🗂️ Requirements: 5+ years of experience in software QA, Minimum 1 year of experience testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent automation frameworks, Experience integrating AI/ML libraries in test automation, Expertise in AI agent frameworks and prompt engineering, Experience evaluating GenAI/LLM systems including grounding and hallucination checks, Knowledge of evaluation metrics: precision, recall, criteria recall, determinism, Experience with Jira, QMetry or TestRail, Experience with version control systems, Experience integrating tests into CI/CD pipelines, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, Git, CI/CD 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing AI agents and LLM-based systems through both manual and automated approaches. The position emphasizes building scalable automation frameworks, LLM test harnesses, and robust evaluation processes for Gen AI features. You will collaborate cross-functionally to ensure reliability, accuracy, and quality of AI-driven applications. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents or LLM-based systems, Experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent framework, Expertise in prompt engineering and LLM evaluation metrics, Experience evaluating Gen AI/LLM applications (grounding, hallucination, determinism), Knowledge of precision, recall, criteria recall metrics, Experience with Jira, QMetry or TestRail, Experience with CI/CD integration and version control systems, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, Jira, QMetry, TestRail, AWS, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

🏢 Summary: Fully remote Senior AI QA Engineer role focused on testing and automating quality assurance for Gen AI and LLM-based applications, including AI agents and integrated systems. The position combines manual and automated testing, development of LLM test harnesses, and evaluation of model quality metrics such as grounding, precision, and recall. Work is conducted in collaboration with cross-functional teams, with working hours aligned to 13:00–21:00 Polish time. 🗂️ Requirements: 5+ years of software QA experience, Minimum 1 year testing AI agents, agentic solutions or LLM-based systems, Hands-on experience in manual and automated testing of AI agents, Strong Python programming skills for test automation, Experience with pytest or equivalent framework, Experience integrating AI/ML libraries in automated tests, Expertise in AI agent frameworks and prompt engineering, Experience evaluating Gen AI/LLM systems (grounding, accuracy, hallucination, determinism), Knowledge of Gen AI/LLM evaluation metrics (precision, recall, criteria recall, efficiency), Experience with Jira, QMetry or TestRail, Experience with version control systems and CI/CD integration, Understanding of AWS cloud environments, Availability to work 13:00–21:00 Polish time 📃 Skills: Python, pytest, LLM, GenAI, LangChain, OpenAI, AWS, Jira, QMetry, TestRail, CI/CD, Git 🏢 Description: We are seeking a skilled Senior AI QA Engineer with strong experience in both manual and automated testing and extensive exposure to AI-based application testing. The ideal candidate will test a variety of applications, including projects involving AI agents and integrations with APIs and databases. You will help ensure our solutions are reliable and accurate and meet business requirements, while also contributing to the development of our automation capabilities. This is a fully remote position with a requirement to work from 13:00 to 21:00 Polish time, due to the client team's location. Responsibilities Research and evolve automation frameworks in line with Gen AI tooling and best practices Design and automate evaluation of Gen AI features — grounding, answer accuracy, determinism/reproducibility, precision, recall, and criteria recall Build automated LLM test harnesses that scale evaluation beyond human-in-the-loop Selection and application of Gen AI evaluation frameworks, measuring answer quality and pipeline efficiency Perform manual testing as needed to validate new features, integrations, and user stories Build and maintain test cases from requirements and user stories Test applications that may include AI agents, APIs, databases, and other integrations Collaborate with product, engineering, and operations teams to understand requirements and deployment environments Track and report test results, defects, and quality metrics Assist with troubleshooting production issues and escalate risks as needed Guide and support team members, including onshore and offshore consultants Requirements 5+ years of experience in software QA, with at least 1 year focused on testing AI agents, agentic solutions or LLM-based systems Hands-on experience with both manual and automated testing of AI agents, including prompt/instruction testing and evaluation of agentic workflows Strong programming skills in Python test automation — pytest or equivalent, scripting and AI/ML library integration Expertise in AI agent frameworks, prompt engineering and evaluation metrics for LLM-based systems Demonstrated experience testing and evaluating Gen AI / LLM applications — grounding, answer accuracy and hallucination/determinism checks Applied knowledge of Gen AI / LLM evaluation frameworks and metrics — precision, recall, criteria recall and efficiency Familiarity with issue and test management tools such as Jira, QMetry and TestRail Experience with version control systems and integrating tests into CI/CD pipelines Flexibility to use AI-powered tools for QA such as GitHub Copilot and LLM-based test generation Understanding of cloud environments, particularly AWS Excellent communication, collaboration and leadership skills Nice to have Experience with agentic AI platforms such as LangChain, OpenAI Function Calling or similar Experience with AI safety, bias and reliability testing Experience with test data generation for AI/ML systems We offer We gather like-minded people: Engineering community of industry professionals Friendly team and enjoyable working environment Flexible schedule and opportunity to work remotely within Poland Chance to work abroad for up to 60 days annually Business-driven relocation opportunities We provide growth opportunities: Outstanding career roadmap Leadership development, career advising, soft skills, and well-being programs Certification (GCP, Azure, AWS) Unlimited access to LinkedIn Learning, Get Abstract, Cloud Guru English classes We cover it all: Stable income (Employment Contract or B2B) Participation in the Employee Stock Purchase Plan Benefits package (health insurance, multisport, shopping vouchers) Strategically located offices featuring entertainment and relaxation zones, table tennis and football, free snacks, fantastic coffee, and more Referral bonuses Corporate, social and well-being events Please, note: The set of bonuses might vary based on the role you apply for – specifics will be discussed with our recruiter during the general interview. We will reach out to selected candidates exclusively. EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.