QA Engineer ID66279

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.

Why join us

If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you! :)

About the role

We are looking for a Senior QA Engineer to perform manual testing of web applications and validate AI-generated outputs as part of a product team building LLM-integrated features. You will design and execute test cases across UI, API, and AI behavior scenarios using TestRail, Postman, and JIRA, working within an Agile environment. The role requires a detail-oriented engineer who can evaluate model quality metrics and ensure accuracy, completeness, and alignment across prompt–response workflows.

What you will do

• Perform manual testing of web applications to identify bugs, glitches, and inconsistencies;

• Develop, maintain, and execute detailed manual test cases based on product requirements and specifications;

• Perform thorough cross-browser UI testing to ensure consistent behavior across multiple browsers;

• Validate AI-generated test cases, summaries, and recommendations produced by integrated LLMs;

• Design and execute test scenarios to verify AI behavior, including prompt-response consistency and contextual relevance;

• Evaluate AI model quality using defined acceptance metrics such as precision, recall, hallucination rate, and duplication rate;

• Conduct manual API testing to validate request and response accuracy, error handling, and integration;

• Use TestRail to create, manage, and track test plans, test cases, and test execution results;

• Use Jenkins CI/CD pipelines;

• Report and document defects clearly using JIRA;

• Participate in daily stand-ups, sprint planning, and other Agile team meetings;

• Review user stories and acceptance criteria to understand the scope of testing;

• Estimate own tasks and report risks to the team lead;

• Collaborate with developers and other QA team members to reproduce issues and verify fixes;

• Maintain and update test documentation as needed.

Must haves

• 4+ years of experience with manual testing;

• Basic understanding of methodologies and SDLC and STLC models;

• Knowledge of testing and quality assurance processes;

• Strong experience with test design activities to cover a set of product requirements;

• Knowledge of test design techniques and testing methodologies;

• Defect analysis skills and the ability to write detailed and useful defect reports;

• Experience with Postman for API testing and browser developer tools;

• Experience with UI testing;

• Experience with TestRail as a test management system;

• Basic understanding of CI/CD processes;

• Intermediate English level.

Nice to haves

• Understanding of LLM evaluation principles, including prompt engineering, output validation, and model fine-tuning cycles;

• Familiarity with AI and ML concepts, APIs, and frameworks such as OpenAI API, Azure OpenAI, or LangChain;

• Knowledge of data protection and compliance standards for AI systems, including GDPR, zero-data retention, and PII anonymization;

• Experience with SSO testing;

• Experience in performance or load testing.
  

The benefits of joining us

• Professional growth

Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps

• Competitive compensation

We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities

• A selection of exciting projects

Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands

• Flextime

Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.

Meet Our Recruitment Process

Asynchronous stage – An automated, self-paced track that helps us move faster and give you quicker feedback:

• Short online form to confirm basic requirements

• 30–60 minute skills assessment via Codility – a platform founded in Poland that helps us provide quicker feedback and streamline this stage of the process.

• 5-minute introduction video

Synchronous stage – Live interviews

• Technical interview with our engineering team (scheduled at your convenience)

• Final interview with your future teammates

If it’s a match — you’ll get an offer!