QA & AI Engineer

Quick Facts

• Role: QA & AI Engineer

Description

Design and build QA AI agents for testing web, mobile applications, and APIs. Create evaluation sets and safety guardrails for LLM/agent systems, including detection of regressions, hallucinations, reward hacking, and prompt injection, while validating latency/cost and decision chains. Evolve QA test strategy and the test automation framework, and integrate both classic test suites and agent runs into GitLab CI/CD.

Responsibilities

• Develop and debug autonomous QA AI agents; choose appropriate architecture per task

• Build evaluation suites for regressions, hallucinations, reward hacking; test prompt injection

• Apply guardrails: validate decision chains and manage latency and cost

• Create QA strategies for complex data flows using GraphQL, REST, Kafka, and databases; uncover edge cases

• Maintain and extend the existing automation framework (Playwright, WebdriverIO with Appium, Cypress)

• Reduce test flakiness and speed up regression runs

• Integrate test runs and AI-agent executions into GitLab CI/CD and improve feedback reliability

Requirements

• Proven experience in functional and exploratory testing for web, iOS/Android, and APIs (REST, GraphQL)

• Strong understanding of HTTP/HTTPS and JSON; ability to find logical and hidden defects

• Ability to write maintainable automation code in modern frameworks (Playwright/WebdriverIO/Cypress)

• Experience architecting tests, reducing flakiness, and optimizing regression suite runtime

• Practical work with SQL/NoSQL databases for test data and integrity validation

• Experience testing asynchronous/event-driven flows with Kafka or RabbitMQ

• Experience integrating automated suites into GitLab CI/CD pipelines

• Hands-on LLM/agent engineering experience: LLM API usage, tool-use/function calling

• Familiarity with loop engineering and multi-agent orchestration/graph concepts, including context window management

• Ability to design evaluation suites for accuracy, hallucinations, guardrails, and prompt injection; decide when to build vs buy

• Critical thinking, independence, and willingness to mentor others in ambiguous domains

Benefits

• Opportunity to shape a new area of software testing and agent-based QA

• Dedicated R&D time for experiments with new models and agent frameworks

• Time and support for personal growth and learning

• Annual company bonus and additional discretionary awards

• High-quality equipment (ergonomic workstation, top hardware, phone and unlimited-data plan)

• Modern office environment with amenities (fitness areas, relaxation zones) and parking

• Company events (meetups, monthly sports tournaments, holiday parties)

• Restaurant credits in the building

• Referral bonus and 25 days of vacation