AI Testing Lead

Minimum qualifications:

• Bachelor’s degree or equivalent practical experience.
  
• 5 years of experience as a testing lead, business analyst or similar role in the AI domain.
• Experience with testing processes like automated testing, adversarial testing, regression testing.

Preferred qualifications:

• Experience working with UX designers, product managers, and engineers.
• Experience with AI based product testing.
• Understanding of UX design principles or conversational AI concepts.
• Ability to interpret data and translate it into insights.
• Excellent communication skills, with the ability to effectively communicate information to both technical and non-technical audiences.

About the job

The Consumer Experience Operations (CXO) team at YouTube is responsible for developing user-centric support solutions that address the evolving needs of YouTube's user base across NFL Sunday Ticket, YouTube TV, and YouTube Music and Premium, as well as Shopping and Fan Funding with the goal of delivering an experience and driving customer lifetime value. The team collaborates with cross-functional partners, including Product, Engineering, Vendor Operations, Marketing, Data Science and Legal, to identify and address user pain points and improve the overall consumer support experience. We drive the implementation of tools and user journeys to enable delightful consumer experiences across Agentic, human, and self-service interactions. We are seeking a proactive and detail-oriented AI Testing Lead to join our Consumer Experience Design team. In this role, you will serve as the crucial link between business stakeholders, tech team, and our design team, ensuring through rigorous testing programs that our conversational and user interfaces not only provide exceptional user experiences but also align with key business objectives. You will be instrumental in translating high-level business goals into requirements and using data to inform and validate design decisions. You will report to our UX Design Lead and work closely with our Product Owners and Product Support Managers.

At YouTube, we believe that everyone deserves to have a voice, and that the world is a better place when we listen, share, and build community through our stories. We work together to give everyone the power to share their story, explore what they love, and connect with one another in the process. Working at the intersection of cutting-edge technology and boundless creativity, we move at the speed of culture with a shared goal to show people the world. We explore new ideas, solve real problems, and have fun and we do it all together.

The US base salary range for this full-time position is $110,000-$157,000 + bonus + equity + benefits. Our salary ranges are determined by role, level, and location. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your preferred location during the hiring process.

Please note that the compensation details listed in US role postings reflect the base salary only, and do not include bonus, equity, or benefits. Learn more about benefits at Google.

Responsibilities

• Develop a comprehensive test strategy and define the scope of testing, identify types of tests needed, and creating a plan for continuous evaluation of the model in a production environment.
• Analyze data from AI model performance and user interactions, e.g., conversation transcripts, escalation rates, and user flows, to identify pain points and opportunities for design and technical improvements.
• Work with UX and Conversation Designers to ensure design solutions are aligned with business key performance indicators (KPIs), technical constraints, and long-term product strategy. Act as a liaison between design team and other teams.
• Create clear documentation, including test cases, user stories, process flows, and requirements matrices, to ensure a shared understanding across the product, design, and engineering teams.
• Manage a team of testers, overseeing testing requirements and implementing a prioritized feedback loop for tech and design.