Senior QA Engineer
We are seeking a Senior QA Engineer to validate the performance of AI-driven video analysis systems, with a focus on detecting key moments in sports content. The role combines manual and automation testing, working with pre-labeled video assets provided by the customer.
Responsibilities
- Audit live sports events across leagues such as NBA, MLB, NFL, and NHL to confirm that the AI/Inference Service accurately tracks and labels major sports moments, including touchdowns, home runs, and buzzer-beaters, exactly as they occur
- Apply strong knowledge of sports contexts, terminology, and metrics while working with timecodes, video frames, transcriptions, and captions
- Act as the human expert who identifies when the AI produces errors, hallucinations, or misinterprets a sports rule, partnering directly with AWS engineers to document defects and validate their resolutions
- Review and validate metadata tags generated by the AI to ensure they align with the sport being analyzed and meet advertising industry standards (IAB rules), keeping content brand-safe for ad placements
- Maintain detailed QC documentation, keeping clear records of AI and Inference Service performance, tracking accuracy metrics, and helping establish a smooth testing process that connects traditional sports broadcasting with emerging AI technology
Requirements
- At least 3 years of professional relevant experience in QA engineering
- Experience in both manual and automation QA, preferably within video-focused or AI-driven environments
- Knowledge of testing Large Language Models (LLMs) along with an understanding of the associated workflows
- Automation scripting skills using languages and tools such as Python, Selenium, or similar for comparing JSON outputs to ground truth at scale
- Solid understanding of software development and QA cycles, including defect logging, triage, and reporting
- Ability to interpret labeled data and validate model outputs against expected results
- Familiarity with concepts such as computer vision and transcript analysis, without the need to understand internal model workings
- Strong communication skills for stakeholder interaction and reporting
- Detail-oriented and iterative approach to testing activities
- Fluent English communication skills at a B2 level or higher, both written and verbal
Nice to have
- Hands-on experience with video testing tools or frameworks dedicated to media QA
- Prior work in sports analytics or media technology environments
