We are seeking a highly detail-oriented AI QA Engineer with a mix of manual and automation testing skills to validate the performance of cutting-edge, AI-driven video analysis systems. will focus on verifying how advanced AI, Large Language Models (LLMs), and computer vision models detect key moments and generate metadata for sports content.This position sits at the intersection of traditional sports broadcasting and advanced artificial intelligence. The ideal candidate has a deep passion for sports (understanding rules, metrics, and context), strong technical QA scripting skills, and experience working with video assets, timecodes, and metadata.ResponsibilitiesLive Game Auditing: Monitor and audit live sports broadcasts (NBA, MLB, NFL, NHL) to ensure the AI/Inference Service correctly tracks, frames, and labels major moments (e.g., touchdowns, home runs, buzzer-beaters) in real-timePrecision QA with Media Assets: Work directly with video frames, timecodes, transcriptions, captions, and JSON outputs to ensure pinpoint alignment between AI detections and actual broadcast momentsModel Error Triage & Support: Act as the human-in-the-loop expert to catch AI hallucinations, misinterpretations of complex sports rules, or edge-case errors. Collaborate directly with AWS and core engineering teams to detail bugs and validate model resolutionsMetadata & Brand Safety Validation: Review AI-generated labels/tags to ensure they align with sports context and meet advertising industry compliance standards (e.g., IAB guidelines), ensuring content is brand-safe for monetizationQA Automation: Design and execute automation scripts to compare AI inference outputs against customer-provided ground truth data at scaleIterative Testing & Documentation: Keep meticulous records of inference service performance, track accuracy metrics, log defects, and help streamline iterative testing processes for ongoing model updatesRequirementsHybrid QA Experience: Proven experience ally in video-focused or AI-driven environmentsAutomation Scripting: Hands-on automation scripting skills (e.g., Python, Selenium, or similar testing frameworks) to validate JSON outputs and data payloads against ground truth datasetsSDLC & QA Methodologies: Strong understanding of testing lifecycles, including detailed bug logging, defect triage, regression testing, and quality reportingLLM & CV Familiarity: Solid understanding of testing LLMs (workflows, prompt/response validation) and conceptual familiarity with computer vision and transcription analysisMedia & Video Literacy: Experience working with video frame analysis, timecodes, subtitles/captions, and metadata structuresSports Domain Knowledge: A deep understanding of major sports leagues (NFL, NBA, MLB, NHL), including gameplay rules, terminology, and key metricsCommunication: Strong verbal and written communication skills to act as a bridge between technical development teams and business stakeholdersAnalytical Mindset: An iterative, meticulous approach to data validation, checking model outputs for accuracy, bias, and contextNice to havePrior experience with specialized video testing tools, media processing pipelines, or video player frameworksProfessional background rts media, or digital broadcasting technologyThis Remote Position Cannot be Performed in New York City.EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our clients, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.Engineer the Future with a Career at EPAMThis posting includes a good faith range of the salary EPAM would reasonably expect to pay the selected candidate. The range provided reflects base salary only. Individual compensation offers within the range are based on a variety of factors, including, but not limited to: geographic location, experience, credentials, education, training; the demand for the role; and overall business and labor market considerations. Most candidates are hired at a salary within the range disclosed. Salary range: $100 - $105. details highlighted in this job posting above are a general description of all other expected benefits and compensation for the position.Applications will be accepted on a rolling basis.EPAM will not provide new H-1B visa sponsorship for this position. Candidates with existing transferable H-1B status may be considered.
10 days ago