🚨 We're Hiring | AI/ML Quality & Evaluation Engineer (LLM Evals) | RemoteKey ResponsibilitiesEvaluate LLM/GenAI outputs for accuracy, faithfulness, completeness, and safety.Test RAG (Retrieval-Augmented Generation) systems and identify hallucinations.Evaluate Speech-to-Text (STT), transcription quality (WER), and voice AI performance.Build evaluation datasets, golden references, and scoring rubrics.Develop automated evaluation pipelines using Python.Validate PII detection, masking, and data protection compliance.Define go/no-go quality gates and produce detailed evaluation reports.Required Skills3–5 years of QA experience with at least 1 year in AI/ML, LLM, or GenAI products.Hands-on experience with LLM evaluation, prompt testing, hallucination detection, and RAG evaluation.Strong Python programming skills.Knowledge of transcription quality metrics (WER).Experience working in Microsoft Azure environments.Strong analytical, reporting, and documentation skills.Nice to HaveExperience with Ragas, DeepEval, Promptfoo, LangSmith, or Braintrust.Azure AI Services (Azure OpenAI, Azure AI Speech, Azure AI Search, Azure AI Foundry).Experience with STT/TTS evaluation, speaker diarization, and PII/data protection testing.Cloud Platform: Microsoft Azure (Mandatory)

Heads-up

AI/ML Quality & Evaluation Engineer (LLM Evals)

Emma

Requirements and responsibilities

Skills wanted:

Language(s) required:

Mubina Shaik

Mubina Shaik

About Ravin Software Solutions Pvt Ltd:

www.ravinsol.com/

Admin access needed

Payment confirmed

A member of the Torre team will contact you shortly