Role Description
In this hourly-paid, fully remote contractor role, you will review AI-generated responses and/or generate safety-focused evaluation content, assess reasoning quality and step-by-step problem-solving, and provide expert feedback so outputs are accurate, safe, and clearly explained. You will annotate and evaluate content in English and Arabic; you must be fluent or proficient (near-native or native) in Arabic and able to apply Minimum C1 English to interpret prompts and policies precisely. Your annotations on these explicit tasks will be used to prevent the Large Language Model from generating unintentional or adversarial, toxic, or unsafe outputs. The types of explicit content you may be exposed to may include, but are not limited to, content of a sexual, violent, or psychologically disturbing nature. This role is with SME Careers, a fast-growing AI Data Services company and a subsidiary of SuperAnnotate that provides AI training data for many of the world’s largest AI companies and foundation-model labs, helping improve the world’s premier AI models.
Your Profile
* Bachelor’s degree or higher in a relevant field (e.g., Communications, Linguistics, Psychology, Law/Policy, Security Studies) or equivalent professional experience.
* Near-native or native Arabic proficiency (reading and writing) for high-precision safety labeling and cultural-linguistic nuance.
* Minimum C1 English proficiency (reading and writing) for policy interpretation, prompt understanding, and consistent documentation.
* Experience in Trust & Safety, content moderation, policy enforcement, risk operations, investigations, or safety evaluation work.
* LLM red teaming experience is a must, with a proven ability to probe safety boundaries and document adversarial patterns.
* Strong knowledge of safety domains, including Hate & Harassment, Sexual content, Suicide & Self-Harm, Violence, Bias, Illegal goods and services, malicious activities, malicious code, and deliberate misinformation.
* Emotional resilience, with an understanding that this role requires annotating texts that contain unsafe, explicit, and/or toxic content, including content of a sexual, violent, or psychologically disturbing nature.
* Excellent judgment under ambiguity, with the ability to apply written policies consistently and explain decisions succinctly.
* Comfort working as an hourly contractor, including dependable throughput, clear documentation, and responsiveness across time zones.
* Previous experience with AI data training, annotation, or evaluation is preferred.
* Strong hands-on experience using tools such as Perplexity, Gemini, ChatGPT, and others.
Key Responsibilities
* Curate and label safety-focused training examples, including adversarial and red-team cases, in English and Arabic that probe model behavior across hate and harassment, sexual content, self-harm, violence, bias, illegal services, malicious activity, malicious code, and misinformation, capturing nuance and intent with Minimum C1 English and near-native French proficiency.
* Review, score, and compare multiple model responses against safety policy and quality rubrics, documenting why an output is safe or unsafe and identifying failure modes such as evasion, normalization, escalation, or procedural enablement.
* Continuously stress-test and audit model behavior for policy gaps and edge cases, flag ambiguous scenarios, propose clearer decision rules, and help maintain consistent annotation standards across reviewers.
