Healthcare AI Evaluation & Clinical Data Expert
Remote | Independent Contractor | India | Tier 3 / Tier 4
About Agents Only Technologies Inc.
Agents Only Technologies Inc. is a global workforce and operations company supporting technology, customer experience, business operations, and professional services across multiple industries.
We collaborate with organizations worldwide to deliver high-quality solutions through distributed networks of skilled contributors.
Our global network includes experienced professionals and subject matter specialists who support complex projects requiring domain expertise, structured reasoning, quality evaluation, and attention to detail.
Role Overview
Agents Only Technologies Inc. is building a specialized network of Healthcare Clinical and Reasoning Experts to support an advanced healthcare data and AI evaluation project.
Contributors will use their professional healthcare expertise to create realistic healthcare artifacts, develop complex prompts, establish verified answers, build evaluation rubrics, and assess AI-generated responses for clinical accuracy, reasoning quality, and completeness.
The project may involve multi-hour, complex assignments requiring strong healthcare knowledge, analytical reasoning, and the ability to communicate clearly in written form.
We are particularly seeking Clinical Experts (Tier 4) and Healthcare Reasoning Experts (Tier 3) across the United States and India.
As a Healthcare Expert, you will:
Create realistic synthetic healthcare artifacts, including clinical records, discharge summaries, medication information, insurance documents, and other healthcare-related materials.
Develop complex, clinically accurate prompts designed to evaluate advanced AI systems.
Produce verified gold responses or expected outputs based on professional expertise.
Create detailed, objective rubrics for evaluating AI-generated responses.
Compare and rank multiple AI responses against defined clinical and quality criteria.
Identify inaccuracies, omissions, unsafe recommendations, reasoning gaps, and inconsistencies.
Evaluate healthcare connectors and responses using healthcare data sources or structured healthcare information.
Define expected results for connector-based healthcare tasks.
Perform specialist quality assurance and, where required, adjudicate disagreements between evaluations.
Review and refine challenging examples designed to improve healthcare AI benchmark performance.
Structure completed artifacts, prompts, expected outputs, and evaluation criteria for dataset delivery.
Follow project-specific schemas, quality standards, documentation requirements, and confidentiality guidelines.
Clinical Expert — Tier 4
Tier 4 opportunities are intended for professionals with advanced clinical or specialized healthcare expertise who can independently assess complex healthcare scenarios.
Relevant backgrounds may include:
Physicians and senior clinicians
Registered Nurses
Pharmacists
Advanced Practice Nurses / Nurse Practitioners
Physician Assistants
Clinical specialists
Radiology or other specialized healthcare technicians
Senior clinical reviewers
Other qualified healthcare professionals with relevant subject matter expertise
Candidates should have substantial hands-on experience in their respective healthcare domain and be comfortable evaluating complex clinical information.
Healthcare Reasoning Expert — Tier 3
Tier 3 opportunities are intended for professionals with strong healthcare-domain knowledge and advanced analytical, reasoning, documentation, or healthcare operations expertise.
Relevant backgrounds may include:
Medical coding and billing specialists
Payer operations specialists
Claims specialists
Utilization management specialists
Prior authorization professionals
Healthcare data specialists
Healthcare reasoning and clinical research professionals
FHIR and healthcare-data-format specialists
Healthcare information management professionals
Other experienced healthcare subject matter experts
Strong written communication, analytical reasoning, and the ability to develop structured evaluation criteria are important for this work.
Education / Expertise
Candidates with educational background or professional experience in one or more of the following areas are encouraged to apply:
Medicine
Nursing
Pharmacy
Allied Health
Clinical Sciences
Health Information Management
Medical Coding and Billing
Healthcare Administration
Healthcare Informatics
Health Data Science
Clinical Research
Radiology and Medical Imaging
Healthcare Information Technology
FHIR / Healthcare Data Standards
Payer Operations and Utilization Management
Other relevant healthcare disciplines
Equivalent professional experience may also be considered where appropriate.
Experience Requirements
Preferred candidates will have:
3+ years of relevant professional experience for Tier 3 roles.
5+ years of relevant professional or clinical experience for Tier 4 roles.
Strong knowledge of their healthcare specialty or domain.
Experience interpreting complex healthcare information.
Strong written English communication skills.
Ability to explain reasoning clearly and objectively.
Experience reviewing documentation, clinical scenarios, healthcare data, or professional outputs.
Strong attention to detail and quality.
Ability to work independently on complex assignments.
Familiarity with structured data, evaluation frameworks, or quality-review processes is an advantage.
Experience with healthcare AI, clinical datasets, or AI evaluation is helpful but not required.
Important Notes
This opportunity requires genuine healthcare subject matter expertise.
This is a specialist evaluation and dataset-development project rather than traditional annotation or basic data-labeling work.
Contributors may be assigned complex scenarios requiring significant research, reasoning, and review time.
Strong written communication and structured thinking are essential.
Final task types, platform requirements, data schemas, and evaluation methodology are subject to project confirmation.
Contributors may be required to complete project-specific onboarding, training, calibration, or qualification activities.
All work must follow applicable project confidentiality, data handling, quality, and compliance requirements.
Work Arrangement
Location: Remote
Work Type: Independent Contractor
Markets: Spain, Philippines, United States & India
Schedule: Flexible, subject to project requirements
Scope: Global
Who This Opportunity Is For
This opportunity may be a strong fit if you are a healthcare professional who enjoys going beyond routine tasks and applying your expertise to complex clinical reasoning, quality evaluation, structured problem-solving, and healthcare data creation.
You do not need previous AI experience to be considered.
What matters most is the depth of your professional expertise, your ability to reason through difficult healthcare scenarios, and your ability to clearly explain what makes a response accurate, incomplete, or incorrect.
Equal Opportunity
Agents Only Technologies Inc. is committed to fostering an inclusive and diverse global community.
We welcome talented professionals from a wide range of backgrounds, experiences, and perspectives.
Opportunities are made available based on qualifications, skills, experience, and business needs, with the goal of building a collaborative environment where contributors can learn, grow, and succeed together.
