Shopify Storefront Agent Evaluator / Shop App Agent Annotator

Mercor

(80)

Emma

Emma highlights

This highlight was written by Emma’s AI. Ask Emma to edit it.

Freelance

Recurrent (~5 hours per week)

Compensation

USD40 - 50/hour

Remote (anywhere)

Shared by

6 days ago

ResponsibilitiesTest AI using realistic online-shopping scenarios.Evaluate responses for relevance, accuracy, and safety.Provide concise feedback when the agent gets something wrong.Focus on evaluating AI-powered shopping experiences (Shopify storefront + Shop App).QualificationsStrong attention to detail.Experience with online shopping/e-commerce.AI evaluation, annotation, QA, or rubric-based review experience.No coding required.Mercor is recruiting for a Shopify Storefront Agent Evaluator / Shop App Agent Annotator role focused on evaluating AI-powered shopping experiences.

User since: Jun 2026

About Mercor:

Our vast talent network trains frontier AI models in the same way teachers teach students: by sharing knowledge, experience, and context that can't be captured in code alone. Today, more than 30,000 experts in our network collectively earn over $2 million a day.

mission:

We connect human expertise with leading AI labs and enterprises to train frontier models.

Reputation:

(80)

Jobs posted: 39

Candidate review speed: ~4 hours

Signalers: 1.6K

Feedback summarized by AI: Applicants who reached late stages reported positive, straightforward interviews. While some found the process interesting, others noted unclear instructions, excessive rounds, payment discrepancies, lack of projects, and potential referral farming.Applicants who didn't reach the late stages reported a positive, easy process. Some suggested native language technical tests. However, many noted a lack of communication, hiring transparency concerns, and frustration over indefinite status.

mercor.com/

learn more about Mercor

To edit the organization's information go to Mercor's team genome.

Admin access needed

Mercor has to validate your admin access to edit their information. You won’t be able to edit until it is approved.

Shopify Storefront Agent Evaluator / Shop App Agent Annotator

Emma

Responsibilities

Skills wanted:

Language(s) required:

Jon Norris

Jon Norris

About Mercor:

mission:

mercor.com/

Admin access needed