What We’re BuildingWe create high-quality coding problems for training AI models — drawn from real production repositories, not synthetic puzzles. Your role is to ensure these problems are fair, clear, and technically sound.The RolePart-Time Code Quality Reviewer | Remote | $40/hr | 0-25 hrs/weekWe’re looking for engineers who can trace the intent behind a code change, understand what changes were made and why, and express that clearly in natural language. This is judgment work, not mechanical testing. You’ll make philosophical decisions about fairness and clarity in technical contexts.What You'll Actually DoReview code changes across unfamiliar repositories and tech stacksAssess problem fairness: Could a skilled engineer solve this with only the given context?Evaluate test quality: Do tests properly validate the intended solution?Write clear justifications for your assessments (2-3 sentences per decision)Flag coherence issues: Inconsistent artifacts, unclear requirements, missing contextTime commitment1-4 hours per problem, fully async — work whenever fits your schedule.Who Should ApplyPhD students in CS/computational fields seeking long-term flexible incomeOpen source contributors who care deeply about code qualityUndergrad with at least 1 year in CS, ECE, Computational Biology, or related technical programsProfessional engineers seeking intellectually interesting part-time workYou'll thrive if youHave strong software engineering fundamentalsCan work independently with minimal supervisionWrite clearly and preciselyEnjoy making judgment calls in ambiguous situationsStay focused during 1-3 hour review sessionsRequirements1+ years of software engineering experience (internships, research projects, or professional work)Fluent English; strong writing is a mustResponsive, independent, and attentive to detail0–25 hours per week, flexibleCompensation & Growth$40/hour for all review workPerformance-based advancement: Top contributors can receive promotions and bonusesLong-term engagement: We're building a scalable team, not a temporary workforceApplication Process15-minute async video interview — answer 4 questions about your technical background and work styleSample problem review — demonstrate your assessment skills on a real coding problemOnboarding — join our active problem design engineering community and start reviewingFAQQ: Is this suitable for PhD students?Yes — many of our top contributors are PhD students. The work is fully async and fits around research schedules.Q: Do I need to be available during specific hours?No. Work whenever you want. Some problem design engineers work weekends, others weekday evenings.Q: What if I'm not familiar with the programming language in a problem?That's expected. You're assessing problem fairness and clarity, not implementing solutions. Familiarity with general software engineering principles is what matters.Q: How quickly can I start?If you pass the video interview and sample review, you can start within 1-2 weeks.Q: Is this a path to full-time employment?This is a long-term contract role, not a pipeline to full-time positions. We value contributors who excel at this specific type of work.About the Widget FactoryThe Widget Factory is the quality assurance division of Idler (YC S25), where we ensure coding problems meet research-grade standards for AI training. Our mission is to build the most reliable QA system for AI coding benchmarks.

Heads-up

Problem Design Engineer

Emma

Requirements and responsibilities

Skills wanted:

Language(s) required:

Nalu Concepcion

About Widget Factory:

widgetfactory.ai/

Admin access needed

Payment confirmed

A member of the Torre team will contact you shortly