For companies:
For professionals:
For everyone:
For developers:
Responsibilities and deliverables
Engagement Details:
- Contract, paid hourly.
- No 40-hour minimum.
- Work arrives in sprints, and you take on what suits you.
- Start immediately.
- The client has sample work that needs turning around this week, with a substantially larger programme expected to follow.
About the Work:
- Our client builds high-fidelity datasets used to train and evaluate frontier large language models.
- One of the things the AI labs want most right now is software engineering tasks for reinforcement learning environments, specifically tasks that frontier models cannot solve.
- The RL environments are already built.
- The client needs a steady supply of tasks to run inside them, written by engineers who know what hard, real work actually looks like.
Responsibilities:
- Writing realistic software engineering tasks: go into a repository, change or fix something, apply a patch, clean up afterwards.
- The emphasis is on work an engineer would genuinely be paid to do, not competition-style puzzles or textbook exercises.
- Writing the instructions, test cases, and expected outputs tightly enough that there is exactly one correct interpretation.
- Running frontier models against your tasks and evaluating the responses against rigorous functional and logical standards.
- Where your expertise sits outside Python, helping build out the RL environment for that language.
Performance Standards:
- A good task defeats a frontier model on the merits: the model understood exactly what was being asked and still could not do it.
- A task that wins because the wording was loose does not count.
- If a model returns something you weren’t looking for because you didn’t specify properly, that is a fault in the task, not in the model.
- Holding that line, hard but scrupulously unambiguous, is most of the job.
Requirements:
- Real engineering depth, the kind that comes from shipping and maintaining production systems.
- Years on a CV matter less than the quality of your judgement; some of the client’s strongest contributors are very young.
- Python, or a very good reason not to.
- Python is the primary environment.
- The client is language agnostic: if you have spent a career in Java, C++ or COBOL and never picked up Python, that expertise is genuinely valuable.
- There is real demand for tasks that move legacy code into modern languages.
- Intellectual honesty: you say so when you don’t know something, you raise problems while they are still fixable, and when you say you’re nearly done, you’re nearly done.
- This matters more than almost anything else.
- Clear written English.
- Nearly everything you produce is written specification that someone else has to be able to read without asking you a question.
Helpful, but not required:
- Prior work on AI data platforms, such as Outlier, Alignerr, Scale, Surge or similar RL and data-annotation workflows.
- Experience across several codebases, stacks or domains rather than one.
This job post is closed.