Role Overview: - You will own the evaluation of Honcho end-to-end, which is the instrument that tells a research-driven team whether their identity representations are improving. - Plastic Labs builds the memory and identity layer for the agentic world. - You will operate a complex multi-agent harness with real black-box behavior. - You will define what better means for representations that change over time without ground truth. - You will build the machinery that measures these representations. Key Responsibilities: - Design the evals. - Define the scores and decide what better means for an entity representation that changes over time. - Keep definitions current as the product and methods move. - Build the pipelines and harnesses. - Implement data ingestion, labels, versions, reruns, and judges. - Create the machinery that lets the team ask a new question and get an answer within one week. - Run evaluations and harvest insights. - Read traces and results to find what is actually broken rather than what is easy to measure. - Propose fixes that produce higher-fidelity representations. - Build simulation agents at scale. - Increase iteration speed by raising the number of entities that can be modeled and measured at once. - Own the loop end-to-end, including the question, the pipeline, the rerun, and the write-up. - Execute tasks independently without waiting on other team sprints. Requirements: - Must have designed, built, and ran evaluations end-to-end for LLMs. - Must have production-quality Python experience, including tests, maintenance, and release cadence. - Must have 3+ years of experience designing and owning LLM evaluation. - PyTorch, Hugging Face, and Weights & Biases experience. - Familiarity with data pipeline tooling including Hydra, Kafka, or SQL is a plus. Preferred Qualifications: - Experience with memory, identity, personalization, agent evaluation, or agentic optimization. - Shipped research experience as a first or co-author at NeurIPS, ICML, ICLR, or equivalent. - Self-published open-source work of comparable quality. - Degree in computer science, mathematics, statistics, or a related quantitative field.

Machine Learning Engineer

Emma

Responsibilities

Skills wanted:

Language(s) required:

ISAAC ADEKANYE Verified

About STAAC Professional E-Learning:

mission:

Admin access needed