Not specified

Job Summary

This role focuses on designing and building environments that train agents to reason, act, and recover effectively, as well as creating benchmarks to evaluate genuine agent capabilities. The position involves both investigative analysis and practical construction, requiring the candidate to identify what differentiates environments that foster true capability from those that only yield high scores. The role demands designing tasks, encoding them for automated verification, stress-testing environments, and developing repeatable methods and public benchmark suites.

Responsibilities

• Environment Design

  • Define reusable methodologies and design patterns for reinforcement learning (RL) environments that effectively train and evaluate agents.
  • Specify what each environment measures, constructing tasks, initial states, and action surfaces to ensure precise measurement.
  • Tune task difficulty to discriminate between different capability levels, avoiding trivially solvable or impossible tasks.
  • Incorporate conditions that make training informative, such as partial observability, multi-step dependencies, recoverable mistakes, realistic tooling, and failure states.
  • Conduct stress tests to identify degeneracy issues like shortcut solutions, leaked answers, under-specified instructions, and reward gaming before deployment.

• Encoding and Verification

  • Translate loosely specified tasks into fully encoded environments with container and dependency definitions, deterministic setups, seeded initial states, and reproducible execution paths.
  • Develop verification layers including programmatic checks, test suites, graders, or reward functions to ensure machine-decidable success resistant to gaming.
  • Maintain clear distinctions between checks that must fail before a solution and pass after, and those that must hold throughout task evolution.
  • Validate environments against reference solutions and no-op baselines to confirm scoring accuracy.
  • Ensure environments are hermetic and deterministic to allow reproducibility over time.

• Benchmark Design

  • Design benchmarks starting from the capability under test, defining passing criteria, and building task sets that isolate specific skills.
  • Compose benchmark suites with intentional coverage across domains, task lengths, and difficulty tiers.
  • Prevent contamination and memorization by creating task variants that detect when models have seen answers rather than derived them.
  • Define scoring methodologies including metrics, pass@k conventions, sample counts, partial credit, and variance bounds.
  • Establish baselines across current models, document measurement conditions, and publish benchmarks with methodologies and known limitations.

• Task Quality Rubrics

  • Develop and maintain rubrics defining good task qualities such as specification clarity, verifier soundness, difficulty calibration, originality, reproducibility, and absence of shortcut paths.
  • Apply rubrics during task review, adjudicate borderline cases, and maintain calibration consistency across reviewers.
  • Automate rubric criteria where possible, reserving human judgment for necessary cases.
  • Iterate rubrics based on task performance during training or benchmarking to improve standards.

• Trajectory Analysis

  • Design pipelines to ingest, normalize, and index agent rollouts for comparison across different harnesses and models.
  • Build instrumentation for trajectory legibility including step segmentation, tool-call extraction, intermediate state capture, token and latency accounting, and reward attribution.
  • Develop automated failure mode classification, including large language model (LLM)-as-judge pipelines validated against human labels.
  • Identify meaningful patterns such as agent stalls, failure attribution, earned versus lucky passes, and environments producing useful learning signals.
  • Integrate findings into environment and rubric design and create dashboards and reports for non-technical stakeholders.

• Scale and Collaboration

  • Enhance environment production tooling to maintain quality at increased throughput.
  • Collaborate with engineering and product teams to present benchmark results clearly to customers and the community.
  • Define acceptance criteria and review protocols for the environment portfolio.

Qualifications

• Research and Analysis

  • Strong foundation in machine learning (ML), demonstrated through graduate study (MS/PhD in ML, Computer Science, or related field) or equivalent industry experience.
  • Genuine interest in agent behavior with the discipline to convert hypotheses into testable experiments.
  • Proven ability to reason about complex systems and derive defensible conclusions from noisy data.
  • Patience for detailed analysis, focusing on individual trajectories rather than aggregate metrics.
  • Sound judgment regarding measurement validity and awareness of artifacts caused by evaluation harnesses.

• Technical Skills

  • Proficient in Python programming.
  • Working fluency in at least one modern ML framework such as PyTorch or JAX.
  • Familiarity with reinforcement learning concepts and agent training; formal RL background is a plus but not mandatory.
  • Comfortable with environment stacks including containers, dependency pinning, test frameworks, and continuous integration (CI).
  • Ability to scope, execute, and interpret experiments end-to-end.
  • Experience running workloads on cloud infrastructure platforms like AWS or Google Cloud Platform (GCP).

• Engineering Skills

  • Ability to automate manual research processes for team-wide use.
  • Experience building data pipelines for semi-structured data such as logs, traces, and event streams, along with analytical layers.
  • Proficiency with data tooling and commitment to reproducible workflows.
  • Systematic approach to testing and verification beyond visual inspection.

Preferred Skills

• Direct experience training agents or working within reinforcement learning systems.
• Background in dataset construction, data curation, or benchmark design.
• Experience authoring or maintaining evaluation harnesses, agent scaffolds, or sandboxed execution environments.
• Exposure to AI safety, robustness evaluation, or adversarial testing.
• Publications or open-source projects related to reinforcement learning, agent evaluation, or data-centric machine learning.
• A clear perspective on designing tasks that deliberately provoke specific failure modes.
• Experience releasing research artifacts such as benchmarks, datasets, and evaluation harnesses to the broader community.

Experience

• Experience level not explicitly specified; however, a strong ML foundation through graduate education or comparable industry work is required.
• Demonstrated experience in reasoning about complex systems, data analysis, and automation in research contexts is implied.

Environment

• Work setting, location, and type are not specified.
• The role involves working with cloud infrastructure and requires familiarity with containerized and sandboxed environments.