AI safety research and agent evaluation.

Test what an AI agent actually does

An agent’s final answer is only one part of its behavior. It may use tools, run commands, contact services, or delegate work before it reports a result. AI safety evaluation needs evidence from those actions, as well as the text the agent produces.

DynoLab is our open-source project for testing agents in isolated environments. It records activity outside the agents’ control and compares their reports with execution evidence. This makes it possible to examine differences between a claimed action and an observed action.

Agent sandboxing and independent observation

A useful test specifies the environment, the task, the rules, and the evidence needed to assess the outcome. In DynoLab, agents work in a sandbox with a configurable gateway. Connection attempts can be allowed, flagged, or refused and recorded.

A hidden Observer examines command and gateway logs. In team tests, delegation chains also matter: researchers need to distinguish the agent that performed an action from an agent that requested it. This supports investigation of instruction following, boundary violations, and reporting behavior.

A connection attempt, an allowed connection, and data access are different events. We keep those distinctions explicit when interpreting a run. A refused attempt is evidence of an attempted action, not proof that the protected resource was accessed.

Model behavior and interpretability

Our research also examines internal representations and the effect of interventions on model behavior. Questions include which activations are associated with a behavior, whether that association holds on different examples, and what changes when a representation is modified.

Neuron capture, attribution, ablation, and sparse representations can support these investigations. An attractive visualization or a plausible label is not sufficient evidence on its own. Held-out examples and controlled interventions help test an interpretation rather than simply describing it.

Evaluations that can be checked and repeated

We use evaluations and benchmarks to compare model versions, prompts, and training choices under stated conditions. Repeat runs are important because one result does not establish a reliable rate. The task set, configuration, scoring rules, and uncertainty affect what a comparison can support.

For an agent evaluation, useful evidence includes the task and constraints, the model and tools used, the actions taken, the final report, and the criteria for success or failure. Changes in the environment or tool access should be recorded alongside changes in the model.

A favorable test result does not establish that a model is safe in every setting. Our approach is to identify specific behaviors, make the evidence inspectable, and describe the conditions under which a finding was observed.

From research to deployment decisions

Research findings can inform which behaviors to evaluate before a model is deployed, what evidence to log, and where to retain human review. They also help identify questions that a public benchmark cannot answer for a particular enterprise workflow.

We connect this research with specialized model development and enterprise AI advisory. The objective is to make model selection and adoption decisions using evidence relevant to the intended use, rather than a general capability score alone.