Request a demo

Feature

Triggers

Triggers let an agent notice a moment and decide whether it should act.

Most agent behavior starts with the user saying something. Spatial agents also need to respond to session context: a recognized object, a changed scene, a workflow step, a repeated hesitation, or another bounded signal exposed by the runtime.

In WorldAgents, triggers are agent-owned rules evaluated by the server. A trigger should return a structured decision. The runtime can then apply cooldowns, dedupe repeated observations, respect interruption policy, and expose evidence for review.

Triggers can be useful for:

  • a museum guide noticing a recognized artwork
  • a field coach noticing a visible tool or equipment state
  • a review agent flagging a missing or suspicious visual condition
  • a workflow assistant noticing that a step appears complete or skipped

The trigger does not need to own the whole session. It should describe the moment and the suggested action. The agent and runtime handle the human interaction.

The practical goal is to let teams adapt agent behavior close to the agent, test it in sessions, and avoid turning every customer-specific rule into platform code.

Triggers are files

A trigger is a JavaScript module at data/triggers/<name>.trigger.js in the agent's folder. Every valid file there runs in the agent's sessions; there is no separate list to attach it to and nothing to approve. The module exports schemaVersion, a metadata object with a title and summary for the Triggers list, interests, and evaluate(ctx); configSchema defaults and frameIntervalMs are optional.

Agent Builder writes and edits these files with its ordinary file tools and checks them with validate_trigger. The Triggers section of the configuration lists every file with its title, points at the ones that fail to load, and opens each in its own tab. Deleting the file removes the trigger.

Events

interests names the events a trigger runs on. A trigger runs on nothing else.

  • text.turn: the user sent a chat message. ctx.text.latestUserText holds it, and a speak decision answers instead of the model.
  • visual.proactive_candidate: recognition found objects the conversation has not just covered. ctx.visual.recognizedArtworks lists them.
  • visual.frame: a camera frame. Frames reach the server about once a second.

How often a frame trigger runs

A visual.frame trigger runs at most every frameIntervalMs in a session, measured from the start of its last evaluation. The default is 5000. The floor is 1000, which means every frame, and the ceiling is 60000. A trigger still evaluating when its next frame is due skips that frame, so a slow model call never queues up behind itself.

  export const interests = ["visual.frame"];
export const frameIntervalMs = 2000;

The interval is per trigger. Two frame triggers in one agent each keep their own cadence, and one waiting on the model does not delay the other. A trigger runs one evaluation at a time in a session: a recognition event that arrives while it waits on the model is evaluated right after, and sees the cooldowns that run set.

Deciding with a model

ctx.llm.generate asks the model about the moment: the current frame (includeCurrentFrame: true), the conversation (messages), and any data the trigger passes. With schema, a JSON Schema, the reply has to follow it, and json is that object:

  const { json } = await ctx.llm.generate({
  system: "You watch a museum visitor's view.",
  includeCurrentFrame: true,
  schema: {
    type: "object",
    properties: { shouldSpeak: { type: "boolean" }, about: { type: "string" } },
    required: ["shouldSpeak", "about"],
  },
});

An evaluation may make up to three calls.

Decisions

evaluate returns a list of decisions, or [] to stay quiet:

  • { type: "speak", message } says the message as is. { type: "speak", instruction } asks the agent to say or do something, in its own voice. In voice the instruction wins when both are set; in chat the message does.
  • priority orders decisions across triggers (lower first, default 100). The first speak decision is delivered.
  • cooldowns: [{ key, cooldownMs }] drops a later decision with the same key from the same trigger until the time passes. Inside evaluate, ctx.cooldowns.has(key) and ctx.cooldowns.set(key, ms) work on the same keys.
  • interrupt: true lets a speak decision cut the agent off. Without it, a decision waits until the agent and the user are quiet and is dropped otherwise. An interrupting decision still never talks over the user or a tool call in progress. Voice sessions only for now; in chat it waits like any other.
  • inspector: { title, summary, fields } shows why the trigger acted in the session inspector. { type: "none", inspector } only updates the inspector.

The older text, dedupeKey and top-level cooldownMs fields are rejected: a decision that uses them fails the evaluation, and the session inspector shows the trigger as failed with the replacement to use. An evaluation that runs past its deadline stops its model call.

Where triggers run

Each agent's triggers run in one warm sandbox process, isolated from the API (Bubblewrap in production: no network, read-only system, empty environment). Evaluations run concurrently inside it, each in a fresh context, so a trigger waiting on the model never holds up the agent's other triggers or other sessions.

Older agents

Agents saved before triggers became files carried a triggerRefs list that named which files ran, with per-ref enabled, priority, maxIntensity and config settings. The loader ignores that list now. When such an agent is deployed, every trigger file in its folder runs, including files that were unattached or switched off, and a config override is replaced by the defaults in the file's configSchema. Before deploying, delete the files that should not run and move any override into the file.