agentic-mobile-control/openspec/specs/world-model/spec.md

world-model Specification

Purpose

TBD - created by archiving change world-model-runtime. Update Purpose after archive.

Requirements

Requirement: Persistent per-task WorldState

The system SHALL maintain one WorldState per task, consisting of a current app identifier, a current page identifier, a variables mapping, and a bounded history of recent semantic-scene/action pairs, that persists across the task's steps rather than being re-derived from scratch each step.

Scenario: WorldState survives across steps within a task

• WHEN a task executes multiple steps in sequence
• THEN the WorldState object associated with the task is the same object (or reflects continuously accumulated updates) across those steps, not reset between steps

Scenario: WorldState is scoped to a single task

• WHEN two different tasks run (sequentially or concurrently) against the same or different devices
• THEN each task has its own independent WorldState, and neither task's WorldState reflects the other task's app/page/variables/history

Requirement: Incremental update after each executed step

The system SHALL update a task's WorldState via a hook invoked once per executed step in the agent runtime's step loop, deriving the update from the step's observed Scene, optional SemanticScene, the PlannedStep that was executed, and its StepResult, without introducing a new LLM call or new network I/O.

Scenario: Update runs after a successful step

• WHEN the agent runtime's step loop executes a step and records a successful StepResult
• THEN the task's WorldState update hook is invoked with that step's Scene, SemanticScene (if any), the executed PlannedStep, and the StepResult, and updates the persisted WorldState accordingly

Scenario: Update runs after a failed step

• WHEN the agent runtime's step loop executes a step and records a failed StepResult
• THEN the task's WorldState update hook is still invoked with that step's data, and the update proceeds without raising an exception or blocking the loop's continuation/failure handling

Scenario: Update derivation makes no external calls

• WHEN the WorldState update hook runs for any step
• THEN the update completes using only the data already passed into the hook, without making any LLM call or other network request

Requirement: Current app and page tracking

The system SHALL derive and refresh current_app from successful app-lifecycle actions (e.g. launch_app, terminate_app) and SHALL derive and refresh current_page from the current step's SemanticScene page identity when a SemanticScene is available, leaving each field unchanged when no corresponding signal is present in a given step.

Scenario: Launching an app updates current_app

• WHEN a step executes a successful launch_app action naming an app/bundle identifier
• THEN the task's WorldState.current_app is updated to that identifier

Scenario: A page identity from SemanticScene updates current_page

• WHEN a step's enrichment produces a SemanticScene with a non-empty page value
• THEN the task's WorldState.current_page is updated to that page value

Scenario: No page signal leaves current_page unchanged

• WHEN a step has no SemanticScene available (enrichment disabled, unavailable, or failed for that step)
• THEN the task's WorldState.current_page retains its previous value rather than being cleared or set to an empty/placeholder value

Requirement: Explicit variable memorization

The system SHALL update WorldState.variables only when a step's PlannedStep.args contains an explicit memorization instruction, and SHALL NOT infer or write arbitrary variables from step content otherwise.

Scenario: A step explicitly remembers a value

• WHEN a step's PlannedStep.args includes an explicit key/value pair designated for memorization
• THEN the task's WorldState.variables is updated to include that key/value pair

Scenario: A step without a memorization instruction does not change variables

• WHEN a step's PlannedStep.args contains no explicit memorization instruction
• THEN the task's WorldState.variables is left unchanged by that step's update

Requirement: Bounded history of recent scene/action pairs

The system SHALL maintain WorldState.history as a fixed-size, bounded collection of the most recent semantic-scene-or-scene/action pairs, automatically evicting the oldest entry when a new entry is added past the configured bound, so that history size never grows unboundedly with task length.

Scenario: History accumulates recent entries up to the bound

• WHEN a task executes a number of steps less than or equal to the configured history bound
• THEN WorldState.history contains one entry per executed step, in order from oldest to newest

Scenario: History evicts the oldest entry once the bound is exceeded

• WHEN a task executes more steps than the configured history bound
• THEN WorldState.history retains only the most recent entries up to the bound, with earlier entries evicted, and never exceeds the configured bound in length

Requirement: Read-only WorldState available to the Planner

The system SHALL expose a task's current WorldState to the Planner as an additional, read-only input alongside the current SemanticScene/Scene, without requiring existing Planner implementations or call sites to change to keep working.

Scenario: Planner can read current WorldState

• WHEN the agent runtime invokes the Planner to produce the next steps for a task
• THEN the Planner is given access to the task's current WorldState (current app, current page, variables, bounded history) as of the most recently completed step

Scenario: Existing Planner call sites keep working unmodified

• WHEN an existing caller invokes the Planner's planning entry point without passing any world-state argument
• THEN the call succeeds exactly as it did before this capability existed, with the Planner treating the absence of world-state input as equivalent to "no world state available"

Requirement: World Runtime tracking failure never blocks the task loop

The system SHALL treat any failure or unavailable input during a WorldState update (e.g. missing SemanticScene, missing expected PlannedStep arguments, disabled configuration) as non-fatal, leaving the affected WorldState fields unchanged rather than raising an exception that would interrupt the agent runtime's step loop.

Scenario: Missing expected data during update does not raise

• WHEN the WorldState update hook runs for a step whose data lacks a field an update rule expects (e.g. no app identifier on a launch_app step)
• THEN the update hook completes without raising, leaving the corresponding WorldState field at its prior value

Scenario: World Runtime tracking disabled by configuration

• WHEN World Runtime tracking is disabled in configuration for a task run
• THEN the agent runtime's step loop proceeds normally without invoking the WorldState update hook, and the Planner receives an absence of world-state input rather than a partially-updated or stale WorldState