feat: checkpoint device agent runtime milestones
This commit is contained in:
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-07-06
|
||||
@@ -0,0 +1,73 @@
|
||||
## Context
|
||||
|
||||
`runtime/task.py`'s `TaskRunner.run()` is the Observe→Think→Act→Observe loop (`agent-runtime` capability, `apex-agent-mvp`, code-complete but unapplied): each iteration calls `self.observer(device_id)` to get a `Scene`, appends it to `TaskContext.scenes`, calls `self.planner.plan(goal=..., scene=..., context=...)`, executes each `PlannedStep` via `Executor.execute()`, and appends the `StepResult` to `TaskContext.step_results`. `TaskContext` (`runtime/context.py`) is a flat dataclass — `scenes: list[Scene]`, `step_results: list[StepResult]` — with no derived/summarized state; `context.latest_scene` is the only convenience accessor today. `semantic-scene-runtime` (Milestone 5, drafted alongside this change) adds `enrich_scene(scene) -> SemanticScene | None`, a same-step, non-persisted artifact — its own design.md explicitly leaves open "should `SemanticScene` be recorded in `TaskContext`/the timeline" as a question for this milestone to answer. Today's `Planner` is a two-step stub (`runtime/planner.py`) that emits one hardcoded `describe_screen` step and declares the goal reached once any step has succeeded — it does not yet reason about page identity at all, so nothing currently consumes anything like "have I already navigated here." A future LLM-driven Planner is the actual reason this capability needs to exist: without a persisted, bounded summary of "where am I / what have I already established," a real Planner has no way to skip re-navigating to a screen it visited two steps ago except by re-scanning the entire `scenes`/`step_results` history and re-deriving semantics itself, every single step.
|
||||
|
||||
Two stakeholders: `runtime/task.py`'s `TaskRunner` (must gain a well-defined, optional integration point without changing behavior when `WorldModel` is not configured), and a future LLM-driven Planner (not built in this change) that will be the actual consumer reading `WorldState` to make cheaper plans.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Maintain one `WorldState` per task run — `current_app`, `current_page`, a `variables` dict, and a bounded `history` of recent semantic-scene/action pairs — updated incrementally after each executed step.
|
||||
- Make the update purely a **derivation over data the loop already produces** (the just-observed `Scene`, the optional `SemanticScene`, the `PlannedStep`, and its `StepResult`) — no new I/O, no new LLM call, no new external dependency introduced by this change.
|
||||
- Keep `history` genuinely bounded (a fixed-size ring buffer) so a long-running task (up to `TaskRunnerConfig.max_steps`, currently 20, but not contractually capped) cannot grow `WorldState` without limit.
|
||||
- Expose `WorldState` to the Planner strictly **read-only** and **additively** — `Planner.plan()` gains an optional keyword argument with a safe default, so every existing caller and test keeps working unmodified.
|
||||
- Make `WorldModel` degrade sensibly when `SemanticScene` is absent (semantic enrichment disabled, per `semantic-scene-runtime`'s default-disabled config) — page/app tracking falls back to a raw-`Scene`/action heuristic rather than going stale or raising.
|
||||
|
||||
**Non-Goals:**
|
||||
- No cross-task or cross-device world sharing — `WorldState` is scoped to exactly one `Task`/`TaskContext` and is discarded when the task ends; a follow-on "world persistence" or "world sharing across a device's tasks" capability is a separate future decision, not made here.
|
||||
- No UI for viewing or editing world state — that belongs to `web-console`'s `console-status-api`/`console-config-api`/`web-console-ui` capabilities (a separate pending change), not touched here.
|
||||
- No real LLM-driven Planner — this change only makes `WorldState` available as a Planner input; teaching a Planner to actually *use* it (e.g. skip a navigation step because `current_page` already matches) is future Planner-capability work, out of scope here.
|
||||
- No second LLM call to summarize/update world state — updates are deterministic rules over already-available per-step data, not a new enrichment call (see D2).
|
||||
- No persistence of `WorldState` into `storage/timeline.py` or `task-memory`'s timeline format — `WorldState` lives only in the in-memory `TaskContext` for the duration of a run (see Open Questions).
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1: New `world/` package, sibling to `semantic/`, not folded into `runtime/context.py`
|
||||
`WorldState`/`WorldModel` get their own package (`world/models.py`, `world/model.py`, `world/config.py`) rather than growing `runtime/context.py` in place. `TaskContext` gains only a thin `world: WorldState | None` field pointing at an object `world/` owns and updates; the update *rules* (how `current_page` is derived from a `SemanticScene`, how `history` eviction works) live in `world/model.py`, not in `runtime/`. This mirrors `semantic-scene-runtime`'s own D1 (a sibling package rather than folding into the layer below) and keeps `runtime/` focused on orchestration (loop shape, retries, tool dispatch) rather than accumulating every future kind of derived state inline.
|
||||
**Alternative considered**: add `current_app`/`current_page`/`variables`/`history` fields directly onto `TaskContext` and put the update logic in `runtime/task.py`'s `TaskRunner.run()`. Rejected — `TaskContext` would become a grab-bag mixing raw per-step history (`scenes`, `step_results`, owned by `agent-runtime`) with derived cross-step summarization (owned by this new capability); a future World-related change (persistence, cross-task sharing) would then have to reach into `runtime/` internals instead of a self-contained `world/` package, repeating the exact "grab-bag package" problem `device-agent-runtime-foundation` already diagnosed and fixed for `core/`.
|
||||
|
||||
### D2: Deterministic rule-based update, not a second LLM call
|
||||
`WorldModel.observe(scene, semantic_scene, step, result)` is pure Python: if `semantic_scene` is present, `current_page = semantic_scene.page`; `current_app` is refreshed whenever `step.action in {"launch_app", "terminate_app"}` succeeds, using `step.args["bundle_id"]`/`step.args["app_id"]` (falling back to leaving `current_app` unchanged if the action fails); `variables` are updated only when `step.args` contains an explicit `"remember"` mapping (`{key: value}`) a Planner opted into; `history` appends one `WorldEvent(semantic_scene_or_scene, step.action, result.success)` per call and evicts the oldest entry once past the configured bound. No network call, no additional latency, no new failure mode beyond "field stays stale if inputs are absent."
|
||||
**Alternative considered**: a second structured-output LLM call (reusing `semantic-scene-runtime`'s `llm_client.py` pattern) that looks at the accumulated history and produces a compact world-state summary each step, similar to how `SemanticScene` itself is produced. Rejected for this milestone — doubling the per-step LLM call count (one for `SemanticScene`, one for `WorldState`) doubles latency/cost for a problem (bookkeeping already-known fields) that does not need model reasoning to solve; a rule-based derivation over the same `SemanticScene`/`PlannedStep`/`StepResult` data that already exists is sufficient and strictly cheaper. This can be revisited if a future milestone's Planner needs richer world summarization than simple field-tracking + bounded history provides.
|
||||
|
||||
### D3: `WorldState` exposed via `TaskContext.world`, not a second context object threaded separately
|
||||
`TaskRunner` sets `context.world = world_model.state` once (or updates it in place) rather than introducing a `WorldContext` parameter threaded alongside `TaskContext` through `Planner.plan()`/`Executor.execute()`. `Planner.plan()`'s new `world: WorldState | None = None` keyword is a convenience mirror of `context.world` (some future Planner implementations may prefer an explicit parameter over reaching into `context`), but the source of truth is always `TaskContext.world`.
|
||||
**Alternative considered**: keep `WorldState` entirely outside `TaskContext`, passed as a wholly separate argument to `Planner.plan()`/`Executor.execute()`/`TaskRunner.run()`. Rejected — `agent-runtime`'s existing requirement is "task context/memory available during a run" as one accessible object; splitting per-run state across two parallel objects (`TaskContext` for raw history, a bare `WorldState` for derived summary) makes every future call site that needs "everything about this run" thread two parameters rather than one, for no benefit since `WorldState` is inherently `TaskContext`-scoped (one task, one world) anyway.
|
||||
|
||||
### D4: Bounded `history` via a fixed-size ring buffer, not unbounded list or time-based eviction
|
||||
`WorldState.history: deque[WorldEvent]` is a `collections.deque(maxlen=N)` with `N` from `world/config.py` (default 10), so the oldest event is dropped automatically once the buffer is full — no separate cleanup pass, no unbounded memory growth even if a task runs far beyond `TaskRunnerConfig.max_steps`'s current default of 20 (e.g. if a future change raises that ceiling).
|
||||
**Alternative considered**: keep an unbounded `list[WorldEvent]` (simplest, matches `TaskContext.scenes`/`step_results` which are also unbounded lists today). Rejected — `scenes`/`step_results` are raw historical record (their unboundedness is intentional, mirroring the timeline), whereas `history` here exists specifically to give the Planner a *recent* window without asking it to reason about a growing list; an explicit bound keeps the read-only Planner-facing contract ("recent" is a stated, fixed size) rather than an implicit "whatever the task length happens to be."
|
||||
|
||||
### D5: World Runtime tracking defaults to enabled (unlike Semantic Scene's default-disabled enrichment)
|
||||
`world/config.py`'s enable flag defaults to `True`, and `TaskRunner`'s optional `world_model` argument, when left `None`, still gets a default `WorldModel` constructed internally (not skipped) unless the config flag is explicitly off. This differs from `semantic-scene-runtime`'s enrichment, which defaults to **disabled** specifically because it makes a paid network call every step.
|
||||
**Alternative considered**: default World Runtime to disabled as well, for consistency with the Semantic Scene precedent and to minimize behavior change on apply. Rejected — `WorldModel.observe()` has no network call and no meaningful cost/latency (it is a few dict/deque operations over data the loop already holds), so the reason Semantic Scene defaults off (protect cost/latency-sensitive environments from a silent new LLM bill) does not apply here; defaulting on means the capability's read-only benefit is available to a Planner immediately, and an explicit off-switch remains for anyone who wants to opt out (e.g. a test asserting exact `TaskContext` shape pre-this-change).
|
||||
|
||||
### D6: `Planner.plan()` gains an optional `world` kwarg with a safe default; the existing stub `Planner` ignores it
|
||||
`runtime/planner.py`'s `Planner.plan(self, *, goal, scene, context, world=None)` accepts but does not use `world` — the existing stub planner's behavior (one hardcoded `describe_screen` step, then done) is unchanged. This is intentionally a no-op integration point in this change: making a Planner *smart enough* to skip steps using `WorldState` is real LLM-Planner work reserved for a future milestone/change, not implied by adding the parameter.
|
||||
**Alternative considered**: leave `Planner.plan()`'s signature untouched and require future Planner implementations to read `context.world` directly instead of a dedicated parameter. Rejected — an explicit `world` parameter documents, at the call boundary, that world-state read access is a first-class, expected part of planning (matching how `scene`/`context` are already explicit parameters rather than everything being reached off one context blob), which matters for future callers/tests constructing a `Planner` subclass against a stable, self-documenting signature.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **[Risk]** A rule-based `current_app`/`current_page` derivation can drift from reality faster than an LLM-based one would (e.g. a page transition not captured by any `launch_app`/`terminate_app` action, or `SemanticScene.page` phrased inconsistently step to step since `semantic-scene-runtime`'s Open Questions leave intents/page labels open-vocabulary) → **Mitigation**: `WorldState` is documented and enforced as **read-only, advisory** context for the Planner, never a source of truth the Executor trusts blindly; a future Planner consuming it is expected to still verify against the current `Scene`/`SemanticScene` before skipping a step, not skip purely on stale `WorldState` says-so. This change does not build that Planner logic, only the state it would read.
|
||||
- **[Risk]** Enabling World Runtime by default (D5) means every existing `TaskRunner` caller/test that doesn't explicitly configure `world_model=None` starts accumulating `WorldState` it previously didn't have → **Mitigation**: this is additive-only (`TaskContext.world` is a new field with no effect on `scenes`/`step_results`/existing assertions), the update hook cannot raise (all derivation is defensive: missing `SemanticScene`, missing `step.args` keys, etc. are all "no-op, leave field unchanged," never an exception), and `world/config.py` provides an explicit off-switch for any caller/test that wants exact pre-this-change `TaskContext` shape.
|
||||
- **[Risk]** `variables`'s `step.args["remember"]` convention has no schema/validation — a Planner could stuff arbitrary or unbounded data into `variables` over many steps → **Mitigation**: out of scope to solve generally in this change (no Planner exists yet that writes to it); `world/config.py` can later gain a max-`variables`-size guard if a real Planner's usage pattern demands it, but there is no real caller to validate against yet, so adding a limit now would be speculative.
|
||||
- **[Trade-off]** `WorldState` is not persisted (Non-Goal) — if a task fails partway and is retried, or if `task-memory`'s timeline is later replayed, the accumulated world state from the failed run is not recoverable, only re-derivable from scratch on the retry → acceptable per this milestone's explicit scope (single task's world only); persistence is a `task-memory`-capability change to propose separately if needed, matching how `semantic-scene-runtime` deferred the same question for its own artifact.
|
||||
- **[Trade-off]** Choosing rule-based derivation (D2) over LLM-based world summarization (rejected alternative) means `WorldState`'s quality is bounded by how good `SemanticScene.page`/action-name heuristics are — if a future milestone finds this insufficient (e.g. genuinely needs to reason "the user probably navigated back," not just track the last known page), upgrading to an LLM-based summarizer is a `world/model.py`-internal change, not a `WorldState` shape change, since the shape (`current_app`/`current_page`/`variables`/`history`) is intentionally derivation-method-agnostic.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
This is purely additive with optional, defensively-defaulted integration points:
|
||||
1. Add the `world/` package (`models.py`: `WorldState`, `WorldEvent`; `model.py`: `WorldModel.observe()`; `config.py`: enable flag + history size).
|
||||
2. Add `TaskContext.world: WorldState | None = None` to `runtime/context.py`.
|
||||
3. Add an optional `world_model: WorldModel | None` constructor argument to `TaskRunner` (`runtime/task.py`); when `None` and World Runtime is enabled in config, `TaskRunner` constructs a default `WorldModel` internally; call `world_model.observe(scene, semantic_scene, step, result)` once per executed step, immediately after the existing `context.add_step_result(result)` line, and set/refresh `context.world` from the model's current state.
|
||||
4. Add the optional `world: WorldState | None = None` keyword argument to `Planner.plan()` (`runtime/planner.py`); `TaskRunner` passes `context.world` through; the existing stub `Planner` implementation ignores it (no behavior change).
|
||||
5. Add `world*` to `pyproject.toml`'s `[tool.setuptools.packages.find].include` list; no new third-party dependency (no LLM SDK, no new I/O library — `collections.deque` is stdlib).
|
||||
6. Run `pytest` — the full existing suite must remain green with zero test-content changes (only additive assertions in new `tests/test_world_model.py`-style tests are expected, not edits to existing tests), confirming the default-enabled World Runtime tracking does not alter any existing task-completion/failure behavior.
|
||||
7. Rollback: since this is a net-new package plus two small additive fields/kwargs with safe defaults, reverting is `git revert` of the commit(s); no data migration, no persisted format touched, no external system involved.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Whether `WorldState` should eventually be persisted into `storage/timeline.py`'s entries (so a replayed/inspected task shows what the runtime "believed" at each step, not just the raw `Scene`/tool-call/result) — left open per this change's Non-Goals; would be a `task-memory`-capability change to propose separately once there is a concrete consumer (e.g. `web-console`'s status API wanting to show "current believed page").
|
||||
- Whether `variables` needs any validation, size bound, or namespacing convention once a real Planner starts writing to it via the `"remember"` args convention — deferred until a real Planner (a future milestone) exists to observe actual usage patterns against.
|
||||
- Whether a future Skill-synthesis milestone would want `WorldState.history`'s bounded window to be configurable per-task rather than one global default — left as a global-default-only setting for this milestone, matching `semantic-scene-runtime`'s equivalent "leaning global-default-only" stance on its own model-selection config.
|
||||
- Whether `current_page` derivation should fall back to something richer than "last known value, unchanged" when `SemanticScene` is absent (e.g. a raw-`Scene`-shape heuristic keyed on element text/layout signature) — this change only specifies the fallback as "leave unchanged," which is safe but potentially stale; worth revisiting once semantic enrichment's real-world enable rate is known.
|
||||
@@ -0,0 +1,28 @@
|
||||
## Why
|
||||
|
||||
`agent-runtime`'s `TaskRunner` (from `apex-agent-mvp`, code-complete but unapplied) re-derives everything about a task from scratch every step: `TaskContext` accumulates a flat list of `Scene`s and `StepResult`s, but nothing in the loop distills "what app am I in," "what page am I on," "am I already logged in," or "did I already navigate into the chat with Zhang San" into a queryable form. Even with `semantic-scene-runtime`'s per-step `SemanticScene` (page identity, intents, widget purposes), that artifact is deliberately ephemeral and same-step only — it is discarded, not accumulated, so a real Planner (a later milestone) still cannot ask "have I already done this" without re-scanning raw scene/step history itself. This change adds a **World Runtime**: a `WorldState` that persists across a task's steps (current app, current page, a small variables dict, and a bounded history of recent semantic scenes/actions), updated incrementally by a hook in `TaskRunner`'s step loop after each executed step, and exposed read-only to the Planner alongside the current `SemanticScene` so plans can skip redundant navigation or re-discovery. This is Milestone 6 (World) of the device-agnostic runtime roadmap established by `device-agent-runtime-foundation`, sitting directly on top of Milestone 5's `semantic-scene` capability.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add a new `world/` package that defines a `WorldState` dataclass (`current_app: str | None`, `current_page: str | None`, `variables: dict[str, Any]`, a bounded `history: deque[WorldEvent]` of recent `(semantic_scene | scene, action)` pairs) and a `WorldModel` that owns one `WorldState` per task and knows how to update it.
|
||||
- Introduce an **update hook** (`WorldModel.observe(scene, semantic_scene, step, result)`) called once per executed step from `runtime/task.py`'s `TaskRunner.run()` loop, after `context.add_step_result(result)`, so `WorldState` is derived incrementally from exactly the same per-step data the timeline already records — no new I/O, no new LLM call in this change.
|
||||
- Define the **update rule set** as a small, deterministic, rule-based derivation (not a second LLM call): app/page fields are refreshed from the current `SemanticScene.page` (falling back to a heuristic derived from the raw `Scene`/last `launch_app` action when semantic enrichment is disabled or unavailable), `variables` are updated only via an explicit `PlannedStep.args["remember"]` convention a Planner can opt into, and `history` is a fixed-size ring buffer (bounded, oldest evicted first) so `WorldState` cannot grow unboundedly across a long-running task.
|
||||
- Expose `WorldState` **read-only** to the Planner: extend `Planner.plan()`'s call signature with an optional `world: WorldState | None` keyword argument (default `None`, so the existing stub `Planner` and any test constructing `PlannedStep`s directly keep working unmodified) that a future LLM-driven Planner (not built in this change) can read to decide "already there, skip this step."
|
||||
- Add `TaskContext.world` (a `WorldState | None` field, populated by `TaskRunner` when a `WorldModel` is configured) so a single object continues to carry all per-run state the Planner/Executor need, matching `agent-runtime`'s existing "task context/memory available during a run" requirement instead of introducing a second parallel context object.
|
||||
- Add configuration to enable/disable World Runtime tracking globally (default **enabled**, since this is a pure derivation over data the loop already produces — unlike `semantic-scene`'s LLM call, there is no cost/latency reason to default it off) and to size the bounded history (default a small fixed window, e.g. 10 events).
|
||||
- **BREAKING**: none. `WorldState`/`WorldModel` are additive; `TaskContext.world` defaults to `None` when no `WorldModel` is configured, `Planner.plan()`'s new `world` kwarg defaults to `None`, and `TaskRunner`'s constructor accepts an optional `world_model` with `None` preserving today's behavior exactly.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `world-model`: A `WorldState` store (current app, current page, a variables dict, and a bounded history of recent semantic-scene/action pairs) for a single task, updated by a hook in the agent runtime's step loop after each executed step, and exposed read-only to the Planner as additional context alongside the current `SemanticScene`, so plans can skip redundant navigation or re-discovery.
|
||||
|
||||
### Modified Capabilities
|
||||
(none — `agent-runtime`'s Observe→Think→Act→Observe loop shape and `semantic-scene`'s `SemanticScene` output are read-only inputs to this change; neither capability's existing requirements are altered. `agent-runtime`'s "task context/memory available during a run" requirement is extended in spirit — `TaskContext` gains a `world` field — but this change does not itself modify `agent-runtime`'s spec deltas since `openspec/specs/` has no applied baseline for it yet; see Impact.)
|
||||
|
||||
## Impact
|
||||
|
||||
- **New package**: `world/` — `models.py` (`WorldState`, `WorldEvent` dataclasses), `model.py` (`WorldModel`, the per-task owner + `observe()` update hook + rule-based derivation logic), `config.py` (enable/disable + history-size settings).
|
||||
- **Modified**: `runtime/context.py` (`TaskContext` gains a `world: WorldState | None = None` field); `runtime/task.py` (`TaskRunner` gains an optional `world_model: WorldModel | None` constructor argument, calls `world_model.observe(...)` once per executed step inside the existing loop, and passes `context.world` into `self.planner.plan(...)`); `runtime/planner.py` (`Planner.plan()` gains an optional `world: WorldState | None = None` keyword argument, unused by today's stub `Planner` but available to a future LLM-driven Planner).
|
||||
- **No change** to `core/models.py`'s `Scene`/`Task`/`Step`, `semantic/`'s `SemanticScene` shape or enrichment logic, `storage/timeline.py`'s persisted format, or any existing tool signature — `WorldState` is derived in-memory per task run and is not persisted by this change (see `design.md` Open Questions for whether a later change should persist it).
|
||||
- **Out of scope**: no cross-task or cross-device world sharing (single task's world only, discarded when the task ends); no UI for viewing/editing world state (that is `web-console`'s domain, not touched here); no change to `skill-catalog-subscription`'s or `web-console`'s pending capabilities.
|
||||
@@ -0,0 +1,86 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Persistent per-task WorldState
|
||||
The system SHALL maintain one `WorldState` per task, consisting of a current app identifier, a current page identifier, a `variables` mapping, and a bounded history of recent semantic-scene/action pairs, that persists across the task's steps rather than being re-derived from scratch each step.
|
||||
|
||||
#### Scenario: WorldState survives across steps within a task
|
||||
- **WHEN** a task executes multiple steps in sequence
|
||||
- **THEN** the `WorldState` object associated with the task is the same object (or reflects continuously accumulated updates) across those steps, not reset between steps
|
||||
|
||||
#### Scenario: WorldState is scoped to a single task
|
||||
- **WHEN** two different tasks run (sequentially or concurrently) against the same or different devices
|
||||
- **THEN** each task has its own independent `WorldState`, and neither task's `WorldState` reflects the other task's app/page/variables/history
|
||||
|
||||
### Requirement: Incremental update after each executed step
|
||||
The system SHALL update a task's `WorldState` via a hook invoked once per executed step in the agent runtime's step loop, deriving the update from the step's observed `Scene`, optional `SemanticScene`, the `PlannedStep` that was executed, and its `StepResult`, without introducing a new LLM call or new network I/O.
|
||||
|
||||
#### Scenario: Update runs after a successful step
|
||||
- **WHEN** the agent runtime's step loop executes a step and records a successful `StepResult`
|
||||
- **THEN** the task's `WorldState` update hook is invoked with that step's `Scene`, `SemanticScene` (if any), the executed `PlannedStep`, and the `StepResult`, and updates the persisted `WorldState` accordingly
|
||||
|
||||
#### Scenario: Update runs after a failed step
|
||||
- **WHEN** the agent runtime's step loop executes a step and records a failed `StepResult`
|
||||
- **THEN** the task's `WorldState` update hook is still invoked with that step's data, and the update proceeds without raising an exception or blocking the loop's continuation/failure handling
|
||||
|
||||
#### Scenario: Update derivation makes no external calls
|
||||
- **WHEN** the `WorldState` update hook runs for any step
|
||||
- **THEN** the update completes using only the data already passed into the hook, without making any LLM call or other network request
|
||||
|
||||
### Requirement: Current app and page tracking
|
||||
The system SHALL derive and refresh `current_app` from successful app-lifecycle actions (e.g. `launch_app`, `terminate_app`) and SHALL derive and refresh `current_page` from the current step's `SemanticScene` page identity when a `SemanticScene` is available, leaving each field unchanged when no corresponding signal is present in a given step.
|
||||
|
||||
#### Scenario: Launching an app updates current_app
|
||||
- **WHEN** a step executes a successful `launch_app` action naming an app/bundle identifier
|
||||
- **THEN** the task's `WorldState.current_app` is updated to that identifier
|
||||
|
||||
#### Scenario: A page identity from SemanticScene updates current_page
|
||||
- **WHEN** a step's enrichment produces a `SemanticScene` with a non-empty `page` value
|
||||
- **THEN** the task's `WorldState.current_page` is updated to that `page` value
|
||||
|
||||
#### Scenario: No page signal leaves current_page unchanged
|
||||
- **WHEN** a step has no `SemanticScene` available (enrichment disabled, unavailable, or failed for that step)
|
||||
- **THEN** the task's `WorldState.current_page` retains its previous value rather than being cleared or set to an empty/placeholder value
|
||||
|
||||
### Requirement: Explicit variable memorization
|
||||
The system SHALL update `WorldState.variables` only when a step's `PlannedStep.args` contains an explicit memorization instruction, and SHALL NOT infer or write arbitrary variables from step content otherwise.
|
||||
|
||||
#### Scenario: A step explicitly remembers a value
|
||||
- **WHEN** a step's `PlannedStep.args` includes an explicit key/value pair designated for memorization
|
||||
- **THEN** the task's `WorldState.variables` is updated to include that key/value pair
|
||||
|
||||
#### Scenario: A step without a memorization instruction does not change variables
|
||||
- **WHEN** a step's `PlannedStep.args` contains no explicit memorization instruction
|
||||
- **THEN** the task's `WorldState.variables` is left unchanged by that step's update
|
||||
|
||||
### Requirement: Bounded history of recent scene/action pairs
|
||||
The system SHALL maintain `WorldState.history` as a fixed-size, bounded collection of the most recent semantic-scene-or-scene/action pairs, automatically evicting the oldest entry when a new entry is added past the configured bound, so that history size never grows unboundedly with task length.
|
||||
|
||||
#### Scenario: History accumulates recent entries up to the bound
|
||||
- **WHEN** a task executes a number of steps less than or equal to the configured history bound
|
||||
- **THEN** `WorldState.history` contains one entry per executed step, in order from oldest to newest
|
||||
|
||||
#### Scenario: History evicts the oldest entry once the bound is exceeded
|
||||
- **WHEN** a task executes more steps than the configured history bound
|
||||
- **THEN** `WorldState.history` retains only the most recent entries up to the bound, with earlier entries evicted, and never exceeds the configured bound in length
|
||||
|
||||
### Requirement: Read-only WorldState available to the Planner
|
||||
The system SHALL expose a task's current `WorldState` to the Planner as an additional, read-only input alongside the current `SemanticScene`/`Scene`, without requiring existing Planner implementations or call sites to change to keep working.
|
||||
|
||||
#### Scenario: Planner can read current WorldState
|
||||
- **WHEN** the agent runtime invokes the Planner to produce the next steps for a task
|
||||
- **THEN** the Planner is given access to the task's current `WorldState` (current app, current page, variables, bounded history) as of the most recently completed step
|
||||
|
||||
#### Scenario: Existing Planner call sites keep working unmodified
|
||||
- **WHEN** an existing caller invokes the Planner's planning entry point without passing any world-state argument
|
||||
- **THEN** the call succeeds exactly as it did before this capability existed, with the Planner treating the absence of world-state input as equivalent to "no world state available"
|
||||
|
||||
### Requirement: World Runtime tracking failure never blocks the task loop
|
||||
The system SHALL treat any failure or unavailable input during a `WorldState` update (e.g. missing `SemanticScene`, missing expected `PlannedStep` arguments, disabled configuration) as non-fatal, leaving the affected `WorldState` fields unchanged rather than raising an exception that would interrupt the agent runtime's step loop.
|
||||
|
||||
#### Scenario: Missing expected data during update does not raise
|
||||
- **WHEN** the `WorldState` update hook runs for a step whose data lacks a field an update rule expects (e.g. no app identifier on a `launch_app` step)
|
||||
- **THEN** the update hook completes without raising, leaving the corresponding `WorldState` field at its prior value
|
||||
|
||||
#### Scenario: World Runtime tracking disabled by configuration
|
||||
- **WHEN** World Runtime tracking is disabled in configuration for a task run
|
||||
- **THEN** the agent runtime's step loop proceeds normally without invoking the `WorldState` update hook, and the Planner receives an absence of world-state input rather than a partially-updated or stale `WorldState`
|
||||
@@ -0,0 +1,44 @@
|
||||
## 1. Package scaffolding
|
||||
|
||||
- [x] 1.1 Create the `world/` package (`__init__.py`, `models.py`, `model.py`, `config.py`)
|
||||
- [x] 1.2 Add `world*` to `[tool.setuptools.packages.find].include` in `pyproject.toml` (no new third-party dependency)
|
||||
- [x] 1.3 Add World Runtime configuration in `world/config.py`: an enabled/disabled flag (default enabled) and a history-size bound (default 10), sourced from environment/config in one place `world/` reads from
|
||||
- [x] 1.4 Extend the project's smoke test (that imports every package) to import `world`
|
||||
|
||||
## 2. WorldState data model (capability: world-model)
|
||||
|
||||
- [x] 2.1 Implement `world/models.py`: `WorldEvent` (`scene_summary: SemanticScene | Scene`, `action: str`, `success: bool`, `timestamp: datetime`) and `WorldState` (`current_app: str | None`, `current_page: str | None`, `variables: dict[str, Any]`, `history: deque[WorldEvent]`) dataclasses, with `to_dict()` mirroring the style of `core/models.py` (for future inspection/debugging use, not persistence in this change)
|
||||
- [x] 2.2 Implement `WorldState.history` as a `collections.deque(maxlen=<configured bound>)` so oldest entries are evicted automatically once the bound is exceeded
|
||||
- [x] 2.3 Write unit tests for `WorldState`/`WorldEvent` construction and for `history`'s bounded-eviction behavior (append past the configured `maxlen` and assert the oldest entry is gone, length stays at the bound)
|
||||
|
||||
## 3. WorldModel update hook (capability: world-model)
|
||||
|
||||
- [x] 3.1 Implement `world/model.py`: `WorldModel` owning one `WorldState` per task, with `observe(scene: Scene, semantic_scene: SemanticScene | None, step: PlannedStep, result: StepResult) -> None` as the single update entry point
|
||||
- [x] 3.2 Implement the `current_page` update rule: set `current_page = semantic_scene.page` when `semantic_scene` is not `None` and its `page` is non-empty; leave unchanged otherwise
|
||||
- [x] 3.3 Implement the `current_app` update rule: on a successful `StepResult` for `step.action in {"launch_app", "terminate_app"}`, set/clear `current_app` from `step.args` (e.g. `bundle_id`/`app_id`); leave unchanged for any other action or a failed result
|
||||
- [x] 3.4 Implement the `variables` update rule: merge `step.args["remember"]` (a `dict`) into `WorldState.variables` when present; leave `variables` unchanged when absent
|
||||
- [x] 3.5 Implement the `history` update rule: append one `WorldEvent` per call to `observe()`, using `semantic_scene` when available and falling back to `scene` otherwise
|
||||
- [x] 3.6 Make every update rule defensive: missing/malformed expected fields (e.g. no `bundle_id` on a `launch_app` step, non-dict `remember` value) are logged and skipped, never raised, so `observe()` never raises for any input shape
|
||||
- [x] 3.7 Write unit tests for `WorldModel.observe()` covering: page update from `SemanticScene`, page unchanged when `semantic_scene` is `None`, app update on successful `launch_app`, app unchanged on failed `launch_app` or unrelated actions, `variables` merge via `remember`, `variables` unchanged without `remember`, and history append/eviction across repeated `observe()` calls
|
||||
|
||||
## 4. TaskContext and TaskRunner integration (capability: world-model)
|
||||
|
||||
- [x] 4.1 Add `world: WorldState | None = None` field to `TaskContext` in `runtime/context.py`
|
||||
- [x] 4.2 Add an optional `world_model: WorldModel | None = None` constructor argument to `TaskRunner` in `runtime/task.py`; when `None` and World Runtime is enabled in config, construct a default `WorldModel` internally; when World Runtime is disabled in config, leave `context.world` as `None` and skip the update hook entirely
|
||||
- [x] 4.3 In `TaskRunner.run()`'s step loop, call `world_model.observe(scene, semantic_scene, step, result)` once per executed step, immediately after the existing `context.add_step_result(result)` line, and refresh `context.world` from the model's current `WorldState`
|
||||
- [x] 4.4 Confirm `TaskRunner.run()`'s existing control flow (retry/failure/max-steps handling) is unaffected by the new hook call — the hook must never change whether a step is treated as success/failure
|
||||
- [x] 4.5 Write unit tests for `TaskRunner` covering: `context.world` populated after a step when World Runtime is enabled (default), `context.world` remaining `None` when explicitly disabled via config, and a task run completing normally (unchanged pass/fail outcome) whether World Runtime is enabled or disabled
|
||||
|
||||
## 5. Planner integration (capability: world-model)
|
||||
|
||||
- [x] 5.1 Add an optional `world: WorldState | None = None` keyword argument to `Planner.plan()` in `runtime/planner.py`; the existing stub `Planner` implementation accepts but does not use it
|
||||
- [x] 5.2 Update `TaskRunner.run()`'s call to `self.planner.plan(...)` to pass `world=context.world`
|
||||
- [x] 5.3 Write a unit test asserting the existing stub `Planner.plan()` call sites (with and without a `world` argument) both continue to return the same steps as before this change
|
||||
- [x] 5.4 Write a unit test asserting `TaskRunner` passes the current `context.world` into `Planner.plan()`'s `world` argument by the second step of a multi-step task (using a custom test `Planner` subclass that records the `world` value it was given)
|
||||
|
||||
## 6. End-to-end validation
|
||||
|
||||
- [x] 6.1 Write an end-to-end test running a multi-step task through `TaskRunner` with a mocked `Driver`/`Scene`/`SemanticScene` sequence, asserting `WorldState.current_app`/`current_page`/`history` reflect the expected values after each step
|
||||
- [x] 6.2 Write an end-to-end test confirming World Runtime tracking failure modes (missing `SemanticScene`, missing expected step args) never fail or interrupt the task loop, only leave the corresponding `WorldState` field unchanged
|
||||
- [x] 6.3 Confirm World Runtime tracking is enabled by default after applying this change, and that disabling it via config fully restores pre-this-change `TaskContext`/`Planner.plan()` call behavior (no `world` state populated or passed)
|
||||
- [x] 6.4 Run the full test suite (`pytest`) and confirm no existing test in `tests/` needed a behavior change, only additive new tests
|
||||
Reference in New Issue
Block a user