29 lines
6.2 KiB
Markdown
29 lines
6.2 KiB
Markdown
## Why
|
|
|
|
`agent-runtime`'s `TaskRunner` (from `apex-agent-mvp`, code-complete but unapplied) re-derives everything about a task from scratch every step: `TaskContext` accumulates a flat list of `Scene`s and `StepResult`s, but nothing in the loop distills "what app am I in," "what page am I on," "am I already logged in," or "did I already navigate into the chat with Zhang San" into a queryable form. Even with `semantic-scene-runtime`'s per-step `SemanticScene` (page identity, intents, widget purposes), that artifact is deliberately ephemeral and same-step only — it is discarded, not accumulated, so a real Planner (a later milestone) still cannot ask "have I already done this" without re-scanning raw scene/step history itself. This change adds a **World Runtime**: a `WorldState` that persists across a task's steps (current app, current page, a small variables dict, and a bounded history of recent semantic scenes/actions), updated incrementally by a hook in `TaskRunner`'s step loop after each executed step, and exposed read-only to the Planner alongside the current `SemanticScene` so plans can skip redundant navigation or re-discovery. This is Milestone 6 (World) of the device-agnostic runtime roadmap established by `device-agent-runtime-foundation`, sitting directly on top of Milestone 5's `semantic-scene` capability.
|
|
|
|
## What Changes
|
|
|
|
- Add a new `world/` package that defines a `WorldState` dataclass (`current_app: str | None`, `current_page: str | None`, `variables: dict[str, Any]`, a bounded `history: deque[WorldEvent]` of recent `(semantic_scene | scene, action)` pairs) and a `WorldModel` that owns one `WorldState` per task and knows how to update it.
|
|
- Introduce an **update hook** (`WorldModel.observe(scene, semantic_scene, step, result)`) called once per executed step from `runtime/task.py`'s `TaskRunner.run()` loop, after `context.add_step_result(result)`, so `WorldState` is derived incrementally from exactly the same per-step data the timeline already records — no new I/O, no new LLM call in this change.
|
|
- Define the **update rule set** as a small, deterministic, rule-based derivation (not a second LLM call): app/page fields are refreshed from the current `SemanticScene.page` (falling back to a heuristic derived from the raw `Scene`/last `launch_app` action when semantic enrichment is disabled or unavailable), `variables` are updated only via an explicit `PlannedStep.args["remember"]` convention a Planner can opt into, and `history` is a fixed-size ring buffer (bounded, oldest evicted first) so `WorldState` cannot grow unboundedly across a long-running task.
|
|
- Expose `WorldState` **read-only** to the Planner: extend `Planner.plan()`'s call signature with an optional `world: WorldState | None` keyword argument (default `None`, so the existing stub `Planner` and any test constructing `PlannedStep`s directly keep working unmodified) that a future LLM-driven Planner (not built in this change) can read to decide "already there, skip this step."
|
|
- Add `TaskContext.world` (a `WorldState | None` field, populated by `TaskRunner` when a `WorldModel` is configured) so a single object continues to carry all per-run state the Planner/Executor need, matching `agent-runtime`'s existing "task context/memory available during a run" requirement instead of introducing a second parallel context object.
|
|
- Add configuration to enable/disable World Runtime tracking globally (default **enabled**, since this is a pure derivation over data the loop already produces — unlike `semantic-scene`'s LLM call, there is no cost/latency reason to default it off) and to size the bounded history (default a small fixed window, e.g. 10 events).
|
|
- **BREAKING**: none. `WorldState`/`WorldModel` are additive; `TaskContext.world` defaults to `None` when no `WorldModel` is configured, `Planner.plan()`'s new `world` kwarg defaults to `None`, and `TaskRunner`'s constructor accepts an optional `world_model` with `None` preserving today's behavior exactly.
|
|
|
|
## Capabilities
|
|
|
|
### New Capabilities
|
|
- `world-model`: A `WorldState` store (current app, current page, a variables dict, and a bounded history of recent semantic-scene/action pairs) for a single task, updated by a hook in the agent runtime's step loop after each executed step, and exposed read-only to the Planner as additional context alongside the current `SemanticScene`, so plans can skip redundant navigation or re-discovery.
|
|
|
|
### Modified Capabilities
|
|
(none — `agent-runtime`'s Observe→Think→Act→Observe loop shape and `semantic-scene`'s `SemanticScene` output are read-only inputs to this change; neither capability's existing requirements are altered. `agent-runtime`'s "task context/memory available during a run" requirement is extended in spirit — `TaskContext` gains a `world` field — but this change does not itself modify `agent-runtime`'s spec deltas since `openspec/specs/` has no applied baseline for it yet; see Impact.)
|
|
|
|
## Impact
|
|
|
|
- **New package**: `world/` — `models.py` (`WorldState`, `WorldEvent` dataclasses), `model.py` (`WorldModel`, the per-task owner + `observe()` update hook + rule-based derivation logic), `config.py` (enable/disable + history-size settings).
|
|
- **Modified**: `runtime/context.py` (`TaskContext` gains a `world: WorldState | None = None` field); `runtime/task.py` (`TaskRunner` gains an optional `world_model: WorldModel | None` constructor argument, calls `world_model.observe(...)` once per executed step inside the existing loop, and passes `context.world` into `self.planner.plan(...)`); `runtime/planner.py` (`Planner.plan()` gains an optional `world: WorldState | None = None` keyword argument, unused by today's stub `Planner` but available to a future LLM-driven Planner).
|
|
- **No change** to `core/models.py`'s `Scene`/`Task`/`Step`, `semantic/`'s `SemanticScene` shape or enrichment logic, `storage/timeline.py`'s persisted format, or any existing tool signature — `WorldState` is derived in-memory per task run and is not persisted by this change (see `design.md` Open Questions for whether a later change should persist it).
|
|
- **Out of scope**: no cross-task or cross-device world sharing (single task's world only, discarded when the task ends); no UI for viewing/editing world state (that is `web-console`'s domain, not touched here); no change to `skill-catalog-subscription`'s or `web-console`'s pending capabilities.
|