30 lines
7.1 KiB
Markdown
30 lines
7.1 KiB
Markdown
## Why
|
|
|
|
`agent-runtime` (`apex-agent-mvp`, code-complete but unapplied) runs a single Observe→Think→Act loop: `TaskRunner.run()` calls `Planner.plan()` once per iteration and hands every produced `PlannedStep` to `Executor.execute()`, whose only failure signal is "did the tool call raise" — retried with bounded backoff. That is a *mechanical* success signal, not a *semantic* one: a `tap` on the send button can return `success=True` from the driver while the message never actually sent (wrong element, stale screen, silent app-level rejection), and nothing in the current loop would ever notice, because no role ever asks "did the intended effect actually happen" using anything richer than the tool call's own return value. As the runtime grows richer state to reason with — `SemanticScene` (Milestone 5, page identity/intents/widget purposes) and `WorldState` (Milestone 6, current app/page/variables/bounded history) — a single Planner+Executor pair has no natural place to *use* that richer state for verification or recovery; today it is planning input only. This change splits the loop's responsibilities across five collaborating roles — Observer, Planner, Executor, Verifier, Reflector — so that "did this step actually work" and "what should we do about it if not" become first-class, semantically-informed steps in the loop, layered above (not replacing) the Executor's existing low-level retry/backoff safety net.
|
|
|
|
## What Changes
|
|
|
|
- Add a new `agents/` package implementing three new roles — **Observer** (perceives and summarizes current device state via `SemanticScene`/`WorldState`, producing an `Observation`), **Verifier** (checks whether an already-*executed* step actually achieved its intended effect, comparing pre-step and post-step `Observation`s against the `PlannedStep`'s stated intent, distinct from and running after the Executor's own low-level retry), and **Reflector** (invoked only when the Verifier flags a step as not-achieved; analyzes the likely failure cause from the `Observation`/`PlannedStep`/`StepResult`/Verifier verdict and proposes either a bounded recovery action or a replan request, never a blind re-issue of the same step).
|
|
- Add a `CollaborativeTaskRunner` (`agents/collab_runner.py`) that composes the existing, **unmodified** `Planner` and `Executor` (`agent-runtime`) with the three new roles in a defined handoff loop: `Observer → Planner → Executor → Verifier → (Reflector only on a Verifier-flagged failure) → back to Observer`. It reuses `runtime/task.py`'s `TaskRunner` as the underlying step-execution engine (composed by import, not forked) rather than reimplementing planning/execution/timeline-append logic.
|
|
- Define the **handoff protocol** as typed data passed between roles (`Observation`, `VerificationVerdict`, `ReflectionOutcome`), each a plain dataclass with no role-specific side effects, so the protocol itself — not any one role's internal implementation — is the stable, spec'd contract.
|
|
- Add configuration to enable/disable multi-agent collaboration per task run, defaulting to **disabled** (mirrors `semantic-scene-runtime`'s default-off precedent) so that applying this change does not silently change any existing task's cost, latency, or step count; when disabled, a task runs exactly as `runtime/task.py`'s existing `TaskRunner.run()` today.
|
|
- Bound Reflector-triggered replanning with an explicit ceiling (a configurable max reflection-recovery attempts per task, separate from and on top of the Executor's own `max_retries`), so a persistently-failing step cannot loop forever between Verifier and Reflector.
|
|
- **BREAKING**: none. `runtime/planner.py`'s `Planner`, `runtime/executor.py`'s `Executor`, and `runtime/task.py`'s `TaskRunner` are not modified by this change; multi-agent collaboration is an opt-in alternate driver of the same underlying Task/Planner/Executor contracts, not a replacement.
|
|
|
|
## Capabilities
|
|
|
|
### New Capabilities
|
|
- `multi-agent-collaboration`: Defines the Observer/Verifier/Reflector roles, their `Observation`/`VerificationVerdict`/`ReflectionOutcome` data contracts, and the Observer→Planner→Executor→Verifier→(Reflector)→Observer handoff protocol that composes with the existing Planner/Executor (`agent-runtime`) as a higher semantic-verification layer above (not a replacement for) the Executor's own bounded low-level retries, usable both for a single-goal task and for one step of a multi-step Workflow (`workflow-orchestration`).
|
|
|
|
### Modified Capabilities
|
|
(none — `agent-runtime`'s `Planner`, `Executor`, and `TaskRunner` (`apex-agent-mvp`) keep their existing requirements unchanged; `CollaborativeTaskRunner` composes them strictly by import as an opt-in alternate driver, and neither `semantic-scene` nor `world-model` has their specified behavior altered by this change, see Impact.)
|
|
|
|
## Impact
|
|
|
|
- **New package**: `agents/` — `models.py` (`Observation`, `VerificationVerdict`, `ReflectionOutcome`, `ReflectionAction` dataclasses), `observer.py` (`Observer.observe(...) -> Observation`), `verifier.py` (`Verifier.verify(...) -> VerificationVerdict`), `reflector.py` (`Reflector.reflect(...) -> ReflectionOutcome`), `collab_runner.py` (`CollaborativeTaskRunner`, the handoff loop), `config.py` (enable flag, max-reflection-attempts ceiling).
|
|
- **No change** to `runtime/planner.py`, `runtime/executor.py`, `runtime/task.py`, `core/models.py`, `storage/timeline.py`, or any existing tool in `tools/` — all existing tests and callers keep working unmodified; `CollaborativeTaskRunner` composes `TaskRunner`/`Planner`/`Executor` strictly by import, as `workflow-orchestration-runtime`'s `WorkflowRunner` already does for its planned-goal steps.
|
|
- **Reads from pending capabilities (read-only composition, no spec changes to them)**: `semantic-scene` (Milestone 5) for `SemanticScene` as Observer/Verifier input (with the existing skip/fallback degrade path honored — Observer must work when `SemanticScene` is `None`), `world-model` (Milestone 6) for `WorldState` as additional Observer/Verifier context, and `agent-runtime` (`apex-agent-mvp`) for `Task`/`PlannedStep`/`StepResult`/`TaskContext` as the underlying vocabulary every new role's inputs/outputs are built from.
|
|
- **Composable with `workflow-orchestration`** (Milestone 8): a `WorkflowDefinition`'s planned-goal step may be driven by `CollaborativeTaskRunner` instead of the plain `TaskRunner`, without any change to `workflow/runner.py`'s step-handler contract (`run(task) -> Task` in, `Task` out) — this change does not itself modify `workflow/`.
|
|
- **Config**: `pyproject.toml` gains an `agents*` entry in `[tool.setuptools.packages.find].include`; no new third-party dependency (roles reuse `semantic/llm_client.py`'s existing LLM client abstraction for Verifier/Reflector reasoning, added in Milestone 5, not a new SDK).
|
|
- **Out of scope**: no new LLM provider/vendor integration; no removal or modification of the Executor's existing retry/backoff logic; no persistence of `Observation`/`VerificationVerdict`/`ReflectionOutcome` into `storage/timeline.py` (left as an open question, mirroring `world-model-runtime`'s deferred `WorldState` persistence question); no change to `skill-catalog-subscription`, `web-console`, or `skill-learning-runtime`.
|