40 lines
6.9 KiB
Markdown
40 lines
6.9 KiB
Markdown
## Why
|
|
|
|
Host Agent now wires a local `TaskMetadataStore` and `Timeline` into its `TaskRunner`, but its Cloud-assignment path still creates a `Task` and immediately calls `runner.run(task)`. `TaskRunner` only updates an existing metadata row, so every update affects zero rows and the Host Agent task page remains empty. The standalone Runtime REST service/UI is a separate process with unrelated storage, so it cannot be the inspection surface for Host Agent executions. An operator who submits or receives a task therefore has no authoritative web view of the actions that actually ran on that Host.
|
|
|
|
## What Changes
|
|
|
|
- Make `TaskRunner.run()` create its metadata row idempotently before its first status update, so every execution path, including workflow-owned tasks, persists history when a metadata store is configured. Have Host Agent register Cloud task/attempt correlation before goal execution so operators can match a submitted Cloud task to its local execution record.
|
|
- Expose that step-level detail through Host Agent's local console: extend `AgentStatusTracker`/`/api/status` (or add a focused endpoint) with current step index, step status, and a short in-progress step log; render it in the dashboard's "Current assignment" section instead of the static started-at-only view.
|
|
- Add a progress-reporting path from Host Agent to Cloud Control Plane so the control plane learns step-level state near-real-time rather than only at claim/heartbeat/terminal-result. Extend the existing `cloud.internal_api.models` Pydantic schema (the single source of truth Host Agent already imports directly) rather than introducing a parallel schema.
|
|
- Persist and expose the latest per-assignment progress on the Cloud Control Plane side, and surface it in Cloud Console so an operator watching a remote task sees live step progress, not just "dispatched" / "succeeded" / "failed".
|
|
- Make Host Agent's own authenticated, server-rendered `:8765/tasks` pages the authoritative web entry point for actual execution history. They render the shared Timeline records directly and retain the existing same-origin session/CSRF boundary.
|
|
- All three surfaces continue to use polling (matching current behavior); this change does not introduce SSE/WebSocket infrastructure unless design.md finds a compelling reason to.
|
|
- Fix the shared `Timeline`/`TaskRunner`/`AIPlanner` recording path so a persisted step's "prompt" is the *actual* prompt sent to the LLM for that step (not the task's overall goal) and the model's resulting decision is captured too — this pre-existing gap affects Runtime and Host Agent alike and undermines the step-level detail this change otherwise adds.
|
|
- Persist a durable, per-step log of full LLM prompt/response content on the Cloud Control Plane, for centralized troubleshooting — reusing the already-existing `cloud-planner-proxy` decide endpoint as the capture point (no new protocol/endpoint) rather than the coarse lease-renewal piggyback used for live index/status. This durable log is populated only for hosts using the `cloud` planner transport; hosts on the `direct` transport still get only the coarse index/status via lease renewal. Cloud Console gains a view to browse a task's full LLM interaction history.
|
|
- Keep before/after screenshots, operation detail, raw OCR observations, and normalized UI-tree results in the shared Runtime Timeline; render all of that evidence in the Host Agent task-detail page.
|
|
- Retire the standalone Runtime REST service, its unauthenticated console/UI, and Host Agent Runtime-supervision settings. Preserve the shared `runtime/`, `storage/`, and non-REST `api/` library modules used by the Host Agent and MCP integrations.
|
|
|
|
## Capabilities
|
|
|
|
### New Capabilities
|
|
- `host-agent-task-progress`: Host Agent captures step-level execution progress for every in-process task, correlates Cloud assignments with local records, and exposes it through its local console/API.
|
|
- `cloud-task-progress-visibility`: Cloud Control Plane receives, persists, and exposes near-real-time step-level progress for assignments it has dispatched to a Host Agent, and Cloud Console renders it.
|
|
- `host-agent-console-task-pages`: Host Agent's local server-rendered console gains the complete before/after evidence, OCR, and UI-tree views for Host-Agent-executed tasks and is the only web inspection surface for those executions.
|
|
- `runtime-task-evidence`: Runtime task history retains complete pre/post action evidence, raw OCR observations, and existing UI-tree inspection results for Host Agent rendering.
|
|
- `runtime-standalone-service`: the standalone Runtime REST service and its UI are removed; Runtime remains an execution library rather than a second operational console.
|
|
|
|
### Modified Capabilities
|
|
- `host-agent-protocol`: add a requirement that the Host Agent reports in-progress step-level status updates to the control plane (in addition to the existing heartbeat/claim/renewal/result operations), and that the control plane accepts and stores them per active assignment.
|
|
- `cloud-planner-proxy`: the existing planner-decision endpoint additionally persists each resolved decision's prompt/response into a durable, bounded-retention per-step log, instead of discarding it after the response is returned.
|
|
|
|
## Impact
|
|
|
|
- `apps/device-host-agent/host_agent/app.py`, `execution.py`, `assignment.py`, `status.py`, `web/app.py` — wire a real metadata/timeline store into the in-process `TaskRunner`, extend status tracking and the local console UI/API.
|
|
- `runtime/task.py`, `runtime/ai_planner.py`, `runtime/tool_calling_client.py`, `storage/timeline.py`, `storage/artifact_store.py`, `core/models.py`, `perception/scene_builder.py` — fix the shared step-recording path so the real per-step prompt and the model's response are captured, retain pre/post action screenshots and raw OCR observations, not just the task goal and parsed tool call.
|
|
- `packages/cloud-platform/cloud/internal_api/models.py`, `packages/cloud-platform/cloud/internal_api/api.py` (`decide_planner_call`), `repository.py`/`sql_repository.py`, a new Alembic migration — new progress-reporting request/response models and a persistence + query path for latest per-assignment coarse progress, *and* a new durable per-step `planner_decision_log` table (with its own retention job) populated from the existing planner-decision endpoint.
|
|
- `cloud-console/` (Vue3 SPA) — new UI to render live per-assignment progress, and a new view to browse a task's full LLM interaction history.
|
|
- `api/rest.py`, `api/console.py`, `api/console_web.py`, `api/templates/runtime_console/*`, `api/static/runtime_console/*`, their tests, package data, and Runtime-supervision configuration — removed.
|
|
- `apps/device-host-agent/host_agent/assignment.py`, `execution.py`, `web/app.py`, and its task templates — create correlated execution records and render the complete shared Timeline evidence.
|
|
- `runtime/task.py`, `storage/task_metadata.py`, and `storage/timeline.py` — make durable task creation an execution invariant while retaining generic storage boundaries.
|