This commit is contained in:
@@ -12,6 +12,7 @@ Nobody can see a task while it is running. Host Agent's local console only shows
|
||||
- All three surfaces continue to use polling (matching current behavior); this change does not introduce SSE/WebSocket infrastructure unless design.md finds a compelling reason to.
|
||||
- Fix the shared `Timeline`/`TaskRunner`/`AIPlanner` recording path so a persisted step's "prompt" is the *actual* prompt sent to the LLM for that step (not the task's overall goal) and the model's resulting decision is captured too — this pre-existing gap affects Runtime and Host Agent alike and undermines the step-level detail this change otherwise adds.
|
||||
- Persist a durable, per-step log of full LLM prompt/response content on the Cloud Control Plane, for centralized troubleshooting — reusing the already-existing `cloud-planner-proxy` decide endpoint as the capture point (no new protocol/endpoint) rather than the coarse lease-renewal piggyback used for live index/status. This durable log is populated only for hosts using the `cloud` planner transport; hosts on the `direct` transport still get only the coarse index/status via lease renewal. Cloud Console gains a view to browse a task's full LLM interaction history.
|
||||
- Extend the local Runtime timeline so every executed action retains a before screenshot, operation detail, after screenshot, and any available raw OCR observations; render that evidence in the Runtime task-detail UI and expose it from the existing Runtime console API. When a step uses the existing UI-tree tool, render its normalized node result as a structured, collapsible view as well.
|
||||
|
||||
## Capabilities
|
||||
|
||||
@@ -19,6 +20,7 @@ Nobody can see a task while it is running. Host Agent's local console only shows
|
||||
- `host-agent-task-progress`: Host Agent captures step-level execution progress for its in-flight assignment (via a wired `metadata_store`/`timeline`) and exposes it through its local console/API.
|
||||
- `cloud-task-progress-visibility`: Cloud Control Plane receives, persists, and exposes near-real-time step-level progress for assignments it has dispatched to a Host Agent, and Cloud Console renders it.
|
||||
- `host-agent-console-task-pages`: Host Agent's local server-rendered console gains read-only task list/detail/timeline pages (mirroring `api/console.py`'s task query shape) so an operator can inspect a Host-Agent-executed task's progress and history without needing the separate Runtime `console/` SPA or violating the `runtime/`-package host/cloud isolation boundary.
|
||||
- `runtime-task-evidence`: Runtime task history retains and renders pre/post action evidence, raw OCR observations, and existing UI-tree inspection results.
|
||||
|
||||
### Modified Capabilities
|
||||
- `host-agent-protocol`: add a requirement that the Host Agent reports in-progress step-level status updates to the control plane (in addition to the existing heartbeat/claim/renewal/result operations), and that the control plane accepts and stores them per active assignment.
|
||||
@@ -27,8 +29,8 @@ Nobody can see a task while it is running. Host Agent's local console only shows
|
||||
## Impact
|
||||
|
||||
- `apps/device-host-agent/host_agent/app.py`, `execution.py`, `assignment.py`, `status.py`, `web/app.py` — wire a real metadata/timeline store into the in-process `TaskRunner`, extend status tracking and the local console UI/API.
|
||||
- `runtime/task.py`, `runtime/ai_planner.py`, `runtime/tool_calling_client.py`, `storage/timeline.py` — fix the shared step-recording path so the real per-step prompt and the model's response are captured, not just the task goal and parsed tool call (a correctness fix in already-existing, shared code, not new behavior scope).
|
||||
- `runtime/task.py`, `runtime/ai_planner.py`, `runtime/tool_calling_client.py`, `storage/timeline.py`, `storage/artifact_store.py`, `core/models.py`, `perception/scene_builder.py` — fix the shared step-recording path so the real per-step prompt and the model's response are captured, retain pre/post action screenshots and raw OCR observations, not just the task goal and parsed tool call.
|
||||
- `packages/cloud-platform/cloud/internal_api/models.py`, `packages/cloud-platform/cloud/internal_api/api.py` (`decide_planner_call`), `repository.py`/`sql_repository.py`, a new Alembic migration — new progress-reporting request/response models and a persistence + query path for latest per-assignment coarse progress, *and* a new durable per-step `planner_decision_log` table (with its own retention job) populated from the existing planner-decision endpoint.
|
||||
- `cloud-console/` (Vue3 SPA) — new UI to render live per-assignment progress, and a new view to browse a task's full LLM interaction history.
|
||||
- `apps/device-host-agent/host_agent/web/app.py` — new read-only task list/detail/timeline pages backed by the Host-Agent-local `TaskMetadataStore`/`Timeline`; no changes anticipated to `console/` (Vue3 SPA).
|
||||
- `api/console.py`, `api/console_web.py`, `api/templates/runtime_console/*` — expose and render Runtime-local step evidence; Host Agent task pages receive the data through the shared timeline shape.
|
||||
- No changes anticipated to `driver/`, `device/`, `core/` device-control internals.
|
||||
|
||||
Reference in New Issue
Block a user