chore(openspec): archive task execution visibility

This commit is contained in:
2026-07-15 12:10:14 +08:00
parent 77d4813bb2
commit 96e403ee47
17 changed files with 372 additions and 64 deletions
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-07-14
@@ -0,0 +1,176 @@
## Context
Host Agent executes Cloud assignments in-process through the shared
`runtime.TaskRunner`. Its `HostAgentApplication` already constructs a
Host-local `TaskMetadataStore` and `Timeline` and injects them into the
runner factory. The missing link is task creation: `AssignmentExecutor`
constructs a `Task`, calls `runner.run(task)`, and `TaskRunner` only
issues updates. SQLite therefore receives updates for a row that does not
exist, so the Host Agent `/tasks` UI is empty.
The separate `api.rest` process was never on this execution path. It owned a
different database and artifact root, so its REST endpoints and UI could only
show tasks submitted directly to that unrelated process. Running it alongside
a Host Agent created two competing operator entry points without transferring
any history between them.
## Goals / Non-Goals
**Goals:**
- Make durable task-row creation an invariant of `TaskRunner.run()` whenever
a metadata store is configured.
- Correlate a Host-local goal execution with the Cloud task ID and attempt that
caused it, without making shared Runtime or storage packages import Cloud
models.
- Make the authenticated Host Agent console at port `8765` the authoritative
web view for actual task execution: task list, detail, live status, before
screenshot, operation, after screenshot, OCR observations, and normalized
UI-tree results.
- Preserve the shared Runtime Timeline as the evidence model and keep its
before/after screenshot, OCR, and UI-tree capture behavior.
- Retire the standalone Runtime REST service/UI and its Host Agent supervisor
configuration while retaining Runtime, storage, MCP, and skill-sync library
modules.
- Retain the existing Cloud latest-progress and Cloud-proxy planner-decision
history behavior. Cloud remains a fleet-level view and never stores
screenshots.
**Non-Goals:**
- No SSE/WebSocket push; Host and Cloud consoles keep their existing polling.
- No duplicate full evidence upload to Cloud. Screenshots and Timeline
artifacts remain Host-local.
- No new Host/Cloud dependencies in `runtime/`, `storage/`, `driver/`, or
`device/`.
- No replacement general-purpose device-control REST API. Operators use Host
Agent device management and task pages for the managed execution workflow.
## Decisions
### D1: TaskRunner owns idempotent task metadata creation
`TaskRunner.run()` will call `TaskMetadataStore.create_task(task)` before
the first running-state update. `create_task()` will use idempotent insert
semantics so callers that already created a direct Runtime task remain
compatible and workflow-created tasks are captured automatically.
This places the invariant at the shared execution boundary rather than relying
on every caller to remember an out-of-band persistence call. It fixes goal
assignments and prevents the same failure for `WorkflowRunner` planned-goal
steps.
### D2: Host assignment correlation stays in the Host adapter
Before executing a Cloud goal assignment, `AssignmentExecutor` creates the
local task record with optional generic `source_task_id` and
`source_attempt` metadata. Shared storage uses generic names and does not
import Cloud types. The Host Agent task list/detail renders those fields as the
Cloud task ID and attempt.
The local `Task.id` remains generated by the Runtime. Reusing the Cloud task
ID as an artifact directory name would make retries overwrite each other and
would admit unsafe path characters from an external identifier.
### D3: Host Agent console is the authoritative evidence UI
The Host Agent already owns the device manager, assignment executor,
metadata store, Timeline, local account, session, and CSRF boundary. Its
same-origin `/tasks` and `/tasks/{task_id}` pages therefore render the
shared Timeline directly. The task list is named for Host executions rather
than "Local Runtime tasks", and submission feedback tells the operator that
the submitted Cloud task appears there when this Host begins execution.
No browser needs to point a separate frontend at the Host Agent. This keeps
the conservative local-account/session model and does not add CORS.
### D4: Complete per-step evidence is rendered by the Host Agent
Timeline records retain distinct pre-action and post-action screenshot paths,
the action description and arguments, the execution result, raw OCR
observations, and the existing normalized UI-tree result. The Host Agent page
creates data URIs only for available local artifacts and supports legacy
records where the single `screenshot_path` is the post-action image.
OCR is rendered when present. UI-tree output is rendered only for
`get_ui_tree` and `ui_tree` records that contain normalized nodes; it uses a
collapsible structured view while retaining the JSON result. No tool contract
or duplicate persistence field is introduced.
### D5: Host-local retention remains bounded
The existing Host retention pass continues to remove metadata rows, Timeline
records, and artifacts according to the configured count and age thresholds.
The new task creation invariant must use that same store so it cannot create
an unbounded second history source.
### D6: Cloud progress remains a latest snapshot on lease renewal
The Host execution thread writes a bounded latest-progress holder. The lease
renewal path optionally carries its step index, status, and summary; Cloud
stores only the latest snapshot for an active assignment and stops exposing it
after terminal completion. No screenshot or scene payload enters this
protocol.
### D7: Cloud-proxy planner decisions remain durable Cloud history
For `AI_PLANNER_TRANSPORT=cloud`, Cloud's existing planner-decision endpoint
persists successful system/user prompts, tool calls, arguments, and a
task/attempt-scoped step index. The direct transport intentionally produces no
such Cloud history. This log has bounded terminal-task retention and never
persists request screenshot bytes.
### D8: Retire the standalone Runtime REST service and UI
Remove `api/rest.py`, `api/console.py`, `api/console_web.py`, their
templates/static assets, their package-data declarations, and their
service/UI tests. Preserve `api/mcp.py`, `api/errors.py`, skill-sync, and
skill-catalog modules because they are independent library integrations.
Remove `HOST_AGENT_RUNTIME_SUPERVISED`,
`HOST_AGENT_RUNTIME_HOST`, and `HOST_AGENT_RUNTIME_PORT`. The optional
dependency supervisor continues to support Appium only. Configuration with a
removed Runtime-supervision variable fails with an actionable migration error
instead of silently doing nothing.
### D9: Documentation points operators to Host Agent
Operator documentation no longer instructs users to start `uvicorn
api.rest:create_app` or browse port `8000`. It identifies the Host Agent
console at `http://127.0.0.1:8765/tasks` as the execution-history authority,
explains its local authentication, and documents that the Cloud Console is a
fleet/progress and Cloud-proxy LLM-history surface rather than a screenshot
store.
## Risks / Trade-offs
- Per-step metadata writes add small SQLite I/O. This is the same local store
already selected for Host history, and bounded retention limits growth.
- A Host-local evidence record is only available while retained on that Host.
This is intentional: it reflects the actual device execution and avoids
sending screenshots to Cloud.
- Removing the unauthenticated Runtime REST service is a breaking operator
change. Clear configuration errors and documentation avoid a silent
fallback to a nonexistent inspection surface.
- Full Cloud-proxy prompts can contain visible screen text. This is the
previously accepted Cloud troubleshooting trade-off; screenshot bytes remain
excluded.
## Migration Plan
1. Upgrade the Host Agent code. Existing task databases gain nullable source
correlation columns on startup; legacy Timeline records remain readable.
2. Remove any `HOST_AGENT_RUNTIME_*` environment variables and stop any
`api.rest` process. Start or browse only the Host Agent console for local
execution evidence.
3. Confirm a completed Host assignment appears at `/tasks` with its Cloud
task/attempt correlation and complete Timeline evidence.
4. Roll back only by restoring the prior release. The retired REST/UI routes
are deliberately not kept as a compatibility alias because their storage
was not connected to Host execution.
## Open Questions
- Manual verification still requires a real Host Agent, Appium, and device.
Automated coverage verifies persistence, correlation, rendering, and
service removal; real hardware validates screenshots and OCR availability.
@@ -0,0 +1,39 @@
## Why
Host Agent now wires a local `TaskMetadataStore` and `Timeline` into its `TaskRunner`, but its Cloud-assignment path still creates a `Task` and immediately calls `runner.run(task)`. `TaskRunner` only updates an existing metadata row, so every update affects zero rows and the Host Agent task page remains empty. The standalone Runtime REST service/UI is a separate process with unrelated storage, so it cannot be the inspection surface for Host Agent executions. An operator who submits or receives a task therefore has no authoritative web view of the actions that actually ran on that Host.
## What Changes
- Make `TaskRunner.run()` create its metadata row idempotently before its first status update, so every execution path, including workflow-owned tasks, persists history when a metadata store is configured. Have Host Agent register Cloud task/attempt correlation before goal execution so operators can match a submitted Cloud task to its local execution record.
- Expose that step-level detail through Host Agent's local console: extend `AgentStatusTracker`/`/api/status` (or add a focused endpoint) with current step index, step status, and a short in-progress step log; render it in the dashboard's "Current assignment" section instead of the static started-at-only view.
- Add a progress-reporting path from Host Agent to Cloud Control Plane so the control plane learns step-level state near-real-time rather than only at claim/heartbeat/terminal-result. Extend the existing `cloud.internal_api.models` Pydantic schema (the single source of truth Host Agent already imports directly) rather than introducing a parallel schema.
- Persist and expose the latest per-assignment progress on the Cloud Control Plane side, and surface it in Cloud Console so an operator watching a remote task sees live step progress, not just "dispatched" / "succeeded" / "failed".
- Make Host Agent's own authenticated, server-rendered `:8765/tasks` pages the authoritative web entry point for actual execution history. They render the shared Timeline records directly and retain the existing same-origin session/CSRF boundary.
- All three surfaces continue to use polling (matching current behavior); this change does not introduce SSE/WebSocket infrastructure unless design.md finds a compelling reason to.
- Fix the shared `Timeline`/`TaskRunner`/`AIPlanner` recording path so a persisted step's "prompt" is the *actual* prompt sent to the LLM for that step (not the task's overall goal) and the model's resulting decision is captured too — this pre-existing gap affects Runtime and Host Agent alike and undermines the step-level detail this change otherwise adds.
- Persist a durable, per-step log of full LLM prompt/response content on the Cloud Control Plane, for centralized troubleshooting — reusing the already-existing `cloud-planner-proxy` decide endpoint as the capture point (no new protocol/endpoint) rather than the coarse lease-renewal piggyback used for live index/status. This durable log is populated only for hosts using the `cloud` planner transport; hosts on the `direct` transport still get only the coarse index/status via lease renewal. Cloud Console gains a view to browse a task's full LLM interaction history.
- Keep before/after screenshots, operation detail, raw OCR observations, and normalized UI-tree results in the shared Runtime Timeline; render all of that evidence in the Host Agent task-detail page.
- Retire the standalone Runtime REST service, its unauthenticated console/UI, and Host Agent Runtime-supervision settings. Preserve the shared `runtime/`, `storage/`, and non-REST `api/` library modules used by the Host Agent and MCP integrations.
## Capabilities
### New Capabilities
- `host-agent-task-progress`: Host Agent captures step-level execution progress for every in-process task, correlates Cloud assignments with local records, and exposes it through its local console/API.
- `cloud-task-progress-visibility`: Cloud Control Plane receives, persists, and exposes near-real-time step-level progress for assignments it has dispatched to a Host Agent, and Cloud Console renders it.
- `host-agent-console-task-pages`: Host Agent's local server-rendered console gains the complete before/after evidence, OCR, and UI-tree views for Host-Agent-executed tasks and is the only web inspection surface for those executions.
- `runtime-task-evidence`: Runtime task history retains complete pre/post action evidence, raw OCR observations, and existing UI-tree inspection results for Host Agent rendering.
- `runtime-standalone-service`: the standalone Runtime REST service and its UI are removed; Runtime remains an execution library rather than a second operational console.
### Modified Capabilities
- `host-agent-protocol`: add a requirement that the Host Agent reports in-progress step-level status updates to the control plane (in addition to the existing heartbeat/claim/renewal/result operations), and that the control plane accepts and stores them per active assignment.
- `cloud-planner-proxy`: the existing planner-decision endpoint additionally persists each resolved decision's prompt/response into a durable, bounded-retention per-step log, instead of discarding it after the response is returned.
## Impact
- `apps/device-host-agent/host_agent/app.py`, `execution.py`, `assignment.py`, `status.py`, `web/app.py` — wire a real metadata/timeline store into the in-process `TaskRunner`, extend status tracking and the local console UI/API.
- `runtime/task.py`, `runtime/ai_planner.py`, `runtime/tool_calling_client.py`, `storage/timeline.py`, `storage/artifact_store.py`, `core/models.py`, `perception/scene_builder.py` — fix the shared step-recording path so the real per-step prompt and the model's response are captured, retain pre/post action screenshots and raw OCR observations, not just the task goal and parsed tool call.
- `packages/cloud-platform/cloud/internal_api/models.py`, `packages/cloud-platform/cloud/internal_api/api.py` (`decide_planner_call`), `repository.py`/`sql_repository.py`, a new Alembic migration — new progress-reporting request/response models and a persistence + query path for latest per-assignment coarse progress, *and* a new durable per-step `planner_decision_log` table (with its own retention job) populated from the existing planner-decision endpoint.
- `cloud-console/` (Vue3 SPA) — new UI to render live per-assignment progress, and a new view to browse a task's full LLM interaction history.
- `api/rest.py`, `api/console.py`, `api/console_web.py`, `api/templates/runtime_console/*`, `api/static/runtime_console/*`, their tests, package data, and Runtime-supervision configuration — removed.
- `apps/device-host-agent/host_agent/assignment.py`, `execution.py`, `web/app.py`, and its task templates — create correlated execution records and render the complete shared Timeline evidence.
- `runtime/task.py`, `storage/task_metadata.py`, and `storage/timeline.py` — make durable task creation an execution invariant while retaining generic storage boundaries.
@@ -0,0 +1,38 @@
## REMOVED Requirements
### Requirement: Planner-decision requests are not durably persisted
## ADDED Requirements
### Requirement: Cloud Control Plane persists each resolved planner decision for later retrieval
The Cloud Control Plane's planner-decision endpoint SHALL, for each request it successfully resolves to a tool-call decision, persist the request's system prompt, user prompt, the resolved tool name and arguments, and an assigned step index (scoped to the request's `task_id`/`attempt`) to a durable, bounded-retention store, in addition to returning the decision to the requesting Host Agent. The Cloud Control Plane SHALL NOT persist screenshot bytes from these requests.
#### Scenario: A planner-decision request resolves successfully
- **WHEN** the Cloud Control Plane resolves a planner-decision request to a tool-call decision
- **THEN** it persists the request's system prompt, user prompt, the resolved tool name and arguments, and a step index for that `task_id`/`attempt`, before returning the decision to the Host Agent
#### Scenario: A planner-decision request fails
- **WHEN** the configured provider call fails and the endpoint returns a structured failure response
- **THEN** no row is persisted for that request
#### Scenario: A screenshot was included in the request
- **WHEN** a planner-decision request includes a screenshot
- **THEN** the screenshot bytes are used only to call the LLM provider and are not written to the persisted decision log
### Requirement: Persisted planner decisions are retained within a bounded window
The Cloud Control Plane SHALL prune persisted planner decisions once their owning task has been in a terminal state for longer than a configurable retention window, so that indefinite operation does not cause unbounded growth of the decision log.
#### Scenario: A task's retention window has elapsed since reaching a terminal state
- **WHEN** a task reached a terminal state more than the configured retention window ago
- **THEN** the Cloud Control Plane removes that task's persisted planner decisions
#### Scenario: A task is still active or within its retention window
- **WHEN** a task is still active, or reached a terminal state less than the configured retention window ago
- **THEN** its persisted planner decisions remain available for query
### Requirement: Cloud-side planner decision history is scoped to hosts using the cloud-proxy transport
The Cloud Control Plane's persisted planner decision log SHALL only ever contain entries for hosts whose planner calls were routed through the cloud-proxy transport; it SHALL NOT contain entries, synthesized or otherwise, for hosts using the direct-to-provider transport.
#### Scenario: A host uses the direct-to-provider transport
- **WHEN** a Host Agent configured for the direct-to-provider transport executes an assignment
- **THEN** no planner decision entries for that assignment appear in the Cloud Control Plane's persisted decision log
@@ -0,0 +1,38 @@
## ADDED Requirements
### Requirement: Cloud Control Plane exposes the latest in-progress step status for an active assignment
The Cloud Control Plane's existing task query surface SHALL include the latest reported step index, step status, and summary for any assignment that has an in-progress Host Agent execution, alongside the task's existing status fields.
#### Scenario: An assignment has reported progress
- **WHEN** an operator queries a task that has an active, in-progress assignment with previously reported step progress
- **THEN** the response includes that step's index, status, and summary alongside the task's existing fields
#### Scenario: An assignment has no reported progress yet
- **WHEN** an operator queries a task whose active assignment has not yet reported any progress
- **THEN** the response omits progress fields rather than showing stale or default values
#### Scenario: An assignment has reached a terminal state
- **WHEN** an operator queries a task whose assignment has already completed (succeeded or failed)
- **THEN** the response does not present the last in-progress step as current status; the task's terminal status and result take precedence
### Requirement: Cloud Console renders live step progress for in-progress tasks
Cloud Console SHALL display the current step index, status, and summary for a task with an active, in-progress Host Agent execution, refreshed on its existing polling interval, without requiring a new push channel.
#### Scenario: Operator views an in-progress task
- **WHEN** an operator opens a task detail view for a task with an active in-progress assignment
- **THEN** Cloud Console shows the latest known step index, status, and summary, updating on subsequent polls as new progress is reported
#### Scenario: Operator views a task with no in-progress execution
- **WHEN** an operator opens a task detail view for a queued, terminal, or otherwise not-currently-executing task
- **THEN** Cloud Console does not display stale in-progress step information
### Requirement: Cloud Console displays a task's full LLM interaction history
Cloud Console SHALL provide a view, for a given task, listing each persisted planner decision in step order, including its full prompt and resulting decision, sourced from the Cloud Control Plane's persisted planner decision log.
#### Scenario: Task has persisted planner decisions
- **WHEN** an operator opens the LLM interaction history view for a task that has one or more persisted planner decisions
- **THEN** Cloud Console shows each decision in step order with its prompt and resulting tool call
#### Scenario: Task's host used the direct-to-provider transport
- **WHEN** an operator opens the LLM interaction history view for a task whose host used the direct-to-provider transport
- **THEN** Cloud Console indicates that no LLM interaction history is available because the host does not report it, rather than showing an empty history with no explanation
@@ -0,0 +1,67 @@
## MODIFIED Requirements
### Requirement: Host Agent local console exposes read-only task history with per-step detail and screenshots
The Host Agent's local console SHALL provide authenticated, read-only pages
listing recently executed local tasks and, for a selected task, its full
per-step history from the Host-local metadata store and Timeline. The task
detail SHALL show available before and after screenshots, operation details and
arguments, execution result, OCR observations, and normalized UI-tree results.
It SHALL render legacy Timeline records that only have a single screenshot as a
post-action image.
#### Scenario: Operator lists recent Host executions
- **WHEN** an authenticated operator opens the Host Agent local console's task
list page
- **THEN** it shows local executions most recent first, including terminal
tasks and any available Cloud task ID and attempt correlation
#### Scenario: Operator inspects a completed task's step history
- **WHEN** an authenticated operator opens the detail page for a completed
Host execution
- **THEN** the page shows each recorded step in order with its tool call,
result, and available before/after screenshots
#### Scenario: OCR was captured for a step
- **WHEN** the selected Timeline record contains OCR observations
- **THEN** the detail page shows each observation's text, confidence, and
bounds
#### Scenario: A UI-tree tool returned normalized nodes
- **WHEN** the selected Timeline record invoked `get_ui_tree` or `ui_tree`
and its result contains normalized nodes
- **THEN** the detail page exposes a structured, collapsible node view while
retaining the persisted result JSON
#### Scenario: A legacy timeline record is displayed
- **WHEN** a Timeline record has only `screenshot_path`
- **THEN** the detail page renders it as the post-action image without failing
#### Scenario: Unauthenticated request
- **WHEN** a request to the task list or task detail pages is made without a
valid Host Agent console session
- **THEN** the Host Agent rejects the request the same way it rejects
unauthenticated requests to its other console pages
### Requirement: Host Agent local console task pages require no new cross-origin surface
The Host Agent local console's task pages SHALL be served same-origin from the
Host Agent's existing web application, without introducing new CORS allowances
or a dependency on a separate Runtime frontend.
#### Scenario: Task pages are requested
- **WHEN** an operator's browser requests the Host Agent local console's task
pages
- **THEN** the pages are served by the Host Agent's own application using its
existing session/CSRF protections, with no additional cross-origin
configuration required
## ADDED Requirements
### Requirement: Host Agent console is the authority for actual execution evidence
The Host Agent local console SHALL be the web authority for task evidence
produced by that Host's in-process execution path. A standalone Runtime
service/UI SHALL NOT be required or consulted to inspect a Host execution.
#### Scenario: A Cloud task is executed by a Host Agent
- **WHEN** an operator opens that Host Agent's task page after execution starts
- **THEN** the page reads the same Host-local metadata and Timeline that the
executing `TaskRunner` writes
@@ -0,0 +1,51 @@
## MODIFIED Requirements
### Requirement: Supervisor is opt-in and disabled by default
The Host Agent SHALL NOT start, adopt-check, or supervise Appium unless
`HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED` is explicitly set to true. Appium
supervision SHALL additionally require `HOST_AGENT_APPIUM_SUPERVISED=true`
and SHALL default to false.
#### Scenario: Default configuration behaves exactly as before
- **WHEN** a Host Agent starts with no
`HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED` or related Appium environment
variable set
- **THEN** the Host Agent does not attempt to connect to, probe, or spawn
Appium, and its heartbeat/claim behavior is unchanged
#### Scenario: Top-level flag on and Appium flag off
- **WHEN** `HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED=true` and
`HOST_AGENT_APPIUM_SUPERVISED=false`
- **THEN** the Host Agent does not probe, adopt, or spawn Appium
### Requirement: Spawn supervised dependencies that are not already running
The Host Agent SHALL spawn Appium as a child process when Appium supervision is
enabled and no healthy Appium instance is adopted, via
`appium --address <host> --port <port>` and SHALL forward the child
process's stdout/stderr into the Host Agent's own logging, tagged by dependency
name.
#### Scenario: Appium is not running at Host Agent startup
- **WHEN** Appium supervision is enabled and no healthy Appium instance is
already listening
- **THEN** the Host Agent spawns Appium before proceeding to its first
device-connect attempt, and its output is visible in Host Agent logs
#### Scenario: Spawn fails because the executable is missing
- **WHEN** the Host Agent attempts to spawn Appium but `appium` is not found
on `PATH`
- **THEN** the Host Agent logs a dependency-supervisor-specific startup error
naming the missing dependency, distinct from a runtime crash of an
already-started process
## ADDED Requirements
### Requirement: Runtime supervision settings are retired
The Host Agent SHALL reject `HOST_AGENT_RUNTIME_SUPERVISED`,
`HOST_AGENT_RUNTIME_HOST`, and `HOST_AGENT_RUNTIME_PORT` because the
standalone Runtime service no longer exists.
#### Scenario: A legacy Runtime supervision variable is set
- **WHEN** startup configuration includes any removed Runtime supervision
variable
- **THEN** configuration fails with an actionable migration error
@@ -0,0 +1,23 @@
## ADDED Requirements
### Requirement: Host Agent reports execution progress alongside lease renewal
The Host Agent SHALL optionally include a bounded, screenshot-free progress summary (current step index, step status, and a short plain-text summary) in its periodic lease-renewal request for an active assignment, and the control plane SHALL accept and store only the most recent such summary per active assignment.
#### Scenario: Progress is available at renewal time
- **WHEN** the Host Agent renews the lease for an in-progress assignment and has a current step index, status, and summary available
- **THEN** the renewal request includes that progress summary and the control plane overwrites any previously stored progress for that assignment with it
#### Scenario: Progress is not available at renewal time
- **WHEN** the Host Agent renews a lease without a progress summary available (e.g. before the first step completes)
- **THEN** the renewal request omits the progress field and any previously stored progress for that assignment is left unchanged
#### Scenario: Assignment reaches a terminal state
- **WHEN** an assignment's terminal result is recorded
- **THEN** the control plane's stored progress for that assignment is no longer treated as current and is not exposed as an in-progress status
### Requirement: Progress reports exclude screenshot and scene payloads
The control plane SHALL reject or ignore any progress field on a renewal request that includes screenshot, scene, or other bulk payload data beyond the bounded step index, status, and short text summary.
#### Scenario: Renewal request includes an oversized or non-text summary
- **WHEN** a Host Agent submits a progress summary exceeding the configured length bound
- **THEN** the control plane truncates or rejects the oversized field without failing the underlying lease renewal
@@ -0,0 +1,74 @@
## MODIFIED Requirements
### Requirement: Host Agent records step-level execution detail for its in-process TaskRunner
The Host Agent SHALL construct its in-process `TaskRunner` with a durable
metadata store and Timeline. `TaskRunner.run()` SHALL create the task's
metadata row idempotently before its first status update, so every execution
path persists its task status and evidence rather than discarding updates for a
missing row. Every completed step SHALL retain its index, actual per-step LLM
prompt and decision when available, tool call, result, distinct before/after
screenshots when captured, raw OCR observations when available, and normalized
UI-tree result when the invoked tool returned one.
#### Scenario: A goal assignment starts execution
- **WHEN** the Host Agent's `AssignmentExecutor` invokes its `TaskRunner`
- **THEN** the task metadata row exists before the runner records its running
status
#### Scenario: A step completes during goal execution
- **WHEN** the Host Agent's `TaskRunner` completes a step while executing an
assigned goal
- **THEN** the step's status, index, actual per-step LLM prompt and response,
tool call, result, and available evidence are persisted before the next step
begins
#### Scenario: A workflow creates a planned-goal task
- **WHEN** a `WorkflowRunner` invokes a Host Agent-configured
`TaskRunner` for a planned-goal step
- **THEN** that task is persisted without requiring the workflow caller to
create a metadata row separately
#### Scenario: An assignment finishes
- **WHEN** an assignment reaches a terminal state (succeeded or failed)
- **THEN** its full step history remains queryable from the Host Agent's local
store after the in-memory `Task` object is discarded
### Requirement: Host-Agent-local task history is retained within a bounded window
The Host Agent SHALL prune persisted task metadata, Timeline records, and
associated screenshot artifacts once they exceed a configurable retention
window or count, so that indefinite process uptime does not cause unbounded
local disk growth.
#### Scenario: Retention window is exceeded
- **WHEN** a persisted task's age or position exceeds the configured retention
threshold
- **THEN** the Host Agent removes that task's metadata row, Timeline records,
and screenshot artifacts from local storage
#### Scenario: Retention has not been exceeded
- **WHEN** a persisted task is within the configured retention threshold
- **THEN** its metadata, Timeline records, and screenshot artifacts remain
available for query
## ADDED Requirements
### Requirement: Host Agent correlates local execution records with Cloud assignments
For a Cloud-dispatched goal assignment, the Host Agent SHALL persist the Cloud
task ID and attempt alongside its generated local Runtime task ID before
execution starts. The correlation fields SHALL remain optional and generic in
the shared storage layer.
#### Scenario: A Cloud goal assignment begins
- **WHEN** the Host Agent begins executing a Cloud goal assignment
- **THEN** the local task row records that assignment's Cloud task ID and
attempt
#### Scenario: A task is not Cloud-dispatched
- **WHEN** a shared Runtime caller executes a task without Host/Cloud
assignment context
- **THEN** the task metadata row is created and the optional source
correlation fields remain empty
## REMOVED Requirements
### Requirement: Host Agent local task storage is isolated from an unrelated local Runtime
@@ -0,0 +1,33 @@
## ADDED Requirements
### Requirement: Runtime is not exposed as a standalone REST service or web console
The repository SHALL not ship a standalone Runtime REST application, its
unauthenticated web console, or JSON console routes. The shared Runtime and
storage packages SHALL remain reusable execution libraries for the Host Agent
and other in-process callers.
#### Scenario: An operator needs to inspect a Host-executed task
- **WHEN** an operator needs task evidence for a Host Agent execution
- **THEN** the operator uses the authenticated Host Agent console rather than
starting or querying a separate Runtime service
#### Scenario: A package uses shared Runtime execution
- **WHEN** the Host Agent or another in-process caller creates a
`TaskRunner`
- **THEN** it continues to use the shared Runtime and storage packages without
importing a REST or UI adapter
### Requirement: Host Agent does not supervise a retired Runtime service
The Host Agent SHALL not expose Runtime-supervision configuration or spawn a
Runtime REST subprocess. It MAY continue to optionally supervise Appium.
#### Scenario: Host Agent dependency supervision is enabled
- **WHEN** `HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED=true` and Appium
supervision is enabled
- **THEN** the Host Agent probes and supervises Appium only
#### Scenario: A removed Runtime-supervision variable is configured
- **WHEN** a Host Agent configuration includes a removed
`HOST_AGENT_RUNTIME_*` variable
- **THEN** startup fails with a message directing the operator to the Host
Agent console and Appium-only supervision
@@ -0,0 +1,61 @@
## ADDED Requirements
### Requirement: Runtime persists complete evidence for each executed action
The shared Runtime SHALL persist, for each action it attempts, a screenshot
captured immediately before the executor call, the action description and
arguments, the execution result, and a screenshot captured immediately after
the executor call. Existing Timeline records that contain only the legacy
single screenshot SHALL remain readable, with that screenshot treated as the
post-action image.
#### Scenario: An action succeeds
- **WHEN** the Runtime executes an action for a task
- **THEN** its Timeline record includes distinct before and after screenshots,
action detail, and execution result
#### Scenario: An action fails
- **WHEN** the Runtime executor exhausts its retries for an action
- **THEN** the Timeline record still includes any captured screenshots and the
failure result before the task is marked failed
#### Scenario: A legacy Timeline record is read
- **WHEN** a Timeline record has only the prior `screenshot_path` field
- **THEN** the Runtime exposes it as the post-action screenshot without
failing to render the record
### Requirement: Runtime task evidence retains available OCR observations
The shared Runtime SHALL persist raw OCR observations associated with the scene
used to plan an action when available, without adding duplicate OCR data to the
LLM-facing normalized Scene payload. The Host Agent task-detail UI SHALL render
available OCR text, confidence, and bounds, and SHALL render normally when no
OCR result exists.
#### Scenario: OCR found text while planning an action
- **WHEN** perception produced one or more OCR observations for the action's
planning scene
- **THEN** the corresponding Timeline record includes those observations and
the Host Agent task-detail page displays them
#### Scenario: OCR was unavailable or found no text
- **WHEN** perception yields no OCR observations
- **THEN** the Runtime records the action evidence and the Host Agent task
detail renders without an OCR result list
### Requirement: Runtime task evidence retains UI-tree inspection results
The Runtime SHALL retain a UI-tree inspection result when a step invokes the
existing `get_ui_tree` or `ui_tree` tool and the result contains normalized
nodes. The Host Agent task-detail UI SHALL render those nodes in a structured,
collapsible view while retaining the recorded JSON result. The Runtime SHALL
NOT change the tool response contract or duplicate the result in a separate
persistence field.
#### Scenario: UI-tree inspection succeeds
- **WHEN** a task step uses `get_ui_tree` or `ui_tree` and returns one or
more normalized nodes
- **THEN** the Host Agent task-detail page displays each node's type, visible
text or identifier, bounds, and available confidence
#### Scenario: A non-UI-tree step is displayed
- **WHEN** a task step did not invoke a UI-tree tool
- **THEN** the Host Agent task-detail page does not render an empty UI-tree
section
@@ -0,0 +1,86 @@
## 1. Host Agent local task storage (D1, D2)
- [x] 1.1 Add `HOST_AGENT_TASK_PROGRESS_DB_PATH` (and matching artifact directory config) to `HostAgentConfig`/`load_host_agent_config()`, with Host-Agent-specific defaults for durable execution history.
- [x] 1.2 In `HostAgentApplication`'s builder (`app.py:218-223`), construct a `TaskMetadataStore`/`Timeline` from that config and pass them into `create_execution_factories(..., metadata_store=..., timeline=...)`.
- [x] 1.3 Add delete/prune methods to `storage/task_metadata.py::TaskMetadataStore` and `storage/timeline.py::Timeline` (remove a task's row, timeline records, and screenshot artifacts).
- [x] 1.4 Implement a bounded retention pass in the Host Agent (default: keep the newer of "last 50 tasks" or "7 days", whichever keeps fewer rows) that runs after each assignment finishes, calling the new prune methods.
- [x] 1.5 Add unit tests: step transitions persist during execution; a task's history is queryable after the assignment completes and the in-memory `Task` is discarded; retention prunes tasks beyond the configured threshold; Host Agent task metadata and artifacts use their configured local paths.
## 2. Host Agent reports progress during lease renewal (D4)
- [x] 2.1 Add `TaskProgressModel` (`step_index: int`, `step_status`, `summary: str` with a bounded max length) to `packages/cloud-platform/cloud/internal_api/models.py`, and an optional `progress: TaskProgressModel | None` field on `LeaseRenewalRequest`.
- [x] 2.2 Add a thread-safe "latest progress" holder written by the Host Agent's execution thread (hook into `TaskRunner`/`AssignmentExecutor` per-step completion) and read by `ActiveAssignmentRunner._renew_while_running` (`lease.py`) just before each renewal call.
- [x] 2.3 Update `HostAgentClient.renew()` to include the current progress snapshot (if any) in the renewal request.
- [x] 2.4 Add unit tests: renewal includes progress once a step has completed; renewal omits progress before any step completes; an oversized summary is truncated/rejected client-side before sending.
## 3. Cloud Control Plane persists latest progress (D5)
- [x] 3.1 Add progress columns (`progress_step_index`, `progress_step_status`, `progress_summary`, `progress_updated_at`, all nullable) via a new Alembic migration (`0008_task_progress_columns`) with a down-revision that drops them.
- [x] 3.2 Add the equivalent columns/handling to the SQLite schema path.
- [x] 3.3 Extend `renew_lease()` in both `repository.py` and `sql_repository.py` to accept and overwrite the optional progress fields under the same row lock already taken for the lease-expiry update.
- [x] 3.4 Update `renew_assignment` in `internal_api/api.py` to pass the optional `payload.progress` fields through to `renew_lease()`, rejecting/truncating an oversized `summary` without failing the underlying renewal.
- [x] 3.5 Ensure progress fields are treated as stale/ignored once `record_task_result` records a terminal outcome for that task/attempt.
- [x] 3.6 Add/extend repository contract tests (both SQLite and PostgreSQL where the existing test setup covers it) so the two backends stay in parity for the new columns.
## 4. Cloud Console displays live progress (D6)
- [x] 4.1 Extend the Cloud API's existing task list/detail response model with the optional latest-progress fields from section 3.
- [x] 4.2 Extend `cloud-console/`'s task list/detail view to render step index, status, and summary for a task with an active in-progress assignment, refreshed on its existing polling interval; show nothing when no progress is present or the task is terminal.
- [x] 4.3 Add Vitest coverage for the new progress rendering (present, absent, and terminal-task-hides-stale-progress cases).
## 5. Host Agent local console task pages (D3)
- [x] 5.1 Extend `AgentStatusTracker`/`/api/status` (or a small addition alongside it) to surface the current step index/status/summary for the in-progress assignment, sourced from the same progress holder built in section 2, and update the dashboard's "Current assignment" rendering to show it.
- [x] 5.2 Add authenticated, read-only task list and task detail/timeline routes to `host_agent/web/app.py`, querying the Host-Agent-local `TaskMetadataStore`/`Timeline` and rendering server-side Jinja HTML consistent with the existing console, including inlined screenshots on the detail/timeline page.
- [x] 5.3 Gate the new routes behind the existing Host Agent console session/CSRF protection; verify no new CORS configuration is introduced.
- [x] 5.4 Add tests: unauthenticated requests to the new routes are rejected the same way as other console routes; task list/detail/timeline pages render expected data including screenshots for a completed task.
## 6. Documentation and verification
- [x] 6.1 Update `docs/MACOS_IPHONE_SETUP.md` and/or `docs/CLOUD_DEPLOYMENT.md` with the new Host Agent config vars (`HOST_AGENT_TASK_PROGRESS_DB_PATH` and retention settings) and a short note on where to view live/historical task progress in each console.
- [x] 6.2 Run `uv run --all-packages pytest -m "not integration"` and targeted Cloud API / Host Agent test suites; run the Cloud Console Vitest suite for its touched frontend.
- [x] 6.3 Run Ruff check/format and `compileall` across touched packages.
- [x] 6.4 Run `openspec validate --strict` for this change.
- [x] 6.5 Manual verification (requires a real Host Agent + Appium/device setup per `docs/MACOS_IPHONE_SETUP.md`): run a real task end-to-end and confirm step progress appears live in the Host Agent console and Cloud Console, and that full step history with screenshots is browsable afterward in the Host Agent console.
## 7. Real per-step LLM prompt/response recorded locally (D9)
- [x] 7.1 Extend `ToolCallDecision` (`runtime/tool_calling_client.py:23-27`) with the actual prompt content given to that call (or return it via a small side-channel from `AIPlanner.plan()`, not by changing the `Planner.plan()` return type used by other planners).
- [x] 7.2 Update `TaskRunner._append_timeline()` (`runtime/task.py:296-319`) to pass the real per-step prompt and the model's resulting decision into `Timeline.append()`, instead of `task.goal`.
- [x] 7.3 Update `storage/timeline.py`'s `Timeline`/`TimelineRecord` field(s) so the persisted meaning is unambiguously "the prompt actually sent to the LLM for this step" (rename or add a field; keep backward-compatible read of any already-persisted rows if a Runtime dev DB might already have old-shaped rows).
- [x] 7.4 Update the Host Agent console task pages to show the corrected field without requiring a separate Runtime renderer.
- [x] 7.5 Add unit tests: a persisted step's prompt matches what `AIPlanner.plan()` actually sent for that step (not the task goal), across at least one multi-step task.
## 8. Cloud persists full per-step LLM decision history via the existing cloud-planner-proxy endpoint (D7, D8)
- [x] 8.1 Add a new `planner_decision_log` table (`id`, `host_id`, `task_id`, `attempt`, `step_index`, `system_prompt`, `user_prompt`, `tool_name`, `arguments_json`, `created_at`) via a new Alembic migration, plus the equivalent SQLite schema/table, on both `repository.py` and `sql_repository.py`.
- [x] 8.2 Add a repository method (e.g. `record_planner_decision(...)`) on both backends that inserts one row, assigning `step_index` as the next value scoped to `(task_id, attempt)`.
- [x] 8.3 Extend `decide_planner_call` (`packages/cloud-platform/cloud/internal_api/api.py:367-488`) to call the new repository method with the resolved decision, immediately after `settle_host_token_reservation`, on the success path only (never on the `ToolCallUnavailable`/502 path).
- [x] 8.4 Add a retention/prune job for `planner_decision_log` (default: prune a task's rows once it has been terminal for longer than a configurable window), mirroring section 1's Host Agent retention approach.
- [x] 8.5 Add/extend repository contract tests so SQLite and PostgreSQL stay in parity for the new table and prune job.
- [x] 8.6 Add unit tests: a successful decision is persisted with the correct prompt/tool_name/arguments and an incrementing step_index; a failed decision persists nothing; screenshot bytes are never written to the log even when the request included one; retention prunes rows for long-terminal tasks.
## 9. Cloud Console browses full LLM interaction history (D7/D8 UI)
- [x] 9.1 Add a Cloud API query endpoint (or extend an existing task-detail endpoint) to list `planner_decision_log` rows for a task in step order.
- [x] 9.2 Add a `cloud-console/` view rendering a task's full LLM interaction history (prompt + resulting decision per step).
- [x] 9.3 When a task's host used the `direct` transport (no persisted decisions and the host's configured transport is known to be `direct`), show an explicit "not reported by this host's transport" state rather than an empty list.
- [x] 9.4 Add Vitest coverage for populated history, empty-but-cloud-transport (task hasn't produced any decisions yet), and direct-transport-hidden cases.
## 10. Shared Runtime per-step evidence (D4)
- [x] 10.1 Extend the shared Scene/Timeline/ArtifactStore model to retain raw OCR observations and separate before/after screenshots while preserving compatibility with existing single-screenshot records.
- [x] 10.2 Capture before and after screenshots around every TaskRunner executor call, then persist the action, result, and available OCR observations in the same timeline record.
- [x] 10.3 Preserve the shared Timeline representation needed to render before/after screenshots, operation details, and available OCR results without changing Runtime execution contracts.
- [x] 10.4 Preserve persisted normalized UI-tree tool results without changing the tool contract or duplicating stored data.
- [x] 10.5 Add focused Timeline, TaskRunner, perception, and UI-tree regression coverage; run the relevant format, lint, test, and strict OpenSpec validation commands.
## 11. Host Agent execution authority and Runtime service retirement (D1-D9)
- [x] 11.1 Make `TaskRunner.run()` create a task metadata row idempotently before status updates; add optional generic source task/attempt fields to task metadata and record Cloud assignment correlation in the Host adapter.
- [x] 11.2 Add regression coverage for Host-dispatched goal execution, workflow-owned TaskRunner execution, retry-safe metadata creation, and Cloud task/attempt correlation.
- [x] 11.3 Extend Host Agent `/tasks` and task-detail rendering so the actual execution list and detail page show correlation, before/after screenshots, operation detail, OCR observations, and structured normalized UI-tree results, including legacy screenshot compatibility.
- [x] 11.4 Remove the standalone Runtime REST service/UI, its package data and dedicated tests, and remove Runtime supervision from Host Agent configuration/supervision with an actionable legacy-config error.
- [x] 11.5 Update operator documentation and OpenSpec artifacts to direct execution inspection to Host Agent `:8765/tasks` and remove port `8000` instructions.
- [x] 11.6 Run focused Host Agent and shared Runtime tests, workspace non-integration tests, Ruff, compileall, and strict OpenSpec validation.
- [x] 11.7 Manual verification (requires a real Host Agent + Appium/device setup): submit or dispatch a task, then confirm the Host Agent console is the only local execution-history UI and shows the complete retained evidence.