Files
agentic-mobile-control/openspec/changes/task-execution-progress-visibility/tasks.md
T
2026-07-15 11:46:27 +08:00

11 KiB

1. Host Agent local task storage (D1, D2)

  • 1.1 Add HOST_AGENT_TASK_PROGRESS_DB_PATH (and matching artifact directory config) to HostAgentConfig/load_host_agent_config(), with Host-Agent-specific defaults for durable execution history.
  • 1.2 In HostAgentApplication's builder (app.py:218-223), construct a TaskMetadataStore/Timeline from that config and pass them into create_execution_factories(..., metadata_store=..., timeline=...).
  • 1.3 Add delete/prune methods to storage/task_metadata.py::TaskMetadataStore and storage/timeline.py::Timeline (remove a task's row, timeline records, and screenshot artifacts).
  • 1.4 Implement a bounded retention pass in the Host Agent (default: keep the newer of "last 50 tasks" or "7 days", whichever keeps fewer rows) that runs after each assignment finishes, calling the new prune methods.
  • 1.5 Add unit tests: step transitions persist during execution; a task's history is queryable after the assignment completes and the in-memory Task is discarded; retention prunes tasks beyond the configured threshold; Host Agent task metadata and artifacts use their configured local paths.

2. Host Agent reports progress during lease renewal (D4)

  • 2.1 Add TaskProgressModel (step_index: int, step_status, summary: str with a bounded max length) to packages/cloud-platform/cloud/internal_api/models.py, and an optional progress: TaskProgressModel | None field on LeaseRenewalRequest.
  • 2.2 Add a thread-safe "latest progress" holder written by the Host Agent's execution thread (hook into TaskRunner/AssignmentExecutor per-step completion) and read by ActiveAssignmentRunner._renew_while_running (lease.py) just before each renewal call.
  • 2.3 Update HostAgentClient.renew() to include the current progress snapshot (if any) in the renewal request.
  • 2.4 Add unit tests: renewal includes progress once a step has completed; renewal omits progress before any step completes; an oversized summary is truncated/rejected client-side before sending.

3. Cloud Control Plane persists latest progress (D5)

  • 3.1 Add progress columns (progress_step_index, progress_step_status, progress_summary, progress_updated_at, all nullable) via a new Alembic migration (0008_task_progress_columns) with a down-revision that drops them.
  • 3.2 Add the equivalent columns/handling to the SQLite schema path.
  • 3.3 Extend renew_lease() in both repository.py and sql_repository.py to accept and overwrite the optional progress fields under the same row lock already taken for the lease-expiry update.
  • 3.4 Update renew_assignment in internal_api/api.py to pass the optional payload.progress fields through to renew_lease(), rejecting/truncating an oversized summary without failing the underlying renewal.
  • 3.5 Ensure progress fields are treated as stale/ignored once record_task_result records a terminal outcome for that task/attempt.
  • 3.6 Add/extend repository contract tests (both SQLite and PostgreSQL where the existing test setup covers it) so the two backends stay in parity for the new columns.

4. Cloud Console displays live progress (D6)

  • 4.1 Extend the Cloud API's existing task list/detail response model with the optional latest-progress fields from section 3.
  • 4.2 Extend cloud-console/'s task list/detail view to render step index, status, and summary for a task with an active in-progress assignment, refreshed on its existing polling interval; show nothing when no progress is present or the task is terminal.
  • 4.3 Add Vitest coverage for the new progress rendering (present, absent, and terminal-task-hides-stale-progress cases).

5. Host Agent local console task pages (D3)

  • 5.1 Extend AgentStatusTracker//api/status (or a small addition alongside it) to surface the current step index/status/summary for the in-progress assignment, sourced from the same progress holder built in section 2, and update the dashboard's "Current assignment" rendering to show it.
  • 5.2 Add authenticated, read-only task list and task detail/timeline routes to host_agent/web/app.py, querying the Host-Agent-local TaskMetadataStore/Timeline and rendering server-side Jinja HTML consistent with the existing console, including inlined screenshots on the detail/timeline page.
  • 5.3 Gate the new routes behind the existing Host Agent console session/CSRF protection; verify no new CORS configuration is introduced.
  • 5.4 Add tests: unauthenticated requests to the new routes are rejected the same way as other console routes; task list/detail/timeline pages render expected data including screenshots for a completed task.

6. Documentation and verification

  • 6.1 Update docs/MACOS_IPHONE_SETUP.md and/or docs/CLOUD_DEPLOYMENT.md with the new Host Agent config vars (HOST_AGENT_TASK_PROGRESS_DB_PATH and retention settings) and a short note on where to view live/historical task progress in each console.
  • 6.2 Run uv run --all-packages pytest -m "not integration" and targeted Cloud API / Host Agent test suites; run the Cloud Console Vitest suite for its touched frontend.
  • 6.3 Run Ruff check/format and compileall across touched packages.
  • 6.4 Run openspec validate --strict for this change.
  • 6.5 Manual verification (requires a real Host Agent + Appium/device setup per docs/MACOS_IPHONE_SETUP.md): run a real task end-to-end and confirm step progress appears live in the Host Agent console and Cloud Console, and that full step history with screenshots is browsable afterward in the Host Agent console.

7. Real per-step LLM prompt/response recorded locally (D9)

  • 7.1 Extend ToolCallDecision (runtime/tool_calling_client.py:23-27) with the actual prompt content given to that call (or return it via a small side-channel from AIPlanner.plan(), not by changing the Planner.plan() return type used by other planners).
  • 7.2 Update TaskRunner._append_timeline() (runtime/task.py:296-319) to pass the real per-step prompt and the model's resulting decision into Timeline.append(), instead of task.goal.
  • 7.3 Update storage/timeline.py's Timeline/TimelineRecord field(s) so the persisted meaning is unambiguously "the prompt actually sent to the LLM for this step" (rename or add a field; keep backward-compatible read of any already-persisted rows if a Runtime dev DB might already have old-shaped rows).
  • 7.4 Update the Host Agent console task pages to show the corrected field without requiring a separate Runtime renderer.
  • 7.5 Add unit tests: a persisted step's prompt matches what AIPlanner.plan() actually sent for that step (not the task goal), across at least one multi-step task.

8. Cloud persists full per-step LLM decision history via the existing cloud-planner-proxy endpoint (D7, D8)

  • 8.1 Add a new planner_decision_log table (id, host_id, task_id, attempt, step_index, system_prompt, user_prompt, tool_name, arguments_json, created_at) via a new Alembic migration, plus the equivalent SQLite schema/table, on both repository.py and sql_repository.py.
  • 8.2 Add a repository method (e.g. record_planner_decision(...)) on both backends that inserts one row, assigning step_index as the next value scoped to (task_id, attempt).
  • 8.3 Extend decide_planner_call (packages/cloud-platform/cloud/internal_api/api.py:367-488) to call the new repository method with the resolved decision, immediately after settle_host_token_reservation, on the success path only (never on the ToolCallUnavailable/502 path).
  • 8.4 Add a retention/prune job for planner_decision_log (default: prune a task's rows once it has been terminal for longer than a configurable window), mirroring section 1's Host Agent retention approach.
  • 8.5 Add/extend repository contract tests so SQLite and PostgreSQL stay in parity for the new table and prune job.
  • 8.6 Add unit tests: a successful decision is persisted with the correct prompt/tool_name/arguments and an incrementing step_index; a failed decision persists nothing; screenshot bytes are never written to the log even when the request included one; retention prunes rows for long-terminal tasks.

9. Cloud Console browses full LLM interaction history (D7/D8 UI)

  • 9.1 Add a Cloud API query endpoint (or extend an existing task-detail endpoint) to list planner_decision_log rows for a task in step order.
  • 9.2 Add a cloud-console/ view rendering a task's full LLM interaction history (prompt + resulting decision per step).
  • 9.3 When a task's host used the direct transport (no persisted decisions and the host's configured transport is known to be direct), show an explicit "not reported by this host's transport" state rather than an empty list.
  • 9.4 Add Vitest coverage for populated history, empty-but-cloud-transport (task hasn't produced any decisions yet), and direct-transport-hidden cases.

10. Shared Runtime per-step evidence (D4)

  • 10.1 Extend the shared Scene/Timeline/ArtifactStore model to retain raw OCR observations and separate before/after screenshots while preserving compatibility with existing single-screenshot records.
  • 10.2 Capture before and after screenshots around every TaskRunner executor call, then persist the action, result, and available OCR observations in the same timeline record.
  • 10.3 Preserve the shared Timeline representation needed to render before/after screenshots, operation details, and available OCR results without changing Runtime execution contracts.
  • 10.4 Preserve persisted normalized UI-tree tool results without changing the tool contract or duplicating stored data.
  • 10.5 Add focused Timeline, TaskRunner, perception, and UI-tree regression coverage; run the relevant format, lint, test, and strict OpenSpec validation commands.

11. Host Agent execution authority and Runtime service retirement (D1-D9)

  • 11.1 Make TaskRunner.run() create a task metadata row idempotently before status updates; add optional generic source task/attempt fields to task metadata and record Cloud assignment correlation in the Host adapter.
  • 11.2 Add regression coverage for Host-dispatched goal execution, workflow-owned TaskRunner execution, retry-safe metadata creation, and Cloud task/attempt correlation.
  • 11.3 Extend Host Agent /tasks and task-detail rendering so the actual execution list and detail page show correlation, before/after screenshots, operation detail, OCR observations, and structured normalized UI-tree results, including legacy screenshot compatibility.
  • 11.4 Remove the standalone Runtime REST service/UI, its package data and dedicated tests, and remove Runtime supervision from Host Agent configuration/supervision with an actionable legacy-config error.
  • 11.5 Update operator documentation and OpenSpec artifacts to direct execution inspection to Host Agent :8765/tasks and remove port 8000 instructions.
  • 11.6 Run focused Host Agent and shared Runtime tests, workspace non-integration tests, Ruff, compileall, and strict OpenSpec validation.
  • 11.7 Manual verification (requires a real Host Agent + Appium/device setup): submit or dispatch a task, then confirm the Host Agent console is the only local execution-history UI and shows the complete retained evidence.