177 lines
8.4 KiB
Markdown
177 lines
8.4 KiB
Markdown
## Context
|
|
|
|
Host Agent executes Cloud assignments in-process through the shared
|
|
`runtime.TaskRunner`. Its `HostAgentApplication` already constructs a
|
|
Host-local `TaskMetadataStore` and `Timeline` and injects them into the
|
|
runner factory. The missing link is task creation: `AssignmentExecutor`
|
|
constructs a `Task`, calls `runner.run(task)`, and `TaskRunner` only
|
|
issues updates. SQLite therefore receives updates for a row that does not
|
|
exist, so the Host Agent `/tasks` UI is empty.
|
|
|
|
The separate `api.rest` process was never on this execution path. It owned a
|
|
different database and artifact root, so its REST endpoints and UI could only
|
|
show tasks submitted directly to that unrelated process. Running it alongside
|
|
a Host Agent created two competing operator entry points without transferring
|
|
any history between them.
|
|
|
|
## Goals / Non-Goals
|
|
|
|
**Goals:**
|
|
|
|
- Make durable task-row creation an invariant of `TaskRunner.run()` whenever
|
|
a metadata store is configured.
|
|
- Correlate a Host-local goal execution with the Cloud task ID and attempt that
|
|
caused it, without making shared Runtime or storage packages import Cloud
|
|
models.
|
|
- Make the authenticated Host Agent console at port `8765` the authoritative
|
|
web view for actual task execution: task list, detail, live status, before
|
|
screenshot, operation, after screenshot, OCR observations, and normalized
|
|
UI-tree results.
|
|
- Preserve the shared Runtime Timeline as the evidence model and keep its
|
|
before/after screenshot, OCR, and UI-tree capture behavior.
|
|
- Retire the standalone Runtime REST service/UI and its Host Agent supervisor
|
|
configuration while retaining Runtime, storage, MCP, and skill-sync library
|
|
modules.
|
|
- Retain the existing Cloud latest-progress and Cloud-proxy planner-decision
|
|
history behavior. Cloud remains a fleet-level view and never stores
|
|
screenshots.
|
|
|
|
**Non-Goals:**
|
|
|
|
- No SSE/WebSocket push; Host and Cloud consoles keep their existing polling.
|
|
- No duplicate full evidence upload to Cloud. Screenshots and Timeline
|
|
artifacts remain Host-local.
|
|
- No new Host/Cloud dependencies in `runtime/`, `storage/`, `driver/`, or
|
|
`device/`.
|
|
- No replacement general-purpose device-control REST API. Operators use Host
|
|
Agent device management and task pages for the managed execution workflow.
|
|
|
|
## Decisions
|
|
|
|
### D1: TaskRunner owns idempotent task metadata creation
|
|
|
|
`TaskRunner.run()` will call `TaskMetadataStore.create_task(task)` before
|
|
the first running-state update. `create_task()` will use idempotent insert
|
|
semantics so callers that already created a direct Runtime task remain
|
|
compatible and workflow-created tasks are captured automatically.
|
|
|
|
This places the invariant at the shared execution boundary rather than relying
|
|
on every caller to remember an out-of-band persistence call. It fixes goal
|
|
assignments and prevents the same failure for `WorkflowRunner` planned-goal
|
|
steps.
|
|
|
|
### D2: Host assignment correlation stays in the Host adapter
|
|
|
|
Before executing a Cloud goal assignment, `AssignmentExecutor` creates the
|
|
local task record with optional generic `source_task_id` and
|
|
`source_attempt` metadata. Shared storage uses generic names and does not
|
|
import Cloud types. The Host Agent task list/detail renders those fields as the
|
|
Cloud task ID and attempt.
|
|
|
|
The local `Task.id` remains generated by the Runtime. Reusing the Cloud task
|
|
ID as an artifact directory name would make retries overwrite each other and
|
|
would admit unsafe path characters from an external identifier.
|
|
|
|
### D3: Host Agent console is the authoritative evidence UI
|
|
|
|
The Host Agent already owns the device manager, assignment executor,
|
|
metadata store, Timeline, local account, session, and CSRF boundary. Its
|
|
same-origin `/tasks` and `/tasks/{task_id}` pages therefore render the
|
|
shared Timeline directly. The task list is named for Host executions rather
|
|
than "Local Runtime tasks", and submission feedback tells the operator that
|
|
the submitted Cloud task appears there when this Host begins execution.
|
|
|
|
No browser needs to point a separate frontend at the Host Agent. This keeps
|
|
the conservative local-account/session model and does not add CORS.
|
|
|
|
### D4: Complete per-step evidence is rendered by the Host Agent
|
|
|
|
Timeline records retain distinct pre-action and post-action screenshot paths,
|
|
the action description and arguments, the execution result, raw OCR
|
|
observations, and the existing normalized UI-tree result. The Host Agent page
|
|
creates data URIs only for available local artifacts and supports legacy
|
|
records where the single `screenshot_path` is the post-action image.
|
|
|
|
OCR is rendered when present. UI-tree output is rendered only for
|
|
`get_ui_tree` and `ui_tree` records that contain normalized nodes; it uses a
|
|
collapsible structured view while retaining the JSON result. No tool contract
|
|
or duplicate persistence field is introduced.
|
|
|
|
### D5: Host-local retention remains bounded
|
|
|
|
The existing Host retention pass continues to remove metadata rows, Timeline
|
|
records, and artifacts according to the configured count and age thresholds.
|
|
The new task creation invariant must use that same store so it cannot create
|
|
an unbounded second history source.
|
|
|
|
### D6: Cloud progress remains a latest snapshot on lease renewal
|
|
|
|
The Host execution thread writes a bounded latest-progress holder. The lease
|
|
renewal path optionally carries its step index, status, and summary; Cloud
|
|
stores only the latest snapshot for an active assignment and stops exposing it
|
|
after terminal completion. No screenshot or scene payload enters this
|
|
protocol.
|
|
|
|
### D7: Cloud-proxy planner decisions remain durable Cloud history
|
|
|
|
For `AI_PLANNER_TRANSPORT=cloud`, Cloud's existing planner-decision endpoint
|
|
persists successful system/user prompts, tool calls, arguments, and a
|
|
task/attempt-scoped step index. The direct transport intentionally produces no
|
|
such Cloud history. This log has bounded terminal-task retention and never
|
|
persists request screenshot bytes.
|
|
|
|
### D8: Retire the standalone Runtime REST service and UI
|
|
|
|
Remove `api/rest.py`, `api/console.py`, `api/console_web.py`, their
|
|
templates/static assets, their package-data declarations, and their
|
|
service/UI tests. Preserve `api/mcp.py`, `api/errors.py`, skill-sync, and
|
|
skill-catalog modules because they are independent library integrations.
|
|
|
|
Remove `HOST_AGENT_RUNTIME_SUPERVISED`,
|
|
`HOST_AGENT_RUNTIME_HOST`, and `HOST_AGENT_RUNTIME_PORT`. The optional
|
|
dependency supervisor continues to support Appium only. Configuration with a
|
|
removed Runtime-supervision variable fails with an actionable migration error
|
|
instead of silently doing nothing.
|
|
|
|
### D9: Documentation points operators to Host Agent
|
|
|
|
Operator documentation no longer instructs users to start `uvicorn
|
|
api.rest:create_app` or browse port `8000`. It identifies the Host Agent
|
|
console at `http://127.0.0.1:8765/tasks` as the execution-history authority,
|
|
explains its local authentication, and documents that the Cloud Console is a
|
|
fleet/progress and Cloud-proxy LLM-history surface rather than a screenshot
|
|
store.
|
|
|
|
## Risks / Trade-offs
|
|
|
|
- Per-step metadata writes add small SQLite I/O. This is the same local store
|
|
already selected for Host history, and bounded retention limits growth.
|
|
- A Host-local evidence record is only available while retained on that Host.
|
|
This is intentional: it reflects the actual device execution and avoids
|
|
sending screenshots to Cloud.
|
|
- Removing the unauthenticated Runtime REST service is a breaking operator
|
|
change. Clear configuration errors and documentation avoid a silent
|
|
fallback to a nonexistent inspection surface.
|
|
- Full Cloud-proxy prompts can contain visible screen text. This is the
|
|
previously accepted Cloud troubleshooting trade-off; screenshot bytes remain
|
|
excluded.
|
|
|
|
## Migration Plan
|
|
|
|
1. Upgrade the Host Agent code. Existing task databases gain nullable source
|
|
correlation columns on startup; legacy Timeline records remain readable.
|
|
2. Remove any `HOST_AGENT_RUNTIME_*` environment variables and stop any
|
|
`api.rest` process. Start or browse only the Host Agent console for local
|
|
execution evidence.
|
|
3. Confirm a completed Host assignment appears at `/tasks` with its Cloud
|
|
task/attempt correlation and complete Timeline evidence.
|
|
4. Roll back only by restoring the prior release. The retired REST/UI routes
|
|
are deliberately not kept as a compatibility alias because their storage
|
|
was not connected to Host execution.
|
|
|
|
## Open Questions
|
|
|
|
- Manual verification still requires a real Host Agent, Appium, and device.
|
|
Automated coverage verifies persistence, correlation, rendering, and
|
|
service removal; real hardware validates screenshots and OCR availability.
|