feat: checkpoint device agent runtime milestones
This commit is contained in:
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-07-06
|
||||
@@ -0,0 +1,46 @@
|
||||
## Context
|
||||
|
||||
The backend (`api/rest.py`, `api/mcp.py`) currently exposes only device-control and task-execution primitives (`/devices`, `/agent/task`, `/task/{id}`), all consumed by an LLM/MCP client. Two pieces of already-implemented backend functionality are not exposed over HTTP at all: `TaskMetadataStore.list_tasks()` (storage/task_metadata.py) and `Timeline.read(task_id)` / `ArtifactStore.read_steps(task_id)` (storage/timeline.py, storage/artifact_store.py). Device registration (`DeviceManager.register_device`, core/device_manager.py) requires a Python `driver_factory` callable and is purely in-memory — there is no config file or persistence, so every device must be (re-)registered in code on each process start. There is no operator-facing UI at all today.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Expose existing device/task/timeline state over a read-only console REST API.
|
||||
- Let an operator register/remove devices and adjust simple runtime parameters (`max_steps`) through a REST API, with changes surviving process restarts.
|
||||
- Ship an independent Vue 3 SPA that consumes these APIs for a status dashboard + configuration screens.
|
||||
|
||||
**Non-Goals:**
|
||||
- Authentication/authorization for console endpoints (assumed trusted network for now).
|
||||
- Editing planner/agent logic, prompts, or tool registration from the UI.
|
||||
- Concurrent multi-operator conflict resolution for config edits.
|
||||
- Any change to the existing MCP tool surface or `/agent/task` execution semantics.
|
||||
- Supporting driver types beyond `wda` in the config UI (Android etc. remain out of scope, consistent with `apex-agent-mvp`).
|
||||
|
||||
## Decisions
|
||||
|
||||
- **New `/console` router, shared state.** Add `api/console.py` exposing an `APIRouter` mounted under `/console` inside the existing `create_app()` in `api/rest.py`, reusing the same `device_manager`, `metadata_store`, and `task_runner` instances already constructed there. Rejected alternative: a second standalone FastAPI app — would duplicate state wiring and require its own process/port for no benefit.
|
||||
- **New persisted device-config store, additive to `DeviceManager`.** Add `storage/device_config.py` (SQLite, same pattern as `TaskMetadataStore`) storing `device_id, name, driver_type, connection_info(json)`. On `create_app()` startup, read all rows and call `device_manager.register_device(...)` with a `driver_factory` built via a small `driver_type -> factory` registry (today: `{"wda": WDADriverConfig-based factory}`). `DeviceManager` itself is not modified — it stays a pure in-memory runtime registry; the config store is the source of truth for "which devices should exist," the manager is the source of truth for "current connection state." Rejected alternative: teaching `DeviceManager` to persist itself — would blur its existing single responsibility and touch a capability (`device-management`) this change intends to leave unmodified.
|
||||
- **Runtime parameters persisted in the same store.** Add a small `settings(key, value)` key-value table to the same SQLite file for `max_steps` (and future simple scalars), read at startup to construct `TaskRunnerConfig`, and updated in-place on the live `TaskRunner.config` object when changed via `PUT /console/config` — so changes apply immediately without a restart and still survive one. Rejected alternative: in-memory-only (rejected: contradicts "simple configuration" persisting across restarts, which is the point of the config API); a separate config file (rejected: adds a second persistence mechanism for no benefit over reusing the SQLite file already needed for device configs).
|
||||
- **Screenshots returned as base64 JSON, not static files.** `GET /console/tasks/{id}/timeline` reads screenshots via `ArtifactStore`/`Timeline` and inlines them as base64, mirroring the existing `/devices/{id}/screenshot` convention. Rejected alternative: mounting `tasks/history` as a static file directory — simpler but opens a directory-listing/path-traversal surface for no strong benefit at this stage; base64 keeps the same trust boundary as the rest of the API.
|
||||
- **Driver type allow-list validated at registration time.** `POST /console/devices` rejects any `driver_type` not in the small supported set (`{"wda"}`) with `400`, rather than silently accepting arbitrary strings that would fail later at `connect()` time.
|
||||
- **Independent Vue 3 SPA, no backend template rendering.** Per explicit choice, the console frontend is its own project (own build/deploy, e.g. Vite), calling the JSON APIs over HTTP. `create_app()` enables permissive CORS for local development only (see Open Questions for production hosting).
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [No auth on console endpoints] → Mitigation: document as trusted-network-only for this change; flag auth as explicit follow-up before any non-trusted deployment.
|
||||
- [`max_steps` change applied to a running `TaskRunner` while a task is mid-execution] → Mitigation: change only affects the loop bound checked at the top of each iteration in `TaskRunner.run`; an in-flight task simply picks up the new bound on its next iteration, no partial-state corruption possible.
|
||||
- [SQLite device-config store and `TaskMetadataStore` are two separate files/schemas] → Mitigation: acceptable for this change's scope (mirrors existing pattern); revisit consolidation only if a third store is added later.
|
||||
- [Base64 screenshot inlining increases timeline payload size for long tasks] → Mitigation: acceptable for MVP console scope (tasks capped at `max_steps`, typically ≤ 20 steps); revisit pagination/lazy-loading if task length grows.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. Add `storage/device_config.py` (device configs + settings key-value table).
|
||||
2. Add `api/console.py` with status endpoints (read-only) first; mount into `create_app()`.
|
||||
3. Add config endpoints (device register/unregister, runtime param get/update) to `api/console.py`; wire startup reload of persisted devices/settings into `create_app()`.
|
||||
4. Scaffold the independent Vue 3 SPA project consuming the above.
|
||||
5. Rollback: the change is purely additive (new router, new tables, new frontend project) — rollback is removing the router mount and the new frontend project; no migration of existing `tasks.sqlite3` or `tasks/history` data is required or affected.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Where should the Vue 3 SPA project live — a `console/` subfolder in this repo, or a separate sibling repository? Defaulting to an in-repo `console/` subfolder for now (simplest to keep in sync with the backend during early iteration); revisit if it needs independent versioning/deploy cadence.
|
||||
- Should console endpoints eventually sit behind the same auth mechanism as a future MCP-facing auth layer, or get their own? Deferred until authentication is actually scoped.
|
||||
@@ -0,0 +1,28 @@
|
||||
## Why
|
||||
|
||||
Apex Agent currently exposes device control and task execution only through the REST/MCP tool surface (`api/rest.py`, `api/mcp.py`) — there is no way for a human operator to see, at a glance, which devices are connected, what a running task is doing, or to register a new device/tweak a runtime setting without editing code and restarting the process. As the platform moves past a single-developer MVP, an operator needs a lightweight status/config surface: view device and task state (including step-by-step timeline/screenshot replay for debugging), and perform simple configuration (register/remove devices, adjust runtime parameters) through a web UI instead of the Python API directly.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add a **console status API**: `GET` endpoints to list devices with live status, list/filter tasks, fetch task detail, and fetch a task's per-step timeline (tool call, result, scene, screenshot) built on top of the existing `TaskMetadataStore.list_tasks()` and `Timeline.read()`, which are implemented but not yet exposed over HTTP.
|
||||
- Add a **console config API**: endpoints to register a new device (driver_type + connection_info, e.g. WDA `server_url`/`udid`/`wda_local_port`), unregister a device, and view/update adjustable runtime parameters (currently just `TaskRunnerConfig.max_steps`).
|
||||
- Add a **device config store** (new persistence, SQLite like `TaskMetadataStore`) so devices registered through the console survive process restarts — on startup the app reloads persisted device configs and re-registers them with `DeviceManager`, instead of devices existing only in the in-memory dict as today.
|
||||
- Add an **independent web console frontend** (separate Vue 3 SPA project with its own build/deploy) that consumes the two APIs above to render: a device status dashboard, a task list/detail view with timeline + screenshot replay, and configuration screens for device management and runtime parameters.
|
||||
- Explicitly out of scope: authentication/authorization for the console (assumed to run on a trusted network for now), editing agent/planner logic from the UI, multi-user concurrent config editing safeguards, and any change to the existing MCP tool surface or `/agent/task` execution semantics.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `console-status-api`: Read-only REST endpoints exposing device status, task list/detail, and per-task execution timeline (including screenshot retrieval) for the web console.
|
||||
- `console-config-api`: REST endpoints for device registration/deregistration (backed by a persisted device config store) and viewing/updating simple runtime parameters (e.g. task runner `max_steps`).
|
||||
- `web-console-ui`: Independent frontend SPA providing the status dashboard and configuration pages, consuming `console-status-api` and `console-config-api`.
|
||||
|
||||
### Modified Capabilities
|
||||
(none — this change is additive on top of `device-management`, `task-memory`, and `mcp-tool-server`; no existing requirements change)
|
||||
|
||||
## Impact
|
||||
|
||||
- **New code**: `api/console.py` (or similar) for the new REST routes; `storage/device_config.py` for the persisted device config store; a new top-level frontend project (e.g. `console/` or a sibling repo) for the Vue 3 SPA.
|
||||
- **Modified code**: `api/rest.py` to mount the console routes and to reload persisted device configs at `create_app()` startup; `runtime/task.py`/`TaskRunnerConfig` to allow `max_steps` to be read/updated at runtime.
|
||||
- **Dependencies**: no new backend dependencies expected (reuses FastAPI/SQLite already in `pyproject.toml`); the new frontend project brings its own Node/Vue 3 toolchain, separate from the Python package.
|
||||
- **Impact on existing behavior**: `DeviceManager` in-memory registration behavior is unchanged; the console config store only adds a reload-on-startup convenience layer on top of it. Existing `/devices`, `/agent/task`, `/task/{id}` endpoints and MCP tools are unaffected.
|
||||
@@ -0,0 +1,45 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Register a device
|
||||
The system SHALL expose `POST /console/devices` accepting `driver_type`, `connection_info`, and an optional `name`, persist the configuration, and register the device with the running `DeviceManager`. Unsupported `driver_type` values SHALL be rejected.
|
||||
|
||||
#### Scenario: Register a supported device
|
||||
- **WHEN** an operator posts `{"driver_type": "wda", "connection_info": {"server_url": "http://127.0.0.1:4723", "udid": "abc123"}}` to `POST /console/devices`
|
||||
- **THEN** the response is a `201` with the new device's id and status `idle`, the device appears in `GET /console/devices`, and its configuration is persisted
|
||||
|
||||
#### Scenario: Reject unsupported driver type
|
||||
- **WHEN** an operator posts a device with `driver_type` not in the supported set (currently `wda`)
|
||||
- **THEN** the response is a `400` and no device is registered or persisted
|
||||
|
||||
### Requirement: Unregister a device
|
||||
The system SHALL expose `DELETE /console/devices/{device_id}` that disconnects and removes the device from `DeviceManager` and deletes its persisted configuration, or returns a `404` if the device is unknown.
|
||||
|
||||
#### Scenario: Unregister a known device
|
||||
- **WHEN** an operator calls `DELETE /console/devices/{device_id}` for a registered device
|
||||
- **THEN** the response is a `204`, the device no longer appears in `GET /console/devices`, and its persisted configuration is removed
|
||||
|
||||
#### Scenario: Unregister an unknown device
|
||||
- **WHEN** an operator calls `DELETE /console/devices/{device_id}` for a device id that is not registered
|
||||
- **THEN** the response is a `404`
|
||||
|
||||
### Requirement: Persisted devices reload on startup
|
||||
The system SHALL re-register every persisted device configuration with `DeviceManager` automatically when the application starts, without requiring manual re-registration.
|
||||
|
||||
#### Scenario: Restart with previously registered devices
|
||||
- **WHEN** the application starts and one or more device configurations exist in the persisted device config store
|
||||
- **THEN** each persisted device appears in `GET /console/devices` immediately after startup, without any additional operator action
|
||||
|
||||
### Requirement: View and update runtime parameters
|
||||
The system SHALL expose `GET /console/config` returning current adjustable runtime parameters (currently the task runner's `max_steps`) and `PUT /console/config` to update them, applying the change to the running task runner immediately and persisting it across restarts.
|
||||
|
||||
#### Scenario: View current runtime parameters
|
||||
- **WHEN** an operator calls `GET /console/config`
|
||||
- **THEN** the response is a `200` including the current `max_steps` value
|
||||
|
||||
#### Scenario: Update max_steps
|
||||
- **WHEN** an operator calls `PUT /console/config` with `{"max_steps": 30}`
|
||||
- **THEN** the response is a `200` with the updated value, subsequently started tasks (and the next iteration of any in-flight task) use the new `max_steps`, and the value is persisted so it survives a restart
|
||||
|
||||
#### Scenario: Reject invalid runtime parameter value
|
||||
- **WHEN** an operator calls `PUT /console/config` with a non-positive `max_steps` (e.g. `0` or `-1`)
|
||||
- **THEN** the response is a `400` and the previous value remains in effect
|
||||
@@ -0,0 +1,49 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: List device status
|
||||
The system SHALL expose `GET /console/devices` returning every registered device's id, name, status (`idle`/`busy`/`offline`/`error`), and driver type, reusing `DeviceManager.list_devices()`.
|
||||
|
||||
#### Scenario: Devices are registered
|
||||
- **WHEN** an operator calls `GET /console/devices` while one or more devices are registered
|
||||
- **THEN** the response is a `200` with a JSON array containing one entry per device with its current `status`
|
||||
|
||||
#### Scenario: No devices registered
|
||||
- **WHEN** an operator calls `GET /console/devices` while no devices are registered
|
||||
- **THEN** the response is a `200` with an empty JSON array
|
||||
|
||||
### Requirement: List tasks
|
||||
The system SHALL expose `GET /console/tasks` returning all tasks known to `TaskMetadataStore`, most recently created first, with optional `device_id` and `status` query filters.
|
||||
|
||||
#### Scenario: List all tasks
|
||||
- **WHEN** an operator calls `GET /console/tasks` with no query parameters
|
||||
- **THEN** the response is a `200` with a JSON array of tasks ordered by `created_at` descending
|
||||
|
||||
#### Scenario: Filter by device
|
||||
- **WHEN** an operator calls `GET /console/tasks?device_id=<id>`
|
||||
- **THEN** the response contains only tasks whose `device_id` matches `<id>`
|
||||
|
||||
#### Scenario: Filter by status
|
||||
- **WHEN** an operator calls `GET /console/tasks?status=running`
|
||||
- **THEN** the response contains only tasks whose `status` equals `running`
|
||||
|
||||
### Requirement: Task detail
|
||||
The system SHALL expose `GET /console/tasks/{task_id}` returning the full task record, or a `404` if the task does not exist.
|
||||
|
||||
#### Scenario: Task exists
|
||||
- **WHEN** an operator calls `GET /console/tasks/{task_id}` for a known task id
|
||||
- **THEN** the response is a `200` with the task's goal, device_id, status, timestamps, and failure_reason (if any)
|
||||
|
||||
#### Scenario: Task does not exist
|
||||
- **WHEN** an operator calls `GET /console/tasks/{task_id}` for an unknown task id
|
||||
- **THEN** the response is a `404`
|
||||
|
||||
### Requirement: Task timeline replay
|
||||
The system SHALL expose `GET /console/tasks/{task_id}/timeline` returning the ordered list of execution steps recorded for the task, each including its scene, prompt, tool call, result, timestamp, and a base64-encoded screenshot when one was captured for that step.
|
||||
|
||||
#### Scenario: Task has recorded steps
|
||||
- **WHEN** an operator calls `GET /console/tasks/{task_id}/timeline` for a task that executed at least one step
|
||||
- **THEN** the response is a `200` with a JSON array ordered by step `index` ascending, each entry including `image_base64` when a screenshot was captured for that step
|
||||
|
||||
#### Scenario: Task has no recorded steps
|
||||
- **WHEN** an operator calls `GET /console/tasks/{task_id}/timeline` for a task with no timeline history (e.g. it failed before its first step)
|
||||
- **THEN** the response is a `200` with an empty JSON array
|
||||
@@ -0,0 +1,45 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Device status dashboard
|
||||
The web console SHALL display a list of registered devices with their current status, fetched from `console-status-api`, and SHALL show an explicit empty state when no devices are registered.
|
||||
|
||||
#### Scenario: Devices registered
|
||||
- **WHEN** an operator opens the dashboard while devices are registered
|
||||
- **THEN** each device is shown with its name/id and current status, refreshed without a full page reload
|
||||
|
||||
#### Scenario: No devices registered
|
||||
- **WHEN** an operator opens the dashboard while no devices are registered
|
||||
- **THEN** an empty-state message is shown inviting the operator to add a device
|
||||
|
||||
### Requirement: Task list and timeline replay
|
||||
The web console SHALL let an operator browse the task list, open a task's detail view, and step through its recorded timeline including screenshots.
|
||||
|
||||
#### Scenario: Browse tasks
|
||||
- **WHEN** an operator opens the task list view
|
||||
- **THEN** tasks are shown most-recent-first with their goal, device, and status, and can be filtered by device or status
|
||||
|
||||
#### Scenario: Replay a task's timeline
|
||||
- **WHEN** an operator opens a completed or failed task's detail view
|
||||
- **THEN** the console renders each recorded step in order, showing its tool call, result, and screenshot (when available)
|
||||
|
||||
### Requirement: Device configuration
|
||||
The web console SHALL let an operator register a new device (choosing a supported driver type and entering its connection info) and remove an existing device, surfacing any validation error returned by `console-config-api`.
|
||||
|
||||
#### Scenario: Add a device successfully
|
||||
- **WHEN** an operator submits the add-device form with valid driver type and connection info
|
||||
- **THEN** the new device appears in the dashboard without a page reload
|
||||
|
||||
#### Scenario: Add a device with invalid input
|
||||
- **WHEN** an operator submits the add-device form with an unsupported driver type
|
||||
- **THEN** the console displays the error returned by the API and does not add the device to the list
|
||||
|
||||
#### Scenario: Remove a device
|
||||
- **WHEN** an operator confirms removal of a registered device
|
||||
- **THEN** the device disappears from the dashboard without a page reload
|
||||
|
||||
### Requirement: Runtime parameter configuration
|
||||
The web console SHALL let an operator view and edit adjustable runtime parameters (currently `max_steps`) through a settings form.
|
||||
|
||||
#### Scenario: Update a runtime parameter
|
||||
- **WHEN** an operator changes `max_steps` in the settings form and saves
|
||||
- **THEN** the console shows the updated value on success, or the validation error if the API rejects it
|
||||
@@ -0,0 +1,36 @@
|
||||
## 1. Device config & settings store
|
||||
|
||||
- [x] 1.1 Add `storage/device_config.py` with a SQLite-backed `DeviceConfigStore` (`device_id, name, driver_type, connection_info json`) following the `TaskMetadataStore` pattern, with `add`, `remove`, `list`, and `get` methods
|
||||
- [x] 1.2 Add a `settings(key, value)` table to the same store with `get_setting`/`set_setting` helpers, seeded with a default `max_steps`
|
||||
- [x] 1.3 Add unit tests for `DeviceConfigStore` (add/remove/list, settings get/set, unknown key/device handling)
|
||||
|
||||
## 2. Console status API
|
||||
|
||||
- [x] 2.1 Add `api/console.py` with an `APIRouter`; implement `GET /console/devices` (list + status via `DeviceManager.list_devices()`)
|
||||
- [x] 2.2 Implement `GET /console/tasks` (via `TaskMetadataStore.list_tasks()`) with optional `device_id`/`status` query filters
|
||||
- [x] 2.3 Implement `GET /console/tasks/{task_id}` (404 on unknown id)
|
||||
- [x] 2.4 Implement `GET /console/tasks/{task_id}/timeline` reading `Timeline.read(task_id)` / `ArtifactStore`, inlining screenshots as base64
|
||||
- [x] 2.5 Add tests for all four endpoints covering populated and empty states, and the 404 case
|
||||
|
||||
## 3. Console config API
|
||||
|
||||
- [x] 3.1 Implement `POST /console/devices`: validate `driver_type` against a supported-driver registry (`{"wda": ...}`), build the `driver_factory`, call `DeviceManager.register_device`, persist via `DeviceConfigStore`; return `400` for unsupported driver types
|
||||
- [x] 3.2 Implement `DELETE /console/devices/{device_id}`: call `DeviceManager.unregister_device` and remove the persisted config; return `404` for unknown device ids
|
||||
- [x] 3.3 Implement `GET /console/config` and `PUT /console/config` for `max_steps`, validating positive values (`400` otherwise), applying the change to the live `TaskRunner.config` and persisting it via the settings table
|
||||
- [x] 3.4 Wire startup reload: in `create_app()`, read all persisted device configs and the persisted `max_steps` setting, register devices and construct `TaskRunnerConfig` accordingly, before returning the app
|
||||
- [x] 3.5 Mount the `console` router into `create_app()` in `api/rest.py` and enable permissive CORS for local frontend development
|
||||
- [x] 3.6 Add tests for register/unregister (success + validation errors), config get/update (success + invalid value), and startup reload of persisted devices/settings
|
||||
|
||||
## 4. Web console frontend (independent SPA)
|
||||
|
||||
- [x] 4.1 Scaffold an independent Vue 3 + Vite SPA project (in-repo `console/` per design's default) with its own `package.json`/build tooling
|
||||
- [x] 4.2 Implement the device status dashboard view (list + empty state) against `GET /console/devices`
|
||||
- [x] 4.3 Implement the task list view (with device/status filters) and task detail + timeline replay view (screenshots) against the console-status-api endpoints
|
||||
- [x] 4.4 Implement the device configuration screen (add/remove device forms, surfacing API validation errors) against the console-config-api endpoints
|
||||
- [x] 4.5 Implement the runtime parameter settings form (`max_steps`) against `GET/PUT /console/config`
|
||||
- [x] 4.6 Document how to run the frontend dev server against the local backend (base URL config, CORS expectations)
|
||||
|
||||
## 5. Verification
|
||||
|
||||
- [x] 5.1 Run the full backend test suite (`pytest`) and confirm no regressions to existing REST/MCP endpoints
|
||||
- [ ] 5.2 Manually verify end-to-end: register a device via the UI, run a task via the existing `/agent/task` endpoint, and confirm its status/timeline/screenshots render correctly in the console
|
||||
Reference in New Issue
Block a user