agentic-mobile-control/openspec/changes/web-console/design.md

Context

The backend (api/rest.py, api/mcp.py) currently exposes only device-control and task-execution primitives (/devices, /agent/task, /task/{id}), all consumed by an LLM/MCP client. Two pieces of already-implemented backend functionality are not exposed over HTTP at all: TaskMetadataStore.list_tasks() (storage/task_metadata.py) and Timeline.read(task_id) / ArtifactStore.read_steps(task_id) (storage/timeline.py, storage/artifact_store.py). Device registration (DeviceManager.register_device, core/device_manager.py) requires a Python driver_factory callable and is purely in-memory — there is no config file or persistence, so every device must be (re-)registered in code on each process start. There is no operator-facing UI at all today.

Goals / Non-Goals

Goals:

• Expose existing device/task/timeline state over a read-only console REST API.
• Let an operator register/remove devices and adjust simple runtime parameters (max_steps) through a REST API, with changes surviving process restarts.
• Ship an independent Vue 3 SPA that consumes these APIs for a status dashboard + configuration screens.

Non-Goals:

• Authentication/authorization for console endpoints (assumed trusted network for now).
• Editing planner/agent logic, prompts, or tool registration from the UI.
• Concurrent multi-operator conflict resolution for config edits.
• Any change to the existing MCP tool surface or /agent/task execution semantics.
• Supporting driver types beyond wda in the config UI (Android etc. remain out of scope, consistent with apex-agent-mvp).

Decisions

• New /console router, shared state. Add api/console.py exposing an APIRouter mounted under /console inside the existing create_app() in api/rest.py, reusing the same device_manager, metadata_store, and task_runner instances already constructed there. Rejected alternative: a second standalone FastAPI app — would duplicate state wiring and require its own process/port for no benefit.
• New persisted device-config store, additive to DeviceManager. Add storage/device_config.py (SQLite, same pattern as TaskMetadataStore) storing device_id, name, driver_type, connection_info(json). On create_app() startup, read all rows and call device_manager.register_device(...) with a driver_factory built via a small driver_type -> factory registry (today: {"wda": WDADriverConfig-based factory}). DeviceManager itself is not modified — it stays a pure in-memory runtime registry; the config store is the source of truth for "which devices should exist," the manager is the source of truth for "current connection state." Rejected alternative: teaching DeviceManager to persist itself — would blur its existing single responsibility and touch a capability (device-management) this change intends to leave unmodified.
• Runtime parameters persisted in the same store. Add a small settings(key, value) key-value table to the same SQLite file for max_steps (and future simple scalars), read at startup to construct TaskRunnerConfig, and updated in-place on the live TaskRunner.config object when changed via PUT /console/config — so changes apply immediately without a restart and still survive one. Rejected alternative: in-memory-only (rejected: contradicts "simple configuration" persisting across restarts, which is the point of the config API); a separate config file (rejected: adds a second persistence mechanism for no benefit over reusing the SQLite file already needed for device configs).
• Screenshots returned as base64 JSON, not static files. GET /console/tasks/{id}/timeline reads screenshots via ArtifactStore/Timeline and inlines them as base64, mirroring the existing /devices/{id}/screenshot convention. Rejected alternative: mounting tasks/history as a static file directory — simpler but opens a directory-listing/path-traversal surface for no strong benefit at this stage; base64 keeps the same trust boundary as the rest of the API.
• Driver type allow-list validated at registration time. POST /console/devices rejects any driver_type not in the small supported set ({"wda"}) with 400, rather than silently accepting arbitrary strings that would fail later at connect() time.
• Independent Vue 3 SPA, no backend template rendering. Per explicit choice, the console frontend is its own project (own build/deploy, e.g. Vite), calling the JSON APIs over HTTP. create_app() enables permissive CORS for local development only (see Open Questions for production hosting).

Risks / Trade-offs

• [No auth on console endpoints] → Mitigation: document as trusted-network-only for this change; flag auth as explicit follow-up before any non-trusted deployment.
• [max_steps change applied to a running TaskRunner while a task is mid-execution] → Mitigation: change only affects the loop bound checked at the top of each iteration in TaskRunner.run; an in-flight task simply picks up the new bound on its next iteration, no partial-state corruption possible.
• [SQLite device-config store and TaskMetadataStore are two separate files/schemas] → Mitigation: acceptable for this change's scope (mirrors existing pattern); revisit consolidation only if a third store is added later.
• [Base64 screenshot inlining increases timeline payload size for long tasks] → Mitigation: acceptable for MVP console scope (tasks capped at max_steps, typically ≤ 20 steps); revisit pagination/lazy-loading if task length grows.

Migration Plan

1. Add storage/device_config.py (device configs + settings key-value table).
2. Add api/console.py with status endpoints (read-only) first; mount into create_app().
3. Add config endpoints (device register/unregister, runtime param get/update) to api/console.py; wire startup reload of persisted devices/settings into create_app().
4. Scaffold the independent Vue 3 SPA project consuming the above.
5. Rollback: the change is purely additive (new router, new tables, new frontend project) — rollback is removing the router mount and the new frontend project; no migration of existing tasks.sqlite3 or tasks/history data is required or affected.

Open Questions

• Where should the Vue 3 SPA project live — a console/ subfolder in this repo, or a separate sibling repository? Defaulting to an in-repo console/ subfolder for now (simplest to keep in sync with the backend during early iteration); revisit if it needs independent versioning/deploy cadence.
• Should console endpoints eventually sit behind the same auth mechanism as a future MCP-facing auth layer, or get their own? Deferred until authentication is actually scoped.