agentic-mobile-control/openspec/changes/runtime-console-jinja2-templates/design.md

## Context

The root Runtime API currently exposes console state and mutations through
`api/console.py` under `/console/*`. Its human interface is a separate
`console/` Vue/Vite SPA. `api/rest.py` only serves that SPA when
`RUNTIME_CONSOLE_STATIC_DIR` points to a built distribution, uses a
SPA-specific 404 fallback, and enables wildcard CORS for cross-origin Vite
development.

The existing `web-console` change is still unarchived and contains the
opposite design decision: an independent SPA with no backend templates. There
is no canonical Runtime-console spec in `openspec/specs/`, so this change
introduces a new capability rather than modifying a pending change's delta
spec. The Host Agent already uses Jinja2, but it is a distinct application
with local-account authentication and must not become a dependency of the
Runtime API.

The Runtime layering rule requires all HTTP, template, static-asset, and form
parsing concerns to remain in `api`. `core`, `driver`, `device`, `tools`,
`perception`, `storage`, and `runtime` retain their current framework-free
contracts.

## Goals / Non-Goals

**Goals:**

- Serve the Runtime operator console from the same FastAPI process at `/ui/`
  without Node, Vite, a prebuilt SPA directory, or a second development
  process.
- Preserve the observable behavior of the existing device, task, timeline,
  and runtime-config console workflows, while retaining `/console/*` JSON
  endpoints for programmatic clients.
- Ensure every HTML interpolation is protected by Jinja2 autoescaping and
  ship templates and assets in the Runtime wheel.
- Keep mutations server-owned through ordinary POST/Redirect/GET form flows
  and reuse one API-local implementation for page and JSON routes.

**Non-Goals:**

- Add authentication, authorization, sessions, CSRF protection, rate limits,
  or a public-network deployment model for the Runtime console.
- Change `/agent/task`, `/devices`, MCP, `DeviceManager`, `TaskRunner`,
  `TaskMetadataStore`, `Timeline`, or persisted data formats.
- Add new driver types, editable planner settings, Cloud Console features, or
  Host Agent console features.
- Retain a generic client-side application framework or a Node build pipeline.
- Edit or archive the pending `web-console` change as part of this change.

## Decisions

### D1: Add a Runtime page router in `api/console_web.py`

`api/console_web.py` will expose `create_console_web_router()` with the `/ui`
prefix. `api/rest.py::create_app()` will build the existing Runtime stores and
manager once, construct the console service once, and include both the JSON
router and the page router in the same FastAPI application. `GET /` will
always redirect to `/ui/` because the console is no longer optional build
output.

The page surface will use stable server routes:

- `GET /ui/` for the device/status dashboard;
- `GET /ui/tasks` for task filtering and browsing;
- `GET /ui/tasks/{task_id}` for one task and its ordered timeline;
- `GET /ui/config` for device registration/removal and `max_steps`;
- POST routes below `/ui/` for device registration, device removal, and
  configuration updates.

Successful POST handlers use `303 See Other` redirects. Validation failures
return the originating page with an HTML `400` response and preserved safe
form values. This is preferred over trying to preserve the SPA's client-side
state machine because normal browser navigation and forms are sufficient for
the console's low-frequency operational actions.

An alternative of replacing `/console/*` JSON endpoints with HTML endpoints
was rejected: the JSON API has explicit tests and remains useful to scripts
and future clients. An alternative of a separate FastAPI app was rejected:
it would duplicate the `DeviceManager`, stores, and `TaskRunner` composition.

### D2: Share API-local console operations rather than issuing loopback HTTP
requests

`api/console.py` will gain a small API-local `ConsoleService` (or equivalent
typed operation object) that owns the current console reads and mutations:
device listing/registration/removal, task listing/detail/timeline lookup, and
runtime configuration reads/updates. The existing JSON router and the new
page router receive the same service instance.

The service stays in `api` and delegates to the existing injected
`DeviceManager`, `TaskMetadataStore`, `Timeline`, `DeviceConfigStore`, and
`TaskRunner`. Page handlers MUST NOT make HTTP requests to the process's own
`/console/*` endpoints. This keeps validation, driver allow-listing,
persistence, error mapping, and state mutation single-sourced without moving
web concepts into a lower layer.

Duplicating the route bodies was rejected because future changes could make
JSON and form behavior diverge. Moving the service into `runtime` or
`storage` was rejected because it would introduce HTTP/UI-oriented
application concerns below the API adapter boundary.

### D3: Use a module-level Jinja2 environment and package-owned assets

`api/console_web.py` will construct one module-level Jinja2 `Environment`
using `FileSystemLoader` rooted at `api/templates/runtime_console/` and
`select_autoescape(["html", "xml"])`. Route handlers will render with
`get_template(...).render(...)` and return `HTMLResponse`; they will not
build HTML through f-strings, string concatenation, or post-process rendered
HTML.

Templates will include a shared `base.html`, dashboard, task list, task
detail, configuration page, and any small server-rendered dashboard fragment.
CSS and the narrow polling script will be regular package assets mounted under
`/ui/assets`, not Vite output. The root package will explicitly include the
template, CSS, and JavaScript globs in setuptools package data so an installed
wheel behaves like an editable checkout.

`jinja2>=3.1` and `python-multipart` will be direct root dependencies. The
latter is declared directly even though it is currently available through an
unrelated transitive dependency, because `Request.form()` is part of this
feature's runtime contract.

`starlette.templating.Jinja2Templates` was rejected because direct Jinja2
rendering is explicit, matches the existing Host Agent convention, and avoids
coupling the implementation to helper signature changes. Inline all-CSS HTML
was rejected because a dedicated static stylesheet keeps base templates
readable without restoring a frontend build system.

### D4: Retain server-owned live status with a narrow rendered fragment

The dashboard will keep the current approximately 10-second live-status
refresh through a small static browser script. It fetches an HTML fragment
rendered by the same Jinja2 environment and replaces only the dashboard's
live-status region. The browser does not fetch JSON and recreate console DOM
state; all stateful markup still originates on the server.

Task filters, task selection, timeline step selection, device mutations, and
configuration updates use normal GET/POST navigation. This retains useful
status freshness without reintroducing a SPA framework or making a full-page
reload the only refresh mechanism.

A zero-JavaScript periodic refresh was rejected because it would require full
page reloads and regress the existing dashboard behavior. Reusing the Vue
reactivity layer was rejected because it retains the independent build and
deployment boundary this change removes.

### D5: Preserve JSON compatibility while retiring SPA deployment wiring

The `/console/*` paths, methods, success payloads, error status codes, and
persistence semantics remain unchanged. Existing `tests/test_console_api.py`
continues to be the compatibility baseline. `api/rest.py` removes
`RUNTIME_CONSOLE_STATIC_DIR`, `SpaStaticFiles`, and its SPA-only fallback.
The broad CORS middleware added only for cross-origin Vite development is
removed; same-origin `/ui/` must not depend on it.

The top-level `console/` directory, Vite environment files, package lockfile,
and npm instructions are removed. Root documentation changes to a single
Runtime startup command followed by `/ui/`. The unrelated `cloud-console/`
SPA and its deployment remain unchanged.

Keeping `RUNTIME_CONSOLE_STATIC_DIR` as a deprecated fallback was rejected:
it would preserve an unsupported second rendering path and force every future
console change to be tested twice. A backward-compatible redirect from the
old external Vite development port is impossible because it is a separate
process, so the operator migration is documented as a breaking change.

### D6: Apply autoescaping uniformly and retain the trusted-network boundary

All interpolated device names, IDs, connection values, task goals, failure
reasons, timeline text, JSON-like tool/result data, and errors pass through
the shared autoescaping environment. Structured values use Jinja's `tojson`
filter only in safe text contexts; no current template uses `|safe` or a
global autoescape opt-out. Screenshot bytes remain data sourced from the
existing timeline and are rendered only as the established PNG data URI.

Removing wildcard CORS reduces the old SPA development surface but does not
provide authentication and does not by itself prevent cross-site HTML form
submission. The Runtime console therefore remains documented as
trusted-network-only. Authentication/session/CSRF design is intentionally
separate, so this migration does not create a misleading partial security
model.

## Risks / Trade-offs

- [A task, device name, or JSON value could carry HTML/script content] -> The
  shared autoescaping environment, `tojson` for structured output, no safe
  bypasses, and XSS regression tests make the protection mechanical.
- [Templates or assets work from a checkout but not an installed wheel] ->
  Explicit setuptools package-data rules and a built-wheel smoke test verify
  deployment behavior.
- [Removing the npm/Vite workflow disrupts an operator's existing runbook] ->
  Mark the removal as breaking, update all Runtime console documentation, and
  retain a Git-revert rollback path with no data migration.
- [Removing CORS breaks an undiscovered browser client] -> The existing CORS
  configuration was introduced for the deleted Vite development flow; JSON
  clients outside a browser remain unaffected. A future browser integration
  must add an explicit origin policy rather than restore a wildcard.
- [Base64 screenshots make large task-detail responses expensive] -> Preserve
  the existing bounded-task behavior and do not change timeline storage or
  transfer format in this rendering migration.
- [The pending `web-console` change still contains a SPA decision and one
  manual task] -> Record this change as the rendering-mechanism successor;
  reconcile the older change only after the new server-rendered browser
  workflow has been manually verified.

## Migration Plan

1. Add direct dependencies and package-data declarations, then create the
   Jinja2 environment, templates, static assets, API-local console service,
   and `/ui/` page routes while preserving existing JSON-route tests.
2. Add focused template, page-route, mutation/PRG, fragment-refresh, and XSS
   tests. Build and install the Runtime wheel in an isolated environment to
   verify package resources are present.
3. Remove `console/`, SPA static mounting, the static-directory environment
   variable, Vite-specific CORS, and obsolete ignore/configuration files.
4. Update root Runtime and macOS setup documentation to direct operators to
   `/ui/`, and document the removed npm/static-directory workflow.
5. Run formatting, linting, non-integration Runtime tests, wheel build/smoke
   checks, strict OpenSpec validation, and a browser walkthrough covering
   dashboard refresh, task replay, device mutation, and `max_steps` update.

Rollback is a source revert. It restores the Vue sources and static mount if
needed and does not alter device configuration, task metadata, or timeline
data, so no data rollback or schema migration is required.

## Open Questions

- The remaining manual verification task in the pending `web-console` change
  must be reconciled before that older change is archived. It is not safe to
  mark it complete solely because this proposal exists; the new `/ui/`
  browser walkthrough supplies the replacement evidence after implementation.
- Authentication and CSRF protection remain deliberately deferred. Any plan to
  expose the Runtime console beyond a trusted local network requires a
  separate threat model and change proposal.