feat(cloud): remove static credentials and add host console
Tests / Test No test results found

This commit is contained in:
2026-07-13 19:45:53 +08:00
parent efeb3eb926
commit c162c2501b
61 changed files with 3118 additions and 1221 deletions
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-07-13
@@ -0,0 +1,71 @@
## Context
`apps/device-host-agent` is currently a headless asyncio process (`HostAgentApplication.run_async` in `host_agent/app.py`): a heartbeat task and a claim/execute/report loop, both driven purely by outbound HTTP calls to the Cloud API. It has no inbound listener of any kind, and its `pyproject.toml` declares only `httpx`, `device-agent-runtime`, and `device-cloud-platform` as direct dependencies — no web framework.
The root `device-agent-runtime` package (a workspace dependency of `device-host-agent`) already depends on `fastapi>=0.115.0` and `uvicorn[standard]>=0.30.0` (used by `api/rest.py` for the local single-machine Runtime API and by `console/`'s backend). Those packages are therefore already present in the resolved `uv.lock` and importable from `device-host-agent` today, even though `device-host-agent` does not declare them directly.
Local operator-facing state currently lives in three separate stores on the edge machine: `host_agent/local_account.py::LocalAccountStore` (PBKDF2 credential), `host_agent/identity.py::HostIdentityStore` (enrollment identity/host_id), and `storage/device_config.py::DeviceConfigStore` (locally registered devices, SQLite). None of it is visible or editable except by reading/editing these files directly or via `device-host-agent setup` (account creation only). `openspec/changes/edge-host-self-enrollment` (not yet implemented) defines the CLI-only first-account-creation flow; this change does not alter that requirement.
## Goals / Non-Goals
**Goals:**
- Give an operator on the edge machine a same-host web page to see heartbeat/enrollment/device/assignment status at a glance, and to perform the small set of actions that currently require hand-editing files: add/edit/remove a local device, change the local account password, review recent assignment/heartbeat outcomes.
- Keep the default deployment posture unchanged: console off by default; when enabled, bound to loopback only unless the operator explicitly opts into a wider bind address.
- Reuse the existing local account as the only credential — no second user/credential system.
- Server-rendered HTML, not a SPA: no new frontend build tooling, no JS framework, minimal inline `fetch()` calls only for the few sections that benefit from polling refresh (heartbeat status, current assignment progress).
**Non-Goals:**
- No TLS termination, reverse proxy, or certificate management built into the Host Agent — LAN exposure beyond loopback is an explicit, documented operator opt-in with the risk called out, not a feature this change makes safe by default.
- No multi-user accounts, roles, or audit logging beyond the bounded local history — the existing `local_account.py` model is single-account, and this change keeps it that way.
- No syncing of the new local assignment/heartbeat history to the Cloud Console; it is a purely local, best-effort operational aid, not a system of record (the Cloud API/`cloud-console` already own durable task/attempt history).
- No change to the outbound `host-agent-protocol` capability or to how devices are enrolled with the cloud; the console only calls the same local `DeviceConfigStore`/enrollment client code paths that `host_agent/app.py::_configured_device_manager` already uses at startup.
## Decisions
### Reuse FastAPI + Starlette's `HTMLResponse`, not a new micro-framework, not Jinja2
FastAPI/Starlette are already transitively resolved via `device-agent-runtime`. Adding them as **explicit** direct dependencies of `device-host-agent` (rather than relying on the transitive edge) is the only `pyproject.toml` change needed — no new third-party web framework enters the dependency graph. Pages are built with small Python functions returning `HTMLResponse(content=...)` from hand-written f-string templates with `html.escape()` on every interpolated value (no Jinja2: the page count is small — login, dashboard, devices, history — and a templating engine is unjustified surface area for a handful of server-rendered fragments). This matches the user's explicit direction: server-rendered pages, not the `console/`/`cloud-console/` SPA pattern, and not a new heavyweight dependency.
Alternative considered: `http.server`/stdlib-only implementation. Rejected — would duplicate routing, form parsing, and cookie handling that FastAPI/Starlette already provide for free given they're already in the dependency graph.
### Run the console server in the same asyncio loop as the heartbeat/claim loop
`HostAgentApplication.run_async` gains a third concurrent task (alongside `heartbeat_task` and the claim/process loop) that runs a `uvicorn.Server` configured with `install_signal_handlers=False` when `config.console_enabled`. It is started and stopped using the same `stop_requested`/`finally` shutdown sequence already used for the heartbeat task, so `Ctrl+C`/service-stop behavior is unchanged when the console is off (the default) and cleanly tears down the extra task when it's on.
Alternative considered: separate process/thread running its own event loop. Rejected — the console needs live references to the same `DeviceManager`, `HostAgentClient`, and in-flight assignment state that the main loop owns; a separate process would need its own IPC layer to read that state, which is unjustified complexity for a same-host admin page.
### Cookie session issued at login, not per-request Basic Auth
Login is a normal HTML form POST to `/login` that calls `LocalAccountStore.verify()` once (PBKDF2, 600 000 iterations — intentionally expensive, on the order of ~100ms+, which is fine for one login but would be a real cost if paid on every polled `fetch()`). On success the server issues a random opaque session token (`secrets.token_urlsafe`), stored in an in-memory `dict[str, SessionState]` (process-local; a restart invalidates all sessions, which is acceptable for a single-operator local admin page), and sets it as an `HttpOnly`, `SameSite=Strict`, `Secure`-when-not-loopback session cookie with a sliding expiry (e.g. 12h idle timeout). All other routes require a valid session and redirect to `/login` otherwise. This mirrors the cookie+session shape already validated in the sibling `cloud-console-user-authentication` change, applied here to a single local account instead of a multi-user table.
### CSRF token bound to the session, required on all mutating requests
Because authentication is a cookie the browser attaches automatically, every state-changing endpoint (device add/edit/remove, password change, logout) requires a per-session CSRF token — rendered into the page/forms and also required as a request header on the small number of `fetch()`-based mutations — checked against the value stored alongside the session. Read-only status/history polling endpoints do not require it.
### Local device CRUD updates the live `DeviceManager` in the same request, not just `DeviceConfigStore`
`device/manager.py::DeviceManager` already exposes `register_device`/`unregister_device`. The console's device-CRUD handlers call the same sequence `host_agent/app.py::_configured_device_manager` uses at startup (persist to `DeviceConfigStore`, call `enrollment_client.enroll_device(...)` when `config.enrollment_managed`, then `manager.register_device(...)`) so a device added or removed through the web page takes effect immediately, without requiring a Host Agent restart. This existing sequence is extracted into a small shared helper used by both the startup path and the new console routes, rather than duplicated.
### New bounded local history store for recent assignments/heartbeats
The Host Agent currently discards assignment outcomes once reported to the control plane and keeps no heartbeat history at all. A new local-only SQLite table (e.g. `tasks/host_console_history.sqlite3`, following the existing `storage.device_config` pattern of a small dedicated SQLite file under `tasks/`) records the last N (configurable, default e.g. 200) assignment results and heartbeat syncs. `AssignmentProcessor` and `HeartbeatSynchronizer` accept an optional recorder callback (no-op when the console is disabled, so there is zero overhead in the default configuration) that appends a row after each terminal report / heartbeat sync; the console's history page reads from this table. Retention is enforced by pruning beyond the configured cap on write, not by a separate cron/background task.
### Config additions, all opt-in and backward compatible
`HostAgentConfig` gains: `console_enabled: bool = False`, `console_bind_host: str = "127.0.0.1"`, `console_port: int = 8765`, `console_allow_non_loopback: bool = False`, `console_session_ttl_seconds: float = 43200.0` (12h), `console_history_limit: int = 200` — all with matching `HOST_AGENT_CONSOLE_*` environment variables following the existing `_positive_float`/`_positive_int` validation helpers in `config.py`. Loading raises `HostAgentConfigurationError` if `console_bind_host` resolves to a non-loopback address while `console_allow_non_loopback` is not set, so the risky configuration requires two explicit affirmative settings, not one.
## Risks / Trade-offs
- **[Risk] No TLS by default; a non-loopback bind sends the session cookie and form-posted password over cleartext HTTP on the LAN.** → Mitigation: loopback-only by default; non-loopback requires the explicit second opt-in flag; document the recommended alternative (SSH local port-forward to keep the console loopback-only while still reachable remotely) in `docs/CLOUD_DEPLOYMENT.md`/`docs/MACOS_IPHONE_SETUP.md` rather than building TLS support into this change.
- **[Risk] In-memory session store means every Host Agent process restart forces re-login.** → Accepted: restarts are infrequent for a background service and re-login is a low-friction PBKDF2 verify; avoids adding a persistent session store and its own cleanup/expiry code for a single-operator page.
- **[Risk] Blocking SQLite calls (`DeviceConfigStore`, new history store) on the same asyncio loop that runs heartbeat/claim could add latency under concurrent console use.** → Mitigation: wrap console route handlers' store calls in `asyncio.to_thread`, consistent with the existing `asyncio.to_thread(self.executor.execute, ...)` pattern in `host_agent/lease.py`; SQLite operations here are small and infrequent (one operator, occasional page loads) so this is a low-severity concern even without the wrapping, but the pattern costs nothing to apply consistently.
- **[Trade-off] Hand-written HTML via f-strings instead of a templating engine is more verbose per-page and pushes escaping discipline onto the author.** → Mitigation: a single small `escape()`-wrapping helper used for every interpolated value, and a lint/review checklist item (covered in tasks.md) rather than relying on an engine's autoescaping; the page count is small enough that this remains manageable.
- **[Trade-off] New local-only history duplicates, in miniature, information the Cloud Console already owns durably.** → Accepted: this history exists specifically for operators without (or before) Cloud Console access, or debugging when the control plane itself is unreachable; it is explicitly not a system of record (Non-Goals).
## Migration Plan
1. Add `fastapi`/`uvicorn[standard]` as explicit direct dependencies in `apps/device-host-agent/pyproject.toml` (versions already pinned in the shared `uv.lock` via the transitive edge — no version drift expected).
2. Add the new `HostAgentConfig` fields with the safe defaults above; existing deployments that don't set any `HOST_AGENT_CONSOLE_*` variable see no behavior change.
3. Implement the console module and wire its optional startup/shutdown into `HostAgentApplication.run_async`, gated on `config.console_enabled`.
4. Add the new bounded history store and the optional recorder hooks to `AssignmentProcessor`/`HeartbeatSynchronizer`, no-op by default.
5. Document how to enable the console (env vars, loopback-only default, SSH port-forward recommendation for remote access) in `docs/CLOUD_DEPLOYMENT.md` and `docs/MACOS_IPHONE_SETUP.md`.
6. Rollback: unset/leave `HOST_AGENT_CONSOLE_ENABLED` at its default `false`. No schema or state migration is introduced for existing stores (`DeviceConfigStore`, `LocalAccountStore`, `HostIdentityStore` are all read via their existing APIs, unchanged); the new history SQLite file is purely additive and can be deleted with no effect on Host Agent operation.
## Open Questions
- Exact default/max value for `console_history_limit` (row cap) — proposed default 200, may need tuning once real usage is observed.
- Whether a future change should let the Cloud Console optionally pull this local history for remote debugging (explicitly out of scope here; would need a new outbound protocol surface and its own review).
- Whether single-shared-account is sufficient long-term for edge machines with multiple physical operators, or whether that should be revisited alongside any future change to `local_account.py`'s single-account model.
@@ -0,0 +1,25 @@
## Why
`apps/device-host-agent` is a headless outbound worker: it has no HTTP server, no static assets, and no way to inspect or manage a running instance except by reading log output or editing `tasks/*.sqlite3`/`tasks/*.json` files by hand on the edge machine. Operators installing a new Host on an edge device (Mac/iPhone rig, etc.) currently must use `device-host-agent setup` (terminal-only, `getpass`) to create the local account, and have no local way to see heartbeat/enrollment status, review or edit locally registered devices, or check why the last assignment failed, without SSH-ing in and reading raw state files or cross-referencing the Cloud Console (which only shows what the Host last reported, not local-only state like unenrolled devices). A minimal local web page closes that operational gap.
## What Changes
- Add an embedded, server-rendered local web console to the Host Agent process: plain HTML responses from a lightweight HTTP server (no separate frontend build, no SPA framework), with a handful of endpoints returning small JSON fragments that a few inline `<script>` blocks poll to refresh sections of the page without a full reload.
- Web console covers: status/monitoring (heartbeat/last-seen, enrollment/identity state, registered local devices and their status, current assignment/execution progress, sanitized effective config such as `control_plane_url` and `host_id` with `token` never rendered), local device management (add/edit/remove entries in `storage/device_config.py`'s `DeviceConfigStore`), account settings (change the local account password in place; creating the *first* account remains the job of `device-host-agent setup`), and recent assignment/heartbeat history (a new bounded local log, since the Host Agent does not currently retain any local record of past assignments after reporting results to the control plane).
- New `HostAgentConfig` fields to gate and bind the console: disabled by default, and when enabled defaults to binding `127.0.0.1` only; binding to a non-loopback address is possible but requires an explicit opt-in and is treated as a documented, operator-accepted risk (no built-in TLS or rate limiting — see design.md threat model).
- Web login reuses the existing `host_agent/local_account.py` PBKDF2 credential (same account as `device-host-agent setup` creates/resets); no second credential store.
- `device-host-agent` gains a new optional dependency on a minimal ASGI/WSGI server library to host the embedded HTTP server; `HostAgentApplication` starts/stops it alongside the existing heartbeat and claim loop.
## Capabilities
### New Capabilities
- `host-agent-local-console`: embedded local-only web UI for the Host Agent covering status monitoring, local device CRUD, local account password change, and bounded recent-assignment/heartbeat history, authenticated against the existing local account and disabled/loopback-bound by default.
### Modified Capabilities
(none — `host-agent-protocol` covers the outbound cloud protocol and is unaffected; this change only adds a local-only inbound surface)
## Impact
- Affected code: `apps/device-host-agent/host_agent/` (new `web` module/package, `app.py` wiring, `config.py` new fields), `apps/device-host-agent/pyproject.toml` (new HTTP server dependency), `storage/device_config.py` (consumed for device CRUD, no schema break expected), new local history storage (new SQLite table or file, scoped to the Host Agent).
- Not affected: `cloud.*`, `apps/cloud-api`, `cloud-console/`, `console/`, `host-agent-protocol` outbound behavior, `openspec/changes/edge-host-self-enrollment` (its CLI-only local-account bootstrap requirement is unchanged and remains the only way to create the *first* account; this change only adds a way to change the password afterward through the web UI).
- Operational impact: a new local listening port on edge devices when explicitly enabled; default-off and loopback-only by default keep the default deployment posture unchanged.
@@ -0,0 +1,118 @@
## ADDED Requirements
### Requirement: The local console is disabled by default and loopback-bound when enabled
The Host Agent SHALL NOT start any local web console listener unless explicitly enabled by configuration, and SHALL bind that listener to a loopback address unless a separate, explicit configuration setting authorizes a non-loopback bind address.
#### Scenario: Default configuration starts no console
- **WHEN** the Host Agent starts with no console-related configuration set
- **THEN** no local web console listener is started and existing heartbeat/claim behavior is unaffected
#### Scenario: Console enabled with default bind
- **WHEN** the console is enabled without a non-loopback opt-in
- **THEN** the console listener binds only to a loopback address
#### Scenario: Non-loopback bind requested without opt-in
- **WHEN** the console is configured to bind a non-loopback address without the separate non-loopback opt-in setting
- **THEN** the Host Agent fails configuration loading with a clear error and does not start
#### Scenario: Non-loopback bind requested with explicit opt-in
- **WHEN** the console is configured to bind a non-loopback address and the non-loopback opt-in setting is also set
- **THEN** the console listener binds to the configured address
### Requirement: Console authentication reuses the existing local account
The local web console SHALL authenticate operators against the same local account credential used by the Host Agent's `setup` command, and SHALL NOT introduce a separate credential store.
#### Scenario: No local account exists
- **WHEN** the console is enabled and no local account file is present
- **THEN** the console's login page reports that no account exists and directs the operator to run the setup command, without accepting any login attempt
#### Scenario: Valid login
- **WHEN** an operator submits the correct local account username and password to the console
- **THEN** the console establishes an authenticated session and grants access to console pages
#### Scenario: Invalid login
- **WHEN** an operator submits an incorrect password to the console
- **THEN** the console rejects the attempt without revealing whether the username or password was wrong and does not establish a session
#### Scenario: Unauthenticated access to a console page
- **WHEN** a request without a valid session reaches any console page other than the login page
- **THEN** the console redirects the request to the login page without exposing any status or device data
### Requirement: Console sessions are cookie-based, session-scoped, and CSRF-protected
The console SHALL issue an opaque session token on successful login, SHALL require a valid, unexpired session for all non-login requests, and SHALL require a session-bound CSRF token on every state-changing request.
#### Scenario: Session expires
- **WHEN** a console session has been idle longer than the configured session timeout
- **THEN** subsequent requests using that session are treated as unauthenticated and redirected to login
#### Scenario: Mutating request without CSRF token
- **WHEN** an authenticated session submits a device change, password change, or logout request without a valid CSRF token
- **THEN** the console rejects the request and makes no change
#### Scenario: Process restart invalidates sessions
- **WHEN** the Host Agent process restarts while the console is enabled
- **THEN** previously issued session tokens are no longer accepted and operators must log in again
### Requirement: Console displays current heartbeat, enrollment, device, and assignment status
The console SHALL present, on an authenticated status page, the most recent heartbeat outcome, the current enrollment/identity state, the list of locally registered devices with their status, the in-progress assignment (if any) and its execution status, and the effective control-plane configuration with credential values never rendered.
#### Scenario: Status page reflects current state
- **WHEN** an operator loads the authenticated status page
- **THEN** it shows the last heartbeat time and outcome, enrollment/host identity state, each registered device and its status, and whether an assignment is currently executing
#### Scenario: Credentials are never rendered
- **WHEN** the status page renders the effective control-plane configuration
- **THEN** the host token, local account password, and any other credential value are omitted or masked, never shown in full
#### Scenario: Status page refreshes without a full reload
- **WHEN** an operator keeps the status page open
- **THEN** the page periodically fetches updated status fragments and updates the displayed heartbeat/assignment state without a full page navigation
### Requirement: Console supports adding, editing, and removing local devices
The console SHALL allow an authenticated operator to add, edit, and remove locally registered device entries, and each change SHALL take effect on the running Host Agent immediately, without requiring a process restart.
#### Scenario: Add a device
- **WHEN** an operator submits a new device's driver type and connection info through the console
- **THEN** the device is persisted to local device configuration and becomes immediately available to the running Host Agent, including cloud enrollment when the Host is enrollment-managed
#### Scenario: Edit a device
- **WHEN** an operator updates an existing device's connection info through the console
- **THEN** the persisted configuration and the running Host Agent's device registration both reflect the update without a restart
#### Scenario: Remove a device
- **WHEN** an operator removes a device through the console
- **THEN** the device is deleted from local device configuration and unregistered from the running Host Agent immediately
### Requirement: Console supports changing the local account password
The console SHALL allow an authenticated operator to change the local account password after re-confirming the current password, and SHALL NOT allow creating the first local account through the web console.
#### Scenario: Successful password change
- **WHEN** an operator submits the correct current password along with a new password and confirmation
- **THEN** the local account credential is updated and future console logins require the new password
#### Scenario: Incorrect current password
- **WHEN** an operator submits an incorrect current password while attempting to change it
- **THEN** the console rejects the change and the existing credential remains valid
#### Scenario: No console path to create the first account
- **WHEN** no local account exists
- **THEN** the console offers no form to create one, and only the Host Agent's setup command can create it
### Requirement: Console shows a bounded local history of recent assignments and heartbeats
The Host Agent SHALL retain a bounded, local-only history of recent assignment outcomes and heartbeat syncs, and the console SHALL display this history to an authenticated operator, independent of whether the Cloud Console is reachable.
#### Scenario: Assignment outcome recorded
- **WHEN** the Host Agent reports a terminal assignment result to the control plane
- **THEN** a corresponding entry recording the outcome is added to the local history and becomes visible on the console's history page
#### Scenario: Heartbeat recorded
- **WHEN** the Host Agent completes a heartbeat sync
- **THEN** a corresponding entry is added to the local history
#### Scenario: History is bounded
- **WHEN** the number of recorded history entries exceeds the configured retention limit
- **THEN** the oldest entries are pruned so the stored history does not grow unbounded
#### Scenario: History available without console enabled
- **WHEN** the console is disabled
- **THEN** the Host Agent does not record local history and incurs no related overhead
@@ -0,0 +1,55 @@
## 1. Config and dependencies
- [ ] 1.1 Add `fastapi` and `uvicorn[standard]` as explicit direct dependencies in `apps/device-host-agent/pyproject.toml` (versions matching those already pinned in `uv.lock`)
- [ ] 1.2 Add `console_enabled`, `console_bind_host`, `console_port`, `console_allow_non_loopback`, `console_session_ttl_seconds`, `console_history_limit` fields to `HostAgentConfig` in `host_agent/config.py`, plus matching `HOST_AGENT_CONSOLE_*` env vars in `load_host_agent_config`, reusing the existing `_positive_float`/`_positive_int` validators
- [ ] 1.3 Add validation that raises `HostAgentConfigurationError` when `console_bind_host` is non-loopback and `console_allow_non_loopback` is not set
- [ ] 1.4 Add unit tests in `apps/device-host-agent/tests/test_config.py` for defaults, env var parsing, and the non-loopback-without-opt-in rejection
## 2. Shared device-registration helper
- [ ] 2.1 Extract the device-add sequence (`DeviceConfigStore.add`/`set_cloud_device_id` + optional `enrollment_client.enroll_device` + `manager.register_device`) currently inlined in `host_agent/app.py::_configured_device_manager` into a small shared function usable by both startup and the console
- [ ] 2.2 Add a matching shared function for device removal (`DeviceConfigStore.remove` + `manager.unregister_device`)
- [ ] 2.3 Update `_configured_device_manager` to use the extracted add helper; confirm existing `test_app.py` startup tests still pass unchanged
## 3. Local history store
- [ ] 3.1 Add a new `host_agent/history.py` module with a `ConsoleHistoryStore` backed by a small SQLite file (e.g. `tasks/host_console_history.sqlite3`), supporting `record_assignment(...)`, `record_heartbeat(...)`, and `list_recent(limit)`, pruning beyond `console_history_limit` on write
- [ ] 3.2 Add an optional recorder hook to `AssignmentProcessor.process` (host_agent/processor.py) invoked after a terminal result is reported, no-op when no recorder is configured
- [ ] 3.3 Add an optional recorder hook to `HeartbeatSynchronizer.sync_once` (host_agent/heartbeat.py) invoked after each successful sync, no-op when no recorder is configured
- [ ] 3.4 Unit tests for `ConsoleHistoryStore` (write, prune-on-overflow, ordering) and for the processor/heartbeat recorder hooks firing with the expected data and being skipped when absent
## 4. Session and authentication
- [ ] 4.1 Add `host_agent/web/auth.py` with an in-memory session store (opaque token → session state with expiry), login verification against `LocalAccountStore`, and CSRF token issuance/validation bound to the session
- [ ] 4.2 Implement session cookie handling (`HttpOnly`, `SameSite=Strict`, `Secure` when bind host is non-loopback) and sliding expiry per `console_session_ttl_seconds`
- [ ] 4.3 Implement an auth dependency/middleware that redirects unauthenticated requests to `/login` and rejects mutating requests lacking a valid CSRF token
- [ ] 4.4 Unit tests: successful login, wrong password, no-account-yet state, session expiry, CSRF rejection on a mutating route, redirect-to-login for an unauthenticated GET
## 5. Console pages and routes
- [ ] 5.1 Add `host_agent/web/app.py` building a FastAPI sub-application with hand-written HTML responses (f-string templates + a shared `escape()` helper for every interpolated value) for: `/login`, `/` (status dashboard), `/devices`, `/account`, `/history`
- [ ] 5.2 Implement `/login` (GET form, POST verify+establish session) per spec scenarios, including the "no local account exists" state
- [ ] 5.3 Implement the status dashboard: last heartbeat outcome/time, enrollment/identity state, device list with status, current assignment/execution state, sanitized effective config (no token/password rendered); add a small JSON status-fragment endpoint polled via inline `fetch()` for refresh without full reload
- [ ] 5.4 Implement `/devices`: list, add, edit, remove forms wired to the section-2 shared helpers, taking effect on the live `DeviceManager` immediately
- [ ] 5.5 Implement `/account`: change-password form requiring current password re-entry, calling `LocalAccountStore.create` (or an equivalent update path) only after verifying the current credential
- [ ] 5.6 Implement `/history`: read-only table of recent assignment/heartbeat entries from `ConsoleHistoryStore`
- [ ] 5.7 Implement `/logout` (CSRF-protected POST) invalidating the session
## 6. Lifecycle wiring
- [ ] 6.1 In `host_agent/app.py::create_application`, construct the console app, session store, and `ConsoleHistoryStore` only when `config.console_enabled`, and wire the recorder hooks from section 3 into the constructed `AssignmentProcessor`/`HeartbeatSynchronizer`
- [ ] 6.2 In `HostAgentApplication.run_async`, start a `uvicorn.Server` task (bound to `console_bind_host`/`console_port`, `install_signal_handlers=False`) alongside the heartbeat task when the console is configured, and stop it in the existing `finally` shutdown sequence
- [ ] 6.3 Integration test exercising the full lifecycle with the console enabled: process starts, console responds on the configured loopback port, process shuts down cleanly and stops the console server
- [ ] 6.4 Integration test confirming that with the console left at its default (disabled), no listening socket is opened and existing `test_app.py`/`test_e2e.py` behavior is unaffected
## 7. Documentation
- [ ] 7.1 Document the new `HOST_AGENT_CONSOLE_*` environment variables, default-off/loopback-only posture, and the SSH port-forward recommendation for remote access in `docs/CLOUD_DEPLOYMENT.md`
- [ ] 7.2 Add a short section to `docs/MACOS_IPHONE_SETUP.md` describing how to enable the console on an edge machine and what it shows
## 8. Validation
- [ ] 8.1 Run `uv run --package device-host-agent pytest` (full package suite) and the root non-integration suite; confirm no regressions
- [ ] 8.2 Run Ruff check/format and `python -m compileall` over the changed files
- [ ] 8.3 Manually verify in a browser: login, status dashboard auto-refresh, add/edit/remove a device, change password, view history, logout, and confirm the console refuses to bind non-loopback without the opt-in flag
- [ ] 8.4 Run `openspec validate host-agent-local-console --strict` and confirm it passes
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-07-13
@@ -0,0 +1,118 @@
## Context
The Cloud API currently chains configured bearer credentials, browser user
sessions, and repository-backed Host credentials. Three JSON environment
variables populate the first provider and a second provider protects first Host
enrollment. The deployment image already contains the Console bundle, but
Compose separately supplies its fixed location. The Console also retains a
bearer-token compatibility screen and an administrator-only Users view.
The intended deployment has one trusted Cloud endpoint. A human operator signs
in with a Cloud account created or recovered through `device-cloud-admin`; a
new Host Agent registers itself once, generates its own high-entropy Host
secret, and persists the returned Host identity locally. Subsequent Host
requests continue to authenticate against the stored digest.
## Goals / Non-Goals
**Goals:**
- Start Cloud production without any static credential JSON or startup
requirement for one.
- Authorize browser operations by the existing user session and preserve its
role, CSRF, expiry, audit, and interactive CLI controls.
- Accept direct first Host registration, then strictly bind all later Host
operations to the registered Host identity and persisted secret.
- Remove obsolete Console and Compose configuration paths while keeping the
Jenkins-built Console available at `/console/`.
**Non-Goals:**
- No personal-access-token, service-account, or replacement SDK credential
scheme in this change.
- No user-directory view in the Console. Account provisioning and recovery
remain administration-CLI operations.
- No Host enrollment approval, rate limiting, or multi-tenant trust policy.
Network reachability to the trusted Cloud endpoint remains the boundary
explicitly chosen for this deployment.
- No change to durable task, lease, device, or user database schema.
## Decisions
### D1. Remove configured bearer providers instead of leaving empty compatibility configuration
`CloudControlConfig` will no longer parse the three credential JSON variables
or require credentials in production. The Cloud application will compose public
authorization from `UserSessionAuthProvider` only, while Host operational
authorization remains `RepositoryHostAuthProvider`. An empty production user
table is allowed so an administrator can execute the interactive bootstrap CLI
after migrations complete.
Leaving the variables optional was rejected because it preserves two
authentication modes and makes operators believe a deployed secret is needed.
Adding user-issued API tokens was rejected because the requested outcome is no
manual token configuration and it introduces a separate credential lifecycle.
### D2. Direct enrollment is a single unauthenticated bootstrap operation
The enrollment endpoint accepts a fresh Host's generated instance identifier
and candidate secret without an `Authorization` header. It stores only the
candidate secret digest and returns a generated `host_id`; the Host persists
that result atomically. The configured enrollment-token provider and static
Host configuration are removed. Heartbeat, device enrollment, claim, renewal,
and result endpoints remain Host-bound and require the persisted bearer secret.
An opt-in enrollment flag was rejected because the target deployment always
uses direct trust and the flag is another operational switch that can silently
block first-run setup. Keeping token enrollment as a fallback was rejected for
the same reason as D1.
### D3. Keep user administration off the Console, not out of the control plane
The Console retains login, logout, current-account display, and password
change. It removes bearer-token controls, token storage, and all Users routes,
navigation, client calls, and components. The authenticated user administration
API and `device-cloud-admin` CLI remain available for controlled provisioning
and recovery; the CLI is the documented initial-admin path.
Removing the administration API entirely was rejected because it would make
recovery tooling less complete and is unrelated to eliminating deployment
tokens.
### D4. Treat packaged static assets as an image contract
The Dockerfile sets `CLOUD_CONSOLE_STATIC_DIR=/app/console-static` after copying
the built SPA. Compose does not repeat this invariant. Application code keeps
the variable optional for source-based local development, where no bundled
directory exists.
## Risks / Trade-offs
- **[Risk] Any network caller can create a Host identity.** → The accepted
mitigation is reverse-proxy/firewall control of the trusted Cloud endpoint;
every operation after enrollment still requires the Host-generated secret.
- **[Risk] Existing integrations using static bearer tokens stop working.** →
This is intentional; create Cloud user accounts before deployment and migrate
human workflows to session login.
- **[Risk] An upgrade before an administrator exists leaves no human API
access.** → Run the documented interactive `device-cloud-admin users create`
command immediately after migration and before exposing the Console.
- **[Risk] Existing static or token-enrolled Hosts cannot rely on removed
configuration after upgrade.** → Preserve their already persisted dynamic
identities where present; otherwise perform a fresh direct enrollment.
## Migration Plan
1. Build and deploy the image with the schema migration already included.
2. Start the Cloud API using only database, production, and network settings.
3. Run `device-cloud-admin users create` interactively to bootstrap an
administrator, then verify Console login over HTTPS.
4. Start each new Host Agent with its persistent identity path; it directly
enrolls once and then uses its durable secret for future starts.
5. Remove the legacy credential JSON values from the deployment secret store.
Rollback requires restoring an earlier image and its matching static
credentials before restarting any Host that lacks a durable enrolled identity.
## Open Questions
- None. The user explicitly accepts direct Host enrollment behind the network
perimeter and does not require a Console user-directory surface.
@@ -0,0 +1,49 @@
## Why
The Cloud deployment currently requires operators to hand-maintain public API,
Host, and enrollment bearer tokens in Compose environment variables. Cloud
operator accounts now provide the human authentication boundary, while a Host
Agent can establish its own durable identity by registering directly with the
trusted control plane. Retaining both models makes deployment error-prone and
leaves secrets in configuration without serving the intended workflow.
## What Changes
- **BREAKING** Remove configured public, static Host, and enrollment bearer
credentials from the Cloud Control Plane configuration and deployment
examples.
- Authorize human Cloud API access exclusively through persistent Cloud user
sessions; remove the Console's bearer-token path and user-directory UI.
Interactive `device-cloud-admin` commands remain the account provisioning and
recovery surface.
- Enable a fresh Host Agent to register directly with the trusted Cloud API
without a pre-shared enrollment token, then use its persisted, cloud-bound
secret for all later Host operations.
- Bake the packaged Console static directory into the production image rather
than repeating it in Compose, and remove Compose entries whose values merely
duplicate application defaults or have no runtime effect.
- Update deployment documentation and tests to describe and enforce the
tokenless production flow.
## Capabilities
### New Capabilities
- `cloud-operator-authentication`: Cloud operator sign-in, CLI account
provisioning, and a token-free Console experience.
- `credentialless-host-bootstrap`: Direct Host registration and durable
post-registration identity without static deployment credentials.
### Modified Capabilities
- `platform-sdk`: Replace configured bearer credentials as the production
authorization prerequisite with Cloud user-session authorization.
- `host-agent-protocol`: Add unauthenticated first registration while retaining
host-bound authentication for every subsequent Host operation.
## Impact
Affected areas include the Cloud control configuration and authentication
composition, public and internal Cloud routers, Host Agent configuration and
enrollment client, Console UI, Docker/Compose deployment assets, tests, and
`docs/CLOUD_DEPLOYMENT.md`. Existing deployments using the removed static token
variables must create an administrator and re-enroll Hosts through the new
flow before upgrading.
@@ -0,0 +1,43 @@
## ADDED Requirements
### Requirement: Cloud operator sessions are the only human API authentication path
The system SHALL authorize human Cloud API operations through persistent Cloud
user sessions and SHALL not require configured static bearer credentials for
production startup or normal Console operation.
#### Scenario: Production starts without credential JSON
- **WHEN** the Cloud API starts in production with a current database schema
and no configured public, Host, or enrollment credential JSON
- **THEN** the application starts, exposes health and authentication routes,
and rejects unauthenticated protected API requests
#### Scenario: Signed-in operator uses a protected API route
- **WHEN** an enabled Cloud user has a valid session and calls an operation
allowed by the user's role scopes
- **THEN** the operation is authorized without an Authorization bearer header
### Requirement: The Console contains no token or user-directory management surface
The Console SHALL offer login, logout, current-account display, and password
change, and SHALL not render bearer-token controls, token persistence, Users
navigation, or user-management forms.
#### Scenario: Unauthenticated operator opens the Console
- **WHEN** no valid user session exists
- **THEN** the Console presents the username/password login flow without an API
token alternative
#### Scenario: Administrator opens the Console
- **WHEN** an administrator signs in
- **THEN** the Console presents normal authorized operational views but no
user-directory navigation or account lifecycle form
### Requirement: Account provisioning and recovery remain interactive administration operations
The system SHALL retain interactive, non-echoed administration CLI commands to
create, reset, enable, and revoke Cloud user accounts without accepting
passwords through Compose configuration or command-line arguments.
#### Scenario: Initial administrator is created after deployment
- **WHEN** an operator runs `device-cloud-admin users create` against the
migrated Cloud database and completes the password prompts
- **THEN** an enabled administrator account is created without a deployment
token or plaintext password in process arguments
@@ -0,0 +1,43 @@
## ADDED Requirements
### Requirement: Fresh Hosts register directly with the trusted Cloud API
The Cloud API SHALL accept a first Host registration without a configured
enrollment token or Authorization header, assign a durable `host_id`, and store
only the digest of the Host-generated high-entropy secret.
#### Scenario: Fresh Host registers without static configuration
- **WHEN** a Host Agent with no cached identity sends a valid instance
identifier and generated Host secret to the enrollment endpoint
- **THEN** the Cloud API creates or returns the durable Host identity and does
not require a pre-shared deployment credential
#### Scenario: Registration is retried by the same Host
- **WHEN** the same Host retries registration with its original instance
identifier and candidate secret after losing the response
- **THEN** the Cloud API returns the existing Host identity without creating a
duplicate Host row
### Requirement: Registered Hosts authenticate all later operational requests
The system SHALL require the persisted Host secret for device enrollment,
heartbeat, claim, renewal, and result operations after first registration, and
SHALL bind each accepted request to its registered `host_id`.
#### Scenario: Registered Host sends a heartbeat
- **WHEN** a Host presents its persisted secret for its own Host identity
- **THEN** the Cloud API accepts the heartbeat subject to normal validation
#### Scenario: Caller attempts a Host operation without its secret
- **WHEN** a caller accesses any post-registration Host operation without a
valid secret bound to the path Host identity
- **THEN** the Cloud API rejects the operation without changing Host state
### Requirement: Host Agent configuration contains no static Cloud credential
The Host Agent SHALL use its cached identity when present and otherwise perform
direct registration, without `HOST_AGENT_HOST_ID`, `HOST_AGENT_TOKEN`, or
`HOST_AGENT_ENROLLMENT_TOKEN` configuration.
#### Scenario: Fresh Host starts with only its local state path
- **WHEN** a Host Agent starts with no cached Cloud identity and no static Host
credential environment values
- **THEN** it generates and persists an identity through direct registration
before starting heartbeat or assignment polling
@@ -0,0 +1,39 @@
## ADDED Requirements
### Requirement: Host identity can be established by direct registration
The internal Host Agent API SHALL accept a fresh Host registration without a
pre-shared deployment credential, SHALL assign a durable Host identifier, and
SHALL store only the digest of the Host-generated secret for subsequent
host-scoped authentication.
#### Scenario: Fresh Host establishes an identity
- **WHEN** a fresh Host Agent submits a valid registration request containing
its instance identifier and generated secret
- **THEN** the control plane returns a durable Host identifier and stores only
the secret digest bound to that Host
#### Scenario: Registration request is retried
- **WHEN** the same Host Agent repeats registration with its original instance
identifier and secret after a lost response
- **THEN** the control plane returns the existing Host identifier without
creating a second Host identity
## MODIFIED Requirements
### Requirement: Host identity is authenticated and bound to one host id
After direct registration, the internal Host Agent API SHALL require a
host-scoped bearer principal for all Host operational requests and SHALL reject
any request that attempts to act for a `host_id` different from the
authenticated principal's bound host. The initial registration endpoint is the
only exception and establishes that bound principal.
#### Scenario: Host authenticates as itself after registration
- **WHEN** a Host Agent presents the persisted generated secret bound to its
requested `host_id`
- **THEN** the internal API authorizes permitted heartbeat, claim, renewal, and
result operations
#### Scenario: Host attempts to impersonate another host
- **WHEN** valid credentials bound to host A are used on a request for host B
- **THEN** the internal API rejects the request without reading or modifying
host B's state
@@ -0,0 +1,36 @@
## MODIFIED Requirements
### Requirement: Pluggable authentication hook with a safe default
The system SHALL evaluate every platform SDK route through a configurable
scope-aware `AuthProvider` hook. The deployable Cloud Control Plane SHALL
reject anonymous access outside an explicit insecure-development override, but
production startup SHALL not require configured static bearer credentials.
#### Scenario: Production starts without configured bearer credentials
- **WHEN** the Cloud Control Plane is configured as production with a usable
database and no static bearer credential configuration
- **THEN** startup succeeds and protected platform routes reject unauthenticated
requests
#### Scenario: Explicit local anonymous override
- **WHEN** a non-production operator explicitly enables the insecure
anonymous-development override
- **THEN** platform routes may use an anonymous principal and the application
records that insecure mode is active
#### Scenario: Custom AuthProvider is honored
- **WHEN** a caller configures a custom `AuthProvider` that rejects a request
or omits its required scope
- **THEN** the platform SDK route returns an authentication or authorization
error without executing its handler operation
## REMOVED Requirements
### Requirement: Python SDK supports authenticated requests
**Reason**: The deployment no longer provisions or accepts static bearer
credentials for public platform access; human operations use Cloud user
sessions.
**Migration**: Replace bearer-token SDK workflows with authenticated Console
user-session workflows. A non-human service credential model is outside this
change and must be designed separately before reintroducing SDK automation.
@@ -0,0 +1,30 @@
## 1. Cloud authentication configuration
- [x] 1.1 Remove configured public and static-Host bearer credential parsing and production credential validation from `CloudControlConfig`, while retaining the optional trusted-proxy setting used by login throttling.
- [x] 1.2 Compose public authorization from Cloud user sessions and dynamic repository-backed Host credentials only; remove configured enrollment-token authentication.
- [x] 1.3 Update Cloud configuration and application tests to prove production starts without JSON credentials, protected API routes reject anonymous requests, and user sessions retain scoped authorization.
## 2. Direct Host bootstrap
- [x] 2.1 Remove static Host and enrollment-token settings from Host Agent configuration while retaining persistent identity-file validation and control-plane URL overrides.
- [x] 2.2 Change the Host enrollment client and identity resolution to perform unauthenticated first registration and persist the returned Host identity and generated secret.
- [x] 2.3 Change the Cloud enrollment route to accept direct registration, preserve idempotency for the same instance, and require the persisted Host secret for all later Host operations.
- [x] 2.4 Add Cloud and Host Agent tests for direct registration, idempotent retry, missing/invalid post-registration Host credentials, and absence of static credential settings.
## 3. Console token and user-directory removal
- [x] 3.1 Remove bearer-token compatibility state, controls, client behavior, and related tests from the Cloud Console.
- [x] 3.2 Remove Console Users navigation, views, client calls, and tests while retaining login, logout, current-user, and password-change behavior.
- [x] 3.3 Add or update Console tests covering session-only login and the absence of token and user-directory UI paths.
## 4. Deployment contract and documentation
- [x] 4.1 Bake the Console static directory into the Docker image environment and remove it, credential JSON, static Host credentials, and redundant defaults from Compose and `.env.example`.
- [x] 4.2 Update deployment documentation for user-account bootstrap, direct Host registration, required runtime configuration, rollback, and the actual deploy Compose topology.
- [x] 4.3 Update deployment-configuration tests to enforce the reduced environment contract and image-provided Console path.
## 5. Verification and change validation
- [x] 5.1 Run focused Cloud API, Cloud platform, Host Agent, deployment, and Console test suites; fix failures caused by the removed credential paths.
- [ ] 5.2 Render `compose.deploy.yaml` with representative non-secret settings and verify the Cloud API command, health check, and environment contract.
- [ ] 5.3 Run the full non-integration workspace suite, Console build, and strict OpenSpec validation; record any environment-gated checks that cannot run locally.