feat(host-agent): start local console by default
Tests / Test passed: 662

This commit is contained in:
2026-07-14 07:31:31 +08:00
parent a166ffd8a4
commit 8b7e5a2800
10 changed files with 101 additions and 152 deletions
@@ -10,7 +10,7 @@ Local operator-facing state currently lives in three separate stores on the edge
**Goals:**
- Give an operator on the edge machine a same-host web page to see heartbeat/enrollment/device/assignment status at a glance, and to perform the small set of actions that currently require hand-editing files: add/edit/remove a local device, change the local account password, review recent assignment/heartbeat outcomes.
- Keep the default deployment posture unchanged: console off by default; when enabled, bound to loopback only unless the operator explicitly opts into a wider bind address.
- Start the Console with every Host Agent while binding it to loopback by default; an operator must explicitly opt into a wider bind address.
- Reuse the existing local account as the only credential — no second user/credential system.
- Server-rendered HTML, not a SPA: no new frontend build tooling, no JS framework, minimal inline `fetch()` calls only for the few sections that benefit from polling refresh (heartbeat status, current assignment progress).
@@ -28,7 +28,7 @@ FastAPI/Starlette are already transitively resolved via `device-agent-runtime`.
Alternative considered: `http.server`/stdlib-only implementation. Rejected — would duplicate routing, form parsing, and cookie handling that FastAPI/Starlette already provide for free given they're already in the dependency graph.
### Run the console server in the same asyncio loop as the heartbeat/claim loop
`HostAgentApplication.run_async` gains a third concurrent task (alongside `heartbeat_task` and the claim/process loop) that runs a `uvicorn.Server` configured with `install_signal_handlers=False` when `config.console_enabled`. It is started and stopped using the same `stop_requested`/`finally` shutdown sequence already used for the heartbeat task, so `Ctrl+C`/service-stop behavior is unchanged when the console is off (the default) and cleanly tears down the extra task when it's on.
`HostAgentApplication.run_async` runs a third concurrent task (alongside `heartbeat_task` and the claim/process loop) for a `uvicorn.Server` configured with `install_signal_handlers=False`. It is started and stopped using the same `stop_requested`/`finally` shutdown sequence as the heartbeat task, so `Ctrl+C`/service-stop cleanly tears down the Console too.
Alternative considered: separate process/thread running its own event loop. Rejected — the console needs live references to the same `DeviceManager`, `HostAgentClient`, and in-flight assignment state that the main loop owns; a separate process would need its own IPC layer to read that state, which is unjustified complexity for a same-host admin page.
@@ -44,8 +44,8 @@ Because authentication is a cookie the browser attaches automatically, every sta
### New bounded local history store for recent assignments/heartbeats
The Host Agent currently discards assignment outcomes once reported to the control plane and keeps no heartbeat history at all. A new local-only SQLite table (e.g. `tasks/host_console_history.sqlite3`, following the existing `storage.device_config` pattern of a small dedicated SQLite file under `tasks/`) records the last N (configurable, default e.g. 200) assignment results and heartbeat syncs. `AssignmentProcessor` and `HeartbeatSynchronizer` accept an optional recorder callback (no-op when the console is disabled, so there is zero overhead in the default configuration) that appends a row after each terminal report / heartbeat sync; the console's history page reads from this table. Retention is enforced by pruning beyond the configured cap on write, not by a separate cron/background task.
### Config additions, all opt-in and backward compatible
`HostAgentConfig` gains: `console_enabled: bool = False`, `console_bind_host: str = "127.0.0.1"`, `console_port: int = 8765`, `console_allow_non_loopback: bool = False`, `console_session_ttl_seconds: float = 43200.0` (12h), `console_history_limit: int = 200` all with matching `HOST_AGENT_CONSOLE_*` environment variables following the existing `_positive_float`/`_positive_int` validation helpers in `config.py`. Loading raises `HostAgentConfigurationError` if `console_bind_host` resolves to a non-loopback address while `console_allow_non_loopback` is not set, so the risky configuration requires two explicit affirmative settings, not one.
### Mandatory startup with loopback defaults
`HostAgentConfig` gains: `console_bind_host: str = "127.0.0.1"`, `console_port: int = 8765`, `console_allow_non_loopback: bool = False`, `console_session_ttl_seconds: float = 43200.0` (12h), `console_history_limit: int = 200` — with matching `HOST_AGENT_CONSOLE_*` environment variables except for an enable flag, because the Console is mandatory. Loading raises `HostAgentConfigurationError` if `console_bind_host` resolves to a non-loopback address while `console_allow_non_loopback` is not set, so the risky configuration requires two explicit affirmative settings, not one.
## Risks / Trade-offs
@@ -58,11 +58,11 @@ The Host Agent currently discards assignment outcomes once reported to the contr
## Migration Plan
1. Add `fastapi`/`uvicorn[standard]` as explicit direct dependencies in `apps/device-host-agent/pyproject.toml` (versions already pinned in the shared `uv.lock` via the transitive edge — no version drift expected).
2. Add the new `HostAgentConfig` fields with the safe defaults above; existing deployments that don't set any `HOST_AGENT_CONSOLE_*` variable see no behavior change.
3. Implement the console module and wire its optional startup/shutdown into `HostAgentApplication.run_async`, gated on `config.console_enabled`.
2. Add the new `HostAgentConfig` fields with the safe loopback defaults above; every Host Agent starts the Console.
3. Implement the console module and wire its startup/shutdown into `HostAgentApplication.run_async` for every Host Agent process.
4. Add the new bounded history store and the optional recorder hooks to `AssignmentProcessor`/`HeartbeatSynchronizer`, no-op by default.
5. Document how to enable the console (env vars, loopback-only default, SSH port-forward recommendation for remote access) in `docs/CLOUD_DEPLOYMENT.md` and `docs/MACOS_IPHONE_SETUP.md`.
6. Rollback: unset/leave `HOST_AGENT_CONSOLE_ENABLED` at its default `false`. No schema or state migration is introduced for existing stores (`DeviceConfigStore`, `LocalAccountStore`, `HostIdentityStore` are all read via their existing APIs, unchanged); the new history SQLite file is purely additive and can be deleted with no effect on Host Agent operation.
5. Document mandatory Console startup, loopback-only defaults, and the SSH port-forward recommendation for remote access in `docs/CLOUD_DEPLOYMENT.md` and `docs/MACOS_IPHONE_SETUP.md`.
6. Rollback: deploy a prior Host Agent release if the embedded Console must be removed. No schema or state migration is introduced for existing stores (`DeviceConfigStore`, `LocalAccountStore`, `HostIdentityStore` are all read via their existing APIs, unchanged); the new history SQLite file is purely additive and can be deleted with no effect on Host Agent operation.
## Open Questions
@@ -6,14 +6,14 @@
- Add an embedded, server-rendered local web console to the Host Agent process: plain HTML responses from a lightweight HTTP server (no separate frontend build, no SPA framework), with a handful of endpoints returning small JSON fragments that a few inline `<script>` blocks poll to refresh sections of the page without a full reload.
- Web console covers: status/monitoring (heartbeat/last-seen, enrollment/identity state, registered local devices and their status, current assignment/execution progress, sanitized effective config such as `control_plane_url` and `host_id` with `token` never rendered), local device management (add/edit/remove entries in `storage/device_config.py`'s `DeviceConfigStore`), account settings (change the local account password in place; creating the *first* account remains the job of `device-host-agent setup`), and recent assignment/heartbeat history (a new bounded local log, since the Host Agent does not currently retain any local record of past assignments after reporting results to the control plane).
- New `HostAgentConfig` fields to gate and bind the console: disabled by default, and when enabled defaults to binding `127.0.0.1` only; binding to a non-loopback address is possible but requires an explicit opt-in and is treated as a documented, operator-accepted risk (no built-in TLS or rate limiting — see design.md threat model).
- New `HostAgentConfig` fields to bind and retain console state: the Console starts with every Host Agent and binds `127.0.0.1` by default; binding to a non-loopback address is possible but requires an explicit opt-in and is treated as a documented, operator-accepted risk (no built-in TLS or rate limiting — see design.md threat model).
- Web login reuses the existing `host_agent/local_account.py` PBKDF2 credential (same account as `device-host-agent setup` creates/resets); no second credential store.
- `device-host-agent` gains a new optional dependency on a minimal ASGI/WSGI server library to host the embedded HTTP server; `HostAgentApplication` starts/stops it alongside the existing heartbeat and claim loop.
## Capabilities
### New Capabilities
- `host-agent-local-console`: embedded local-only web UI for the Host Agent covering status monitoring, local device CRUD, local account password change, and bounded recent-assignment/heartbeat history, authenticated against the existing local account and disabled/loopback-bound by default.
- `host-agent-local-console`: embedded local web UI for the Host Agent covering status monitoring, local device CRUD, local account password change, and bounded recent-assignment/heartbeat history, authenticated against the existing local account and loopback-bound by default.
### Modified Capabilities
(none — `host-agent-protocol` covers the outbound cloud protocol and is unaffected; this change only adds a local-only inbound surface)
@@ -22,4 +22,4 @@
- Affected code: `apps/device-host-agent/host_agent/` (new `web` module/package, `app.py` wiring, `config.py` new fields), `apps/device-host-agent/pyproject.toml` (new HTTP server dependency), `storage/device_config.py` (consumed for device CRUD, no schema break expected), new local history storage (new SQLite table or file, scoped to the Host Agent).
- Not affected: `cloud.*`, `apps/cloud-api`, `cloud-console/`, `console/`, `host-agent-protocol` outbound behavior, `openspec/changes/edge-host-self-enrollment` (its CLI-only local-account bootstrap requirement is unchanged and remains the only way to create the *first* account; this change only adds a way to change the password afterward through the web UI).
- Operational impact: a new local listening port on edge devices when explicitly enabled; default-off and loopback-only by default keep the default deployment posture unchanged.
- Operational impact: every Host Agent opens a local listener on startup; loopback-only binding by default limits that new surface to the edge machine itself.
@@ -1,15 +1,11 @@
## ADDED Requirements
### Requirement: The local console is disabled by default and loopback-bound when enabled
The Host Agent SHALL NOT start any local web console listener unless explicitly enabled by configuration, and SHALL bind that listener to a loopback address unless a separate, explicit configuration setting authorizes a non-loopback bind address.
### Requirement: The local console starts with every Host Agent and is loopback-bound by default
The Host Agent SHALL start its local web console listener on every process start and SHALL bind that listener to a loopback address unless a separate, explicit configuration setting authorizes a non-loopback bind address.
#### Scenario: Default configuration starts no console
#### Scenario: Default configuration starts a loopback console
- **WHEN** the Host Agent starts with no console-related configuration set
- **THEN** no local web console listener is started and existing heartbeat/claim behavior is unaffected
#### Scenario: Console enabled with default bind
- **WHEN** the console is enabled without a non-loopback opt-in
- **THEN** the console listener binds only to a loopback address
- **THEN** the local web console listener starts on `127.0.0.1:8765` and existing heartbeat/claim behavior continues alongside it
#### Scenario: Non-loopback bind requested without opt-in
- **WHEN** the console is configured to bind a non-loopback address without the separate non-loopback opt-in setting
@@ -23,7 +19,7 @@ The Host Agent SHALL NOT start any local web console listener unless explicitly
The local web console SHALL authenticate operators against the same local account credential used by the Host Agent's `setup` command, and SHALL NOT introduce a separate credential store.
#### Scenario: No local account exists
- **WHEN** the console is enabled and no local account file is present
- **WHEN** the Console starts and no local account file is present
- **THEN** the console's login page reports that no account exists and directs the operator to run the setup command, without accepting any login attempt
#### Scenario: Valid login
@@ -50,7 +46,7 @@ The console SHALL issue an opaque session token on successful login, SHALL requi
- **THEN** the console rejects the request and makes no change
#### Scenario: Process restart invalidates sessions
- **WHEN** the Host Agent process restarts while the console is enabled
- **WHEN** the Host Agent process restarts while the Console is running
- **THEN** previously issued session tokens are no longer accepted and operators must log in again
### Requirement: Console displays current heartbeat, enrollment, device, and assignment status
@@ -112,7 +108,3 @@ The Host Agent SHALL retain a bounded, local-only history of recent assignment o
#### Scenario: History is bounded
- **WHEN** the number of recorded history entries exceeds the configured retention limit
- **THEN** the oldest entries are pruned so the stored history does not grow unbounded
#### Scenario: History available without console enabled
- **WHEN** the console is disabled
- **THEN** the Host Agent does not record local history and incurs no related overhead
@@ -1,55 +1,55 @@
## 1. Config and dependencies
- [ ] 1.1 Add `fastapi` and `uvicorn[standard]` as explicit direct dependencies in `apps/device-host-agent/pyproject.toml` (versions matching those already pinned in `uv.lock`)
- [ ] 1.2 Add `console_enabled`, `console_bind_host`, `console_port`, `console_allow_non_loopback`, `console_session_ttl_seconds`, `console_history_limit` fields to `HostAgentConfig` in `host_agent/config.py`, plus matching `HOST_AGENT_CONSOLE_*` env vars in `load_host_agent_config`, reusing the existing `_positive_float`/`_positive_int` validators
- [ ] 1.3 Add validation that raises `HostAgentConfigurationError` when `console_bind_host` is non-loopback and `console_allow_non_loopback` is not set
- [ ] 1.4 Add unit tests in `apps/device-host-agent/tests/test_config.py` for defaults, env var parsing, and the non-loopback-without-opt-in rejection
- [x] 1.1 Add `fastapi` and `uvicorn[standard]` as explicit direct dependencies in `apps/device-host-agent/pyproject.toml` (versions matching those already pinned in `uv.lock`)
- [x] 1.2 Add `console_bind_host`, `console_port`, `console_allow_non_loopback`, `console_session_ttl_seconds`, and `console_history_limit` fields to `HostAgentConfig` in `host_agent/config.py`, plus matching `HOST_AGENT_CONSOLE_*` env vars except an enable flag in `load_host_agent_config`, reusing the existing `_positive_float`/`_positive_int` validators
- [x] 1.3 Add validation that raises `HostAgentConfigurationError` when `console_bind_host` is non-loopback and `console_allow_non_loopback` is not set
- [x] 1.4 Add unit tests in `apps/device-host-agent/tests/test_config.py` for defaults, env var parsing, and the non-loopback-without-opt-in rejection
## 2. Shared device-registration helper
- [ ] 2.1 Extract the device-add sequence (`DeviceConfigStore.add`/`set_cloud_device_id` + optional `enrollment_client.enroll_device` + `manager.register_device`) currently inlined in `host_agent/app.py::_configured_device_manager` into a small shared function usable by both startup and the console
- [ ] 2.2 Add a matching shared function for device removal (`DeviceConfigStore.remove` + `manager.unregister_device`)
- [ ] 2.3 Update `_configured_device_manager` to use the extracted add helper; confirm existing `test_app.py` startup tests still pass unchanged
- [x] 2.1 Extract the device-add sequence (`DeviceConfigStore.add`/`set_cloud_device_id` + optional `enrollment_client.enroll_device` + `manager.register_device`) currently inlined in `host_agent/app.py::_configured_device_manager` into a small shared function usable by both startup and the console
- [x] 2.2 Add a matching shared function for device removal (`DeviceConfigStore.remove` + `manager.unregister_device`)
- [x] 2.3 Update `_configured_device_manager` to use the extracted add helper; confirm existing `test_app.py` startup tests still pass unchanged
## 3. Local history store
- [ ] 3.1 Add a new `host_agent/history.py` module with a `ConsoleHistoryStore` backed by a small SQLite file (e.g. `tasks/host_console_history.sqlite3`), supporting `record_assignment(...)`, `record_heartbeat(...)`, and `list_recent(limit)`, pruning beyond `console_history_limit` on write
- [ ] 3.2 Add an optional recorder hook to `AssignmentProcessor.process` (host_agent/processor.py) invoked after a terminal result is reported, no-op when no recorder is configured
- [ ] 3.3 Add an optional recorder hook to `HeartbeatSynchronizer.sync_once` (host_agent/heartbeat.py) invoked after each successful sync, no-op when no recorder is configured
- [ ] 3.4 Unit tests for `ConsoleHistoryStore` (write, prune-on-overflow, ordering) and for the processor/heartbeat recorder hooks firing with the expected data and being skipped when absent
- [x] 3.1 Add a new `host_agent/history.py` module with a `ConsoleHistoryStore` backed by a small SQLite file (e.g. `tasks/host_console_history.sqlite3`), supporting `record_assignment(...)`, `record_heartbeat(...)`, and `list_recent(limit)`, pruning beyond `console_history_limit` on write
- [x] 3.2 Add an optional recorder hook to `AssignmentProcessor.process` (host_agent/processor.py) invoked after a terminal result is reported, no-op when no recorder is configured
- [x] 3.3 Add an optional recorder hook to `HeartbeatSynchronizer.sync_once` (host_agent/heartbeat.py) invoked after each successful sync, no-op when no recorder is configured
- [x] 3.4 Unit tests for `ConsoleHistoryStore` (write, prune-on-overflow, ordering) and for the processor/heartbeat recorder hooks firing with the expected data and being skipped when absent
## 4. Session and authentication
- [ ] 4.1 Add `host_agent/web/auth.py` with an in-memory session store (opaque token → session state with expiry), login verification against `LocalAccountStore`, and CSRF token issuance/validation bound to the session
- [ ] 4.2 Implement session cookie handling (`HttpOnly`, `SameSite=Strict`, `Secure` when bind host is non-loopback) and sliding expiry per `console_session_ttl_seconds`
- [ ] 4.3 Implement an auth dependency/middleware that redirects unauthenticated requests to `/login` and rejects mutating requests lacking a valid CSRF token
- [ ] 4.4 Unit tests: successful login, wrong password, no-account-yet state, session expiry, CSRF rejection on a mutating route, redirect-to-login for an unauthenticated GET
- [x] 4.1 Add `host_agent/web/auth.py` with an in-memory session store (opaque token → session state with expiry), login verification against `LocalAccountStore`, and CSRF token issuance/validation bound to the session
- [x] 4.2 Implement session cookie handling (`HttpOnly`, `SameSite=Strict`, `Secure` when bind host is non-loopback) and sliding expiry per `console_session_ttl_seconds`
- [x] 4.3 Implement an auth dependency/middleware that redirects unauthenticated requests to `/login` and rejects mutating requests lacking a valid CSRF token
- [x] 4.4 Unit tests: successful login, wrong password, no-account-yet state, session expiry, CSRF rejection on a mutating route, redirect-to-login for an unauthenticated GET
## 5. Console pages and routes
- [ ] 5.1 Add `host_agent/web/app.py` building a FastAPI sub-application with hand-written HTML responses (f-string templates + a shared `escape()` helper for every interpolated value) for: `/login`, `/` (status dashboard), `/devices`, `/account`, `/history`
- [ ] 5.2 Implement `/login` (GET form, POST verify+establish session) per spec scenarios, including the "no local account exists" state
- [ ] 5.3 Implement the status dashboard: last heartbeat outcome/time, enrollment/identity state, device list with status, current assignment/execution state, sanitized effective config (no token/password rendered); add a small JSON status-fragment endpoint polled via inline `fetch()` for refresh without full reload
- [ ] 5.4 Implement `/devices`: list, add, edit, remove forms wired to the section-2 shared helpers, taking effect on the live `DeviceManager` immediately
- [ ] 5.5 Implement `/account`: change-password form requiring current password re-entry, calling `LocalAccountStore.create` (or an equivalent update path) only after verifying the current credential
- [ ] 5.6 Implement `/history`: read-only table of recent assignment/heartbeat entries from `ConsoleHistoryStore`
- [ ] 5.7 Implement `/logout` (CSRF-protected POST) invalidating the session
- [x] 5.1 Add `host_agent/web/app.py` building a FastAPI sub-application with hand-written HTML responses (f-string templates + a shared `escape()` helper for every interpolated value) for: `/login`, `/` (status dashboard), `/devices`, `/account`, `/history`
- [x] 5.2 Implement `/login` (GET form, POST verify+establish session) per spec scenarios, including the "no local account exists" state
- [x] 5.3 Implement the status dashboard: last heartbeat outcome/time, enrollment/identity state, device list with status, current assignment/execution state, sanitized effective config (no token/password rendered); add a small JSON status-fragment endpoint polled via inline `fetch()` for refresh without full reload
- [x] 5.4 Implement `/devices`: list, add, edit, remove forms wired to the section-2 shared helpers, taking effect on the live `DeviceManager` immediately
- [x] 5.5 Implement `/account`: change-password form requiring current password re-entry, calling `LocalAccountStore.create` (or an equivalent update path) only after verifying the current credential
- [x] 5.6 Implement `/history`: read-only table of recent assignment/heartbeat entries from `ConsoleHistoryStore`
- [x] 5.7 Implement `/logout` (CSRF-protected POST) invalidating the session
## 6. Lifecycle wiring
- [ ] 6.1 In `host_agent/app.py::create_application`, construct the console app, session store, and `ConsoleHistoryStore` only when `config.console_enabled`, and wire the recorder hooks from section 3 into the constructed `AssignmentProcessor`/`HeartbeatSynchronizer`
- [ ] 6.2 In `HostAgentApplication.run_async`, start a `uvicorn.Server` task (bound to `console_bind_host`/`console_port`, `install_signal_handlers=False`) alongside the heartbeat task when the console is configured, and stop it in the existing `finally` shutdown sequence
- [ ] 6.3 Integration test exercising the full lifecycle with the console enabled: process starts, console responds on the configured loopback port, process shuts down cleanly and stops the console server
- [ ] 6.4 Integration test confirming that with the console left at its default (disabled), no listening socket is opened and existing `test_app.py`/`test_e2e.py` behavior is unaffected
- [x] 6.1 In `host_agent/app.py::create_application`, always construct the console app, session store, and `ConsoleHistoryStore`, and wire the recorder hooks from section 3 into the constructed `AssignmentProcessor`/`HeartbeatSynchronizer`
- [x] 6.2 In `HostAgentApplication.run_async`, start a `uvicorn.Server` task (bound to `console_bind_host`/`console_port`, `install_signal_handlers=False`) alongside the heartbeat task and stop it in the existing `finally` shutdown sequence
- [x] 6.3 Integration test exercising the full lifecycle: process starts, console responds on the configured loopback port, process shuts down cleanly and stops the console server
- [x] 6.4 Integration test confirming that the Console is constructed by default and existing `test_app.py`/`test_e2e.py` behavior is unaffected
## 7. Documentation
- [ ] 7.1 Document the new `HOST_AGENT_CONSOLE_*` environment variables, default-off/loopback-only posture, and the SSH port-forward recommendation for remote access in `docs/CLOUD_DEPLOYMENT.md`
- [ ] 7.2 Add a short section to `docs/MACOS_IPHONE_SETUP.md` describing how to enable the console on an edge machine and what it shows
- [x] 7.1 Document the `HOST_AGENT_CONSOLE_*` environment variables excluding an enable flag, mandatory loopback-bound startup, and the SSH port-forward recommendation for remote access in `docs/CLOUD_DEPLOYMENT.md`
- [x] 7.2 Add a short section to `docs/MACOS_IPHONE_SETUP.md` describing mandatory Console startup on an edge machine and what it shows
## 8. Validation
- [ ] 8.1 Run `uv run --package device-host-agent pytest` (full package suite) and the root non-integration suite; confirm no regressions
- [ ] 8.2 Run Ruff check/format and `python -m compileall` over the changed files
- [x] 8.1 Run `uv run --package device-host-agent pytest` (full package suite) and the root non-integration suite; confirm no regressions
- [x] 8.2 Run Ruff check/format and `python -m compileall` over the changed files
- [ ] 8.3 Manually verify in a browser: login, status dashboard auto-refresh, add/edit/remove a device, change password, view history, logout, and confirm the console refuses to bind non-loopback without the opt-in flag
- [ ] 8.4 Run `openspec validate host-agent-local-console --strict` and confirm it passes
- [x] 8.4 Run `openspec validate host-agent-local-console --strict` and confirm it passes