agentic-mobile-control/openspec/changes/archive/2026-07-14-host-agent-dependency-supervisor/design.md

## Context

The Host Agent (`apps/device-host-agent/host_agent/`) is a headless outbound worker: `HostAgentApplication.run_async()` (`app.py:41-90`) runs a heartbeat loop and a claim/execute loop, and (since `host-agent-local-console`) an embedded local web console bound to `127.0.0.1:8765` by default. None of this touches Appium or the local Runtime API.

Real device control depends on two processes the Host Agent does not manage today:
- **Appium server**, default `http://127.0.0.1:4723` (`docs/MACOS_IPHONE_SETUP.md:209`, `driver/wda_driver.py:12`, `driver/android_driver.py:22`). `Driver.connect()` opens a `webdriver.Remote` session against it; on failure it raises `DeviceOfflineError`, which `DeviceManager.connect()` (`device/manager.py:68-108`) catches and turns into device status `"offline"`/`"error"` — this is exactly what happened in the live incident that motivated this change: Appium wasn't running, the device stayed offline, and every assigned task queued forever with no error surfaced anywhere.
- **Local Runtime API** (`api.rest:create_app`, default `127.0.0.1:8000`), a fully separate FastAPI app/process used to inspect device/task/world state locally. It shares no runtime coupling with the Host Agent process today ("Runtime 不依赖/打包 cloud 或 application" — established repo convention) and this change does not introduce any.

Appium already owns the entire WDA session lifecycle end to end (`docs/MACOS_IPHONE_SETUP.md:176`, `:455` — "本项目会让 Appium 的 XCUITest Driver 创建和管理 WDA session"; "常规单机使用优先让 Appium 管理 WDA，不要一开始就引入 iproxy 或手工 WDA 生命周期"). Any supervision the Host Agent adds must sit *outside* that boundary — it manages whether the Appium **process** is running, never what Appium does internally with WDA.

## Goals / Non-Goals

**Goals:**
- Let the Host Agent optionally start Appium and the local Runtime API as plain subprocesses when they aren't already running, so a forgotten manual step turns into "it just works" instead of "device silently offline forever."
- Make the current state (spawned vs. adopted-existing vs. failed-to-start) visible in the Host Agent's own log/console, so this remains debuggable rather than adding a second silent-failure mode on top of the one it fixes.
- Never conflict with an already-running instance: detect and adopt (read-only) rather than double-spawn or steal a port.

**Non-Goals:**
- No change to Appium's ownership of WDA session lifecycle — the supervisor never touches WDA directly, never passes `webDriverAgentUrl`, never signs/builds anything.
- No Docker/Compose story. `compose.yaml`/`compose.deploy.yaml` are untouched; this is native-process-only, scoped to the macOS single-machine real-device workflow described in `docs/MACOS_IPHONE_SETUP.md`.
- No change to `driver/registry.py`, `driver/wda_driver.py`, `driver/android_driver.py`, or the outbound cloud protocol (`host-agent-protocol`).
- No process supervision beyond these two known dependencies (not a general-purpose process manager).
- No remote/cloud configuration of this feature — it's a local opt-in, matching how `host_agent/config.py` is already entirely env-var driven per-Host.

## Decisions

**1. Default: entire supervisor is off (`HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED=false`).**
Unlike `host-agent-local-console` (which is now forced-on because it's a passive, loopback-bound, low-risk read/manage surface), this feature actively spawns and can restart external processes — including, in the crash-restart path, potentially interrupting an in-flight Appium session/WDA connection if a spawned Appium process is killed and restarted mid-task. That's a materially different risk profile from serving a status page, so it defaults off and is opt-in per Host, consistent with how `AI_PLANNER_ENABLED` defaulting decisions in this repo have been made deliberately per-surface rather than blanket. Each of the two dependencies (Appium, Runtime) also has its own enable flag nested under the top-level flag, so an operator can supervise Appium only, Runtime only, or both.

**2. Adopt-don't-fight: TCP connect + HTTP health probe before spawn.**
Before spawning either process, the supervisor does a plain TCP connect to `(host, port)`. If something answers, it issues the same health check used post-spawn (Appium: `GET /status` expects HTTP 200 with `ready`/build JSON per `docs/MACOS_IPHONE_SETUP.md:218`; Runtime: `GET /health` or equivalent existing endpoint on `api.rest`). If the health check passes, the supervisor logs "adopted existing instance at host:port" and does **not** spawn a subprocess, does **not** track it as "ours," and will never kill or restart it. If the port is occupied but the health check fails (something else is listening, or it's an unhealthy/half-started instance), the supervisor logs an error for that dependency and leaves it un-started rather than guessing — port conflicts get surfaced, not silently resolved. This directly implements the concern raised during proposal review that an already-running manually-started Appium must not be duplicated or fought over.

**3. Only supervisor-spawned processes are restarted; adopted processes are never touched.**
This is the direct mitigation for the WDA-session-interruption risk: an operator who already has their own long-lived Appium running (e.g., manually, or from a previous supervised run in another terminal) is never at risk of the Host Agent killing/restarting it. Restart-on-crash only applies to a `subprocess.Popen` handle the supervisor itself holds.

**4. Crash-restart uses capped exponential backoff with a per-run attempt ceiling.**
Backoff schedule (e.g. 1s, 2s, 4s, 8s, capped at 30s) with a max attempt count per dependency per Host Agent process lifetime (e.g. 5); once exhausted, the supervisor logs that it's giving up on that dependency and stops trying, rather than restart-looping forever. This directly answers the "限流/退避避免死循环重启风暴" concern. The counter resets only on a fresh Host Agent process start, not periodically — a persistently-crashing Appium install shouldn't be retried indefinitely in the background.

**5. Spawn mechanism: stdlib `subprocess.Popen`, no new dependency.**
Appium is invoked as `appium --address <host> --port <port>` (assumes `appium` is already on `PATH`, exactly as the manual instructions already require — this change does not install or vendor Appium). Runtime is invoked as the existing `uvicorn api.rest:create_app --factory --host <host> --port <port>` command, run as a subprocess rather than in-process, preserving the existing "Runtime is a fully separate process" boundary — this is process supervision, not a code-level dependency between the two packages. Both children's stdout/stderr are captured and forwarded into the Host Agent's own logging (prefixed per-process) so a crash's root cause doesn't require the operator to have had a spare terminal open.

**6. Config fields follow the existing `HostAgentConfig`/`HOST_AGENT_*` convention.**
New fields on `HostAgentConfig` (mirroring the `console_*` fields added by `host-agent-local-console`):
- `dependency_supervisor_enabled: bool` ← `HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED` (default `false`)
- `appium_supervised: bool` ← `HOST_AGENT_APPIUM_SUPERVISED` (default `false`)
- `appium_host: str` ← `HOST_AGENT_APPIUM_HOST` (default `127.0.0.1`)
- `appium_port: int` ← `HOST_AGENT_APPIUM_PORT` (default `4723`, matching the documented default)
- `runtime_supervised: bool` ← `HOST_AGENT_RUNTIME_SUPERVISED` (default `false`)
- `runtime_host: str` ← `HOST_AGENT_RUNTIME_HOST` (default `127.0.0.1`)
- `runtime_port: int` ← `HOST_AGENT_RUNTIME_PORT` (default `8000`, matching the documented default)
- `dependency_restart_max_attempts: int` ← `HOST_AGENT_DEPENDENCY_RESTART_MAX_ATTEMPTS` (default `5`)

**7. Wiring point: alongside the existing heartbeat/claim loop, not inside it.**
`HostAgentApplication` starts the supervisor (if enabled) before starting the heartbeat loop and `connect_devices()` pass, and stops it during shutdown alongside the existing console/heartbeat teardown in `run_async()` — so a supervised Appium is up before the first `DeviceManager.connect()` attempt in the same startup sequence that already exists.

## Risks / Trade-offs

- **[Risk] A supervisor-spawned Appium restart mid-task interrupts an active WDA session, failing whatever task is running.** → Mitigation: restart only triggers on the spawned process actually exiting (crash), not on a health-check hiccup; the underlying task still fails cleanly through the existing `DeviceOfflineError` path (same as today when Appium isn't running at all) rather than in some new/undefined way. This is a pre-existing failure mode (Appium can already crash unsupervised) — the difference is the Host Agent now also tries to bring it back instead of leaving the device offline until a human notices.
- **[Risk] Supervising the Runtime API blurs the "fully independent process" boundary this repo has deliberately maintained.** → Mitigation: it's spawned as a subprocess via its existing CLI entry point, not imported/mounted in-process — no new Python-level dependency from `device-host-agent` onto `api.rest`, no shared code path. The boundary that matters (no import coupling, no shared FastAPI app) is preserved; only process *lifecycle* (start/stop timing) is now optionally shared.
- **[Risk] `appium` not on `PATH`, or wrong Appium driver installed (per `docs/MACOS_IPHONE_SETUP.md` xcuitest/uiautomator2 driver setup).** → Mitigation: spawn failure (e.g. `FileNotFoundError` from `subprocess.Popen`) is logged clearly as a dependency-supervisor error, distinct from a normal Appium crash, so the operator isn't left thinking Appium crashed when it never started.
- **[Risk] Health-check false positive: something unrelated happens to be listening on 4723/8000.** → Mitigation: the health check requires the actual expected response shape (Appium `/status` JSON, Runtime health endpoint), not just "port is open" — an unrelated listener fails the check and is treated as a conflict, not adopted.
- **[Trade-off] This only covers macOS native real-device usage; Docker/CI paths get no benefit.** → Accepted per proposal scope — those paths don't have this failure mode in the first place (no interactive multi-terminal manual step to forget).

## Migration Plan

- Fully additive and opt-in: existing Host Agent deployments/configs are unaffected with zero config changes (all new fields default to off/disabled).
- Rollout is per-Host: an operator turns it on by setting `HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED=true` (and the relevant `*_SUPERVISED` flags) in their existing `.env`/environment, no schema or protocol changes involved.
- Rollback: unset the env vars (or set back to `false`) and restart the Host Agent — reverts to today's fully-manual behavior with no residual state (the supervisor holds no persistent storage).
- Documentation: `docs/MACOS_IPHONE_SETUP.md` gains a section presenting supervised mode as an optional alternative alongside the existing manual steps, which remain the documented default path.

## Open Questions

- Should `dependency_restart_max_attempts` exhaustion eventually surface in the local console (`host-agent-local-console`) status page rather than only in logs? Left for a future change if operators find log-only insufficient in practice.
- Android/UiAutomator2 real-device validation for this supervisor is out of scope for the same reason `android-driver` itself shipped without real-device validation — should be revisited once Android hardware validation happens.