agentic-mobile-control/openspec/changes/host-agent-dependency-supervisor/design.md

Context

The Host Agent (apps/device-host-agent/host_agent/) is a headless outbound worker: HostAgentApplication.run_async() (app.py:41-90) runs a heartbeat loop and a claim/execute loop, and (since host-agent-local-console) an embedded local web console bound to 127.0.0.1:8765 by default. None of this touches Appium or the local Runtime API.

Real device control depends on two processes the Host Agent does not manage today:

• Appium server, default http://127.0.0.1:4723 (docs/MACOS_IPHONE_SETUP.md:209, driver/wda_driver.py:12, driver/android_driver.py:22). Driver.connect() opens a webdriver.Remote session against it; on failure it raises DeviceOfflineError, which DeviceManager.connect() (device/manager.py:68-108) catches and turns into device status "offline"/"error" — this is exactly what happened in the live incident that motivated this change: Appium wasn't running, the device stayed offline, and every assigned task queued forever with no error surfaced anywhere.
• Local Runtime API (api.rest:create_app, default 127.0.0.1:8000), a fully separate FastAPI app/process used to inspect device/task/world state locally. It shares no runtime coupling with the Host Agent process today ("Runtime 不依赖/打包 cloud 或 application" — established repo convention) and this change does not introduce any.

Appium already owns the entire WDA session lifecycle end to end (docs/MACOS_IPHONE_SETUP.md:176, :455 — "本项目会让 Appium 的 XCUITest Driver 创建和管理 WDA session"; "常规单机使用优先让 Appium 管理 WDA，不要一开始就引入 iproxy 或手工 WDA 生命周期"). Any supervision the Host Agent adds must sit outside that boundary — it manages whether the Appium process is running, never what Appium does internally with WDA.

Goals / Non-Goals

Goals:

• Let the Host Agent optionally start Appium and the local Runtime API as plain subprocesses when they aren't already running, so a forgotten manual step turns into "it just works" instead of "device silently offline forever."
• Make the current state (spawned vs. adopted-existing vs. failed-to-start) visible in the Host Agent's own log/console, so this remains debuggable rather than adding a second silent-failure mode on top of the one it fixes.
• Never conflict with an already-running instance: detect and adopt (read-only) rather than double-spawn or steal a port.

Non-Goals:

• No change to Appium's ownership of WDA session lifecycle — the supervisor never touches WDA directly, never passes webDriverAgentUrl, never signs/builds anything.
• No Docker/Compose story. compose.yaml/compose.deploy.yaml are untouched; this is native-process-only, scoped to the macOS single-machine real-device workflow described in docs/MACOS_IPHONE_SETUP.md.
• No change to driver/registry.py, driver/wda_driver.py, driver/android_driver.py, or the outbound cloud protocol (host-agent-protocol).
• No process supervision beyond these two known dependencies (not a general-purpose process manager).
• No remote/cloud configuration of this feature — it's a local opt-in, matching how host_agent/config.py is already entirely env-var driven per-Host.

Decisions

1. Default: entire supervisor is off (HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED=false). Unlike host-agent-local-console (which is now forced-on because it's a passive, loopback-bound, low-risk read/manage surface), this feature actively spawns and can restart external processes — including, in the crash-restart path, potentially interrupting an in-flight Appium session/WDA connection if a spawned Appium process is killed and restarted mid-task. That's a materially different risk profile from serving a status page, so it defaults off and is opt-in per Host, consistent with how AI_PLANNER_ENABLED defaulting decisions in this repo have been made deliberately per-surface rather than blanket. Each of the two dependencies (Appium, Runtime) also has its own enable flag nested under the top-level flag, so an operator can supervise Appium only, Runtime only, or both.

2. Adopt-don't-fight: TCP connect + HTTP health probe before spawn. Before spawning either process, the supervisor does a plain TCP connect to (host, port). If something answers, it issues the same health check used post-spawn (Appium: GET /status expects HTTP 200 with ready/build JSON per docs/MACOS_IPHONE_SETUP.md:218; Runtime: GET /health or equivalent existing endpoint on api.rest). If the health check passes, the supervisor logs "adopted existing instance at host:port" and does not spawn a subprocess, does not track it as "ours," and will never kill or restart it. If the port is occupied but the health check fails (something else is listening, or it's an unhealthy/half-started instance), the supervisor logs an error for that dependency and leaves it un-started rather than guessing — port conflicts get surfaced, not silently resolved. This directly implements the concern raised during proposal review that an already-running manually-started Appium must not be duplicated or fought over.

3. Only supervisor-spawned processes are restarted; adopted processes are never touched. This is the direct mitigation for the WDA-session-interruption risk: an operator who already has their own long-lived Appium running (e.g., manually, or from a previous supervised run in another terminal) is never at risk of the Host Agent killing/restarting it. Restart-on-crash only applies to a subprocess.Popen handle the supervisor itself holds.

4. Crash-restart uses capped exponential backoff with a per-run attempt ceiling. Backoff schedule (e.g. 1s, 2s, 4s, 8s, capped at 30s) with a max attempt count per dependency per Host Agent process lifetime (e.g. 5); once exhausted, the supervisor logs that it's giving up on that dependency and stops trying, rather than restart-looping forever. This directly answers the "限流/退避避免死循环重启风暴" concern. The counter resets only on a fresh Host Agent process start, not periodically — a persistently-crashing Appium install shouldn't be retried indefinitely in the background.

5. Spawn mechanism: stdlib subprocess.Popen, no new dependency. Appium is invoked as appium --address <host> --port <port> (assumes appium is already on PATH, exactly as the manual instructions already require — this change does not install or vendor Appium). Runtime is invoked as the existing uvicorn api.rest:create_app --factory --host <host> --port <port> command, run as a subprocess rather than in-process, preserving the existing "Runtime is a fully separate process" boundary — this is process supervision, not a code-level dependency between the two packages. Both children's stdout/stderr are captured and forwarded into the Host Agent's own logging (prefixed per-process) so a crash's root cause doesn't require the operator to have had a spare terminal open.

6. Config fields follow the existing HostAgentConfig/HOST_AGENT_* convention. New fields on HostAgentConfig (mirroring the console_* fields added by host-agent-local-console):

• dependency_supervisor_enabled: bool ← HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED (default false)
• appium_supervised: bool ← HOST_AGENT_APPIUM_SUPERVISED (default false)
• appium_host: str ← HOST_AGENT_APPIUM_HOST (default 127.0.0.1)
• appium_port: int ← HOST_AGENT_APPIUM_PORT (default 4723, matching the documented default)
• runtime_supervised: bool ← HOST_AGENT_RUNTIME_SUPERVISED (default false)
• runtime_host: str ← HOST_AGENT_RUNTIME_HOST (default 127.0.0.1)
• runtime_port: int ← HOST_AGENT_RUNTIME_PORT (default 8000, matching the documented default)
• dependency_restart_max_attempts: int ← HOST_AGENT_DEPENDENCY_RESTART_MAX_ATTEMPTS (default 5)

7. Wiring point: alongside the existing heartbeat/claim loop, not inside it. HostAgentApplication starts the supervisor (if enabled) before starting the heartbeat loop and connect_devices() pass, and stops it during shutdown alongside the existing console/heartbeat teardown in run_async() — so a supervised Appium is up before the first DeviceManager.connect() attempt in the same startup sequence that already exists.

Risks / Trade-offs

• [Risk] A supervisor-spawned Appium restart mid-task interrupts an active WDA session, failing whatever task is running. → Mitigation: restart only triggers on the spawned process actually exiting (crash), not on a health-check hiccup; the underlying task still fails cleanly through the existing DeviceOfflineError path (same as today when Appium isn't running at all) rather than in some new/undefined way. This is a pre-existing failure mode (Appium can already crash unsupervised) — the difference is the Host Agent now also tries to bring it back instead of leaving the device offline until a human notices.
• [Risk] Supervising the Runtime API blurs the "fully independent process" boundary this repo has deliberately maintained. → Mitigation: it's spawned as a subprocess via its existing CLI entry point, not imported/mounted in-process — no new Python-level dependency from device-host-agent onto api.rest, no shared code path. The boundary that matters (no import coupling, no shared FastAPI app) is preserved; only process lifecycle (start/stop timing) is now optionally shared.
• [Risk] appium not on PATH, or wrong Appium driver installed (per docs/MACOS_IPHONE_SETUP.md xcuitest/uiautomator2 driver setup). → Mitigation: spawn failure (e.g. FileNotFoundError from subprocess.Popen) is logged clearly as a dependency-supervisor error, distinct from a normal Appium crash, so the operator isn't left thinking Appium crashed when it never started.
• [Risk] Health-check false positive: something unrelated happens to be listening on 4723/8000. → Mitigation: the health check requires the actual expected response shape (Appium /status JSON, Runtime health endpoint), not just "port is open" — an unrelated listener fails the check and is treated as a conflict, not adopted.
• [Trade-off] This only covers macOS native real-device usage; Docker/CI paths get no benefit. → Accepted per proposal scope — those paths don't have this failure mode in the first place (no interactive multi-terminal manual step to forget).

Migration Plan

• Fully additive and opt-in: existing Host Agent deployments/configs are unaffected with zero config changes (all new fields default to off/disabled).
• Rollout is per-Host: an operator turns it on by setting HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED=true (and the relevant *_SUPERVISED flags) in their existing .env/environment, no schema or protocol changes involved.
• Rollback: unset the env vars (or set back to false) and restart the Host Agent — reverts to today's fully-manual behavior with no residual state (the supervisor holds no persistent storage).
• Documentation: docs/MACOS_IPHONE_SETUP.md gains a section presenting supervised mode as an optional alternative alongside the existing manual steps, which remain the documented default path.

Open Questions

• Should dependency_restart_max_attempts exhaustion eventually surface in the local console (host-agent-local-console) status page rather than only in logs? Left for a future change if operators find log-only insufficient in practice.
• Android/UiAutomator2 real-device validation for this supervisor is out of scope for the same reason android-driver itself shipped without real-device validation — should be revisited once Android hardware validation happens.