docs(openspec): correct host-agent-single-instance-lock's Why section
This change was originally proposed as the root-cause fix for the
2026-07-14 DeviceNotFoundError incident. That diagnosis was wrong: it was
subsequently confirmed only one Host Agent process was running at the time,
ruling out the duplicate-process precondition this change addresses. The
actual root cause was execution.py's create_task_runner() omitting manager=
when wiring TaskRunner (see 08cef7c). Reframe the Why section: this change
stands on its own as independent duplicate-process hardening, not as a fix
for an incident it turned out not to have caused.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,6 +1,8 @@
|
||||
## Why
|
||||
|
||||
A production task dispatch failed with `DeviceNotFoundError: unknown device device-8967d09f0d5f4cf49f7361a9f0dcb0ce` (host `host-311f68...`, 2026-07-14) even though the Host Agent's own console showed that same device as `connected`. Root-cause tracing confirmed: `DeviceManager` (`device/manager.py`) is in-process, in-memory, and per-process, with no persistence and no reactive removal. `TaskScheduler.assign()` (`cloud/scheduler.py`) dispatches to a `host_id` based on a periodically-synced `DevicePool` snapshot, with no concept of *which OS process* is currently answering that host's long-poll. If two Host Agent processes ever run concurrently under the same host identity — e.g. a supervisor starting a replacement before a hung previous instance has fully exited, or an operator accidentally leaving a second instance running in another terminal/tmux pane — both heartbeat and both claim under the same `host_id`, and a claim can land on whichever instance answers next, including one whose own in-memory `DeviceManager` never registered the device. Nothing in the codebase today prevents this: a repo-wide search confirms no pidfile, lockfile, or `flock`-style guard exists anywhere in `apps/device-host-agent`, and both `identity_path` and the device-config store default to process-working-directory-relative paths, which makes an accidental duplicate launch easy to trigger unnoticed.
|
||||
Duplicate Host Agent processes sharing the same host identity are a real, unguarded risk: a repo-wide search confirms no pidfile, lockfile, or `flock`-style guard exists anywhere in `apps/device-host-agent`, and both `identity_path` and the device-config store default to process-working-directory-relative paths, which makes an accidental duplicate launch easy to trigger unnoticed (e.g. a supervisor starting a replacement before a hung previous instance has fully exited, or an operator accidentally leaving a second instance running in another terminal/tmux pane). If it ever happens, the failure mode is severe and confusing: `DeviceManager` (`device/manager.py`) is in-process, in-memory, and per-process, with no persistence and no reactive removal, while `TaskScheduler.assign()` (`cloud/scheduler.py`) dispatches to a `host_id` based on a periodically-synced `DevicePool` snapshot with no concept of *which OS process* is currently answering that host's long-poll — both instances would heartbeat and claim under the same `host_id`, and a claim could land on whichever instance answers next, including one whose own in-memory `DeviceManager` never registered the device.
|
||||
|
||||
This change was originally proposed as the root-cause fix for a production incident (`DeviceNotFoundError: unknown device device-8967d09f0d5f4cf49f7361a9f0dcb0ce`, host `host-311f68...`, 2026-07-14). That diagnosis was **wrong**: it was subsequently confirmed that only one Host Agent process was running at the time, which rules out the duplicate-process precondition this change addresses. The actual root cause was `create_task_runner()` (`apps/device-host-agent/host_agent/execution.py`) omitting `manager=` when wiring `TaskRunner`'s `observer`/`screenshot_provider`, so both silently fell back to the process-global `DEFAULT_MANAGER` singleton instead of the Host Agent's real, device-populated `DeviceManager` — fixed directly in code, without an OpenSpec proposal, since it was a small, fully-diagnosed, single-file wiring bug. This proposal is kept anyway, on its own merits: duplicate-process protection is a legitimate, independent hardening measure against a real gap the investigation surfaced along the way, not a fix for an incident it turned out not to have caused.
|
||||
|
||||
## What Changes
|
||||
|
||||
|
||||
Reference in New Issue
Block a user