q792602257andClaude Opus 4.6
2f8f4a36c6
Design for mounting a Streamable HTTP MCP server inside the host-agent process so Hermes Agent (or any MCP client) can drive devices directly. Reuses the existing console FastAPI + uvicorn on port 8765, adds bearer- token auth, per-device session-level locks with 60s TTL, and cloud coordination via a new heartbeat field. Cloud scheduler skips devices reported as MCP-busy. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Device Agent Runtime
Device Agent Runtime is a device-agnostic runtime for LLM-driven automation. It gives agents a stable way to observe, decide, and act against real devices through a small set of domain models, driver contracts, tools, perception providers, and runtime orchestration.
iPhone automation through WebDriverAgent/Appium is the first driver, not the platform boundary. Future drivers can target Android, browsers, desktop environments, or other device surfaces without changing the runtime's core contracts.
Current Shape
core/: shared domain models and runtime errors.driver/: theDrivercontract, concrete driver adapters, and driver-type registry.device/: device lifecycle and active driver management.tools/: device capabilities exposed to Runtime and adapter layers.perception/: screen-to-Sceneperception behindPerceptionProvider.runtime/: planning and execution orchestration.api/: MCP and supporting integration adapters.storage/: timeline, task, and device configuration persistence.packages/cloud-platform/: cloud scheduling, device pooling, plugins, and the Python cloud SDK as thedevice-cloud-platformworkspace member.apps/cloud-api/: deployable authenticated Cloud Control Plane with PostgreSQL/SQLite persistence, scheduling, leases, and health endpoints.apps/device-host-agent/: outbound Host Agent that synchronizes configured devices and executes leased tasks through the existing Runtime/workflow.
Python Workspace
The repository uses a uv workspace with one committed lockfile. From the repository root, synchronize every Python member with:
uv sync --locked --all-packages
Run the complete local test suite in a workspace environment:
uv run --all-packages pytest -m "not integration"
Select one member when running package-specific commands:
uv run --package device-agent-runtime python -c "import runtime"
uv run --package device-cloud-platform python -c "import cloud"
uv run --package device-cloud-api device-cloud-api --help
uv run --package device-host-agent device-host-agent --help
uv build --package device-agent-runtime
uv build --package device-cloud-platform
Runtime is an in-process execution library, not a standalone HTTP service. The
Host Agent console at http://127.0.0.1:8765/tasks is the authenticated
operator view for the tasks that actually execute on that Host, including
per-step screenshots, OCR observations, and UI-tree results. The Cloud Console
remains the fleet-level view for dispatch status and Cloud-proxy planner history.
Project Direction
The durable roadmap is in docs/ROADMAP.md. The architecture invariants future changes must preserve are in docs/CONSTITUTION.md.
Operator Guides
- Cloud Control Plane deployment: run local SQLite or deployed PostgreSQL, configure credentials and Runtime AI planning, and perform orderly shutdown or rollback.
- macOS migration and real iPhone setup: install Xcode, Appium/XCUITest, sign WebDriverAgent, verify a real device, and run a connected Host Agent.