100 Commits
Author SHA1 Message Date
q792602257 433ab41f95 Merge branch 'worktree-host-agent-mcp-server' — Host-Agent MCP Server
Tests / Test apps.device-host-agent.tests.test_mcp_token.test_load_or_create_concurrent_calls_do_not_corrupt failed
Adds a Streamable HTTP MCP server (mount /mcp, port 8765) to the
device-host-agent process so Hermes Agent (or any MCP client) can
drive devices directly, coexisting with the Cloud Control Plane
worker path. Per-device session-level locking with 20s TTL,
independent bearer-token auth, and bidirectional cloud ↔ MCP
coordination via a new heartbeat field.

Implementation:
- 4 new modules (mcp_token, mcp_lock, web/mcp_auth, web/mcp)
- Console mount at /mcp with bearer auth sub-app
- Cloud heartbeat payload + scheduler skip MCP-busy devices
- AssignmentExecutor fail-fast reverse check
- CLI mcp-token subcommand
- docs/MCP_INTEGRATION.md + MACOS_IPHONE_SETUP.md section

Spec: docs/superpowers/specs/2026-07-21-host-agent-mcp-server-design.md
Plan: docs/superpowers/plans/2026-07-21-host-agent-mcp-server.md

18 implementation commits ( Tasks 1-15 + final fix wave).
Spec/plan cherry-picks are detected as already-applied via patch-id.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

# Conflicts:
#	docs/superpowers/specs/2026-07-21-host-agent-mcp-server-design.md
2026-07-21 17:00:29 +08:00
q792602257andClaude Opus 4.6 70e0624a47 fix(host-agent): align MCP integration with mcp SDK 1.28.1 realities
Three final-review deviations closed:

I1 (session-end release): mcp SDK 1.28.1 exposes no per-session
shutdown callback (only a server-level lifespan). Lower the
McpBusyTracker default TTL from 60s to 20s and update spec §6.5,
Q5/R3, D9, and docs/MCP_INTEGRATION.md concurrency section to
document the TTL-only recovery path. 20s is short enough to recover
within one 30s heartbeat interval but long enough that an active
session does not lose its lease during normal operator pauses.

I2 (JSON-RPC error shape): FastMCP Tool.run wraps every non-
UrlElicitationRequiredError exception (including McpError with typed
ErrorData) into ToolError, which the lowlevel call_tool handler
serializes as CallToolResult(isError=true, content=[TextContent(...)]).
There is no public path that surfaces JSON-RPC -32000 with structured
data.busy_owner from a tool call site. Update spec §7 error matrix
and docs/MCP_INTEGRATION.md error table to document the actual wire
shape; busy_owner now lives in the text content.

I3 (typing): mcp_server: Any = None -> FastMCP | None = None via
TYPE_CHECKING, keeping the mcp import lazy (matches precedent
elsewhere in the codebase) while adding static type checking at the
create_console_app boundary.

Tests added (4):
- test_default_ttl_is_20_seconds — locks I1's new default TTL
- test_default_ttl_recovers_dead_session_within_one_window — locks
  I1's recovery semantics (lease sweeped on next read after 20s)
- test_busy_error_wire_shape_is_calltoolresult_iserror — pins I2's
  wire envelope via Tool.run + lowlevel Server._make_error_result
- test_busy_error_text_includes_cloud_assignment_owner — same for
  the cloud_assignment busy_owner branch

Full non-integration suite: 697 passed / 54 deselected (was 693 / 54).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-21 16:25:35 +08:00
q792602257andClaude Opus 4.6 e69cea0245 style: ruff format after MCP server integration
Reformat the files touched by Tasks 1-14 of the host-agent MCP server
plan. No semantic changes; pre-existing format issues in unrelated
files (test_templates, test_skill_sync_wiring, 0010_skill_management,
test_skill_catalog_mcp) left untouched for a separate housekeeping
pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-21 15:52:32 +08:00
q792602257andClaude Opus 4.6 6d9237a592 test: align skill catalog and migration tests with new MCP API
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-21 15:31:51 +08:00
q792602257 6241fb9d6d docs: add MCP integration guide 2026-07-21 15:24:16 +08:00
q792602257andClaude Opus 4.6 2d0c740c88 feat(host-agent): add mcp-token CLI subcommand
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-21 15:21:32 +08:00
q792602257 ce64c4eb47 feat(host-agent): wire MCP server into create_application 2026-07-21 15:17:28 +08:00
q792602257 ce2469616e feat(host-agent): mount /mcp + surface MCP status in console 2026-07-21 15:09:41 +08:00
q792602257andClaude Opus 4.6 dcb4798408 feat(host-agent): fail-fast cloud assignment when MCP holds device
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-21 15:00:54 +08:00
q792602257 ab15218b27 feat(host-agent): include mcp_busy_device_ids in heartbeat payload 2026-07-21 14:56:21 +08:00
q792602257 b0932dd398 feat(cloud): skip MCP-busy devices in scheduler 2026-07-21 14:37:21 +08:00
q792602257 9e3007e7f6 feat(cloud): accept mcp_busy_device_ids in heartbeat payload 2026-07-21 14:32:44 +08:00
q792602257andClaude Opus 4.6 98089b6748 fix(host-agent): use stable ServerSession id for MCP lock identity
The previous _current_session_id() implementation tried to import a
non-existent get_context() helper, so the production code path always
fell through to the empty _TEST_SESSION_ID ContextVar — meaning every
MCP client shared the empty-string identity and there was no per-session
isolation in production.

Use Context.session (the long-lived ServerSession object) as the source
of identity. id(ctx.session) is stable across every tool call the same
client makes within a Streamable HTTP session, which is exactly what the
busy tracker needs to renew leases.

Wire FastMCP to inject the Context into the wrapper by setting
tool.context_kwarg = "ctx" after swapping tool.fn; wrap the swap in a
defensive try/except that surfaces a FastMcpSdkIncompatibilityError on
future SDK layout drift.

Add 4 tests covering the production path: stability across calls in the
same session, isolation between sessions, fallback to _TEST_SESSION_ID
when no Context is supplied, and verification that the registered tool
declares context_kwarg="ctx".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-21 14:28:50 +08:00
q792602257 b73db01626 feat(host-agent): wrap tool_handlers with busy check + status mapping 2026-07-21 14:09:42 +08:00
q792602257 cf8affe4d7 feat(host-agent): add BearerAuthMiddleware for MCP server 2026-07-21 14:02:20 +08:00
q792602257 61c923b92b feat(host-agent): add McpBusyTracker for per-device session locks 2026-07-21 13:58:37 +08:00
q792602257 c7faee8da3 feat(host-agent): add McpTokenStore for MCP bearer token 2026-07-21 13:55:24 +08:00
q792602257 29b9a8c39a refactor(api): make tool_handlers require a DeviceManager
Eliminates the silent fallback to DEFAULT_MANAGER that produced the
DeviceNotFoundError incident. All existing callers already pass
manager explicitly.
2026-07-21 13:52:01 +08:00
q792602257 d1b0fffabb build(host-agent): add mcp as direct dependency 2026-07-21 13:49:17 +08:00
q792602257andClaude Opus 4.6 47eac0f2a7 docs(superpowers): add host-agent MCP server implementation plan
15-task TDD plan implementing the spec committed in 3b62195. Covers
the four new host-agent modules (mcp_token, mcp_lock, web/mcp_auth,
web/mcp), cloud heartbeat + scheduler coordination, console mount
wiring, CLI subcommand, and documentation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-21 13:47:15 +08:00
q792602257andClaude Opus 4.6 2f8f4a36c6 docs(superpowers): add host-agent MCP server design spec
Design for mounting a Streamable HTTP MCP server inside the host-agent
process so Hermes Agent (or any MCP client) can drive devices directly.
Reuses the existing console FastAPI + uvicorn on port 8765, adds bearer-
token auth, per-device session-level locks with 60s TTL, and cloud
coordination via a new heartbeat field. Cloud scheduler skips devices
reported as MCP-busy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-21 13:47:15 +08:00
q792602257andClaude Opus 4.6 28dccb908c docs(superpowers): add host-agent MCP server implementation plan
15-task TDD plan implementing the spec committed in 3b62195. Covers
the four new host-agent modules (mcp_token, mcp_lock, web/mcp_auth,
web/mcp), cloud heartbeat + scheduler coordination, console mount
wiring, CLI subcommand, and documentation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-21 13:46:27 +08:00
q792602257andClaude Opus 4.6 3b62195a36 docs(superpowers): add host-agent MCP server design spec
Design for mounting a Streamable HTTP MCP server inside the host-agent
process so Hermes Agent (or any MCP client) can drive devices directly.
Reuses the existing console FastAPI + uvicorn on port 8765, adds bearer-
token auth, per-device session-level locks with 60s TTL, and cloud
coordination via a new heartbeat field. Cloud scheduler skips devices
reported as MCP-busy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-21 13:18:10 +08:00
q792602257 358f4623ba feat(planner): add execution context to prompts
Tests / Test passed: 977
2026-07-16 10:25:20 +08:00
q792602257 240b7be7b8 fix(cloud-console): refresh task planner history 2026-07-16 09:35:31 +08:00
q792602257 9a17297f1e feat(perception): expose active app metadata in UI tree
Tests / Test passed: 971
2026-07-16 08:13:04 +08:00
q792602257 059fb272bb test(device-host-agent): add conftest.py to disable humanize in tests
Tests / Test passed: 966
The root tests/conftest.py fixture disables APEX_HUMANIZE_ENABLED by default for deterministic assertions, but apps/device-host-agent/tests/ was outside its scope. test_e2e.py::test_public_sdk_reports_fake_device_success_and_runtime_failure failed because tap coordinates were jittered (2.67..., 3.37...) instead of exact (2, 3).

Add the same autouse fixture to apps/device-host-agent/tests/conftest.py so all tests in that directory inherit the deterministic behavior.
2026-07-15 21:19:20 +08:00
q792602257 41006b098a feat(perception): sample OCR text foreground/background colors
Tests / Test apps.device-host-agent.tests.test_e2e.test_public_sdk_reports_fake_device_success_and_runtime_failure failed
PaddleOCR itself returns no color info, only text/bounds/confidence.
Add pixel-level post-processing in perception/ocr.py: crop the
screenshot to each OCR box, split pixels into two luminance clusters
via Otsu threshold, and treat the minority cluster as the text stroke
(foreground) and the majority as the background. New
SceneElement.foreground_color/background_color fields ("#rrggbb",
None when not OCR-sourced or sampling fails) round-trip through
to_dict/from_dict alongside the existing accessibility-state fields.
Planner system prompt documents the new fields as a secondary signal.

pillow is promoted from an implicit paddleocr transitive dependency to
an explicit direct dependency since perception/ocr.py now imports PIL
directly; uv.lock re-resolved with no version change (already locked
at 12.3.0).
2026-07-15 20:58:47 +08:00
q792602257andClaude Opus 4.6 f64f98834f Merge branch 'worktree-gesture-humanize': gesture primitives + humanize
Tests / Test apps.device-host-agent.tests.test_e2e.test_public_sdk_reports_fake_device_success_and_runtime_failure failed
Adds long_press/double_tap atomic gestures, a centralized humanize layer
(coordinate jitter, curved W3C-Actions swipe, timing jitter) gated by
APEX_HUMANIZE_ENABLED, and planner integration. 651 non-integration tests
pass on the branch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 20:11:05 +08:00
q792602257 9da73cc6e3 fix(humanize): preserve gaussian magnitude in jitter_point; unify swipe return shape 2026-07-15 19:58:02 +08:00
q792602257 5a93651db7 feat(planner): expose long_press/double_tap to the AI planner 2026-07-15 19:45:06 +08:00
q792602257 bd6b7e64e2 Merge branch 'worktree-task-cancellation': task cancellation feature
Tests / Test passed: 926
# Conflicts:
#	packages/cloud-platform/cloud/schema.py
2026-07-15 19:39:28 +08:00
q792602257 dda70940c0 feat(tools): humanize swipe into curved W3C path when enabled 2026-07-15 19:38:07 +08:00
q792602257 865c163683 test(tools): assert tap humanize actually jitters coords 2026-07-15 19:33:46 +08:00
q792602257 85f0d6e188 feat(tools): humanize tap coordinates; default off in tests 2026-07-15 19:29:57 +08:00
q792602257 fd6365cf6e Add end-to-end cancellation path test and verify no regressions (task-cancellation 9.1-9.2)
- test_cancellation_full_path_queued_immediate_and_dispatched_collaborative
  exercises the full public-API cancellation path: immediate cancel of a
  queued task, collaborative cancel of a dispatched task surfaced through
  lease renewal and a cancelled terminal report, and visibility of the
  cancelled status via both the get and list endpoints.
- Full backend suite (869 passed, 50 skipped) and cloud-console frontend
  suite (27 passed) + typecheck show no regressions; the only failures are
  4 pre-existing live-LLM integration tests unrelated to this change.
2026-07-15 19:23:35 +08:00
q792602257andClaude Opus 4.6 d8e7be4ccb feat(tools): add double_tap tool with humanize hook
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 19:20:35 +08:00
q792602257andClaude Opus 4.6 c4ee4279ef feat(tools): add long_press tool with humanize hook
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 19:16:25 +08:00
q792602257 1dd24825ca Document task cancellation latency in CLOUD_DEPLOYMENT.md (task-cancellation 8.1) 2026-07-15 19:14:52 +08:00
q792602257 18f053e64b Add Host Agent local console task cancellation (task-cancellation 7.1-7.3)
- New internal API route POST /internal/v1/hosts/{host_id}/tasks/{task_id}/cancel,
  authenticated via the host's own bearer credential (authorize_host) with an
  ownership check, since host tokens carry no scopes and cannot reach the
  public SDK's tasks:submit-scoped cancel endpoint.
- HostAgentClient.cancel_task() calls the new internal route directly.
- create_console_app() gains a cancel_task callable with automatic default
  wiring from host_client, so production app.py needs no changes.
- Local console: POST /tasks/{task_id}/cancel route resolves the local
  execution id to its Cloud source_task_id before cancelling, and the task
  detail page/template show a Cancel button plus notice/error banners.
- Tests across all three layers: internal API route, Jinja2 template
  rendering, and FastAPI console route behavior.
2026-07-15 19:13:40 +08:00
q792602257 8a73edf4db feat(driver): add Driver.double_tap via W3C actions 2026-07-15 19:12:10 +08:00
q792602257 c25ccb491d feat(driver): add W3C actions helper and Driver.swipe_path 2026-07-15 19:04:57 +08:00
q792602257 ff91bd4f70 feat(driver): add Driver.long_press on WDA and Android 2026-07-15 18:57:53 +08:00
q792602257 7d79f677fe feat(humanize): add coordinate/duration/swipe-path jitter module 2026-07-15 18:51:15 +08:00
q792602257 4d04d7ac83 Add Cloud Console cancel action and cancelled status
- Widen TaskStatus to include "cancelled"; add it to TasksView's
  STATUSES filter dropdown.
- Add TaskCancellationResponse type and cancelTask(taskId) to api.ts.
- Add a Cancel button to TasksView's task detail panel, gated on
  tasks:submit and a non-terminal task status; updates the displayed
  status on success and surfaces errors via the existing error path.
- Extract the cancellability rule into a pure taskCancellation.ts
  module (mirroring taskProgress.ts/plannerHistory.ts) with unit
  tests, since the project has no Vue component-mounting test setup.

Task 6/9 of task-cancellation change.
2026-07-15 18:43:55 +08:00
q792602257 6776ac2f2d Add public SDK cancel endpoint and CloudClient method
- POST /v1/tasks/{task_id}/cancel: tasks:submit scoped, 200 for
  immediate/idempotent cancellation, 202 for newly recorded pending
  cancellation, 404 for unknown task, 409 for terminal task.
- TaskCancellationResponse{task_id, status} model.
- Widen list_tasks status_filter Literal to include "cancelled".
- CloudClient.cancel_task(task_id).
- SDK-level tests covering queued/assigned/idempotent/404/409/scope
  cases for both the router and CloudClient.

Task 5/9 of task-cancellation change.
2026-07-15 18:39:00 +08:00
q792602257 d3024b4810 feat(host-agent): stop assignment execution collaboratively on cancellation
- LeaseGuard gains an is_cancellation convenience property
- ActiveAssignmentRunner marks the lease lost with a cancellation
  reason when a renewal response reports cancel_requested
- AssignmentExecutor threads stop_reason through to TaskRunner/
  WorkflowRunner and maps a cancellation-flavored stop to
  AssignmentExecutionResult.status = "cancelled" instead of "failed"
- AssignmentProcessor forwards a three-way done/cancelled/failed
  status when reporting the terminal result
- Add/extend tests across lease, assignment, processor, and client
2026-07-15 18:28:21 +08:00
q792602257andClaude Opus 4.6 a58ded055e docs(superpowers): add gesture primitives + humanize implementation plan
9-task TDD plan: humanize module, Driver.long_press/swipe_path/double_tap
on WDA+Android (+ all Fake subclasses), tool wrappers, tap/swipe hooks,
and planner integration.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 18:26:34 +08:00
q792602257andClaude Opus 4.6 557c8a25ba docs(superpowers): add gesture primitives + humanize design spec
Design for long_press/double_tap atomic gestures and a centralized
humanize layer (coordinate jitter, curved W3C-Actions swipe, timing
jitter) gated by APEX_HUMANIZE_ENABLED.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 18:15:43 +08:00
q792602257 d69be48f96 feat(planner): persist reusable action semantics
Tests / Test passed: 879
2026-07-15 18:14:28 +08:00
q792602257 8a0d48eada feat(cloud): surface cancellation over the internal Host<->Cloud protocol
- LeaseRenewalResponse gains cancel_requested (populated from the
  repository's renew_lease result)
- TerminalResultRequest.status widened to accept "cancelled"
- Add internal API tests for a renewal surfacing cancel_requested=True
  and a cancelled terminal report being accepted/idempotent
2026-07-15 18:13:43 +08:00
q792602257 361dada276 feat(perception): surface accessibility interaction state on UI-tree elements
SceneElement gains enabled/clickable/selected/checked/focused (bool | None),
populated from the literal attributes Appium's XCUITest and UiAutomator2
page_source already emit (iOS: enabled only; Android: all five). None means
"not reported by this platform", not false. to_dict() omits unset fields to
keep the LLM-facing scene JSON compact; planner_prompts.py documents the new
fields so the AI planner knows how to use them (e.g. don't tap disabled
elements, use selected/checked to judge whether a toggle already matches the
goal).
2026-07-15 18:10:53 +08:00
q792602257 19c6669800 feat(cloud): add durable cancellation support to task repository
- Add nullable cancel_requested_at column (migration 0012)
- Widen ScheduledTaskStatus/TerminalTaskStatus to include cancelled
- Add CancellationRequestStatus + request_task_cancellation() to
  CloudRepository protocol and SQLAlchemy implementation
- renew_lease() now returns LeaseRenewalResult, surfacing whether
  cancellation is pending, instead of a bare status string
- reap_expired_leases() resolves pending-cancellation tasks to
  cancelled instead of requeuing/failing them
- record_task_result() accepts cancelled and clears
  cancel_requested_at on any terminal write

Note: internal_api/api.py's renew_assignment route still compares
renew_lease()'s return value against a bare string; it will be
updated in the next task (Internal Host<->Cloud protocol) to consume
LeaseRenewalResult and populate the new cancel_requested wire field.
2026-07-15 18:08:48 +08:00
q792602257 7c6cdc5b67 feat(host-agent): persist OCR/UI overlay toggle preference
Tests / Test passed: 868
Save the "Show OCR/UI-tree bounding boxes" checkbox state to localStorage so it persists across page refreshes in the task detail view.
2026-07-15 18:01:10 +08:00
q792602257 88189770ff feat(runtime): add cancellation-aware stop_reason to TaskRunner and WorkflowRunner
- TaskRunner.run() and WorkflowRunner.run()/resume() accept an optional
  stop_reason callable alongside should_stop, distinguishing a genuine
  cancellation from other stop conditions (e.g. lost lease).
- is_cancellation_reason() shared helper added to runtime/task.py.
- WorkflowRunner._stop_status() now branches cancelled/failed based on
  stop_reason, correcting a prior blanket cancelled-on-any-stop behavior
  that conflicted with the host-agent-protocol spec's requirement to
  distinguish cancellation from lease-loss stops.
- Default behavior (stop_reason=None) is preserved exactly for both
  runners so existing callers/tests are unaffected.
- Task 1 of openspec change task-cancellation.
2026-07-15 17:52:33 +08:00
q792602257 947434b65a docs(openspec): add task-cancellation proposal, design, specs, tasks 2026-07-15 17:38:02 +08:00
q792602257 a25542694d fix test
Tests / Test passed: 868
2026-07-15 17:04:59 +08:00
q792602257 17a709c92f fix(perception): disable PaddleOCR doc-unwarping for screenshots
Tests / Test tests.test_device_config.test_device_config_store_settings_get_set_and_defaults failed
PaddleOCR's OCR.yaml pipeline defaults to use_doc_orientation_classify
and use_doc_unwarping enabled, which are meant for photographed paper
documents. Applied to a flat, upright device screenshot, UVDoc
geometrically warps the image before detection, and returns box
coordinates in that warped space with no inverse mapping back to the
original image.

Verified on a real screenshot: with unwarping on, the same detected
element ("新项目") shifts from y=158 to y=71 versus the original image,
and 2 boxes near the top edge (status bar time/battery) are dropped
entirely. Disabling both flags by default (still overridable via
explicit kwargs) makes detected boxes match the original screenshot.
2026-07-15 16:51:36 +08:00
q792602257 4046c9452d fix(deps): pin paddlepaddle below 3.3.0 to avoid OCR text corruption
paddlepaddle 3.3.1 silently corrupts non-ASCII (CJK) recognized text
into literal U+FFFD replacement characters during rec postprocessing,
while leaving confidence scores high and ASCII/digit text unaffected.
The same release also breaks CPU oneDNN inference on Windows entirely
(NotImplementedError in onednn_instruction.cc). Verified on a real
task screenshot that downgrading to 3.2.x eliminates the corruption
with no other environment changes (same GBK-locale machine).
2026-07-15 16:41:58 +08:00
q792602257 7f439f0db5 fix(perception): reconcile points/pixels scale and stale overlay screenshot
Tests / Test tests.test_device_config.test_device_config_store_settings_get_set_and_defaults failed
Host-agent console showed OCR/UI-tree overlay boxes misaligned with the
displayed screenshot. Two independent causes, both confirmed with real
task data and pixel-level measurement of a user-provided screenshot:

1. perception/ui_parser.py parses XCUITest UI-tree bounds as iOS logical
   points, while scene_builder.py's Scene.width/height (via infer_png_size)
   and OCR bounds are in screenshot pixels, never reconciled (2.0x on
   Retina devices). build_scene() now detects the scale from the first
   x==0,y==0 UI element and rescales OCR bounds down to points-space,
   reporting Scene.width/height in points too. No-op for Android, where
   UiAutomator2 bounds already match pixels 1:1. This also fixes tap()
   landing at the wrong location for OCR-matched text, and lets the IOU
   fusion between UI-tree and OCR elements actually fire on iOS.

2. runtime/task.py captured `scene` (OCR/UI-tree data) before the LLM
   planning call, but re-captured `before_screenshot` for each step
   afterward - a real time gap during which on-screen content (e.g. a
   keyboard) could shift, producing a directional drift between the
   overlay and the displayed image. The first step of each plan batch
   now reuses the screenshot already taken for planning instead of
   capturing a new one; later steps in a multi-step batch still take a
   fresh capture (left unresolved, scoped out by request).

Regression tests added for both the scale reconciliation (using real
828x1792 vs 414x896 numbers) and the screenshot reuse behavior.
2026-07-15 16:12:53 +08:00
q792602257 c50ce1faec OCR不展开
Tests / Test tests.test_device_config.test_device_config_store_settings_get_set_and_defaults failed
2026-07-15 15:06:06 +08:00
q792602257 701983ccdd config: increase max_steps limit from 20 to 999999
Remove the 20-step execution limit that was causing "max steps exceeded" errors for long-running tasks. Increase the default max_steps to 999999 in all configurations, effectively removing the practical limit while maintaining the safety mechanism.

Changes:

- runtime/task.py: TaskRunnerConfig.max_steps 20 → 999999

- agents/collab_runner.py: CollaborativeTaskRunnerConfig.max_steps 20 → 999999

- storage/device_config.py: DEFAULT_MAX_STEPS 20 → 999999
2026-07-15 14:58:06 +08:00
q792602257 8162509158 feat(host-agent): persist UI-tree evidence and add overlay/action visualization
Tests / Test passed: 863
Fixes issue 3: the host-agent console showed OCR results but never real
UI-tree data, because _ui_tree_nodes() checked for a get_ui_tree/ui_tree
tool action that has never existed anywhere in the codebase.

- storage/timeline.py: add a ui_tree_results field to TimelineRecord and
  Timeline.append(), mirroring the existing ocr_results field.
- runtime/task.py: _append_timeline() now extracts scene.elements with
  source == "ui" into ui_tree_results (scene_builder.build_scene() already
  preserved these; they were just never persisted).
- host_agent/web/app.py: _ui_tree_nodes() reads the new field directly
  instead of the dead tool-action check. New _overlay_payload() exposes
  each step's scene dimensions and fused element list for client-side
  rendering.
- task_detail.html: adds a toggle to overlay OCR (orange) and UI-tree
  (blue) bounding boxes on the before-action screenshot, plus a visual
  marker for the actually executed action (tap circle, or an animated
  swipe path) using an SVG viewBox so no manual coordinate-scaling JS is
  needed. Legacy/incomplete records degrade to no overlay, never an error.

Also corrects openspec/specs/runtime-task-evidence and
host-agent-console-task-pages, which had encoded the same nonexistent-tool
assumption, via the new host-agent-console-visual-evidence change.

600 tests passing; ruff/compileall/openspec validate all clean.
2026-07-15 14:39:28 +08:00
q792602257 367fd0d412 fix(planner): allow rationale/thinking by using tool_choice=auto
Tests / Test passed: 855
Forced tool_choice ("any"/"required") makes both Anthropic and OpenAI
skip any text/thinking block before the tool call, which silently made
rationale and thinking always None despite the planner-reflection-history
change's capture code being correct. Switch the primary call to
tool_choice="auto" (Anthropic: type=auto, disable_parallel_tool_use=true;
OpenAI: "auto") so the model can emit its reflection text, and add a
one-time forced retry (Anthropic "any", OpenAI "required", thinking
disabled) if the model responds without a tool call, guaranteeing a step
never stalls. Also add OpenAI text_output capture from message.content,
which was never extracted before (Anthropic-only gap).

Update planner-reflection-history design.md/tasks.md to document the bug
found during the pending manual smoke test (task 8.5) and the fix (new
section 9).
2026-07-15 13:43:39 +08:00
q792602257 24992fc9fb fix test
Tests / Test passed: 851
2026-07-15 12:53:50 +08:00
q792602257 a5aeb8889c feat(runtime): add planner reflection history with rationale and thinking
Tests / Test failed: 2, passed: 849
- ToolCallDecision captures thinking blocks and pre-tool text output
- AnthropicToolCallingClient supports optional extended thinking (budget_tokens + beta header)
- PlannedStep carries rationale and thinking from each LLM decision
- WorldEvent replaces scene_summary with rationale/thinking/page fields (backward-compatible)
- AI planner system prompt instructs reflection before each tool call
- _history_summary() emits compact {page, rationale, action, success} dicts
- Cloud DB migration 0011 adds nullable rationale/thinking columns to planner_decision_log
- OpenAI client extracts reasoning_content into thinking field
2026-07-15 12:43:22 +08:00
q792602257 96e403ee47 chore(openspec): archive task execution visibility 2026-07-15 12:10:14 +08:00
q792602257 77d4813bb2 feat(host-agent): make execution history authoritative
Tests / Test failed: 2, passed: 830
2026-07-15 11:46:27 +08:00
q792602257 ccde30e378 feat(runtime): capture step evidence in console
Tests / Test passed: 862
2026-07-15 10:12:09 +08:00
q792602257 8d5b02e37f fix(perception): degrade to OCR-only when UI tree is unavailable
Tests / Test passed: 858
driver.tree() failures (WDA/Appium session errors) previously raised
uncaught, killing describe_screen() before OCR ever ran. Malformed
tree content (invalid XML) had the same problem inside
parse_ui_tree(). Both are now caught and logged, falling back to an
empty ui_elements list so the scene degrades to OCR-only, mirroring
the existing OCR-failure fallback in run_ocr().
2026-07-15 10:03:30 +08:00
q792602257 778af2da53 fix(cloud): align planner configuration and records
Tests / Test passed: 856
2026-07-15 09:43:14 +08:00
q792602257andClaude Opus 4.6 f8054cb58c chore(skills): docs + ruff format for skill-management-console
Tests / Test passed: 855
Documents Skill Management in CLOUD_DEPLOYMENT.md (cloud-skill store,
per-host entitlement, incremental sync, local authoring/override,
inventory report, skills:admin scope) and applies ruff check/format to
all touched modules. All tasks complete; full non-integration suite
green (593 passed) and openspec validate --strict passes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 08:10:20 +08:00
q792602257andClaude Opus 4.6 fd0ea3a066 feat(console): Cloud Console Skills management view
Tests / Test passed: 855
Adds SkillsView.vue (cloud-skill CRUD, per-host entitlement grant/revoke,
read-only host local-skill inventory), skill API client methods + types,
and wires it into App.vue behind the skills:admin scope. Console
typecheck/build/tests green (20 passed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 08:06:49 +08:00
q792602257andClaude Opus 4.6 8300c3b6b7 Merge branch 'worktree-runtime-console-jinja2-templates'
Server-rendered Jinja2 Runtime console at /ui/, replacing the Vue/Vite SPA.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 08:05:02 +08:00
q792602257andClaude Opus 4.6 e00c50e703 feat(api): server-rendered Jinja2 Runtime console at /ui/
Replaces the separate Vue/Vite `console/` SPA with a same-origin,
server-rendered console built on a module-level Jinja2 Environment
with select_autoescape(["html","xml"]).

- Add api/console_web.py with /ui/ routes (dashboard, tasks, task
  detail/timeline, config) and a _status_fragment polled every 10s.
- Refactor api/console.py into a typed ConsoleService shared by the
  JSON and HTML routers so validation/persistence cannot drift.
- Remove RUNTIME_CONSOLE_STATIC_DIR, SpaStaticFiles, and the wildcard
  CORS middleware from api/rest.py; GET / now redirects to /ui/.
- Delete the top-level console/ project; add jinja2 and python-multipart
  as direct dependencies and ship templates/CSS/JS via package-data.
- Add 31 tests (XSS probes, PRG flows, fragment refresh, no-static-dir
  and no-CORS regressions, wheel-packaging smoke test).

/console/* JSON endpoints remain unchanged. The console keeps the
trusted-network-only boundary; auth/CSRF is intentionally deferred.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 08:03:13 +08:00
q792602257andClaude Opus 4.6 0d944ec97d feat(host-agent): wire skill sync + inventory report into app lifecycle
Adds host_agent/skill_sync.py (HostAgentSkillSync) which constructs the
synced + local skill stores, the Cloud API sync client, and the runner,
then drives them on the host-agent lifecycle: incremental per-host pull
into the synced catalog, fork-on-revocation, and a best-effort local-
skill inventory report to the Cloud (design D7). Wired into
create_application + run_async start/stop. Host-agent suite green
(219 passed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 08:02:31 +08:00
q792602257andClaude Opus 4.6 a8ba2312fc feat(skills): cloud sync client + incremental sync + fork-on-revocation
Adds CloudApiSkillClient (Cloud API per-host sync endpoint + inventory
report), forwards since_version for incremental sync (full-replace on
first/stale), and forks a local override into a standalone local skill
when its cloud skill is revoked (design D9). Skill-side tests green (87
passed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 07:55:08 +08:00
q792602257andClaude Opus 4.6 cbfdb2ae39 feat(cloud): skill management + host-scoped sync REST endpoints
Adds the skills:admin router (cloud/sdk/skill_api.py) for cloud-skill CRUD
and per-host entitlement grant/revoke with CSRF/scope/audit, and a
host-scoped router serving incremental per-host sync deltas plus the
agent local-skill inventory report/readback. Both composed into the
Cloud API app. cloud-api suite green (46 passed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 07:51:27 +08:00
q792602257andClaude Opus 4.6 52e442790a feat(cloud): cloud-managed skill store + per-host entitlement + sync versioning
Tests / Test passed: 819
Adds cloud/skills.py (domain + service), SQLAlchemy models and Alembic
migration 0010_skill_management (cloud_skills, cloud_skill_entitlements,
cloud_skill_sync_state, a per-host changelog, and host_skill_inventory),
and repository methods with a monotonic per-host entitlement_version that
drives correct incremental fetch_host_delta. cloud-api suite green (41
passed); HEAD_REVISION bumped to 0010.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 07:45:22 +08:00
q792602257andClaude Opus 4.6 8baf3a6a8b feat(skills): unified read-merge + authoring/override MCP tools
Adds the merged read surface (api/skill_catalog_view.py) over synced +
local stores with origin discrimination and override precedence, and
extends the skill MCP tools with create_skill/update_skill/delete_skill
that dispatch by origin (edit local skills; create/update/remove local
overrides for cloud skills). Wired into api.mcp.create_mcp_server.
Full non-integration suite green (564 passed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 07:38:12 +08:00
q792602257andClaude Opus 4.6 dd03abbbb0 feat(skills): open skill-management-console change + local skill store
Opens the skill-management-console openspec change (cloud/local skill split
with local override) with proposal, design (D1-D11), four delta specs, and
tasks. Implements the agent-side persistent local skill store
(storage/local_skills.py): authored local skills + cloud-skill overrides in
a physically separate SQLite file, with fork-on-revocation. 10 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 07:28:42 +08:00
q792602257andClaude Opus 4.6 e5a12f9b74 chore(openspec): archive skill-catalog-subscription
Change is complete (24/24 tasks) per its declared scope (read-only local
catalog + MCP tools + sync client contract). Management UI and the
upstream Subscription Platform were explicitly out of scope. Deltas
synced into three new main specs: skill-catalog, skill-mcp-tools,
skill-subscription-sync.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 00:09:24 +08:00
q792602257andClaude Opus 4.6 56f3f96363 chore(openspec): archive database-llm-provider-management
Tests / Test passed: 794
Change is complete (17/17 tasks). Deltas synced: MODIFIED the
cloud-planner-proxy "Endpoint resolves exactly one tool-call decision"
requirement to resolve provider config from the active database profile,
and created a new main spec openspec/specs/llm-provider-management/spec.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-14 23:32:08 +08:00
q792602257andClaude Opus 4.6 8e37b965aa chore(openspec): archive cloud-planner-proxy
Change is complete (20/20 tasks). Deltas synced: MODIFIED the
agent-runtime "Pluggable dual-provider tool-calling abstraction"
requirement (added transport selection), and created a new main spec
openspec/specs/cloud-planner-proxy/spec.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-14 23:30:25 +08:00
q792602257andClaude Opus 4.6 c01dd4c6b2 chore(openspec): archive ai-planner-runtime
Change is complete (22/22 tasks) and its delta spec has been synced
into a new main spec openspec/specs/agent-runtime/spec.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-14 23:28:17 +08:00
q792602257andClaude Opus 4.6 9669b52498 chore(openspec): archive uv-workspace-packaging
Change is complete (17/17 tasks) and its delta spec has been synced
into a new main spec openspec/specs/workspace-packaging/spec.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-14 21:24:59 +08:00
q792602257andClaude Opus 4.6 fa10cccf71 chore(openspec): archive downgrade-python-3-13-paddleocr
Change is complete (19/19 tasks) and its delta spec has been synced
into a new main spec openspec/specs/python-runtime-baseline/spec.md.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-14 21:18:57 +08:00
q792602257andClaude Opus 4.6 989fdbf878 chore(openspec): archive edge-host-enrollment
Tests / Test passed: 794
Sync delta specs into main specs before archiving: modified
cloud-control-plane, device-pool, and host-agent-protocol; created
new edge-host-enrollment capability spec. openspec validate --specs
reports 18/18 passing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-14 21:01:47 +08:00
q792602257 30f09b6268 feat(host-agent): default planner transport to cloud
Tests / Test passed: 794
2026-07-14 20:42:27 +08:00
q792602257 6e511111c4 fix(perception): harden PaddleOCR result handling
Tests / Test passed: 793
2026-07-14 20:07:49 +08:00
q792602257 25ebc10a8a feat: downgrade Python baseline to 3.13 for PaddleOCR compatibility
Tests / Test passed: 789
paddlepaddle has no Python 3.14 (cp314) wheel on PyPI, so host-agent
deployments on 3.14 can never install it, causing OCR to fail at
runtime with RuntimeError. Pin the workspace to Python 3.13 across
all pyproject.toml files, the Docker base image, and the Jenkins CI
image; regenerate uv.lock against 3.13.

Also fixes a pre-existing Python-2-style `except X, Y:` syntax error
(invalid in all Python 3.x) in runtime/task.py and
packages/cloud-platform/cloud/{sql_repository,internal_api/api}.py,
introduced in 22d37ca9 and unrelated to this change's scope, which
blocked the full test suite from collecting on any interpreter
version.

openspec change: downgrade-python-3-13-paddleocr
2026-07-14 18:05:49 +08:00
q792602257 ecb1dba9ff chore(openspec): archive host-agent-console-task-submission
Tests / Test passed: 789
2026-07-14 17:55:27 +08:00
q792602257 a883903b66 style(host-agent): format Console task submission files 2026-07-14 17:52:35 +08:00
q792602257 a46f7d02a1 Merge branch 'opsx/host-agent-console-task-submission' into master
Tests / Test passed: 789
Adds a CSRF-protected task submission form to the local Console Tasks page

# Conflicts:
#	apps/device-host-agent/host_agent/app.py
2026-07-14 17:05:37 +08:00
q792602257 fb09924835 feat(host-agent): add Console task submission with Host self-submission client
Adds a CSRF-protected task submission form to the local Console Tasks page
2026-07-14 17:02:41 +08:00
q792602257andClaude Sonnet 5 bb6f24bbcb docs(openspec): correct host-agent-single-instance-lock's Why section
This change was originally proposed as the root-cause fix for the
2026-07-14 DeviceNotFoundError incident. That diagnosis was wrong: it was
subsequently confirmed only one Host Agent process was running at the time,
ruling out the duplicate-process precondition this change addresses. The
actual root cause was execution.py's create_task_runner() omitting manager=
when wiring TaskRunner (see 08cef7c). Reframe the Why section: this change
stands on its own as independent duplicate-process hardening, not as a fix
for an incident it turned out not to have caused.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-14 16:36:04 +08:00
q792602257andClaude Sonnet 5 08cef7ca3c fix(host-agent): thread device manager into task runner observer/screenshot
create_task_runner() built TaskRunner's observer/screenshot_provider by
calling describe_screen(device_id)/take_screenshot(device_id) without
manager=, so both silently fell back to the process-global DEFAULT_MANAGER
singleton instead of the Host Agent's real, device-populated DeviceManager.
DEFAULT_MANAGER never has any device registered, so every task's first step
raised DeviceNotFoundError even though the console (which does pass
manager=) showed the same device as connected. Deterministic on every task,
independent of process count.

Add regression tests confirming both lambdas now resolve devices via the
configured manager; verified each fails with the original DeviceNotFoundError
symptom when the fix is reverted.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-14 16:35:48 +08:00
q792602257andClaude Opus 4.6 82567fd248 chore(openspec): add host-agent-single-instance-lock change artifacts
Tests / Test passed: 759
Proposal, design, spec, and tasks for the per-installation exclusive
instance lock. 15/16 tasks complete; only manual real-environment
verification (5.4) remains, with semantics covered by unit tests in
test_app.py and test_instance_lock.py.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-14 15:49:45 +08:00
q792602257andClaude Opus 4.6 d00ada67a5 feat(host-agent): add single-instance lock to prevent duplicate-process dispatch races
Acquire an exclusive, non-blocking filelock on the identity state directory
as the first action of create_application(), before resolve_host_identity()
or any enrollment/heartbeat side effect. A second process against the same
identity_path exits immediately with InstanceAlreadyRunningError naming the
lock path; the lock releases automatically on any process exit (including
SIGKILL) via OS-level advisory locking, and explicitly during run_async()'s
shutdown finally block. filelock is promoted from transitive to direct
dependency (version unchanged at 3.29.7).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-14 15:49:30 +08:00
q792602257 99bde4febb test: align deployment and lease contracts
Tests / Test passed: 748
2026-07-14 13:53:00 +08:00