Adds long_press/double_tap atomic gestures, a centralized humanize layer
(coordinate jitter, curved W3C-Actions swipe, timing jitter) gated by
APEX_HUMANIZE_ENABLED, and planner integration. 651 non-integration tests
pass on the branch.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_cancellation_full_path_queued_immediate_and_dispatched_collaborative
exercises the full public-API cancellation path: immediate cancel of a
queued task, collaborative cancel of a dispatched task surfaced through
lease renewal and a cancelled terminal report, and visibility of the
cancelled status via both the get and list endpoints.
- Full backend suite (869 passed, 50 skipped) and cloud-console frontend
suite (27 passed) + typecheck show no regressions; the only failures are
4 pre-existing live-LLM integration tests unrelated to this change.
- New internal API route POST /internal/v1/hosts/{host_id}/tasks/{task_id}/cancel,
authenticated via the host's own bearer credential (authorize_host) with an
ownership check, since host tokens carry no scopes and cannot reach the
public SDK's tasks:submit-scoped cancel endpoint.
- HostAgentClient.cancel_task() calls the new internal route directly.
- create_console_app() gains a cancel_task callable with automatic default
wiring from host_client, so production app.py needs no changes.
- Local console: POST /tasks/{task_id}/cancel route resolves the local
execution id to its Cloud source_task_id before cancelling, and the task
detail page/template show a Cancel button plus notice/error banners.
- Tests across all three layers: internal API route, Jinja2 template
rendering, and FastAPI console route behavior.
- POST /v1/tasks/{task_id}/cancel: tasks:submit scoped, 200 for
immediate/idempotent cancellation, 202 for newly recorded pending
cancellation, 404 for unknown task, 409 for terminal task.
- TaskCancellationResponse{task_id, status} model.
- Widen list_tasks status_filter Literal to include "cancelled".
- CloudClient.cancel_task(task_id).
- SDK-level tests covering queued/assigned/idempotent/404/409/scope
cases for both the router and CloudClient.
Task 5/9 of task-cancellation change.
- LeaseGuard gains an is_cancellation convenience property
- ActiveAssignmentRunner marks the lease lost with a cancellation
reason when a renewal response reports cancel_requested
- AssignmentExecutor threads stop_reason through to TaskRunner/
WorkflowRunner and maps a cancellation-flavored stop to
AssignmentExecutionResult.status = "cancelled" instead of "failed"
- AssignmentProcessor forwards a three-way done/cancelled/failed
status when reporting the terminal result
- Add/extend tests across lease, assignment, processor, and client
- LeaseRenewalResponse gains cancel_requested (populated from the
repository's renew_lease result)
- TerminalResultRequest.status widened to accept "cancelled"
- Add internal API tests for a renewal surfacing cancel_requested=True
and a cancelled terminal report being accepted/idempotent
SceneElement gains enabled/clickable/selected/checked/focused (bool | None),
populated from the literal attributes Appium's XCUITest and UiAutomator2
page_source already emit (iOS: enabled only; Android: all five). None means
"not reported by this platform", not false. to_dict() omits unset fields to
keep the LLM-facing scene JSON compact; planner_prompts.py documents the new
fields so the AI planner knows how to use them (e.g. don't tap disabled
elements, use selected/checked to judge whether a toggle already matches the
goal).
- Add nullable cancel_requested_at column (migration 0012)
- Widen ScheduledTaskStatus/TerminalTaskStatus to include cancelled
- Add CancellationRequestStatus + request_task_cancellation() to
CloudRepository protocol and SQLAlchemy implementation
- renew_lease() now returns LeaseRenewalResult, surfacing whether
cancellation is pending, instead of a bare status string
- reap_expired_leases() resolves pending-cancellation tasks to
cancelled instead of requeuing/failing them
- record_task_result() accepts cancelled and clears
cancel_requested_at on any terminal write
Note: internal_api/api.py's renew_assignment route still compares
renew_lease()'s return value against a bare string; it will be
updated in the next task (Internal Host<->Cloud protocol) to consume
LeaseRenewalResult and populate the new cancel_requested wire field.
- TaskRunner.run() and WorkflowRunner.run()/resume() accept an optional
stop_reason callable alongside should_stop, distinguishing a genuine
cancellation from other stop conditions (e.g. lost lease).
- is_cancellation_reason() shared helper added to runtime/task.py.
- WorkflowRunner._stop_status() now branches cancelled/failed based on
stop_reason, correcting a prior blanket cancelled-on-any-stop behavior
that conflicted with the host-agent-protocol spec's requirement to
distinguish cancellation from lease-loss stops.
- Default behavior (stop_reason=None) is preserved exactly for both
runners so existing callers/tests are unaffected.
- Task 1 of openspec change task-cancellation.
PaddleOCR's OCR.yaml pipeline defaults to use_doc_orientation_classify
and use_doc_unwarping enabled, which are meant for photographed paper
documents. Applied to a flat, upright device screenshot, UVDoc
geometrically warps the image before detection, and returns box
coordinates in that warped space with no inverse mapping back to the
original image.
Verified on a real screenshot: with unwarping on, the same detected
element ("新项目") shifts from y=158 to y=71 versus the original image,
and 2 boxes near the top edge (status bar time/battery) are dropped
entirely. Disabling both flags by default (still overridable via
explicit kwargs) makes detected boxes match the original screenshot.
Host-agent console showed OCR/UI-tree overlay boxes misaligned with the
displayed screenshot. Two independent causes, both confirmed with real
task data and pixel-level measurement of a user-provided screenshot:
1. perception/ui_parser.py parses XCUITest UI-tree bounds as iOS logical
points, while scene_builder.py's Scene.width/height (via infer_png_size)
and OCR bounds are in screenshot pixels, never reconciled (2.0x on
Retina devices). build_scene() now detects the scale from the first
x==0,y==0 UI element and rescales OCR bounds down to points-space,
reporting Scene.width/height in points too. No-op for Android, where
UiAutomator2 bounds already match pixels 1:1. This also fixes tap()
landing at the wrong location for OCR-matched text, and lets the IOU
fusion between UI-tree and OCR elements actually fire on iOS.
2. runtime/task.py captured `scene` (OCR/UI-tree data) before the LLM
planning call, but re-captured `before_screenshot` for each step
afterward - a real time gap during which on-screen content (e.g. a
keyboard) could shift, producing a directional drift between the
overlay and the displayed image. The first step of each plan batch
now reuses the screenshot already taken for planning instead of
capturing a new one; later steps in a multi-step batch still take a
fresh capture (left unresolved, scoped out by request).
Regression tests added for both the scale reconciliation (using real
828x1792 vs 414x896 numbers) and the screenshot reuse behavior.
Fixes issue 3: the host-agent console showed OCR results but never real
UI-tree data, because _ui_tree_nodes() checked for a get_ui_tree/ui_tree
tool action that has never existed anywhere in the codebase.
- storage/timeline.py: add a ui_tree_results field to TimelineRecord and
Timeline.append(), mirroring the existing ocr_results field.
- runtime/task.py: _append_timeline() now extracts scene.elements with
source == "ui" into ui_tree_results (scene_builder.build_scene() already
preserved these; they were just never persisted).
- host_agent/web/app.py: _ui_tree_nodes() reads the new field directly
instead of the dead tool-action check. New _overlay_payload() exposes
each step's scene dimensions and fused element list for client-side
rendering.
- task_detail.html: adds a toggle to overlay OCR (orange) and UI-tree
(blue) bounding boxes on the before-action screenshot, plus a visual
marker for the actually executed action (tap circle, or an animated
swipe path) using an SVG viewBox so no manual coordinate-scaling JS is
needed. Legacy/incomplete records degrade to no overlay, never an error.
Also corrects openspec/specs/runtime-task-evidence and
host-agent-console-task-pages, which had encoded the same nonexistent-tool
assumption, via the new host-agent-console-visual-evidence change.
600 tests passing; ruff/compileall/openspec validate all clean.
Forced tool_choice ("any"/"required") makes both Anthropic and OpenAI
skip any text/thinking block before the tool call, which silently made
rationale and thinking always None despite the planner-reflection-history
change's capture code being correct. Switch the primary call to
tool_choice="auto" (Anthropic: type=auto, disable_parallel_tool_use=true;
OpenAI: "auto") so the model can emit its reflection text, and add a
one-time forced retry (Anthropic "any", OpenAI "required", thinking
disabled) if the model responds without a tool call, guaranteeing a step
never stalls. Also add OpenAI text_output capture from message.content,
which was never extracted before (Anthropic-only gap).
Update planner-reflection-history design.md/tasks.md to document the bug
found during the pending manual smoke test (task 8.5) and the fix (new
section 9).
driver.tree() failures (WDA/Appium session errors) previously raised
uncaught, killing describe_screen() before OCR ever ran. Malformed
tree content (invalid XML) had the same problem inside
parse_ui_tree(). Both are now caught and logged, falling back to an
empty ui_elements list so the scene degrades to OCR-only, mirroring
the existing OCR-failure fallback in run_ocr().
Replaces the separate Vue/Vite `console/` SPA with a same-origin,
server-rendered console built on a module-level Jinja2 Environment
with select_autoescape(["html","xml"]).
- Add api/console_web.py with /ui/ routes (dashboard, tasks, task
detail/timeline, config) and a _status_fragment polled every 10s.
- Refactor api/console.py into a typed ConsoleService shared by the
JSON and HTML routers so validation/persistence cannot drift.
- Remove RUNTIME_CONSOLE_STATIC_DIR, SpaStaticFiles, and the wildcard
CORS middleware from api/rest.py; GET / now redirects to /ui/.
- Delete the top-level console/ project; add jinja2 and python-multipart
as direct dependencies and ship templates/CSS/JS via package-data.
- Add 31 tests (XSS probes, PRG flows, fragment refresh, no-static-dir
and no-CORS regressions, wheel-packaging smoke test).
/console/* JSON endpoints remain unchanged. The console keeps the
trusted-network-only boundary; auth/CSRF is intentionally deferred.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds CloudApiSkillClient (Cloud API per-host sync endpoint + inventory
report), forwards since_version for incremental sync (full-replace on
first/stale), and forks a local override into a standalone local skill
when its cloud skill is revoked (design D9). Skill-side tests green (87
passed).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds the merged read surface (api/skill_catalog_view.py) over synced +
local stores with origin discrimination and override precedence, and
extends the skill MCP tools with create_skill/update_skill/delete_skill
that dispatch by origin (edit local skills; create/update/remove local
overrides for cloud skills). Wired into api.mcp.create_mcp_server.
Full non-integration suite green (564 passed).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Opens the skill-management-console openspec change (cloud/local skill split
with local override) with proposal, design (D1-D11), four delta specs, and
tasks. Implements the agent-side persistent local skill store
(storage/local_skills.py): authored local skills + cloud-skill overrides in
a physically separate SQLite file, with fork-on-revocation. 10 tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Host Agent now persists step-level execution detail locally (via a real
TaskMetadataStore/Timeline wired into TaskRunner) and reports a bounded
in-progress snapshot piggybacked on lease renewal. Cloud persists that
snapshot per active assignment and exposes it through the existing task
list/detail query path; Cloud Console renders it as a live badge. Host
Agent's local console gains authenticated, read-only task list and
detail/timeline pages (same-origin, server-rendered) with inlined
screenshots.
Also fixes a pre-existing gap in the shared Timeline: the actual
per-step LLM prompt is now recorded instead of the task goal, benefiting
both Runtime and Host Agent consoles. When a host uses the cloud planner
transport, each decide call's prompt and resulting tool decision are
durably logged in a new planner_decision_log table (with bounded
retention) and browsable from Cloud Console; direct-transport hosts
explicitly surface a "not reported" state.
Includes Alembic migrations 0008 (progress columns on scheduled_tasks)
and 0009 (planner_decision_log), bounded Host-Agent-local retention,
dual-backend repository parity, and Vitest + pytest coverage. Task 6.5
(manual end-to-end device verification) remains.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements all 19 tasks of the cloud-planner-proxy OpenSpec change:
- Cloud API: cloud.planner_config (CloudPlannerConfig, load/build helpers)
reusing runtime.tool_calling_client provider clients (no new dependency
needed -- device-cloud-platform already depends on device-agent-runtime).
- Cloud API: new host-scoped POST /internal/v1/hosts/{host_id}/planner/decide
internal endpoint, reusing existing bearer auth; logs only metadata
(host id, tool name, latency, error class), never prompt/screenshot
content.
- Host Agent: new AI_PLANNER_TRANSPORT config (direct default | cloud) and
host_agent/cloud_planner_client.py::CloudProxyToolCallingClient, a
synchronous ToolCallingClient implementation (structural, not importing
runtime) that calls the new endpoint via its own httpx.Client -- avoids
bridging the async HostAgentClient across the worker-thread boundary
that AIPlanner.plan() runs in (asyncio.to_thread in lease.py).
- Host Agent wiring: create_execution_factories()/_host_agent_planner()
select the cloud-proxy client only when AI_PLANNER_TRANSPORT=cloud;
direct/unset transport is unchanged (still the default).
- Tests: 22 new tests across Cloud API config, the new endpoint, the new
client, and transport-selection wiring; full non-integration suite
(492 tests) passes with no regressions.
- Docs: docs/CLOUD_DEPLOYMENT.md documents the cloud transport, its
trade-offs, and the credential split between Host Agent and Cloud API.
proposal.md/design.md were corrected during implementation to reflect two
findings: no new anthropic/openai dependency is actually needed, and
CloudProxyToolCallingClient uses its own sync httpx.Client rather than a
new HostAgentClient method, per the thread-boundary reasoning above.
Host Agent:
- One-time local operator account bootstrap (PBKDF2-HMAC-SHA256, atomic
0600-permission write) gating the daemon's first unattended start via a
new `setup` CLI subcommand.
- Default control-plane URL now https://amcp.home.jerryyan.top (env var
override unchanged).
- Enrollment no longer requires a pre-issued token; falls back to
zero-token self-service enrollment when none is configured.
Cloud control plane:
- CLOUD_SELF_SERVICE_ENROLLMENT_ENABLED (default false) opt-in flag.
- SelfServiceEnrollmentAuthProvider + ChainedEnrollmentAuthProvider:
configured tokens still take priority; self-service only applies when
no token matches, preserving edge-host-enrollment's token-bound path.
- Fixed a latent bug in sql_repository.py::enroll_host: the token-conflict
lookup used `== enrollment_token_digest`, which SQLAlchemy compiles to
`IS NULL` when the value is None, so every self-service enrollment after
the first would have falsely collided with an existing NULL-digest host.
Skipped that lookup entirely when the digest is None.
Docs/deploy: .env.example, compose.yaml, compose.deploy.yaml,
CLOUD_DEPLOYMENT.md, MACOS_IPHONE_SETUP.md updated for the new flag,
URL default, and required `device-host-agent setup` step.
Verification: 494 non-integration tests pass; openspec validate --strict
passes. PostgreSQL-backed contract tests and full manual end-to-end
verification were not run (no Postgres/Docker or reachable cloud-api in
this environment); noted as unchecked in tasks.md 7.2/7.4.
q792602257andClaude Opus 4.6