Commit Graph
87 Commits
Author SHA1 Message Date
q792602257 d3024b4810 feat(host-agent): stop assignment execution collaboratively on cancellation
- LeaseGuard gains an is_cancellation convenience property
- ActiveAssignmentRunner marks the lease lost with a cancellation
  reason when a renewal response reports cancel_requested
- AssignmentExecutor threads stop_reason through to TaskRunner/
  WorkflowRunner and maps a cancellation-flavored stop to
  AssignmentExecutionResult.status = "cancelled" instead of "failed"
- AssignmentProcessor forwards a three-way done/cancelled/failed
  status when reporting the terminal result
- Add/extend tests across lease, assignment, processor, and client
2026-07-15 18:28:21 +08:00
q792602257 8a0d48eada feat(cloud): surface cancellation over the internal Host<->Cloud protocol
- LeaseRenewalResponse gains cancel_requested (populated from the
  repository's renew_lease result)
- TerminalResultRequest.status widened to accept "cancelled"
- Add internal API tests for a renewal surfacing cancel_requested=True
  and a cancelled terminal report being accepted/idempotent
2026-07-15 18:13:43 +08:00
q792602257 19c6669800 feat(cloud): add durable cancellation support to task repository
- Add nullable cancel_requested_at column (migration 0012)
- Widen ScheduledTaskStatus/TerminalTaskStatus to include cancelled
- Add CancellationRequestStatus + request_task_cancellation() to
  CloudRepository protocol and SQLAlchemy implementation
- renew_lease() now returns LeaseRenewalResult, surfacing whether
  cancellation is pending, instead of a bare status string
- reap_expired_leases() resolves pending-cancellation tasks to
  cancelled instead of requeuing/failing them
- record_task_result() accepts cancelled and clears
  cancel_requested_at on any terminal write

Note: internal_api/api.py's renew_assignment route still compares
renew_lease()'s return value against a bare string; it will be
updated in the next task (Internal Host<->Cloud protocol) to consume
LeaseRenewalResult and populate the new cancel_requested wire field.
2026-07-15 18:08:48 +08:00
q792602257 88189770ff feat(runtime): add cancellation-aware stop_reason to TaskRunner and WorkflowRunner
- TaskRunner.run() and WorkflowRunner.run()/resume() accept an optional
  stop_reason callable alongside should_stop, distinguishing a genuine
  cancellation from other stop conditions (e.g. lost lease).
- is_cancellation_reason() shared helper added to runtime/task.py.
- WorkflowRunner._stop_status() now branches cancelled/failed based on
  stop_reason, correcting a prior blanket cancelled-on-any-stop behavior
  that conflicted with the host-agent-protocol spec's requirement to
  distinguish cancellation from lease-loss stops.
- Default behavior (stop_reason=None) is preserved exactly for both
  runners so existing callers/tests are unaffected.
- Task 1 of openspec change task-cancellation.
2026-07-15 17:52:33 +08:00
q792602257 a25542694d fix test
Tests / Test passed: 868
2026-07-15 17:04:59 +08:00
q792602257 17a709c92f fix(perception): disable PaddleOCR doc-unwarping for screenshots
Tests / Test tests.test_device_config.test_device_config_store_settings_get_set_and_defaults failed
PaddleOCR's OCR.yaml pipeline defaults to use_doc_orientation_classify
and use_doc_unwarping enabled, which are meant for photographed paper
documents. Applied to a flat, upright device screenshot, UVDoc
geometrically warps the image before detection, and returns box
coordinates in that warped space with no inverse mapping back to the
original image.

Verified on a real screenshot: with unwarping on, the same detected
element ("新项目") shifts from y=158 to y=71 versus the original image,
and 2 boxes near the top edge (status bar time/battery) are dropped
entirely. Disabling both flags by default (still overridable via
explicit kwargs) makes detected boxes match the original screenshot.
2026-07-15 16:51:36 +08:00
q792602257 7f439f0db5 fix(perception): reconcile points/pixels scale and stale overlay screenshot
Tests / Test tests.test_device_config.test_device_config_store_settings_get_set_and_defaults failed
Host-agent console showed OCR/UI-tree overlay boxes misaligned with the
displayed screenshot. Two independent causes, both confirmed with real
task data and pixel-level measurement of a user-provided screenshot:

1. perception/ui_parser.py parses XCUITest UI-tree bounds as iOS logical
   points, while scene_builder.py's Scene.width/height (via infer_png_size)
   and OCR bounds are in screenshot pixels, never reconciled (2.0x on
   Retina devices). build_scene() now detects the scale from the first
   x==0,y==0 UI element and rescales OCR bounds down to points-space,
   reporting Scene.width/height in points too. No-op for Android, where
   UiAutomator2 bounds already match pixels 1:1. This also fixes tap()
   landing at the wrong location for OCR-matched text, and lets the IOU
   fusion between UI-tree and OCR elements actually fire on iOS.

2. runtime/task.py captured `scene` (OCR/UI-tree data) before the LLM
   planning call, but re-captured `before_screenshot` for each step
   afterward - a real time gap during which on-screen content (e.g. a
   keyboard) could shift, producing a directional drift between the
   overlay and the displayed image. The first step of each plan batch
   now reuses the screenshot already taken for planning instead of
   capturing a new one; later steps in a multi-step batch still take a
   fresh capture (left unresolved, scoped out by request).

Regression tests added for both the scale reconciliation (using real
828x1792 vs 414x896 numbers) and the screenshot reuse behavior.
2026-07-15 16:12:53 +08:00
q792602257 8162509158 feat(host-agent): persist UI-tree evidence and add overlay/action visualization
Tests / Test passed: 863
Fixes issue 3: the host-agent console showed OCR results but never real
UI-tree data, because _ui_tree_nodes() checked for a get_ui_tree/ui_tree
tool action that has never existed anywhere in the codebase.

- storage/timeline.py: add a ui_tree_results field to TimelineRecord and
  Timeline.append(), mirroring the existing ocr_results field.
- runtime/task.py: _append_timeline() now extracts scene.elements with
  source == "ui" into ui_tree_results (scene_builder.build_scene() already
  preserved these; they were just never persisted).
- host_agent/web/app.py: _ui_tree_nodes() reads the new field directly
  instead of the dead tool-action check. New _overlay_payload() exposes
  each step's scene dimensions and fused element list for client-side
  rendering.
- task_detail.html: adds a toggle to overlay OCR (orange) and UI-tree
  (blue) bounding boxes on the before-action screenshot, plus a visual
  marker for the actually executed action (tap circle, or an animated
  swipe path) using an SVG viewBox so no manual coordinate-scaling JS is
  needed. Legacy/incomplete records degrade to no overlay, never an error.

Also corrects openspec/specs/runtime-task-evidence and
host-agent-console-task-pages, which had encoded the same nonexistent-tool
assumption, via the new host-agent-console-visual-evidence change.

600 tests passing; ruff/compileall/openspec validate all clean.
2026-07-15 14:39:28 +08:00
q792602257 367fd0d412 fix(planner): allow rationale/thinking by using tool_choice=auto
Tests / Test passed: 855
Forced tool_choice ("any"/"required") makes both Anthropic and OpenAI
skip any text/thinking block before the tool call, which silently made
rationale and thinking always None despite the planner-reflection-history
change's capture code being correct. Switch the primary call to
tool_choice="auto" (Anthropic: type=auto, disable_parallel_tool_use=true;
OpenAI: "auto") so the model can emit its reflection text, and add a
one-time forced retry (Anthropic "any", OpenAI "required", thinking
disabled) if the model responds without a tool call, guaranteeing a step
never stalls. Also add OpenAI text_output capture from message.content,
which was never extracted before (Anthropic-only gap).

Update planner-reflection-history design.md/tasks.md to document the bug
found during the pending manual smoke test (task 8.5) and the fix (new
section 9).
2026-07-15 13:43:39 +08:00
q792602257 a5aeb8889c feat(runtime): add planner reflection history with rationale and thinking
Tests / Test failed: 2, passed: 849
- ToolCallDecision captures thinking blocks and pre-tool text output
- AnthropicToolCallingClient supports optional extended thinking (budget_tokens + beta header)
- PlannedStep carries rationale and thinking from each LLM decision
- WorldEvent replaces scene_summary with rationale/thinking/page fields (backward-compatible)
- AI planner system prompt instructs reflection before each tool call
- _history_summary() emits compact {page, rationale, action, success} dicts
- Cloud DB migration 0011 adds nullable rationale/thinking columns to planner_decision_log
- OpenAI client extracts reasoning_content into thinking field
2026-07-15 12:43:22 +08:00
q792602257 77d4813bb2 feat(host-agent): make execution history authoritative
Tests / Test failed: 2, passed: 830
2026-07-15 11:46:27 +08:00
q792602257 ccde30e378 feat(runtime): capture step evidence in console
Tests / Test passed: 862
2026-07-15 10:12:09 +08:00
q792602257 8d5b02e37f fix(perception): degrade to OCR-only when UI tree is unavailable
Tests / Test passed: 858
driver.tree() failures (WDA/Appium session errors) previously raised
uncaught, killing describe_screen() before OCR ever ran. Malformed
tree content (invalid XML) had the same problem inside
parse_ui_tree(). Both are now caught and logged, falling back to an
empty ui_elements list so the scene degrades to OCR-only, mirroring
the existing OCR-failure fallback in run_ocr().
2026-07-15 10:03:30 +08:00
q792602257 778af2da53 fix(cloud): align planner configuration and records
Tests / Test passed: 856
2026-07-15 09:43:14 +08:00
q792602257andClaude Opus 4.6 8300c3b6b7 Merge branch 'worktree-runtime-console-jinja2-templates'
Server-rendered Jinja2 Runtime console at /ui/, replacing the Vue/Vite SPA.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 08:05:02 +08:00
q792602257andClaude Opus 4.6 e00c50e703 feat(api): server-rendered Jinja2 Runtime console at /ui/
Replaces the separate Vue/Vite `console/` SPA with a same-origin,
server-rendered console built on a module-level Jinja2 Environment
with select_autoescape(["html","xml"]).

- Add api/console_web.py with /ui/ routes (dashboard, tasks, task
  detail/timeline, config) and a _status_fragment polled every 10s.
- Refactor api/console.py into a typed ConsoleService shared by the
  JSON and HTML routers so validation/persistence cannot drift.
- Remove RUNTIME_CONSOLE_STATIC_DIR, SpaStaticFiles, and the wildcard
  CORS middleware from api/rest.py; GET / now redirects to /ui/.
- Delete the top-level console/ project; add jinja2 and python-multipart
  as direct dependencies and ship templates/CSS/JS via package-data.
- Add 31 tests (XSS probes, PRG flows, fragment refresh, no-static-dir
  and no-CORS regressions, wheel-packaging smoke test).

/console/* JSON endpoints remain unchanged. The console keeps the
trusted-network-only boundary; auth/CSRF is intentionally deferred.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 08:03:13 +08:00
q792602257andClaude Opus 4.6 a8ba2312fc feat(skills): cloud sync client + incremental sync + fork-on-revocation
Adds CloudApiSkillClient (Cloud API per-host sync endpoint + inventory
report), forwards since_version for incremental sync (full-replace on
first/stale), and forks a local override into a standalone local skill
when its cloud skill is revoked (design D9). Skill-side tests green (87
passed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 07:55:08 +08:00
q792602257andClaude Opus 4.6 8baf3a6a8b feat(skills): unified read-merge + authoring/override MCP tools
Adds the merged read surface (api/skill_catalog_view.py) over synced +
local stores with origin discrimination and override precedence, and
extends the skill MCP tools with create_skill/update_skill/delete_skill
that dispatch by origin (edit local skills; create/update/remove local
overrides for cloud skills). Wired into api.mcp.create_mcp_server.
Full non-integration suite green (564 passed).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 07:38:12 +08:00
q792602257andClaude Opus 4.6 dd03abbbb0 feat(skills): open skill-management-console change + local skill store
Opens the skill-management-console openspec change (cloud/local skill split
with local override) with proposal, design (D1-D11), four delta specs, and
tasks. Implements the agent-side persistent local skill store
(storage/local_skills.py): authored local skills + cloud-skill overrides in
a physically separate SQLite file, with fork-on-revocation. 10 tests pass.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-15 07:28:42 +08:00
q792602257 6e511111c4 fix(perception): harden PaddleOCR result handling
Tests / Test passed: 793
2026-07-14 20:07:49 +08:00
q792602257 99bde4febb test: align deployment and lease contracts
Tests / Test passed: 748
2026-07-14 13:53:00 +08:00
q792602257andClaude Opus 4.6 ec261d57c2 feat: surface task execution progress across Host Agent and Cloud
Host Agent now persists step-level execution detail locally (via a real
TaskMetadataStore/Timeline wired into TaskRunner) and reports a bounded
in-progress snapshot piggybacked on lease renewal. Cloud persists that
snapshot per active assignment and exposes it through the existing task
list/detail query path; Cloud Console renders it as a live badge. Host
Agent's local console gains authenticated, read-only task list and
detail/timeline pages (same-origin, server-rendered) with inlined
screenshots.

Also fixes a pre-existing gap in the shared Timeline: the actual
per-step LLM prompt is now recorded instead of the task goal, benefiting
both Runtime and Host Agent consoles. When a host uses the cloud planner
transport, each decide call's prompt and resulting tool decision are
durably logged in a new planner_decision_log table (with bounded
retention) and browsable from Cloud Console; direct-transport hosts
explicitly surface a "not reported" state.

Includes Alembic migrations 0008 (progress columns on scheduled_tasks)
and 0009 (planner_decision_log), bounded Host-Agent-local retention,
dual-backend repository parity, and Vitest + pytest coverage. Task 6.5
(manual end-to-end device verification) remains.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-14 12:47:49 +08:00
q792602257 9250254dec feat(cloud): support Anthropic provider base URL
Tests / Test passed: 665
2026-07-14 07:58:38 +08:00
q792602257 a166ffd8a4 feat(cloud): manage LLM providers in database
Tests / Test passed: 664
2026-07-14 00:31:47 +08:00
q792602257 b613a315ff docs(cloud): document governance rollout
Tests / Test passed: 665
2026-07-13 23:22:53 +08:00
q792602257 5b64efab53 feat(cloud): report planner transport status 2026-07-13 23:15:46 +08:00
q792602257 40efa53411 feat(cloud): attribute planner token usage 2026-07-13 23:10:21 +08:00
q792602257 b4803f90e6 feat(cloud): enforce host governance budgets
Tests / Test passed: 662
2026-07-13 22:56:31 +08:00
q792602257 2cd314b183 feat(cloud): add targeted task governance foundation
Tests / Test passed: 659
2026-07-13 22:21:12 +08:00
q792602257 a68f609453 Implement cloud-planner-proxy: AI planner routes through Cloud API
Implements all 19 tasks of the cloud-planner-proxy OpenSpec change:

- Cloud API: cloud.planner_config (CloudPlannerConfig, load/build helpers)
  reusing runtime.tool_calling_client provider clients (no new dependency
  needed -- device-cloud-platform already depends on device-agent-runtime).
- Cloud API: new host-scoped POST /internal/v1/hosts/{host_id}/planner/decide
  internal endpoint, reusing existing bearer auth; logs only metadata
  (host id, tool name, latency, error class), never prompt/screenshot
  content.
- Host Agent: new AI_PLANNER_TRANSPORT config (direct default | cloud) and
  host_agent/cloud_planner_client.py::CloudProxyToolCallingClient, a
  synchronous ToolCallingClient implementation (structural, not importing
  runtime) that calls the new endpoint via its own httpx.Client -- avoids
  bridging the async HostAgentClient across the worker-thread boundary
  that AIPlanner.plan() runs in (asyncio.to_thread in lease.py).
- Host Agent wiring: create_execution_factories()/_host_agent_planner()
  select the cloud-proxy client only when AI_PLANNER_TRANSPORT=cloud;
  direct/unset transport is unchanged (still the default).
- Tests: 22 new tests across Cloud API config, the new endpoint, the new
  client, and transport-selection wiring; full non-integration suite
  (492 tests) passes with no regressions.
- Docs: docs/CLOUD_DEPLOYMENT.md documents the cloud transport, its
  trade-offs, and the credential split between Host Agent and Cloud API.

proposal.md/design.md were corrected during implementation to reflect two
findings: no new anthropic/openai dependency is actually needed, and
CloudProxyToolCallingClient uses its own sync httpx.Client rather than a
new HostAgentClient method, per the thread-boundary reasoning above.
2026-07-13 21:27:48 +08:00
q792602257 c162c2501b feat(cloud): remove static credentials and add host console
Tests / Test No test results found
2026-07-13 19:45:53 +08:00
q792602257 efeb3eb926 Implement edge-host-self-enrollment
Tests / Test passed: 581
Host Agent:
- One-time local operator account bootstrap (PBKDF2-HMAC-SHA256, atomic
  0600-permission write) gating the daemon's first unattended start via a
  new `setup` CLI subcommand.
- Default control-plane URL now https://amcp.home.jerryyan.top (env var
  override unchanged).
- Enrollment no longer requires a pre-issued token; falls back to
  zero-token self-service enrollment when none is configured.

Cloud control plane:
- CLOUD_SELF_SERVICE_ENROLLMENT_ENABLED (default false) opt-in flag.
- SelfServiceEnrollmentAuthProvider + ChainedEnrollmentAuthProvider:
  configured tokens still take priority; self-service only applies when
  no token matches, preserving edge-host-enrollment's token-bound path.
- Fixed a latent bug in sql_repository.py::enroll_host: the token-conflict
  lookup used `== enrollment_token_digest`, which SQLAlchemy compiles to
  `IS NULL` when the value is None, so every self-service enrollment after
  the first would have falsely collided with an existing NULL-digest host.
  Skipped that lookup entirely when the digest is None.

Docs/deploy: .env.example, compose.yaml, compose.deploy.yaml,
CLOUD_DEPLOYMENT.md, MACOS_IPHONE_SETUP.md updated for the new flag,
URL default, and required `device-host-agent setup` step.

Verification: 494 non-integration tests pass; openspec validate --strict
passes. PostgreSQL-backed contract tests and full manual end-to-end
verification were not run (no Postgres/Docker or reachable cloud-api in
this environment); noted as unchecked in tasks.md 7.2/7.4.
2026-07-13 18:30:49 +08:00
q792602257 cdef630e67 feat(cloud-console): add user authentication and administration 2026-07-13 17:54:53 +08:00
q792602257 8f74eac50c Jenkins
Tests / Test passed: 545
2026-07-13 16:11:40 +08:00
q792602257andClaude Opus 4.6 50f0b8ade9 feat(driver): add Android UiAutomator2 driver
Tests / Test No test results found
Register `SUPPORTED_DRIVER_TYPES["uiautomator2"]` backed by the new
`AndroidDriver`, mirroring `WDADriver` method-for-method. tap/swipe/home use
the confirmed UiAutomator2 mobile commands (`clickGesture`, `dragGesture`,
`pressKey`); swipe converts `duration_ms` to a drag speed with a zero-distance
guard. Adds 34 mocked unit tests, an env-var-gated integration test, and
corrects the outdated "Android not registered" note in the iPhone setup doc.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-13 14:47:01 +08:00
q792602257andClaude Opus 4.6 2169bb03d9 feat(cloud-console): task listing, attempt history, CORS, and console SPA
Implements the cloud-console OpenSpec change: adds GET /v1/tasks (filterable,
bounded pagination, tasks:read) and GET /v1/tasks/{id}/attempts (404 on unknown
task) to the platform SDK, with matching CloudClient methods and a closed-by-
default CLOUD_CONSOLE_CORS_ORIGINS allow-list wired through CloudControlConfig.
Ships an independent Vue 3 + Vite SPA at cloud-console/ that authenticates with
an operator-supplied bearer token held in sessionStorage, renders tasks with
attempt history, device pool, host registry, and the plugin registry with a
registration form.

Backend test suite: 438 passed (-m "not integration"); cloud-console typecheck
and production build both succeed. PostgreSQL-backed repository tests and
manual end-to-end verification remain pending external infrastructure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-13 14:00:23 +08:00
q792602257 e61dcca801 feat(cloud): add edge host enrollment 2026-07-13 13:54:16 +08:00
q792602257 7b717a05a9 test(openspec): map cloud integration scenarios 2026-07-13 08:21:44 +08:00
q792602257 22d37ca95b chore(cloud): verify integration change 2026-07-13 08:06:15 +08:00
q792602257 e28705b0f2 docs(cloud): add deployment runbook 2026-07-13 07:58:24 +08:00
q792602257 90961bd0a4 fix(host-agent): load configured devices 2026-07-13 07:56:41 +08:00
q792602257 dbd70732e1 build(deploy): add cloud stack containers 2026-07-12 22:58:17 +08:00
q792602257 1f95d23beb feat(cloud-sdk): authenticate client requests 2026-07-12 22:53:34 +08:00
q792602257 16bcbf5a27 feat(cloud-sdk): expose distributed task status 2026-07-12 19:57:03 +08:00
q792602257 ec1e8c20d8 feat(host-agent): renew active assignment leases 2026-07-12 19:06:22 +08:00
q792602257 8131c0124b feat(cloud-api): add correlated lifecycle logging 2026-07-12 18:37:51 +08:00
q792602257 09cfe54dd9 test(host-protocol): verify internal API isolation 2026-07-12 18:17:39 +08:00
q792602257 a24efc1043 feat(host-protocol): renew and complete leases 2026-07-12 18:15:04 +08:00
q792602257 c7f8d0c128 feat(host-protocol): long poll assignments 2026-07-12 18:12:33 +08:00
q792602257 efb754fbe7 feat(host-protocol): accept heartbeat snapshots 2026-07-12 18:10:42 +08:00