4 Commits

Include renames

Author	SHA1	Message	Date
q792602257	5a93651db7	feat(planner): expose long_press/double_tap to the AI planner	2026-07-15 19:45:06 +08:00
q792602257	361dada276	feat(perception): surface accessibility interaction state on UI-tree elements ... SceneElement gains enabled/clickable/selected/checked/focused (bool | None), populated from the literal attributes Appium's XCUITest and UiAutomator2 page_source already emit (iOS: enabled only; Android: all five). None means "not reported by this platform", not false. to_dict() omits unset fields to keep the LLM-facing scene JSON compact; planner_prompts.py documents the new fields so the AI planner knows how to use them (e.g. don't tap disabled elements, use selected/checked to judge whether a toggle already matches the goal).	2026-07-15 18:10:53 +08:00
q792602257	a5aeb8889c	feat(runtime): add planner reflection history with rationale and thinking ... - ToolCallDecision captures thinking blocks and pre-tool text output - AnthropicToolCallingClient supports optional extended thinking (budget_tokens + beta header) - PlannedStep carries rationale and thinking from each LLM decision - WorldEvent replaces scene_summary with rationale/thinking/page fields (backward-compatible) - AI planner system prompt instructs reflection before each tool call - _history_summary() emits compact {page, rationale, action, success} dicts - Cloud DB migration 0011 adds nullable rationale/thinking columns to planner_decision_log - OpenAI client extracts reasoning_content into thinking field	2026-07-15 12:43:22 +08:00
q792602257andClaude Sonnet 5	61ff3b425d	feat(agent-runtime): add LLM-driven AI Planner with dual-provider tool calling ... Replaces the stub Planner's fixed describe_screen/[] behavior with a real decision-maker: AIPlanner uses native tool/function calling (Anthropic or OpenAI, pluggable via AI_PLANNER_PROVIDER) to select exactly one grounded action per turn, with an explicit finish_task(success, reason) tool for completion/failure instead of an ambiguous "no tool call" signal. Default disabled (AI_PLANNER_ENABLED=false) and additive; TaskRunner falls back to the existing stub Planner unchanged when disabled. Amends CONSTITUTION.md's Perception Boundary with one narrow exception: only the AI Planner may receive the current step's raw screenshot bytes alongside Scene, for vision-grounded coordinate grounding. Also fixes a latent gap in TaskRunner.run(): observe/plan exceptions are now caught per iteration and turned into a failed task with a failure_reason, instead of propagating uncaught. openspec change: ai-planner-runtime. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-12 13:48:50 +08:00