feat(planner): persist reusable action semantics
Tests / Test passed: 879

This commit is contained in:
2026-07-15 18:14:28 +08:00
parent 361dada276
commit d69be48f96
41 changed files with 733 additions and 116 deletions
@@ -8,7 +8,8 @@ The AI planner's execution history currently stores raw `scene_summary` (full UI
- **Rationale capture**: `ToolCallDecision` captures the AI's pre-tool text output (`text_output`) and, when extended thinking is enabled, the thinking block (`thinking`). Both flow through `PlannedStep` into `WorldEvent`.
- **Extended thinking support**: `AnthropicToolCallingClient` gains optional `thinking_budget_tokens` config. When set, the Anthropic API is called with `thinking` enabled (interleaved thinking beta); the thinking block is extracted and stored.
- **OpenAI reasoning capture**: `OpenAIToolCallingClient` extracts `reasoning_content` from responses when present (o-series models).
- **WorldEvent schema change**: `scene_summary` is replaced by `rationale` (from `text_output`) and `thinking` (from thinking block), plus `current_page` from `WorldState` for minimal page-level verification context. This reduces per-step history token cost by an order of magnitude.
- **Required action metadata**: every device-action tool call must return a concise `purpose` and observable `expected_outcome`; the Runtime stores them separately from executable tool arguments so successful executions can be reused as semantically meaningful flows.
- **WorldEvent schema change**: `scene_summary` is replaced by rationale/thinking plus the action name, executable arguments, purpose, expected outcome, and `current_page` from `WorldState`, giving later planning and skill reuse a compact but complete action record.
- **History format**: `_history_summary()` in `ai_planner.py` switches from full `WorldEvent.to_dict()` to a compact `{page, rationale, action, success}` format.
- **planner_decision_log extension**: The Cloud-side decision log table adds `thinking` and `rationale` columns to persist these fields alongside existing prompt/tool records.
@@ -28,6 +29,7 @@ The AI planner's execution history currently stores raw `scene_summary` (full UI
- `runtime/tool_calling_client.py``ToolCallDecision`, `AnthropicToolCallingClient`, `OpenAIToolCallingClient`, response parsers
- `runtime/planner.py``PlannedStep`
- `runtime/tool_specs.py` — required purpose/expected-outcome fields for device actions
- `runtime/ai_planner.py``AIPlanner.plan()`, `_history_summary()`
- `runtime/planner_prompts.py``PLANNER_SYSTEM_PROMPT`, `planner_user_prompt`
- `runtime/planner_config.py` — new `thinking_budget_tokens` field
@@ -35,7 +37,8 @@ The AI planner's execution history currently stores raw `scene_summary` (full UI
- `world/model.py``_append_history()`
- `packages/cloud-platform/cloud/db_models.py``planner_decision_log` table
- `packages/cloud-platform/cloud/schema.py` — Alembic migration
- `packages/cloud-platform/cloud/internal_api/api.py``record_planner_decision()`
- `packages/cloud-platform/cloud/internal_api/models.py``PlannerDecisionRecord`
- No changes to `CloudProxyToolCallingClient`thinking/text are surfaced at the local client layer only; the cloud proxy is transparent to them.
- `packages/cloud-platform/cloud/internal_api/api.py``record_planner_decision()` and planner decision response
- `packages/cloud-platform/cloud/internal_api/models.py`planner decision transport models
- `apps/device-host-agent/host_agent/cloud_planner_client.py`return reflection and action metadata from the Cloud proxy
- `skills_learning/` — retain action purpose/expected outcome in synthesized flow steps
- No new external dependencies; Anthropic extended thinking uses existing SDK via beta header.