This commit is contained in:
@@ -36,9 +36,7 @@ Constraints:
|
||||
|
||||
**Non-Goals:**
|
||||
- Not adding a dedicated reflection LLM call (explicit method B) — method C (pre-tool text block) achieves the intent within the existing call budget.
|
||||
- Not changing `CloudProxyToolCallingClient` — it transparently proxies and is not aware of thinking/text blocks.
|
||||
- Not changing the `scene_summary` field in `WorldEvent` for stored/displayed task history in the Console — only the prompt-construction path (`_history_summary()`) is changed.
|
||||
- Not surfacing thinking/rationale in the Cloud Console UI (that's a follow-on concern).
|
||||
- Not making `thinking_budget_tokens` configurable per-task at runtime (only via environment variable).
|
||||
|
||||
## Decisions
|
||||
@@ -112,13 +110,42 @@ Constraints:
|
||||
|
||||
**Why separate columns rather than a JSON blob**: The existing table uses discrete columns for `system_prompt`, `user_prompt`, `tool_name`, `tool_arguments` — consistency favours discrete columns. Both fields are optional (cloud-proxy path only; `direct` transport never produces cloud decision records).
|
||||
|
||||
### D9: Device-action tool calls carry required purpose and expected outcome
|
||||
|
||||
**Decision**: Every device-action schema (`tap`, `swipe`, `input_text`,
|
||||
`launch_app`, and `terminate_app`) adds required non-empty `purpose` and
|
||||
`expected_outcome` string fields. The tool-call response parser removes these
|
||||
metadata fields from executable arguments and assigns them to
|
||||
`ToolCallDecision`; `AIPlanner` carries them on `PlannedStep`. The completion
|
||||
signal remains unchanged because it already has a required terminal `reason`.
|
||||
|
||||
The Cloud planner response returns rationale, thinking, purpose, and expected
|
||||
outcome to the Host. The Cloud decision log persists the four values as
|
||||
nullable, additive columns so historic rows and non-AI callers remain
|
||||
readable. `WorldEvent` retains the executable action arguments together with
|
||||
purpose and expected outcome; its compact history summary includes that action
|
||||
record. Timeline records and learned `FlowStep` instances retain the same
|
||||
arguments and metadata; flow embedding text includes available semantics so
|
||||
retrieval can use them.
|
||||
|
||||
**Why structured tool arguments rather than rationale**: the pre-tool text
|
||||
block is optional by API design and can be suppressed by a forced retry. Tool
|
||||
schemas are the provider-enforced structured-output boundary, so requiring
|
||||
purpose and expected outcome there makes them available for every submitted
|
||||
device action without relying on free-form rationale.
|
||||
|
||||
**Why strip metadata before execution**: device tool functions only accept
|
||||
their physical-action arguments. Keeping metadata off `step.args` preserves
|
||||
their API contracts while still making it available to execution history,
|
||||
verification, Cloud audit, and skill synthesis.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **AI may not always output a text block**: even with `tool_choice: "auto"` (D8), the model is not guaranteed to prefix a text block before the tool call. When absent, `text_output` is `None` and `rationale` is `None`. History degrades gracefully to `{page, rationale: null, action, success}`.
|
||||
- **`tool_choice: "auto"` occasionally yields no tool call at all**: unlike forced `tool_choice`, `"auto"` permits the model to respond with text only and no tool call. D8's forced retry (no thinking, no rationale on that path) guards this case so a step never stalls; this trades away rationale/thinking for that single step, not overall reliability.
|
||||
- **Extended thinking increases latency**: `budget_tokens` directly adds to minimum response time. This is opt-in and accepted by the operator who enables it.
|
||||
- **`WorldEvent` schema divergence from stored data**: Existing `WorldEvent` instances in memory or serialised timelines lack `rationale`/`thinking`. The `to_dict()` method will emit `null` for these fields; downstream consumers should treat `null` as absent, not as a failure.
|
||||
- **Cloud-proxy transport never produces thinking/rationale at the client layer**: The proxy returns only `tool_name`/`arguments`. `ToolCallDecision.thinking` and `.text_output` will always be `None` for cloud-transport tasks. The `planner_decision_log` on the cloud side will be populated from the cloud-proxied call itself (D7), which does see the full LLM response.
|
||||
- **Cloud/Host deployment order**: a new Host requires a Cloud API that returns the additive metadata fields to preserve them locally. The Cloud response models remain nullable so an old peer remains readable during a rolling deployment, but it cannot provide the new semantic records.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
|
||||
Reference in New Issue
Block a user