Forced tool_choice ("any"/"required") makes both Anthropic and OpenAI
skip any text/thinking block before the tool call, which silently made
rationale and thinking always None despite the planner-reflection-history
change's capture code being correct. Switch the primary call to
tool_choice="auto" (Anthropic: type=auto, disable_parallel_tool_use=true;
OpenAI: "auto") so the model can emit its reflection text, and add a
one-time forced retry (Anthropic "any", OpenAI "required", thinking
disabled) if the model responds without a tool call, guaranteeing a step
never stalls. Also add OpenAI text_output capture from message.content,
which was never extracted before (Anthropic-only gap).
Update planner-reflection-history design.md/tasks.md to document the bug
found during the pending manual smoke test (task 8.5) and the fix (new
section 9).
This commit is contained in:
@@ -51,7 +51,15 @@ Constraints:
|
||||
|
||||
**Why over method A (implicit)**: Method A relies on the AI organically reflecting without prompting, which is unreliable. Explicit instruction in the system prompt makes it consistent.
|
||||
|
||||
**Constraint**: `tool_choice: {type: "any"}` already allows text blocks before a tool use block. No API parameter changes needed for text output capture.
|
||||
**Correction (post-implementation, found during manual smoke testing per task 8.5)**: The original assumption that `tool_choice: {type: "any"}` allows text blocks before `tool_use` was wrong. Per Anthropic's API behaviour, forced `tool_choice` (`any` or a specific tool) makes Claude skip any preceding text block entirely, and forced tool_choice is also incompatible with extended thinking. Under the original `tool_choice: {type: "any"}` call, `text_output`/`thinking` were therefore *always* `None` in practice — not merely "sometimes absent" as D1/Risks originally assumed. See D8 for the fix.
|
||||
|
||||
### D8: `tool_choice` must be `"auto"` (with a forced retry fallback) to allow rationale/thinking capture
|
||||
|
||||
**Decision**: `AnthropicToolCallingClient`/`OpenAIToolCallingClient` now call the API with `tool_choice: "auto"` first (Anthropic: `{"type": "auto", "disable_parallel_tool_use": True}`; OpenAI: `"auto"`). Extended thinking (when `thinking_budget_tokens` is set) is only ever requested alongside `"auto"`. If the model responds without any tool call at all, the client retries once with the original forced `tool_choice` (Anthropic `"any"`, OpenAI `"required"`) and no thinking parameter, guaranteeing a tool call is eventually returned. `OpenAIToolCallingClient` also now captures `message.content` as `text_output` (previously never extracted for any provider — OpenAI never had rationale capture at all, forced or not).
|
||||
|
||||
**Why not just always force tool_choice with a "think first" instruction**: Confirmed via Anthropic's official docs/SDK guidance that this combination structurally suppresses the text/thinking Claude would otherwise produce — no amount of prompting fixes it while `tool_choice` stays forced.
|
||||
|
||||
**Why a fallback retry rather than failing the step**: `PLANNER_SYSTEM_PROMPT` already instructs "you must then call exactly one tool", so `tool_choice: "auto"` calls overwhelmingly still return a tool_use block; the retry only guards the rare case where the model responds with pure text. Failing the task step outright on that rare case would regress reliability for a` cosmetic (`rationale`) improvement. The forced retry accepts losing rationale/thinking for that one step rather than losing task progress.
|
||||
|
||||
### D2: Extend `ToolCallDecision` with `thinking` and `text_output`
|
||||
|
||||
@@ -106,7 +114,8 @@ Constraints:
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **AI may not always output a text block**: `tool_choice: {type: "any"}` does not guarantee a text block. When absent, `text_output` is `None` and `rationale` is `None`. History degrades gracefully to `{page, rationale: null, action, success}`. No retries or fallbacks needed.
|
||||
- **AI may not always output a text block**: even with `tool_choice: "auto"` (D8), the model is not guaranteed to prefix a text block before the tool call. When absent, `text_output` is `None` and `rationale` is `None`. History degrades gracefully to `{page, rationale: null, action, success}`.
|
||||
- **`tool_choice: "auto"` occasionally yields no tool call at all**: unlike forced `tool_choice`, `"auto"` permits the model to respond with text only and no tool call. D8's forced retry (no thinking, no rationale on that path) guards this case so a step never stalls; this trades away rationale/thinking for that single step, not overall reliability.
|
||||
- **Extended thinking increases latency**: `budget_tokens` directly adds to minimum response time. This is opt-in and accepted by the operator who enables it.
|
||||
- **`WorldEvent` schema divergence from stored data**: Existing `WorldEvent` instances in memory or serialised timelines lack `rationale`/`thinking`. The `to_dict()` method will emit `null` for these fields; downstream consumers should treat `null` as absent, not as a failure.
|
||||
- **Cloud-proxy transport never produces thinking/rationale at the client layer**: The proxy returns only `tool_name`/`arguments`. `ToolCallDecision.thinking` and `.text_output` will always be `None` for cloud-transport tasks. The `planner_decision_log` on the cloud side will be populated from the cloud-proxied call itself (D7), which does see the full LLM response.
|
||||
|
||||
Reference in New Issue
Block a user