agentic-mobile-control/openspec/changes/planner-reflection-history/specs/agent-runtime/spec.md

MODIFIED Requirements

Requirement: LLM-driven Planner selects exactly one grounded action per turn

The system SHALL provide a Planner implementation that, given a goal, the current Scene, recent task history in compact rationale-based format, and optional prior-step context, uses native LLM tool/function calling to select exactly one action (or the completion signal defined below) per plan() invocation, grounding any coordinates in the current turn's Scene element bounds. The Planner SHALL instruct the LLM to output a short text block (rationale) before the tool call, reflecting on the previous step's outcome and stating the current step's intent.

Scenario: Planner selects a single action for the current turn

• WHEN the AI Planner is invoked with a goal and the current Scene
• THEN it returns at most one PlannedStep, whose action and arguments come from exactly one tool call chosen by the underlying LLM for that turn

Scenario: Planner re-decides every turn from the current Scene

• WHEN the AI Planner is invoked again after a prior step has executed
• THEN its decision is grounded in the newly observed Scene for that turn, not in coordinates or assumptions carried over from a previous turn

Scenario: PlannedStep carries rationale when the LLM outputs a text block

• WHEN the LLM response includes a text block before the tool call
• THEN the returned PlannedStep has a non-None rationale containing that text

Scenario: PlannedStep carries thinking when extended thinking is enabled and the LLM outputs a thinking block

• WHEN extended thinking is enabled and the LLM response includes a thinking block
• THEN the returned PlannedStep has a non-None thinking containing the thinking block text

Scenario: PlannedStep carries None rationale when no text block is present

• WHEN the LLM response contains only a tool call with no preceding text block
• THEN PlannedStep.rationale is None and the step is returned normally

Requirement: Pluggable dual-provider tool-calling abstraction

The system SHALL support at least two interchangeable LLM providers (Anthropic native tool use and OpenAI function calling) for the AI Planner's decision calls, selectable via configuration, with both providers constrained to return exactly one tool call per request. The Anthropic client SHALL additionally support optional extended thinking via a configurable thinking_budget_tokens value. The OpenAI client SHALL capture reasoning_content from responses when present. Independently of provider selection, the system SHALL support at least two transports for making that decision call — direct-to-provider and cloud-proxy — selectable via configuration without requiring any change to AIPlanner's own decision logic.

Scenario: Provider selected via configuration

• WHEN the AI Planner is configured with a given provider identifier
• THEN it constructs and uses the tool-calling client for that provider without requiring any change to AIPlanner's own decision logic

Scenario: Provider response resolves to a single decision

• WHEN either supported provider returns a response to a tool-calling request
• THEN the response is parsed into exactly one tool name and one arguments object, regardless of which provider produced it

Scenario: Transport selected via configuration

• WHEN the Host Agent is configured with a given transport (direct or cloud-proxy)
• THEN AIPlanner is constructed with the tool-calling client for that transport, and its own decision logic is unchanged regardless of which transport is in effect

Scenario: Cloud-proxy transport is the default

• WHEN no transport is explicitly configured
• THEN the AI Planner uses the cloud-proxy transport and the Cloud Control Plane's planner-decision endpoint

Scenario: Direct transport remains available by explicit configuration

• WHEN the Host Agent is configured with the direct transport
• THEN the AI Planner uses the direct-to-provider transport with locally configured credentials

Scenario: Cloud-proxy transport resolves a decision without a local provider client

• WHEN the Host Agent is configured with the cloud-proxy transport
• THEN its tool-calling client sends the decision request to the Cloud Control Plane's planner-decision endpoint instead of constructing a local Anthropic or OpenAI SDK client

Scenario: Cloud-proxy transport returns None for thinking and text_output

• WHEN the Host Agent uses cloud-proxy transport
• THEN ToolCallDecision.thinking and ToolCallDecision.text_output are None because the proxy surface does not expose them