feat(agent-runtime): add LLM-driven AI Planner with dual-provider tool calling

Replaces the stub Planner's fixed describe_screen/[] behavior with a real
decision-maker: AIPlanner uses native tool/function calling (Anthropic or
OpenAI, pluggable via AI_PLANNER_PROVIDER) to select exactly one grounded
action per turn, with an explicit finish_task(success, reason) tool for
completion/failure instead of an ambiguous "no tool call" signal. Default
disabled (AI_PLANNER_ENABLED=false) and additive; TaskRunner falls back to
the existing stub Planner unchanged when disabled.

Amends CONSTITUTION.md's Perception Boundary with one narrow exception:
only the AI Planner may receive the current step's raw screenshot bytes
alongside Scene, for vision-grounded coordinate grounding. Also fixes a
latent gap in TaskRunner.run(): observe/plan exceptions are now caught per
iteration and turned into a failed task with a failure_reason, instead of
propagating uncaught.

openspec change: ai-planner-runtime.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

This commit is contained in:

q792602257

2026-07-12 13:48:50 +08:00

co-authored by

Claude Sonnet 5

parent b94abde92a

commit 61ff3b425d

19 changed files with 1977 additions and 19 deletions

									
										openspec/changes/ai-planner-runtime/.openspec.yaml
									
		+2
		
												View File
												
				@@ -0,0 +1,2 @@

				schema: spec-driven

				created: 2026-07-12