## 1. Configuration and tool face - [x] 1.1 Create `runtime/planner_config.py`: `PlannerConfig` (`enabled`, `provider`, `model`, `timeout`), `load_config()` reading `AI_PLANNER_ENABLED`/`AI_PLANNER_PROVIDER`/`AI_PLANNER_MODEL`/ `AI_PLANNER_TIMEOUT_SECONDS`, `enabled` defaulting `False` - [x] 1.2 Create `runtime/tool_specs.py`: vendor-neutral `ToolSpec` dataclass plus `TAP_SPEC`/`SWIPE_SPEC`/`INPUT_TEXT_SPEC`/`LAUNCH_APP_SPEC`/ `TERMINATE_APP_SPEC`/`FINISH_TASK_SPEC`, matching real tool signatures, exported as `ACTION_TOOL_SPECS` (5) and `ALL_TOOL_SPECS` (6) - [x] 1.3 Create `runtime/planner_prompts.py`: `PLANNER_SYSTEM_PROMPT` and `planner_user_prompt(*, goal, scene_json, history_summary)` ## 2. Dual-provider tool-calling client - [x] 2.1 Create `runtime/tool_calling_client.py`: `ToolCallDecision`, `ToolCallUnavailable`, `ToolCallingClient` Protocol - [x] 2.2 Implement `AnthropicToolCallingClient` (lazy SDK import, injectable transport, forced single tool call via `tool_choice={"type": "any", "disable_parallel_tool_use": True}`, image content block when a screenshot is present) - [x] 2.3 Implement `OpenAIToolCallingClient` (lazy SDK import, injectable transport, `max_completion_tokens`, forced single tool call via `tool_choice="required"` + `parallel_tool_calls=False`, `image_url` data-URI block when a screenshot is present) - [x] 2.4 Implement `build_client(config)` provider selection ## 3. AI Planner and runtime wiring - [x] 3.1 Create `runtime/ai_planner.py::AIPlanner` (single-step decision, `finish_task` → empty plan / `TaskFailedError`, `goal_reached()` permanently `False`) - [x] 3.2 Add `screenshot: bytes | None = None` to base `runtime/planner.py::Planner.plan()` - [x] 3.3 Wire `runtime/task.py::TaskRunner`: `planner_config` constructor param, `_default_planner()` (AI Planner when enabled, stub otherwise), generalize `_planner_accepts_world()` into `_planner_accepts(name)`, add `_planning_screenshot()`, conditionally inject `screenshot=` in `_plan()` - [x] 3.4 Wrap `TaskRunner.run()`'s per-iteration observe+plan in try/except, marking the task `status="failed"` with a `failure_reason` on any exception instead of propagating it uncaught ## 4. Constitution amendment - [x] 4.1 Amend `docs/CONSTITUTION.md`'s Perception Boundary section with the narrow, Planner-only screenshot exception ## 5. openspec change artifacts - [x] 5.1 Write `proposal.md`, `design.md` (with explicit Constitution Compliance section), `tasks.md` - [x] 5.2 Write `specs/agent-runtime/spec.md` (`## ADDED Requirements` only) ## 6. Tests - [x] 6.1 `tests/test_planner_config.py`: env var parsing, defaults, `enabled` defaults `False`, invalid/negative timeout falls back to default - [x] 6.2 `tests/test_tool_specs.py`: each `ToolSpec.parameters` schema (required fields, `additionalProperties: False`), `ALL_TOOL_SPECS` has exactly 6 entries - [x] 6.3 `tests/test_tool_calling_client.py`: fake-transport tests per provider — forced single-tool-call fields present, image block present/absent based on screenshot, successful response parses to `ToolCallDecision`, malformed/error response raises `ToolCallUnavailable` - [x] 6.4 `tests/test_ai_planner.py`: fake `ToolCallingClient` — action decision → single `PlannedStep`; `finish_task(success=True)` → `[]`; `finish_task(success=False, reason=...)` → raises `TaskFailedError`; `goal_reached()` always `False` - [x] 6.5 Added `tests/test_ai_planner_task_runner.py` (kept `tests/test_task_loop.py` untouched rather than extending it): planner exception mid-task → task ends `status="failed"` with a `failure_reason` (not stuck `running`); observer exception is caught the same way; a narrow-signature Planner does not receive an unexpected `screenshot` kwarg while one that declares it does; `PlannerConfig(enabled=False)` (default) still yields the stub `Planner`; `PlannerConfig(enabled=True)` without an explicit `planner=` yields an `AIPlanner` - [x] 6.6 `tests/test_ai_planner_integration.py` (`@pytest.mark.integration`, skipped without a real API key): one real call per provider ## 7. Verification - [x] 7.1 Ran `pytest -m "not integration"`: 314 passed, 6 deselected (up from the pre-existing 269-test baseline plus the 45 new tests added by this change) - [x] 7.2 Confirmed default-off behavior: no `AI_PLANNER_*` environment variables set in the shell → `TaskRunner()` constructs `planner_config=PlannerConfig(enabled=False, ...)` and `type(runner.planner) is Planner`; `uvicorn api.rest:create_app --factory` starts cleanly with no API key configured