q792602257
a5aeb8889c
Tests / Test failed: 2, passed: 849
- ToolCallDecision captures thinking blocks and pre-tool text output
- AnthropicToolCallingClient supports optional extended thinking (budget_tokens + beta header)
- PlannedStep carries rationale and thinking from each LLM decision
- WorldEvent replaces scene_summary with rationale/thinking/page fields (backward-compatible)
- AI planner system prompt instructs reflection before each tool call
- _history_summary() emits compact {page, rationale, action, success} dicts
- Cloud DB migration 0011 adds nullable rationale/thinking columns to planner_decision_log
- OpenAI client extracts reasoning_content into thinking field
49 lines
1.9 KiB
Python
49 lines
1.9 KiB
Python
from __future__ import annotations
|
|
|
|
import json
|
|
from typing import Any
|
|
|
|
PLANNER_SYSTEM_PROMPT = """You are the planning brain of a mobile device automation agent.
|
|
|
|
Each turn you are given a goal, the current screen as a structured Scene (a
|
|
list of UI elements with id, type, text, and pixel bounds), and — when
|
|
available — a screenshot of the same screen and a short history of recent
|
|
actions and their outcomes.
|
|
|
|
Before calling a tool, output a short text block (1-2 sentences):
|
|
1. If this is the first step, state what you intend to do and why.
|
|
2. Otherwise, first assess whether the previous action achieved its intended
|
|
effect based on the current screen, then state the intent of your next action.
|
|
Keep this reflection concise and factual.
|
|
|
|
You must then call exactly one tool:
|
|
- One of `tap`, `swipe`, `input_text`, `launch_app`, `terminate_app` to make
|
|
progress toward the goal.
|
|
- `finish_task` when the goal has been reached, or when it cannot be reached
|
|
and no further action would help.
|
|
|
|
Ground every coordinate you choose in the Scene element bounds (and the
|
|
screenshot, if provided) for the current turn only — never reuse coordinates
|
|
from history, since the screen may have changed. Only call `finish_task` with
|
|
`success=True` when the current Scene shows the goal has actually been
|
|
reached. Call it with `success=False` and a clear `reason` if you are stuck,
|
|
repeating the same action without progress, or the goal is not achievable.
|
|
"""
|
|
|
|
|
|
def planner_user_prompt(
|
|
*,
|
|
goal: str,
|
|
scene_json: dict[str, Any],
|
|
history_summary: list[dict[str, Any]],
|
|
) -> str:
|
|
return (
|
|
"Goal:\n"
|
|
f"{goal}\n\n"
|
|
"Current Scene (JSON):\n"
|
|
f"{json.dumps(scene_json, ensure_ascii=False, sort_keys=True)}\n\n"
|
|
"Recent history, oldest first (JSON):\n"
|
|
f"{json.dumps(history_summary, ensure_ascii=False, sort_keys=True)}\n\n"
|
|
"Call exactly one tool for this turn."
|
|
)
|