q792602257andClaude Opus 4.6
a58ded055e
9-task TDD plan: humanize module, Driver.long_press/swipe_path/double_tap on WDA+Android (+ all Fake subclasses), tool wrappers, tap/swipe hooks, and planner integration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1377 lines
46 KiB
Markdown
1377 lines
46 KiB
Markdown
# 原子手势 + 拟人化抖动 Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** 为 Runtime 新增 `long_press` / `double_tap` 两个原子手势，并引入一个集中的拟人化层，对所有指针动作注入坐标抖动、曲线滑动、时序抖动（`APEX_HUMANIZE_ENABLED` 默认开）。
|
|
|
|
**Architecture:** 纯数学集中在 `tools/humanize.py`；W3C Actions 序列化集中在 `driver/_w3c_actions.py`；driver 仍是薄适配器，只把点序列/单点命令翻译成 Appium 调用；工具层（`tools/*.py`）在调 driver 前套一层 humanize。`swipe` 在 humanize 开时走 `Driver.swipe_path`（多点 W3C），关时走原 `Driver.swipe`（2 点命令，零回归）。
|
|
|
|
**Tech Stack:** Python 3.13+、appium-python-client（W3C Actions / mobile commands）、pytest。
|
|
|
|
## Global Constraints
|
|
|
|
- `APEX_HUMANIZE_ENABLED` 默认 `true`；env 前缀统一 `APEX_HUMANIZE_*`。humanize 关闭时 `tap`/`swipe` 必须走与今天完全相同的 driver 方法，现有精确坐标断言零改动。
|
|
- Driver ABC 新增抽象方法后，**所有** `Driver` 子类必须同步实现，否则无法实例化。全仓库子类清单（每个 driver 任务都要改全）：
|
|
- `driver/wda_driver.py::WDADriver`
|
|
- `driver/android_driver.py::AndroidDriver`
|
|
- `tests/fakes.py::FakeDriver`
|
|
- `tests/test_describe_screen.py::TreeFailingDriver`
|
|
- `apps/device-host-agent/tests/test_execution.py::FakeDriver`
|
|
- `apps/device-host-agent/tests/test_e2e.py::FakeDriver`
|
|
- mobile 命令名（`mobile: touchAndHold` / `mobile: longClickGesture` / W3C `pause`）实现期对照已安装的 appium-python-client 源码核实，不得凭记忆。真机执行无法在本机完成，仅做源码核实 + mock 单测。
|
|
- 所有新文件首行 `from __future__ import annotations`，遵循仓库既有风格。
|
|
- 验证命令：`uv run --all-packages pytest -m "not integration" -q`；lint：`uv run --with ruff ruff check <paths>`（本机无独立 ruff）。
|
|
|
|
---
|
|
|
|
## File Structure
|
|
|
|
| 文件 | 职责 | 任务 |
|
|
|---|---|---|
|
|
| `tools/humanize.py`（新） | 拟人化纯函数 + config（env） | Task 1 |
|
|
| `tests/test_humanize.py`（新） | humanize 单测 | Task 1 |
|
|
| `driver/base.py`（改） | Driver ABC：+`long_press`/+`swipe_path`/+`double_tap` 抽象方法 | Task 2/3/4 |
|
|
| `driver/_w3c_actions.py`（新） | W3C Actions payload 构造 + 发送 seam | Task 3 |
|
|
| `driver/wda_driver.py`（改） | WDA 实现 3 个新方法 | Task 2/3/4 |
|
|
| `driver/android_driver.py`（改） | Android 实现 3 个新方法 | Task 2/3/4 |
|
|
| `tests/fakes.py`（改） | FakeDriver 补 3 方法 | Task 2/3/4 |
|
|
| `tests/test_describe_screen.py`（改） | TreeFailingDriver 补 3 方法 | Task 2/3/4 |
|
|
| `apps/device-host-agent/tests/test_execution.py`（改） | FakeDriver 补 3 方法 | Task 2/3/4 |
|
|
| `apps/device-host-agent/tests/test_e2e.py`（改） | FakeDriver 补 3 方法 | Task 2/3/4 |
|
|
| `tests/test_android_driver.py`（改） | 命令 shape + 预连接守卫 + 错误包装 | Task 2/3/4 |
|
|
| `tests/test_w3c_actions.py`（新） | W3C payload 纯函数单测 | Task 3 |
|
|
| `tools/long_press.py`（新） | long_press 工具 | Task 5 |
|
|
| `tools/double_tap.py`（新） | double_tap 工具 | Task 6 |
|
|
| `tools/tap.py`（改） | 加 humanize 钩子 | Task 7 |
|
|
| `tools/swipe.py`（改） | 加 humanize 钩子（开→swipe_path） | Task 8 |
|
|
| `tests/conftest.py`（改/新） | autouse fixture：测试默认关 humanize | Task 7 |
|
|
| `runtime/tool_specs.py`（改） | +LONG_PRESS_SPEC/+DOUBLE_TAP_SPEC | Task 9 |
|
|
| `runtime/executor.py`（改） | registry 注册 2 个新工具 | Task 9 |
|
|
| `runtime/planner_prompts.py`（改） | prompt 工具枚举补 2 个 | Task 9 |
|
|
|
|
依赖顺序：1 → (2,3,4 独立，但 4 依赖 3 的 helper) → 5 依赖 1+2；6 依赖 1+4；7 依赖 1；8 依赖 1+3；9 依赖 5+6。
|
|
|
|
---
|
|
|
|
### Task 1: `tools/humanize.py` 拟人化模块
|
|
|
|
**Files:**
|
|
- Create: `tools/humanize.py`
|
|
- Test: `tests/test_humanize.py`
|
|
|
|
**Interfaces:**
|
|
- Produces: `HumanizeConfig`、`load_humanize_config(env=None) -> HumanizeConfig`、`get_rng() -> random.Random`、`set_rng(rng|None) -> None`、`jitter_point(x, y, *, radius, rng) -> tuple[float,float]`、`jitter_duration(value_ms, *, spread, rng) -> int`、`swipe_waypoints(start, end, *, curvature, n, rng) -> list[tuple[float,float]]`
|
|
|
|
- [ ] **Step 1: 写失败测试 `tests/test_humanize.py`**
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
import math
|
|
import random
|
|
|
|
import pytest
|
|
|
|
from tools.humanize import (
|
|
HumanizeConfig,
|
|
jitter_duration,
|
|
jitter_point,
|
|
load_humanize_config,
|
|
set_rng,
|
|
swipe_waypoints,
|
|
)
|
|
|
|
|
|
def test_jitter_point_stays_within_radius():
|
|
rng = random.Random(42)
|
|
for _ in range(500):
|
|
x, y = jitter_point(100, 200, radius=5.0, rng=rng)
|
|
assert math.hypot(x - 100, y - 200) <= 5.0
|
|
|
|
|
|
def test_jitter_point_centered_on_input():
|
|
rng = random.Random(0)
|
|
xs = [jitter_point(0, 0, radius=4.0, rng=rng)[0] for _ in range(4000)]
|
|
assert abs(sum(xs) / len(xs)) < 0.3
|
|
|
|
|
|
def test_jitter_duration_within_spread():
|
|
rng = random.Random(3)
|
|
for _ in range(200):
|
|
v = jitter_duration(1000, spread=0.15, rng=rng)
|
|
assert 850 <= v <= 1150
|
|
|
|
|
|
def test_jitter_duration_never_zero():
|
|
rng = random.Random(9)
|
|
for _ in range(100):
|
|
assert jitter_duration(1, spread=0.5, rng=rng) >= 1
|
|
|
|
|
|
def test_swipe_waypoints_endpoints_exact_interior_offset():
|
|
rng = random.Random(7)
|
|
pts = swipe_waypoints((0, 0), (100, 0), curvature=0.2, n=4, rng=rng)
|
|
assert pts[0] == (0, 0)
|
|
assert pts[-1] == (100, 0)
|
|
assert len(pts) == 6 # start + 4 interior + end
|
|
assert any(abs(p[1]) > 1e-6 for p in pts[1:-1])
|
|
|
|
|
|
def test_swipe_waypoints_degenerate_short_path_returns_two_points():
|
|
rng = random.Random(1)
|
|
pts = swipe_waypoints((5, 5), (5, 5), curvature=0.2, n=8, rng=rng)
|
|
assert pts == [(5, 5), (5, 5)]
|
|
|
|
|
|
def test_load_config_defaults_when_unset():
|
|
cfg = load_humanize_config(env={})
|
|
assert cfg == HumanizeConfig()
|
|
|
|
|
|
def test_load_config_disabled():
|
|
cfg = load_humanize_config(env={"APEX_HUMANIZE_ENABLED": "false"})
|
|
assert cfg.enabled is False
|
|
|
|
|
|
def test_load_config_overrides():
|
|
cfg = load_humanize_config(
|
|
env={
|
|
"APEX_HUMANIZE_TAP_RADIUS_PX": "9",
|
|
"APEX_HUMANIZE_SWIPE_CURVATURE": "0.3",
|
|
"APEX_HUMANIZE_SWIPE_WAYPOINTS": "12",
|
|
"APEX_HUMANIZE_DURATION_SPREAD": "0.2",
|
|
}
|
|
)
|
|
assert cfg.tap_radius_px == 9.0
|
|
assert cfg.swipe_curvature == 0.3
|
|
assert cfg.swipe_waypoints == 12
|
|
assert cfg.duration_spread == 0.2
|
|
|
|
|
|
def test_set_rng_makes_output_reproducible():
|
|
set_rng(random.Random(99))
|
|
a = jitter_point(0, 0, radius=5.0, rng=random.Random(99))
|
|
set_rng(None) # reset to default rng
|
|
assert isinstance(a, tuple) and len(a) == 2
|
|
```
|
|
|
|
- [ ] **Step 2: 运行测试确认失败**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_humanize.py -q`
|
|
Expected: FAIL — `ModuleNotFoundError: No module named 'tools.humanize'`
|
|
|
|
- [ ] **Step 3: 写实现 `tools/humanize.py`**
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
import math
|
|
import os
|
|
import random
|
|
from collections.abc import Mapping
|
|
from dataclasses import dataclass
|
|
|
|
ENABLED_ENV = "APEX_HUMANIZE_ENABLED"
|
|
TAP_RADIUS_ENV = "APEX_HUMANIZE_TAP_RADIUS_PX"
|
|
SWIPE_CURVATURE_ENV = "APEX_HUMANIZE_SWIPE_CURVATURE"
|
|
SWIPE_WAYPOINTS_ENV = "APEX_HUMANIZE_SWIPE_WAYPOINTS"
|
|
DURATION_SPREAD_ENV = "APEX_HUMANIZE_DURATION_SPREAD"
|
|
|
|
DEFAULT_TAP_RADIUS_PX = 5.0
|
|
DEFAULT_SWIPE_CURVATURE = 0.15
|
|
DEFAULT_SWIPE_WAYPOINTS = 8
|
|
DEFAULT_DURATION_SPREAD = 0.15
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
class HumanizeConfig:
|
|
enabled: bool = True
|
|
tap_radius_px: float = DEFAULT_TAP_RADIUS_PX
|
|
swipe_curvature: float = DEFAULT_SWIPE_CURVATURE
|
|
swipe_waypoints: int = DEFAULT_SWIPE_WAYPOINTS
|
|
duration_spread: float = DEFAULT_DURATION_SPREAD
|
|
|
|
|
|
def load_humanize_config(env: Mapping[str, str] | None = None) -> HumanizeConfig:
|
|
values = env or os.environ
|
|
return HumanizeConfig(
|
|
enabled=_parse_bool(values.get(ENABLED_ENV), default=True),
|
|
tap_radius_px=_parse_float(values.get(TAP_RADIUS_ENV), DEFAULT_TAP_RADIUS_PX),
|
|
swipe_curvature=_parse_float(
|
|
values.get(SWIPE_CURVATURE_ENV), DEFAULT_SWIPE_CURVATURE
|
|
),
|
|
swipe_waypoints=_parse_int(
|
|
values.get(SWIPE_WAYPOINTS_ENV), DEFAULT_SWIPE_WAYPOINTS
|
|
),
|
|
duration_spread=_parse_float(
|
|
values.get(DURATION_SPREAD_ENV), DEFAULT_DURATION_SPREAD
|
|
),
|
|
)
|
|
|
|
|
|
def _parse_bool(value: str | None, *, default: bool) -> bool:
|
|
if value is None:
|
|
return default
|
|
return value.strip().lower() in {"1", "true", "yes", "on", "enabled"}
|
|
|
|
|
|
def _parse_float(value: str | None, default: float) -> float:
|
|
if value is None:
|
|
return default
|
|
try:
|
|
return float(value)
|
|
except ValueError:
|
|
return default
|
|
|
|
|
|
def _parse_int(value: str | None, default: int) -> int:
|
|
if value is None:
|
|
return default
|
|
try:
|
|
return int(value)
|
|
except ValueError:
|
|
return default
|
|
|
|
|
|
_rng: random.Random | None = None
|
|
|
|
|
|
def get_rng() -> random.Random:
|
|
global _rng
|
|
if _rng is None:
|
|
_rng = random.Random()
|
|
return _rng
|
|
|
|
|
|
def set_rng(rng: random.Random | None) -> None:
|
|
"""Inject a seeded rng (tests). Pass ``None`` to reset to the default."""
|
|
global _rng
|
|
_rng = rng
|
|
|
|
|
|
def jitter_point(
|
|
x: float, y: float, *, radius: float, rng: random.Random
|
|
) -> tuple[float, float]:
|
|
"""Gaussian offset clamped to ``[-radius, radius]`` so the tap never leaves
|
|
a ``radius``-px box around the target (keeps it inside the button)."""
|
|
dx = max(-radius, min(radius, rng.gauss(0.0, radius / 2.0)))
|
|
dy = max(-radius, min(radius, rng.gauss(0.0, radius / 2.0)))
|
|
return (x + dx, y + dy)
|
|
|
|
|
|
def jitter_duration(
|
|
value_ms: int, *, spread: float, rng: random.Random
|
|
) -> int:
|
|
"""Uniform jitter within ``value_ms * spread``; floored at 1ms."""
|
|
delta = value_ms * spread
|
|
return max(1, int(value_ms + rng.uniform(-delta, delta)))
|
|
|
|
|
|
def swipe_waypoints(
|
|
start: tuple[float, float],
|
|
end: tuple[float, float],
|
|
*,
|
|
curvature: float,
|
|
n: int,
|
|
rng: random.Random,
|
|
) -> list[tuple[float, float]]:
|
|
"""Return start + ``n`` interior + end points. Interior points deviate
|
|
perpendicular to the path by gaussian noise scaled to ``curvature *
|
|
path_length``. Degenerate (zero-length) path returns ``[start, end]``."""
|
|
sx, sy = start
|
|
ex, ey = end
|
|
dx = ex - sx
|
|
dy = ey - sy
|
|
length = math.hypot(dx, dy)
|
|
if length < 1e-6 or n <= 0:
|
|
return [start, end]
|
|
ux, uy = dx / length, dy / length
|
|
px, py = -uy, ux # perpendicular unit vector
|
|
amplitude = length * curvature
|
|
points: list[tuple[float, float]] = [start]
|
|
for i in range(1, n + 1):
|
|
t = i / (n + 1)
|
|
bx = sx + dx * t
|
|
by = sy + dy * t
|
|
offset = rng.gauss(0.0, amplitude / 2.0)
|
|
points.append((bx + px * offset, by + py * offset))
|
|
points.append(end)
|
|
return points
|
|
```
|
|
|
|
- [ ] **Step 4: 运行测试确认通过**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_humanize.py -q`
|
|
Expected: PASS (10 passed)
|
|
|
|
- [ ] **Step 5: 提交**
|
|
|
|
```bash
|
|
git add tools/humanize.py tests/test_humanize.py
|
|
git commit -m "feat(humanize): add coordinate/duration/swipe-path jitter module"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: `Driver.long_press`（抽象 + WDA + Android + 全部 Fake）
|
|
|
|
**Files:**
|
|
- Modify: `driver/base.py`（新增抽象方法）
|
|
- Modify: `driver/wda_driver.py`
|
|
- Modify: `driver/android_driver.py`
|
|
- Modify: `tests/fakes.py`、`tests/test_describe_screen.py`、`apps/device-host-agent/tests/test_execution.py`、`apps/device-host-agent/tests/test_e2e.py`
|
|
- Test: `tests/test_android_driver.py`
|
|
|
|
**Interfaces:**
|
|
- Produces: `Driver.long_press(self, x: float, y: float, duration_ms: int = 1200) -> None`（抽象，所有子类必须实现）
|
|
- WDA 命令：`mobile: touchAndHold` `{"x", "y", "duration"}`（duration 秒，float）
|
|
- Android 命令：`mobile: longClickGesture` `{"x", "y"}`（系统固定时长，`duration_ms` 被忽略）
|
|
|
|
- [ ] **Step 1: 核实命令名（源码，不猜测）**
|
|
|
|
Run: `uv run python -c "import appium, os; print(os.path.dirname(appium.__file__))"` 定位 appium 安装目录，然后在该目录下 `grep -rn "touchAndHold\|longClickGesture" <appium_dir>` 确认两个命令的参数键。把核实到的精确参数键填入下面的实现（默认按 spec：WDA `{x,y,duration}`，Android `{x,y}`）。若 appium 未安装导致无法核实，在 commit message 与代码注释里标 `# verify on real Appium`。
|
|
|
|
- [ ] **Step 2: 写失败测试（追加到 `tests/test_android_driver.py`）**
|
|
|
|
```python
|
|
def test_long_press_uses_long_click_gesture() -> None:
|
|
driver = _connected_driver()
|
|
driver.long_press(30, 40, duration_ms=1500)
|
|
driver._client.execute_script.assert_called_once_with(
|
|
"mobile: longClickGesture", {"x": 30, "y": 40}
|
|
)
|
|
|
|
|
|
def test_long_press_wraps_exception_into_driver_error() -> None:
|
|
driver = _connected_driver()
|
|
driver._client.execute_script.side_effect = RuntimeError("boom")
|
|
with pytest.raises(DriverError):
|
|
driver.long_press(1, 2)
|
|
```
|
|
|
|
并把 `test_operation_before_connect_raises_device_offline` 的 parametrize 列表追加：
|
|
```python
|
|
("long_press", {"x": 1, "y": 2}),
|
|
```
|
|
|
|
- [ ] **Step 3: 运行确认失败**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_android_driver.py -q`
|
|
Expected: FAIL — `AttributeError: 'AndroidDriver' object has no attribute 'long_press'`
|
|
|
|
- [ ] **Step 4: 加抽象方法（`driver/base.py`，接在 `tap` 之后）**
|
|
|
|
```python
|
|
@abstractmethod
|
|
def long_press(self, x: float, y: float, duration_ms: int = 1200) -> None:
|
|
"""Press and hold at the given coordinates for ``duration_ms``.
|
|
|
|
Some platforms ignore ``duration_ms`` (fixed system hold duration).
|
|
"""
|
|
```
|
|
|
|
- [ ] **Step 5: WDA 实现（`driver/wda_driver.py`，接在 `tap` 之后）**
|
|
|
|
```python
|
|
def long_press(self, x: float, y: float, duration_ms: int = 1200) -> None:
|
|
client = self._require_client()
|
|
try:
|
|
client.execute_script(
|
|
"mobile: touchAndHold",
|
|
{"x": x, "y": y, "duration": duration_ms / 1000},
|
|
)
|
|
except Exception as exc:
|
|
raise DriverError("long press failed") from exc
|
|
```
|
|
|
|
- [ ] **Step 6: Android 实现（`driver/android_driver.py`，接在 `tap` 之后）**
|
|
|
|
```python
|
|
def long_press(self, x: float, y: float, duration_ms: int = 1200) -> None:
|
|
# Android ``longClickGesture`` uses a system-fixed hold duration; the
|
|
# caller's ``duration_ms`` is intentionally ignored (see design R2).
|
|
client = self._require_client()
|
|
try:
|
|
client.execute_script("mobile: longClickGesture", {"x": x, "y": y})
|
|
except Exception as exc:
|
|
raise DriverError("long press failed") from exc
|
|
```
|
|
|
|
- [ ] **Step 7: 同步全部 Fake/Stub 子类**
|
|
|
|
`tests/fakes.py::FakeDriver`（在 `tap` 之后加，记录调用）：
|
|
```python
|
|
def long_press(self, x: float, y: float, duration_ms: int = 1200) -> None:
|
|
self.calls.append(("long_press", (x, y, duration_ms)))
|
|
```
|
|
|
|
其余三处（`tests/test_describe_screen.py::TreeFailingDriver`、`apps/device-host-agent/tests/test_execution.py::FakeDriver`、`apps/device-host-agent/tests/test_e2e.py::FakeDriver`）各加一个 no-op：
|
|
```python
|
|
def long_press(self, x: float, y: float, duration_ms: int = 1200) -> None:
|
|
return None
|
|
```
|
|
|
|
- [ ] **Step 8: 运行确认通过**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_android_driver.py tests/test_describe_screen.py apps/device-host-agent/tests/test_execution.py apps/device-host-agent/tests/test_e2e.py -q`
|
|
Expected: PASS（含预连接守卫 parametrize 新增项）
|
|
|
|
- [ ] **Step 9: 提交**
|
|
|
|
```bash
|
|
git add driver/base.py driver/wda_driver.py driver/android_driver.py tests/fakes.py tests/test_describe_screen.py tests/test_android_driver.py apps/device-host-agent/tests/test_execution.py apps/device-host-agent/tests/test_e2e.py
|
|
git commit -m "feat(driver): add Driver.long_press on WDA and Android"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: W3C Actions helper + `Driver.swipe_path`
|
|
|
|
**Files:**
|
|
- Create: `driver/_w3c_actions.py`
|
|
- Modify: `driver/base.py`、`driver/wda_driver.py`、`driver/android_driver.py`、4 个 Fake/Stub、`tests/test_android_driver.py`
|
|
- Test: `tests/test_w3c_actions.py`（新）
|
|
|
|
**Interfaces:**
|
|
- Produces: `driver/_w3c_actions.py` 的 `build_swipe_actions(waypoints, total_duration_ms) -> list[dict]`、`build_double_tap_actions(x, y, interval_ms) -> list[dict]`、`perform_actions(client, actions) -> None`；`Driver.swipe_path(self, waypoints: list[tuple[float,float]], duration_ms: int) -> None`（抽象）
|
|
|
|
- [ ] **Step 1: 写失败测试 `tests/test_w3c_actions.py`**
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from driver._w3c_actions import (
|
|
build_double_tap_actions,
|
|
build_swipe_actions,
|
|
)
|
|
|
|
|
|
def test_build_swipe_actions_shape_and_duration_split():
|
|
actions = build_swipe_actions([(0, 0), (50, 10), (100, 0)], 600)
|
|
assert len(actions) == 1
|
|
pointer = actions[0]
|
|
assert pointer["type"] == "pointer"
|
|
assert pointer["id"] == "finger1"
|
|
assert pointer["parameters"]["pointerType"] == "touch"
|
|
seq = pointer["actions"]
|
|
types = [a["type"] for a in seq]
|
|
assert types == [
|
|
"pointerMove",
|
|
"pointerDown",
|
|
"pointerMove",
|
|
"pointerMove",
|
|
"pointerUp",
|
|
]
|
|
# 600ms over 2 segments => 300ms per move after pointerDown
|
|
moves_after_down = [a for a in seq if a["type"] == "pointerMove"][1:]
|
|
assert all(m["duration"] == 300 for m in moves_after_down)
|
|
assert seq[0]["x"] == 0 and seq[0]["y"] == 0
|
|
assert seq[-1]["x"] == 100 and seq[-1]["y"] == 0
|
|
|
|
|
|
def test_build_swipe_actions_requires_two_waypoints():
|
|
with pytest.raises(ValueError):
|
|
build_swipe_actions([(1, 1)], 100)
|
|
|
|
|
|
def test_build_double_tap_actions_shape():
|
|
actions = build_double_tap_actions(10, 20, interval_ms=80)
|
|
seq = actions[0]["actions"]
|
|
types = [a["type"] for a in seq]
|
|
assert types == [
|
|
"pointerMove",
|
|
"pointerDown",
|
|
"pointerUp",
|
|
"pause",
|
|
"pointerDown",
|
|
"pointerUp",
|
|
]
|
|
pause = next(a for a in seq if a["type"] == "pause")
|
|
assert pause["duration"] == 80
|
|
assert seq[0]["x"] == 10 and seq[0]["y"] == 20
|
|
```
|
|
|
|
- [ ] **Step 2: 运行确认失败**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_w3c_actions.py -q`
|
|
Expected: FAIL — `ModuleNotFoundError: No module named 'driver._w3c_actions'`
|
|
|
|
- [ ] **Step 3: 写实现 `driver/_w3c_actions.py`**
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from typing import Any
|
|
|
|
POINTER_ID = "finger1"
|
|
|
|
|
|
def _pointer(*actions: dict[str, Any]) -> list[dict[str, Any]]:
|
|
return [
|
|
{
|
|
"type": "pointer",
|
|
"id": POINTER_ID,
|
|
"parameters": {"pointerType": "touch"},
|
|
"actions": list(actions),
|
|
}
|
|
]
|
|
|
|
|
|
def build_swipe_actions(
|
|
waypoints: list[tuple[float, float]], total_duration_ms: int
|
|
) -> list[dict[str, Any]]:
|
|
"""W3C actions moving through ``waypoints`` in one continuous touch."""
|
|
if len(waypoints) < 2:
|
|
raise ValueError("swipe path requires at least two waypoints")
|
|
segments = len(waypoints) - 1
|
|
per_segment = max(1, total_duration_ms // segments)
|
|
sx, sy = waypoints[0]
|
|
actions: list[dict[str, Any]] = [
|
|
{"type": "pointerMove", "duration": 0, "x": int(sx), "y": int(sy)},
|
|
{"type": "pointerDown", "button": 0},
|
|
]
|
|
for point in waypoints[1:]:
|
|
actions.append(
|
|
{
|
|
"type": "pointerMove",
|
|
"duration": per_segment,
|
|
"x": int(point[0]),
|
|
"y": int(point[1]),
|
|
}
|
|
)
|
|
actions.append({"type": "pointerUp", "button": 0})
|
|
return _pointer(*actions)
|
|
|
|
|
|
def build_double_tap_actions(
|
|
x: float, y: float, interval_ms: int
|
|
) -> list[dict[str, Any]]:
|
|
actions = [
|
|
{"type": "pointerMove", "duration": 0, "x": int(x), "y": int(y)},
|
|
{"type": "pointerDown", "button": 0},
|
|
{"type": "pointerUp", "button": 0},
|
|
{"type": "pause", "duration": max(1, interval_ms)},
|
|
{"type": "pointerDown", "button": 0},
|
|
{"type": "pointerUp", "button": 0},
|
|
]
|
|
return _pointer(*actions)
|
|
|
|
|
|
def perform_actions(client: Any, actions: list[dict[str, Any]]) -> None:
|
|
"""Send a W3C actions payload via the Appium/Selenium command seam.
|
|
|
|
Import is lazy so the pure payload builders stay importable without
|
|
selenium on the path (used by unit tests).
|
|
"""
|
|
from selenium.webdriver.remote.command import Command
|
|
|
|
client.execute(Command.W3C_ACTIONS, {"actions": actions})
|
|
```
|
|
|
|
- [ ] **Step 4: 运行确认通过**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_w3c_actions.py -q`
|
|
Expected: PASS (3 passed)
|
|
|
|
- [ ] **Step 5: 写 swipe_path 失败测试（追加到 `tests/test_android_driver.py`）**
|
|
|
|
```python
|
|
def test_swipe_path_sends_w3c_actions() -> None:
|
|
from driver._w3c_actions import build_swipe_actions
|
|
|
|
driver = _connected_driver()
|
|
waypoints = [(0, 0), (50, 5), (100, 0)]
|
|
driver.swipe_path(waypoints, 600)
|
|
args = driver._client.execute.call_args.args
|
|
assert args[1] == {"actions": build_swipe_actions(waypoints, 600)}
|
|
|
|
|
|
def test_swipe_path_wraps_exception_into_driver_error() -> None:
|
|
driver = _connected_driver()
|
|
driver._client.execute.side_effect = RuntimeError("boom")
|
|
with pytest.raises(DriverError):
|
|
driver.swipe_path([(0, 0), (1, 1)], 100)
|
|
```
|
|
|
|
并把 `test_operation_before_connect_raises_device_offline` 的 parametrize 追加：
|
|
```python
|
|
("swipe_path", {"waypoints": [(0, 0), (1, 1)], "duration_ms": 100}),
|
|
```
|
|
|
|
- [ ] **Step 6: 运行确认失败**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_android_driver.py -q`
|
|
Expected: FAIL — `AttributeError: 'AndroidDriver' object has no attribute 'swipe_path'`
|
|
|
|
- [ ] **Step 7: 加抽象方法（`driver/base.py`，接在 `swipe` 之后）**
|
|
|
|
```python
|
|
@abstractmethod
|
|
def swipe_path(
|
|
self, waypoints: list[tuple[float, float]], duration_ms: int
|
|
) -> None:
|
|
"""Swipe through a sequence of waypoints in one continuous touch."""
|
|
```
|
|
|
|
- [ ] **Step 8: WDA + Android 实现（两处相同，各接在 `swipe` 之后）**
|
|
|
|
```python
|
|
def swipe_path(
|
|
self, waypoints: list[tuple[float, float]], duration_ms: int
|
|
) -> None:
|
|
from driver._w3c_actions import build_swipe_actions, perform_actions
|
|
|
|
client = self._require_client()
|
|
try:
|
|
perform_actions(client, build_swipe_actions(waypoints, duration_ms))
|
|
except Exception as exc:
|
|
raise DriverError("swipe_path failed") from exc
|
|
```
|
|
|
|
- [ ] **Step 9: 同步全部 Fake/Stub 子类**
|
|
|
|
`tests/fakes.py::FakeDriver`：
|
|
```python
|
|
def swipe_path(
|
|
self, waypoints: list[tuple[float, float]], duration_ms: int
|
|
) -> None:
|
|
self.calls.append(("swipe_path", (tuple(waypoints), duration_ms)))
|
|
```
|
|
其余三处各加 no-op：
|
|
```python
|
|
def swipe_path(
|
|
self, waypoints: list[tuple[float, float]], duration_ms: int
|
|
) -> None:
|
|
return None
|
|
```
|
|
|
|
- [ ] **Step 10: 运行确认通过**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_w3c_actions.py tests/test_android_driver.py tests/test_describe_screen.py apps/device-host-agent/tests/test_execution.py apps/device-host-agent/tests/test_e2e.py -q`
|
|
Expected: PASS
|
|
|
|
- [ ] **Step 11: 提交**
|
|
|
|
```bash
|
|
git add driver/_w3c_actions.py driver/base.py driver/wda_driver.py driver/android_driver.py tests/test_w3c_actions.py tests/test_android_driver.py tests/fakes.py tests/test_describe_screen.py apps/device-host-agent/tests/test_execution.py apps/device-host-agent/tests/test_e2e.py
|
|
git commit -m "feat(driver): add W3C actions helper and Driver.swipe_path"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: `Driver.double_tap`（用 Task 3 的 helper）
|
|
|
|
**Files:**
|
|
- Modify: `driver/base.py`、`driver/wda_driver.py`、`driver/android_driver.py`、4 个 Fake/Stub、`tests/test_android_driver.py`
|
|
|
|
**Interfaces:**
|
|
- Produces: `Driver.double_tap(self, x: float, y: float, interval_ms: int = 80) -> None`（抽象）。两平台都用 W3C Actions（`pointerDown→up→pause(interval)→down→up`），单次往返。
|
|
|
|
- [ ] **Step 1: 写失败测试（追加到 `tests/test_android_driver.py`）**
|
|
|
|
```python
|
|
def test_double_tap_sends_w3c_actions() -> None:
|
|
from driver._w3c_actions import build_double_tap_actions
|
|
|
|
driver = _connected_driver()
|
|
driver.double_tap(15, 25, interval_ms=90)
|
|
args = driver._client.execute.call_args.args
|
|
assert args[1] == {"actions": build_double_tap_actions(15, 25, 90)}
|
|
|
|
|
|
def test_double_tap_wraps_exception_into_driver_error() -> None:
|
|
driver = _connected_driver()
|
|
driver._client.execute.side_effect = RuntimeError("boom")
|
|
with pytest.raises(DriverError):
|
|
driver.double_tap(1, 2)
|
|
```
|
|
|
|
并把 parametrize 追加：
|
|
```python
|
|
("double_tap", {"x": 1, "y": 2}),
|
|
```
|
|
|
|
- [ ] **Step 2: 运行确认失败**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_android_driver.py -q`
|
|
Expected: FAIL — `AttributeError: 'AndroidDriver' object has no attribute 'double_tap'`
|
|
|
|
- [ ] **Step 3: 加抽象方法（`driver/base.py`，接在 `long_press` 之后）**
|
|
|
|
```python
|
|
@abstractmethod
|
|
def double_tap(self, x: float, y: float, interval_ms: int = 80) -> None:
|
|
"""Tap twice at the given coordinates with ``interval_ms`` between."""
|
|
```
|
|
|
|
- [ ] **Step 4: WDA + Android 实现（两处相同，各接在 `long_press` 之后）**
|
|
|
|
```python
|
|
def double_tap(self, x: float, y: float, interval_ms: int = 80) -> None:
|
|
from driver._w3c_actions import build_double_tap_actions, perform_actions
|
|
|
|
client = self._require_client()
|
|
try:
|
|
perform_actions(client, build_double_tap_actions(x, y, interval_ms))
|
|
except Exception as exc:
|
|
raise DriverError("double tap failed") from exc
|
|
```
|
|
|
|
- [ ] **Step 5: 同步全部 Fake/Stub 子类**
|
|
|
|
`tests/fakes.py::FakeDriver`：
|
|
```python
|
|
def double_tap(self, x: float, y: float, interval_ms: int = 80) -> None:
|
|
self.calls.append(("double_tap", (x, y, interval_ms)))
|
|
```
|
|
其余三处各加 no-op：
|
|
```python
|
|
def double_tap(self, x: float, y: float, interval_ms: int = 80) -> None:
|
|
return None
|
|
```
|
|
|
|
- [ ] **Step 6: 运行确认通过**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_android_driver.py tests/test_describe_screen.py apps/device-host-agent/tests/test_execution.py apps/device-host-agent/tests/test_e2e.py -q`
|
|
Expected: PASS
|
|
|
|
- [ ] **Step 7: 提交**
|
|
|
|
```bash
|
|
git add driver/base.py driver/wda_driver.py driver/android_driver.py tests/test_android_driver.py tests/fakes.py tests/test_describe_screen.py apps/device-host-agent/tests/test_execution.py apps/device-host-agent/tests/test_e2e.py
|
|
git commit -m "feat(driver): add Driver.double_tap via W3C actions"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 5: `tools/long_press.py`
|
|
|
|
**Files:**
|
|
- Create: `tools/long_press.py`
|
|
- Test: `tests/test_long_press.py`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `tools.humanize`（Task 1）、`Driver.long_press`（Task 2）
|
|
- Produces: `long_press(x, y, *, duration_ms=1200, device_id=None, manager=None) -> dict`
|
|
|
|
- [ ] **Step 1: 写失败测试 `tests/test_long_press.py`**
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from device.manager import DeviceManager
|
|
from tests.fakes import FakeDriver
|
|
|
|
|
|
def _connected(driver: FakeDriver) -> DeviceManager:
|
|
manager = DeviceManager()
|
|
manager.register_device("phone", lambda: driver)
|
|
manager.connect("phone", max_retries=1)
|
|
return manager
|
|
|
|
|
|
def test_long_press_disabled_passes_exact_coords(monkeypatch):
|
|
monkeypatch.setenv("APEX_HUMANIZE_ENABLED", "false")
|
|
from tools.long_press import long_press
|
|
|
|
driver = FakeDriver()
|
|
res = long_press(100, 200, duration_ms=1500, manager=_connected(driver))
|
|
assert driver.calls[-1] == ("long_press", (100, 200, 1500))
|
|
assert res["action"] == "long_press"
|
|
assert res["duration_ms"] == 1500
|
|
|
|
|
|
def test_long_press_enabled_jitters_within_radius(monkeypatch):
|
|
import random
|
|
|
|
monkeypatch.setenv("APEX_HUMANIZE_ENABLED", "true")
|
|
from tools.humanize import set_rng
|
|
from tools.long_press import long_press
|
|
|
|
set_rng(random.Random(5))
|
|
try:
|
|
driver = FakeDriver()
|
|
long_press(100, 200, duration_ms=1500, manager=_connected(driver))
|
|
finally:
|
|
set_rng(None)
|
|
name, args = driver.calls[-1]
|
|
assert name == "long_press"
|
|
px, py, _ = args
|
|
assert abs(px - 100) <= 5.0 and abs(py - 200) <= 5.0
|
|
```
|
|
|
|
- [ ] **Step 2: 运行确认失败**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_long_press.py -q`
|
|
Expected: FAIL — `ModuleNotFoundError: No module named 'tools.long_press'`
|
|
|
|
- [ ] **Step 3: 写实现 `tools/long_press.py`**
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from device.manager import DeviceManager
|
|
from tools._device import get_driver
|
|
from tools.humanize import (
|
|
get_rng,
|
|
jitter_duration,
|
|
jitter_point,
|
|
load_humanize_config,
|
|
)
|
|
|
|
|
|
def long_press(
|
|
x: float,
|
|
y: float,
|
|
*,
|
|
duration_ms: int = 1200,
|
|
device_id: str | None = None,
|
|
manager: DeviceManager | None = None,
|
|
) -> dict[str, object]:
|
|
cfg = load_humanize_config()
|
|
px, py = x, y
|
|
dms = duration_ms
|
|
if cfg.enabled:
|
|
rng = get_rng()
|
|
px, py = jitter_point(x, y, radius=cfg.tap_radius_px, rng=rng)
|
|
dms = jitter_duration(duration_ms, spread=cfg.duration_spread, rng=rng)
|
|
get_driver(device_id, manager=manager).long_press(px, py, dms)
|
|
return {"ok": True, "action": "long_press", "x": px, "y": py, "duration_ms": dms}
|
|
```
|
|
|
|
- [ ] **Step 4: 运行确认通过**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_long_press.py -q`
|
|
Expected: PASS (2 passed)
|
|
|
|
- [ ] **Step 5: 提交**
|
|
|
|
```bash
|
|
git add tools/long_press.py tests/test_long_press.py
|
|
git commit -m "feat(tools): add long_press tool with humanize hook"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: `tools/double_tap.py`
|
|
|
|
**Files:**
|
|
- Create: `tools/double_tap.py`
|
|
- Test: `tests/test_double_tap.py`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `tools.humanize`（Task 1）、`Driver.double_tap`（Task 4）
|
|
- Produces: `double_tap(x, y, *, interval_ms=80, device_id=None, manager=None) -> dict`
|
|
|
|
- [ ] **Step 1: 写失败测试 `tests/test_double_tap.py`**
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from device.manager import DeviceManager
|
|
from tests.fakes import FakeDriver
|
|
|
|
|
|
def _connected(driver: FakeDriver) -> DeviceManager:
|
|
manager = DeviceManager()
|
|
manager.register_device("phone", lambda: driver)
|
|
manager.connect("phone", max_retries=1)
|
|
return manager
|
|
|
|
|
|
def test_double_tap_disabled_passes_exact(monkeypatch):
|
|
monkeypatch.setenv("APEX_HUMANIZE_ENABLED", "false")
|
|
from tools.double_tap import double_tap
|
|
|
|
driver = FakeDriver()
|
|
res = double_tap(50, 60, interval_ms=90, manager=_connected(driver))
|
|
assert driver.calls[-1] == ("double_tap", (50, 60, 90))
|
|
assert res["action"] == "double_tap"
|
|
|
|
|
|
def test_double_tap_enabled_jitters_within_radius(monkeypatch):
|
|
import random
|
|
|
|
monkeypatch.setenv("APEX_HUMANIZE_ENABLED", "true")
|
|
from tools.humanize import set_rng
|
|
from tools.double_tap import double_tap
|
|
|
|
set_rng(random.Random(8))
|
|
try:
|
|
driver = FakeDriver()
|
|
double_tap(50, 60, manager=_connected(driver))
|
|
finally:
|
|
set_rng(None)
|
|
px, py, _ = driver.calls[-1][1]
|
|
assert abs(px - 50) <= 5.0 and abs(py - 60) <= 5.0
|
|
```
|
|
|
|
- [ ] **Step 2: 运行确认失败**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_double_tap.py -q`
|
|
Expected: FAIL — `ModuleNotFoundError`
|
|
|
|
- [ ] **Step 3: 写实现 `tools/double_tap.py`**
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from device.manager import DeviceManager
|
|
from tools._device import get_driver
|
|
from tools.humanize import get_rng, jitter_duration, jitter_point, load_humanize_config
|
|
|
|
|
|
def double_tap(
|
|
x: float,
|
|
y: float,
|
|
*,
|
|
interval_ms: int = 80,
|
|
device_id: str | None = None,
|
|
manager: DeviceManager | None = None,
|
|
) -> dict[str, object]:
|
|
cfg = load_humanize_config()
|
|
px, py = x, y
|
|
interval = interval_ms
|
|
if cfg.enabled:
|
|
rng = get_rng()
|
|
px, py = jitter_point(x, y, radius=cfg.tap_radius_px, rng=rng)
|
|
interval = jitter_duration(interval_ms, spread=cfg.duration_spread, rng=rng)
|
|
get_driver(device_id, manager=manager).double_tap(px, py, interval)
|
|
return {
|
|
"ok": True,
|
|
"action": "double_tap",
|
|
"x": px,
|
|
"y": py,
|
|
"interval_ms": interval,
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: 运行确认通过**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_double_tap.py -q`
|
|
Expected: PASS (2 passed)
|
|
|
|
- [ ] **Step 5: 提交**
|
|
|
|
```bash
|
|
git add tools/double_tap.py tests/test_double_tap.py
|
|
git commit -m "feat(tools): add double_tap tool with humanize hook"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 7: `tools/tap.py` 加 humanize 钩子 + 测试默认关 humanize
|
|
|
|
**Files:**
|
|
- Modify: `tools/tap.py`
|
|
- Modify/Create: `tests/conftest.py`（autouse fixture）
|
|
- Test: `tests/test_tap_humanize.py`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `tools.humanize`（Task 1）
|
|
- 改动后 `tap(x, y, *, device_id=None, manager=None)`：humanize 开则坐标抖动，关则原样
|
|
|
|
- [ ] **Step 1: 写失败测试 `tests/test_tap_humanize.py`**
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from device.manager import DeviceManager
|
|
from tests.fakes import FakeDriver
|
|
|
|
|
|
def _connected(driver: FakeDriver) -> DeviceManager:
|
|
manager = DeviceManager()
|
|
manager.register_device("phone", lambda: driver)
|
|
manager.connect("phone", max_retries=1)
|
|
return manager
|
|
|
|
|
|
def test_tap_default_off_in_tests_passes_exact(monkeypatch):
|
|
# Autouse conftest fixture sets APEX_HUMANIZE_ENABLED=false; reaffirm here.
|
|
monkeypatch.setenv("APEX_HUMANIZE_ENABLED", "false")
|
|
from tools.tap import tap
|
|
|
|
driver = FakeDriver()
|
|
tap(10, 20, manager=_connected(driver))
|
|
assert driver.calls[-1] == ("tap", (10, 20))
|
|
|
|
|
|
def test_tap_enabled_jitters_within_radius(monkeypatch):
|
|
import random
|
|
|
|
monkeypatch.setenv("APEX_HUMANIZE_ENABLED", "true")
|
|
from tools.humanize import set_rng
|
|
from tools.tap import tap
|
|
|
|
set_rng(random.Random(11))
|
|
try:
|
|
driver = FakeDriver()
|
|
tap(10, 20, manager=_connected(driver))
|
|
finally:
|
|
set_rng(None)
|
|
px, py = driver.calls[-1][1]
|
|
assert abs(px - 10) <= 5.0 and abs(py - 20) <= 5.0
|
|
```
|
|
|
|
- [ ] **Step 2: 加 autouse fixture（`tests/conftest.py`，文件不存在则创建）**
|
|
|
|
读取现有 `tests/conftest.py`（若存在）；在文件末尾追加（不破坏既有内容）：
|
|
```python
|
|
import pytest
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _humanize_disabled_by_default_in_tests(monkeypatch):
|
|
"""Humanize defaults ON in production; tests default it OFF so existing
|
|
exact-coordinate assertions stay deterministic. Tests that want to
|
|
exercise humanize call ``monkeypatch.setenv("APEX_HUMANIZE_ENABLED", "true")``
|
|
in their own body, which overrides this fixture (test body runs after
|
|
fixture setup)."""
|
|
monkeypatch.setenv("APEX_HUMANIZE_ENABLED", "false")
|
|
```
|
|
若 conftest.py 已有 `import pytest`，不要重复 import。
|
|
|
|
- [ ] **Step 3: 运行确认失败（实现未改前 tap 不抖动，第二条测试会 FAIL）**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_tap_humanize.py -q`
|
|
Expected: `test_tap_default_off_in_tests_passes_exact` PASS、`test_tap_enabled_jitters_within_radius` FAIL（坐标未偏移）
|
|
|
|
- [ ] **Step 4: 改 `tools/tap.py`**
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from device.manager import DeviceManager
|
|
from tools._device import get_driver
|
|
from tools.humanize import get_rng, jitter_point, load_humanize_config
|
|
|
|
|
|
def tap(
|
|
x: float,
|
|
y: float,
|
|
*,
|
|
device_id: str | None = None,
|
|
manager: DeviceManager | None = None,
|
|
) -> dict[str, object]:
|
|
cfg = load_humanize_config()
|
|
px, py = x, y
|
|
if cfg.enabled:
|
|
px, py = jitter_point(x, y, radius=cfg.tap_radius_px, rng=get_rng())
|
|
get_driver(device_id, manager=manager).tap(px, py)
|
|
return {"ok": True, "action": "tap", "x": px, "y": py}
|
|
```
|
|
|
|
- [ ] **Step 5: 运行新测试 + 回归 tap 相关测试**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_tap_humanize.py tests/test_skill_catalog_e2e.py tests/test_semantic_task_loop.py tests/test_mcp.py -q`
|
|
Expected: PASS（autouse fixture 保护了既有精确坐标断言）
|
|
|
|
- [ ] **Step 6: 提交**
|
|
|
|
```bash
|
|
git add tools/tap.py tests/conftest.py tests/test_tap_humanize.py
|
|
git commit -m "feat(tools): humanize tap coordinates; default off in tests"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 8: `tools/swipe.py` 加 humanize 钩子（开→swipe_path）
|
|
|
|
**Files:**
|
|
- Modify: `tools/swipe.py`
|
|
- Test: `tests/test_swipe_humanize.py`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `tools.humanize`（Task 1）、`Driver.swipe_path`（Task 3）
|
|
- 改动后 `swipe(...)`：humanize 开→`driver.swipe_path(waypoints, jittered_duration)`；关→原 `driver.swipe(...)`
|
|
|
|
- [ ] **Step 1: 写失败测试 `tests/test_swipe_humanize.py`**
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from device.manager import DeviceManager
|
|
from tests.fakes import FakeDriver
|
|
|
|
|
|
def _connected(driver: FakeDriver) -> DeviceManager:
|
|
manager = DeviceManager()
|
|
manager.register_device("phone", lambda: driver)
|
|
manager.connect("phone", max_retries=1)
|
|
return manager
|
|
|
|
|
|
def test_swipe_disabled_uses_two_point_swipe(monkeypatch):
|
|
monkeypatch.setenv("APEX_HUMANIZE_ENABLED", "false")
|
|
from tools.swipe import swipe
|
|
|
|
driver = FakeDriver()
|
|
swipe(0, 0, 100, 0, duration_ms=500, manager=_connected(driver))
|
|
assert driver.calls[-1] == ("swipe", (0, 0, 100, 0, 500))
|
|
|
|
|
|
def test_swipe_enabled_uses_curved_swipe_path(monkeypatch):
|
|
import random
|
|
|
|
monkeypatch.setenv("APEX_HUMANIZE_ENABLED", "true")
|
|
from tools.humanize import set_rng
|
|
from tools.swipe import swipe
|
|
|
|
set_rng(random.Random(21))
|
|
try:
|
|
driver = FakeDriver()
|
|
swipe(0, 0, 100, 0, duration_ms=500, manager=_connected(driver))
|
|
finally:
|
|
set_rng(None)
|
|
name, args = driver.calls[-1]
|
|
assert name == "swipe_path"
|
|
waypoints, _dms = args
|
|
assert waypoints[0] == (0, 0)
|
|
assert waypoints[-1] == (100, 0)
|
|
assert len(waypoints) > 2 # curved: more than just endpoints
|
|
```
|
|
|
|
- [ ] **Step 2: 运行确认失败**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_swipe_humanize.py -q`
|
|
Expected: `test_swipe_disabled_uses_two_point_swipe` PASS、`test_swipe_enabled_uses_curved_swipe_path` FAIL
|
|
|
|
- [ ] **Step 3: 改 `tools/swipe.py`**
|
|
|
|
```python
|
|
from __future__ import annotations
|
|
|
|
from device.manager import DeviceManager
|
|
from tools._device import get_driver
|
|
from tools.humanize import (
|
|
get_rng,
|
|
jitter_duration,
|
|
load_humanize_config,
|
|
swipe_waypoints,
|
|
)
|
|
|
|
|
|
def swipe(
|
|
start_x: float,
|
|
start_y: float,
|
|
end_x: float,
|
|
end_y: float,
|
|
*,
|
|
duration_ms: int = 500,
|
|
device_id: str | None = None,
|
|
manager: DeviceManager | None = None,
|
|
) -> dict[str, object]:
|
|
cfg = load_humanize_config()
|
|
if cfg.enabled:
|
|
rng = get_rng()
|
|
dms = jitter_duration(duration_ms, spread=cfg.duration_spread, rng=rng)
|
|
waypoints = swipe_waypoints(
|
|
(start_x, start_y),
|
|
(end_x, end_y),
|
|
curvature=cfg.swipe_curvature,
|
|
n=cfg.swipe_waypoints,
|
|
rng=rng,
|
|
)
|
|
get_driver(device_id, manager=manager).swipe_path(waypoints, dms)
|
|
return {
|
|
"ok": True,
|
|
"action": "swipe",
|
|
"waypoints": waypoints,
|
|
"duration_ms": dms,
|
|
}
|
|
get_driver(device_id, manager=manager).swipe(
|
|
start_x, start_y, end_x, end_y, duration_ms
|
|
)
|
|
return {
|
|
"ok": True,
|
|
"action": "swipe",
|
|
"start": {"x": start_x, "y": start_y},
|
|
"end": {"x": end_x, "y": end_y},
|
|
"duration_ms": duration_ms,
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: 运行确认通过 + swipe 回归**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_swipe_humanize.py tests/test_android_driver.py -q`
|
|
Expected: PASS
|
|
|
|
- [ ] **Step 5: 提交**
|
|
|
|
```bash
|
|
git add tools/swipe.py tests/test_swipe_humanize.py
|
|
git commit -m "feat(tools): humanize swipe into curved W3C path when enabled"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 9: Planner 集成（tool_specs + executor registry + prompt）
|
|
|
|
**Files:**
|
|
- Modify: `runtime/tool_specs.py`
|
|
- Modify: `runtime/executor.py`
|
|
- Modify: `runtime/planner_prompts.py`
|
|
- Test: `tests/test_tool_specs.py`、`tests/test_executor.py`
|
|
|
|
**Interfaces:**
|
|
- Consumes: `tools.long_press`（Task 5）、`tools.double_tap`（Task 6）
|
|
- Produces: `LONG_PRESS_SPEC`、`DOUBLE_TAP_SPEC` 加入 `ACTION_TOOL_SPECS`；executor registry 多 2 个键；prompt 工具枚举更新
|
|
|
|
- [ ] **Step 1: 写失败测试（追加到 `tests/test_tool_specs.py`）**
|
|
|
|
先读 `tests/test_tool_specs.py` 既有断言风格，再追加：
|
|
```python
|
|
def test_long_press_and_double_tap_are_action_tools():
|
|
from runtime.tool_specs import ACTION_TOOL_NAMES
|
|
|
|
assert "long_press" in ACTION_TOOL_NAMES
|
|
assert "double_tap" in ACTION_TOOL_NAMES
|
|
|
|
|
|
def test_long_press_spec_has_duration_with_default():
|
|
from runtime.tool_specs import LONG_PRESS_SPEC
|
|
|
|
props = LONG_PRESS_SPEC.parameters["properties"]
|
|
assert props["x"]["type"] == "number"
|
|
assert props["y"]["type"] == "number"
|
|
assert props["duration_ms"]["default"] == 1200
|
|
# purpose/expected_outcome are required metadata
|
|
assert "purpose" in LONG_PRESS_SPEC.parameters["required"]
|
|
|
|
|
|
def test_double_tap_spec_has_interval_with_default():
|
|
from runtime.tool_specs import DOUBLE_TAP_SPEC
|
|
|
|
props = DOUBLE_TAP_SPEC.parameters["properties"]
|
|
assert props["interval_ms"]["default"] == 80
|
|
```
|
|
|
|
并在 `tests/test_executor.py` 追加：
|
|
```python
|
|
def test_default_registry_dispatches_long_press_and_double_tap():
|
|
from runtime.executor import default_tool_registry
|
|
|
|
registry = default_tool_registry()
|
|
assert callable(registry["long_press"])
|
|
assert callable(registry["double_tap"])
|
|
```
|
|
|
|
- [ ] **Step 2: 运行确认失败**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_tool_specs.py tests/test_executor.py -q`
|
|
Expected: FAIL — `ImportError: cannot import name 'LONG_PRESS_SPEC'`
|
|
|
|
- [ ] **Step 3: 改 `runtime/tool_specs.py`（在 `TAP_SPEC` 之后插入两个 spec，并把它们加进 `ACTION_TOOL_SPECS`）**
|
|
|
|
在 `TAP_SPEC` 定义之后插入：
|
|
```python
|
|
LONG_PRESS_SPEC = ToolSpec(
|
|
name="long_press",
|
|
description="Press and hold a point on the screen, given in Scene pixel coordinates.",
|
|
parameters=_action_parameters(
|
|
required=["x", "y"],
|
|
properties={
|
|
"x": {"type": "number", "description": "X coordinate in Scene pixel space."},
|
|
"y": {"type": "number", "description": "Y coordinate in Scene pixel space."},
|
|
"duration_ms": {
|
|
"type": "integer",
|
|
"description": "Hold duration in milliseconds (ignored on some platforms).",
|
|
"default": 1200,
|
|
},
|
|
},
|
|
),
|
|
)
|
|
|
|
DOUBLE_TAP_SPEC = ToolSpec(
|
|
name="double_tap",
|
|
description="Tap a point twice quickly, given in Scene pixel coordinates.",
|
|
parameters=_action_parameters(
|
|
required=["x", "y"],
|
|
properties={
|
|
"x": {"type": "number", "description": "X coordinate in Scene pixel space."},
|
|
"y": {"type": "number", "description": "Y coordinate in Scene pixel space."},
|
|
"interval_ms": {
|
|
"type": "integer",
|
|
"description": "Milliseconds between the two taps.",
|
|
"default": 80,
|
|
},
|
|
},
|
|
),
|
|
)
|
|
```
|
|
并把 `ACTION_TOOL_SPECS` 列表改为：
|
|
```python
|
|
ACTION_TOOL_SPECS: list[ToolSpec] = [
|
|
TAP_SPEC,
|
|
LONG_PRESS_SPEC,
|
|
DOUBLE_TAP_SPEC,
|
|
SWIPE_SPEC,
|
|
INPUT_TEXT_SPEC,
|
|
LAUNCH_APP_SPEC,
|
|
TERMINATE_APP_SPEC,
|
|
]
|
|
```
|
|
|
|
- [ ] **Step 4: 改 `runtime/executor.py::default_tool_registry`**
|
|
|
|
在 import 段加：
|
|
```python
|
|
from tools.double_tap import double_tap
|
|
from tools.long_press import long_press
|
|
```
|
|
在返回 dict 内（`"tap": ...` 之后）加：
|
|
```python
|
|
"long_press": _bind_manager(long_press, manager),
|
|
"double_tap": _bind_manager(double_tap, manager),
|
|
```
|
|
|
|
- [ ] **Step 5: 改 `runtime/planner_prompts.py` 的工具枚举**
|
|
|
|
把 `PLANNER_SYSTEM_PROMPT` 里：
|
|
```
|
|
- One of `tap`, `swipe`, `input_text`, `launch_app`, `terminate_app` to make
|
|
progress toward the goal.
|
|
```
|
|
改为：
|
|
```
|
|
- One of `tap`, `long_press`, `double_tap`, `swipe`, `input_text`,
|
|
`launch_app`, `terminate_app` to make progress toward the goal. Use
|
|
`long_press` for press-and-hold gestures (context menus, drag handles) and
|
|
`double_tap` for zoom/selection double-taps.
|
|
```
|
|
|
|
- [ ] **Step 6: 运行确认通过**
|
|
|
|
Run: `uv run --all-packages pytest tests/test_tool_specs.py tests/test_executor.py tests/test_humanize.py tests/test_long_press.py tests/test_double_tap.py tests/test_tap_humanize.py tests/test_swipe_humanize.py tests/test_w3c_actions.py tests/test_android_driver.py -q`
|
|
Expected: PASS
|
|
|
|
- [ ] **Step 7: 全量回归 + lint**
|
|
|
|
Run: `uv run --all-packages pytest -m "not integration" -q`
|
|
Expected: 全绿（既有非集成测试无回归；留意 `test_deployment_config.py` 的预存失败是否仍为预存）
|
|
|
|
Run: `uv run --with ruff ruff check tools/humanize.py tools/long_press.py tools/double_tap.py tools/tap.py tools/swipe.py driver/_w3c_actions.py driver/wda_driver.py driver/android_driver.py driver/base.py runtime/tool_specs.py runtime/executor.py runtime/planner_prompts.py`
|
|
Expected: 无错误
|
|
|
|
- [ ] **Step 8: 提交**
|
|
|
|
```bash
|
|
git add runtime/tool_specs.py runtime/executor.py runtime/planner_prompts.py tests/test_tool_specs.py tests/test_executor.py
|
|
git commit -m "feat(planner): expose long_press/double_tap to the AI planner"
|
|
```
|
|
|
|
---
|
|
|
|
## Self-Review（plan 完成后自查，已执行）
|
|
|
|
1. **Spec coverage**：spec §4.1 两个新原语 → Task 5/6；§4.2 humanize 模块 → Task 1；§4.3 W3C 共享 + swipe_path/double_tap → Task 3/4；§4.4 开关矩阵 → Task 7/8（tap/swipe）+ Task 5/6（新工具自带）；§4.5 tool_specs+registry → Task 9；§4.6 prompt → Task 9；§3 命令矩阵 → Task 2（long_press）+ Task 3（swipe W3C）+ Task 4（double_tap W3C）；§5 env → Task 1；§7 测试 → 每个任务 TDD。无遗漏。
|
|
2. **Placeholder scan**：无 TBD/TODO；每步含完整代码或确切命令。命令名核实步骤（Task 2 Step 1）有可执行指令。
|
|
3. **Type consistency**：`Driver.long_press(x,y,duration_ms=1200)`、`swipe_path(waypoints, duration_ms)`、`double_tap(x,y,interval_ms=80)` 在 ABC/两驱动/Fake/工具/spec 间签名一致；`build_swipe_actions`/`build_double_tap_actions`/`perform_actions` 在 Task 3 定义、Task 4 复用，名称一致。
|
|
|
|
## Execution Handoff
|
|
|
|
Plan complete and saved to `docs/superpowers/plans/2026-07-15-gesture-primitives-humanize.md`.（未提交，下一步先提交此 plan 再执行。）
|