agentic-mobile-control/openspec/changes/archive/2026-07-06-device-agent-runtime-foundation/tasks.md

1. Split core/ into driver/ and device/

• 1.1 git mv core/driver.py driver/base.py, add driver/__init__.py, update its internal imports if any
• 1.2 git mv core/wda_driver.py driver/wda_driver.py, update its import of core.driver → driver.base
• 1.3 git mv core/device_manager.py device/manager.py, add device/__init__.py, update its imports of core.driver/core.errors/core.models → driver.base/core.errors/core.models
• 1.4 Confirm core/__init__.py, core/models.py, core/errors.py remain in place unchanged (shared domain types only)

2. Rename vision/ to perception/

• 2.1 git mv vision perception (carries ocr.py, ui_parser.py, scene_builder.py, icon_detector.py, __init__.py)
• 2.2 Update internal imports within the renamed package if any reference vision.*

3. Introduce the PerceptionProvider port

• 3.1 Create perception/provider.py with a PerceptionProvider ABC (build_scene(screenshot, tree) -> Scene)
• 3.2 Add a default provider that wraps the existing scene_builder.py fusion logic unmodified (same inputs, same Scene output — no behavior change)
• 3.3 Add NullPerceptionProvider returning an empty Scene (correct width/height, zero elements), usable without any OCR dependency installed
• 3.4 Point tools/describe_screen.py (and any other caller of scene_builder directly) at the PerceptionProvider port instead of importing scene_builder directly
• 3.5 Confirm tests/test_scene_builder.py still passes unchanged against the default provider

4. Relocate the Driver Registry

• 4.1 Create driver/registry.py; move SUPPORTED_DRIVER_TYPES, DriverFactoryBuilder, build_wda_driver_factory, and build_driver_factory out of api/console.py into it, importing WDADriver/WDADriverConfig from driver.wda_driver and DriverFactory from device.manager
• 4.2 Update api/console.py to import build_driver_factory (and SUPPORTED_DRIVER_TYPES if referenced) from driver.registry instead of defining them locally
• 4.3 Update tests/test_console_api.py (and any other test importing these names from api.console) to import from driver.registry where appropriate

5. Rename the shared error base

• 5.1 In core/errors.py, rename ApexAgentError → DeviceRuntimeError (subclasses DriverError, DeviceNotFoundError, DeviceOfflineError, DeviceBusyError, ElementNotFoundError, TaskFailedError keep their names, just re-parented)
• 5.2 Update every ApexAgentError reference across api/, tools/, runtime/, core/, driver/, device/, perception/, and tests/ to DeviceRuntimeError

6. Fix up all import sites and re-point package config

• 6.1 Grep the full tree for core.driver, core.device_manager, core.wda_driver, and vision. and update every remaining import (spans tools/, runtime/, storage/, api/, tests/) to the new driver.*/device.*/perception.* paths
• 6.2 Update pyproject.toml: [tool.setuptools.packages.find].include to list driver*, device*, perception* alongside the existing api*, core*, runtime*, storage*, tools*
• 6.3 Run the full pytest suite and fix any remaining import errors until it is fully green with no test-logic changes

7. Rebrand project identity

• 7.1 Update pyproject.toml: name = "device-agent-runtime", add a one-line description reflecting the new positioning
• 7.2 Create root README.md: project positioning as a device-agnostic Device Agent Runtime (iPhone/WDA as the first driver), link to docs/ROADMAP.md and docs/CONSTITUTION.md

8. Planning artifacts

• 8.1 Create docs/ROADMAP.md with the Milestone 0–10 sequence, the three delivery phases, and the long-term "DeviceOS" / Universal Device Runtime direction (v2.0, explicitly not started); explicitly map Milestones 1–4 to already-implemented apex-agent-mvp capabilities (device-management, scene-perception, agent-runtime, task-memory) rather than treating them as separate future work
• 8.2 Create docs/adr/0001-device-agnostic-runtime.md recording this repositioning decision (context, decision, alternatives considered, consequences), referencing this change's design.md decisions
• 8.3 Create docs/adr/0002-layered-hexagonal-architecture.md recording the Hexagonal/Ports-and-Adapters + DDD layering decision (D7): dependency direction (core domain has zero framework/LLM/HTTP deps; driver/device adapt external SDKs; tools is the capability/port layer; perception sits behind PerceptionProvider; runtime is the application layer where an LLM first enters via the Planner; api is the outermost transport adapter) and the bottom-up build order future milestones must respect
• 8.4 Create docs/CONSTITUTION.md capturing durable invariants: Driver is the only device-capability contract; tools/ only calls into device/driver capabilities, never a concrete driver; Scene is the only perception artifact the LLM ever sees, produced through PerceptionProvider; drivers are stateless; Planner produces a plan, Executor is the only thing that calls tools and retries; no LLM, HTTP, or MCP dependency may appear in core, driver, device, or tools — those enter only at runtime (Planner) and api respectively; every future milestone change must state in its design.md how it upholds this boundary
• 8.5 Create research/001-scene-model/README.md through research/007-ui-understanding/README.md (scene-model, world-model, skill, agent, memory, planner, ui-understanding), each a short placeholder stating the track's purpose and that it holds papers/experiments/benchmarks, not code

9. Final verification

• 9.1 Run pytest once more after all doc additions to confirm nothing in tests/ was accidentally affected
• 9.2 Manually skim every touched file's diff to confirm no behavior changed — only paths, names, new port, and net-new docs