feat: checkpoint device agent runtime milestones

2026-07-06 17:24:03 +08:00
parent 2d4251e98e
commit 5658735bca
153 changed files with 8060 additions and 65 deletions
@@ -0,0 +1,88 @@
+# Roadmap
+
+This roadmap positions the project as a Device Agent Runtime: a stable runtime
+for LLM agents to operate real devices through device-agnostic contracts.
+
+## Delivery Phases
+
+### Phase 1: Local Runtime Foundation
+
+Build and stabilize the single-process runtime that can control one real
+device, perceive its screen, execute tool calls, and persist task history.
+
+### Phase 2: Intelligence and Reuse
+
+Add semantic scene understanding, world state, skill learning, workflow
+orchestration, and multi-agent collaboration on top of the runtime boundaries.
+
+### Phase 3: Cloud Runtime
+
+Scale the runtime into a schedulable, multi-device platform with device pools,
+plugin surfaces, and operational APIs.
+
+## Milestones
+
+### Milestone 0: Foundation
+
+Reposition the project as Device Agent Runtime, split driver/device/perception
+packages, move driver selection into `driver/registry.py`, add
+`PerceptionProvider`, and record the architecture invariants.
+
+### Milestone 1: Device Management
+
+Already implemented by `apex-agent-mvp` through the `device-management`
+capability: device registration, status, connection lifecycle, retries, and WDA
+as the first concrete driver.
+
+### Milestone 2: Scene Perception
+
+Already implemented by `apex-agent-mvp` through the `scene-perception`
+capability: screenshot and UI tree fusion into a single `Scene` with OCR and
+deduplication.
+
+### Milestone 3: Agent Runtime
+
+Already implemented by `apex-agent-mvp` through the `agent-runtime` capability:
+Planner/Executor structure, tool execution, retry behavior, and task runner.
+
+### Milestone 4: Task Memory
+
+Already implemented by `apex-agent-mvp` through the `task-memory` capability:
+timeline records, screenshots, prompts, tool calls, and task metadata.
+
+### Milestone 5: Semantic Scene
+
+Add compact semantic summaries on top of `Scene` so prompts consume page
+identity, supported intents, and widget purposes rather than raw geometry.
+
+### Milestone 6: World Model
+
+Maintain task-relevant state across observations, including durable facts about
+apps, workflows, and device state transitions.
+
+### Milestone 7: Skill Learning
+
+Represent reusable skills, version them, retrieve them by embedding or metadata,
+and allow new skills to be authored from successful task traces.
+
+### Milestone 8: Workflow Orchestration
+
+Add explicit workflow definitions, branching, waiting, and reusable execution
+plans above the low-level tool layer.
+
+### Milestone 9: Multi-Agent Runtime
+
+Support multiple collaborating roles, shared context, review loops, and
+coordinated execution while preserving the same device/runtime boundaries.
+
+### Milestone 10: Cloud Runtime
+
+Add device pools, scheduling, plugins, and platform APIs for distributed
+execution across many devices.
+
+## Long-Term Direction
+
+The long-term v2.0 direction is a "DeviceOS" / Universal Device Runtime: one
+agent-facing operating surface for many real-world device types. That direction
+is explicitly not started by this roadmap. Current work remains focused on the
+local Device Agent Runtime and the milestone sequence above.