feat: checkpoint device agent runtime milestones
This commit is contained in:
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-07-06
|
||||
@@ -0,0 +1,90 @@
|
||||
## Context
|
||||
|
||||
Today's Agent Runtime (`agent-runtime`, `apex-agent-mvp`, code-complete but unapplied) is exactly one loop shape: `runtime/task.py`'s `TaskRunner.run(task)` repeats observe→plan→act until the (currently stub) `Planner.goal_reached()` returns true, a step fails, or `TaskRunnerConfig.max_steps` (default 20) is exceeded, for one flat `Task.goal` string. `TaskContext` (`runtime/context.py`) holds `scenes`/`step_results` for that one run and is discarded when it ends. Three other pending, unapplied changes extend what a single step can *mean* but not the *shape* of orchestration above it: `semantic-scene-runtime` enriches a `Scene` into a `SemanticScene` per step; `world-model-runtime` derives a per-task `WorldState` (`current_app`/`current_page`/`variables`/bounded `history`) from that same step loop; `skill-learning-runtime` synthesizes a parameterized `FlowTemplateSkill` from a *completed* task's `Timeline`, but explicitly does not build anything that can *run* a skill — "a synthesized skill is inert data until some future runner resolves parameters and drives `tools/`." None of these three changes, nor `agent-runtime` itself, gives a caller a way to express "do sub-goal A, then run skill B, then wait until condition C, then do sub-goal D" as one persisted, resumable unit — that is this change's entire scope.
|
||||
|
||||
Two stakeholders: a workflow author (human or a future higher-level Agent/orchestrator, out of scope here) who defines a `WorkflowDefinition` up front as data (not interactively via a UI — no visual editor exists or is planned in this change), and the process running `WorkflowRunner`, which must be able to pick a `WorkflowRun` back up after a restart without re-running already-completed mutating steps (a `tap`, `input_text`, or skill invocation is not safely re-runnable the way a read-only `describe_screen` is).
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
- Introduce `WorkflowDefinition` as an ordered, possibly-branching list of `WorkflowStep`s of four kinds: planned-goal, skill-invocation, wait-for-condition, branch.
|
||||
- Introduce `WorkflowRun` as the persisted, checkpointed execution record of one `WorkflowDefinition` run, survivable across process restarts.
|
||||
- Provide a `WorkflowRunner` that composes (imports, never edits or subclasses) the existing `agent-runtime` `TaskRunner`/`Planner`/`Executor` for planned-goal steps.
|
||||
- Provide the first real skill-invocation execution path: resolve a `FlowTemplateSkill`'s parameters against step-supplied argument values and drive the resolved tool calls through `tools/` — the gap `skill-learning-runtime` deliberately left open.
|
||||
- Provide a small, closed, registrable set of wait/branch condition kinds sufficient for common multi-stage flows (text appears, a tracked variable equals a value, elapsed time, prior step succeeded), extensible the same way the Driver Registry (`device-agent-runtime-foundation`) is.
|
||||
- Make resumability real: a `WorkflowRunner.resume(run_id)` reload continues from the last checkpointed step, never re-executing a step already marked completed.
|
||||
|
||||
**Non-Goals:**
|
||||
- No visual workflow editor/UI — `WorkflowDefinition` is authored as data (constructed programmatically or loaded from a file/API a future caller provides); any UI is `web-console`'s domain, untouched here.
|
||||
- No distributed/multi-device workflow execution or cross-node coordination — that is Milestone 10 (Cloud Runtime); this milestone assumes one `WorkflowRunner` process driving one workflow run against one device at a time.
|
||||
- Does not replace or remove the existing single-goal Planner/Executor loop (`agent-runtime`) — a `WorkflowDefinition` with exactly one planned-goal step is a valid but uninteresting workflow; simple one-shot tasks should keep calling `TaskRunner` directly.
|
||||
- No general expression/scripting language for wait/branch conditions — only a closed, registrable set of condition *kinds*, each a small typed evaluator function, not an embedded interpreter or arbitrary `eval`.
|
||||
- No generic skill-kind runner — `skill-learning-runtime`'s `Skill.kind` also allows `knowledge` skills (non-executable reference text); this change's skill-invocation step only executes `flow_template` skills. Attempting to invoke a `knowledge`-kind skill is a step-definition error, not a runtime capability gap to fill here.
|
||||
- No retrieval-driven skill *selection* at workflow-run time — a skill-invocation step names an explicit `skill_id`; using `skill-embedding-retrieval`'s `retrieve_candidate_skills(goal, top_k)` to pick which skill a step should invoke is an authoring-time or future-Planner concern, not built into `WorkflowRunner` here.
|
||||
- No changes to `runtime/`, `storage/`, `tools/`, `skill-catalog-subscription`, or `web-console`'s files or specs.
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1: New `workflow/` package, sibling to `semantic/`/`world/`/`skills_learning/`, not folded into `runtime/`
|
||||
`workflow/` (`models.py`, `runner.py`, `conditions.py`, `skill_exec.py`, `store.py`, `config.py`) is its own package. `runtime/` stays focused on the single-goal Observe-Think-Act-Observe loop shape (its established job per `device-agent-runtime-foundation`'s layering ADR); multi-stage orchestration is a distinct concern one layer above it, matching the sibling-package precedent already set by `semantic-scene-runtime`'s D1, `world-model-runtime`'s D1, and `skill-learning-runtime`'s D1.
|
||||
**Alternative considered**: extend `runtime/task.py`'s `TaskRunner` in place to accept a list of goals/steps instead of one goal. Rejected — `TaskRunner` is a pending, unapplied, code-complete capability (`agent-runtime`) with no applied baseline in `openspec/specs/` to safely diff a `MODIFIED` delta against in this session; treating it as a stable, composed-over dependency (never edited) avoids retroactively changing another change's still-pending contract, and keeps "one flat goal, one loop" and "multi-stage workflow of possibly-heterogeneous steps" as two separate, independently testable concerns.
|
||||
|
||||
### D2: `WorkflowStep` as a tagged union of four concrete step dataclasses, not one flat schema with optional fields
|
||||
`workflow/models.py` defines `PlannedGoalStep{goal: str}`, `SkillInvocationStep{skill_id: str, args: dict}`, `WaitForConditionStep{condition: ConditionSpec, timeout_seconds: float, poll_interval_seconds: float}`, and `BranchStep{condition: ConditionSpec, on_true: str, on_false: str}` (all sharing a common `step_id: str` and optional explicit `next_step_id: str | None` for non-branch kinds — default is "fall through to the next step in `WorkflowDefinition.steps` order"), unioned as `WorkflowStep = PlannedGoalStep | SkillInvocationStep | WaitForConditionStep | BranchStep`. `WorkflowRunner` dispatches on `isinstance`/a `kind` discriminator to one handler function per step type.
|
||||
**Alternative considered**: one flat `WorkflowStep` dataclass with optional fields for every kind (`goal: str | None`, `skill_id: str | None`, `condition: ConditionSpec | None`, `on_true`/`on_false: str | None` all present simultaneously). Rejected — this makes invalid states representable (a step with both `goal` and `condition` populated, ambiguous about which the runner should honor) and pushes validation into every consumer; a tagged union makes constructing a step with two kinds' fields at once a type error, not a runtime validation rule to remember and enforce.
|
||||
|
||||
### D3: `WorkflowRun` checkpointed to `WorkflowStore` after every completed step, not only at start/end
|
||||
`WorkflowRunner.run(definition)` creates a `WorkflowRun{id, definition_id, status, current_step_id, variables, step_results}` row via `WorkflowStore.create_run()`, then after each step completes (success or failure) calls `WorkflowStore.update_run()` to persist the new `current_step_id`/`status`/`step_results` entry before advancing. `WorkflowRunner.resume(run_id)` loads the persisted `WorkflowRun`, looks up its `current_step_id` in the `WorkflowDefinition`, and continues from there.
|
||||
**Alternative considered**: persist only the initial definition and the final result, reconstructing "how far did we get" by scanning `task-memory`'s `Timeline`/`storage/task_metadata.py` for related task ids. Rejected — `Timeline`/task-metadata know nothing about workflow-step boundaries (a workflow step is not a 1:1 mapping to a `Task`; a wait-for-condition or branch step produces no `Task` at all), so re-deriving workflow progress from task-memory records would require workflow-step-boundary metadata to be smuggled into another capability's storage; an explicit, workflow-owned checkpoint after each step is simpler and self-contained.
|
||||
|
||||
### D4: Resumability is "reload-and-continue from last checkpoint," not "replay from step 0 assuming idempotent actions"
|
||||
On `resume(run_id)`, `WorkflowRunner` never re-executes a step whose result is already recorded in the persisted `WorkflowRun.step_results` for `current_step_id`'s predecessors; it resumes execution starting at the step recorded as the current (in-progress or not-yet-started) one.
|
||||
**Alternative considered**: on resume, replay the entire workflow from its first step, relying on `tools/` actions being idempotent (e.g., re-tapping an already-tapped button is harmless) to make re-execution safe. Rejected — many mutating actions are not idempotent (sending a chat message twice creates two messages; launching a payment flow twice is unacceptable), so assuming idempotency across arbitrary drivers/tools is unsafe; explicit step-level checkpointing avoids re-invoking already-completed mutating steps, at the cost (see Risks) of not providing a stronger exactly-once guarantee across a crash *during* a single step's execution.
|
||||
|
||||
### D5: Skill-invocation steps get a first, minimal parameter-resolution + tool-dispatch path in `workflow/skill_exec.py`
|
||||
`skill_exec.run_flow_template_skill(skill: FlowTemplateSkill, args: dict) -> list[StepResult]` validates `args` against `skill.parameters` (missing required parameter or unknown extra key fails the step before any tool call is issued), substitutes `{param}` placeholders in the skill's stored `steps` (tool name + args-template, as produced by `skill-learning-runtime`'s synthesis) with the resolved values, and calls the corresponding `tools/*` functions directly (the same functions `Executor.execute()` already dispatches to for planned-goal steps), returning one `StepResult` per resolved step. This is deliberately the "some future runner" `skill-learning-runtime`'s design.md left as an open non-goal — Workflow Runtime is the first concrete need for it.
|
||||
**Alternative considered**: leave skill-invocation steps declared-but-unimplemented in this change too (a no-op stub, like `Planner.plan()`'s unused `world` kwarg in `world-model-runtime`), deferring actual skill execution to yet another future change. Rejected — a workflow step kind that can never execute would make this proposal's own headline example ("run a Skill instead of re-planning from scratch") impossible to demonstrate; scoping it down instead (only `flow_template`-kind skills, no retry/backoff beyond what a direct `tools/` call already offers, no retrieval-based selection — see Non-Goals) keeps the executable surface small without leaving it entirely unbuilt.
|
||||
|
||||
### D6: `ConditionEvaluator` port + registry for wait/branch conditions, mirroring the Driver Registry pattern
|
||||
`workflow/conditions.py` defines `ConditionEvaluator` (one method, `evaluate(spec: ConditionSpec, *, scene, world_state) -> bool`) and a registry mapping a condition `kind` string (`scene_contains_text`, `world_variable_equals`, `elapsed_seconds`, `step_result_success`) to an evaluator, the same "string key → pluggable implementation" shape as `driver/registry.py`'s `SUPPORTED_DRIVER_TYPES` (`device-agent-runtime-foundation`, D3) and `perception/provider.py`'s `PerceptionProvider` port (D8). A `WaitForConditionStep`/`BranchStep`'s `ConditionSpec{kind: str, params: dict}` is looked up in this registry at evaluation time; adding a new condition kind is a one-function addition to `conditions.py`, not a change to `WorkflowRunner`'s control flow.
|
||||
**Alternative considered**: hardcode condition evaluation as an `if/elif` chain over a `condition_type` string directly inside `WorkflowRunner`. Rejected — this is precisely the extension-point shape the project has already standardized on twice (Driver Registry, `PerceptionProvider`); a third ad hoc if/elif chain for the same "pluggable-by-string-key" problem would be inconsistent with established project convention for no benefit.
|
||||
|
||||
### D7: Branching via explicit `on_true`/`on_false` step-id targets, not a general expression/scripting language
|
||||
`BranchStep.condition` is evaluated via the same `ConditionEvaluator` registry as wait steps; the runner then sets `current_step_id` to `on_true` or `on_false` (both required, explicit step ids present in the same `WorkflowDefinition.steps`) rather than falling through sequentially.
|
||||
**Alternative considered**: support an embedded expression language (e.g., a small boolean-expression evaluator over `WorldState.variables`) for branch conditions, allowing arbitrary author-supplied predicates. Rejected — an embedded expression evaluator large enough to be genuinely useful risks becoming a code-injection-adjacent surface in a system that drives real devices (a crafted expression reaching into more than intended), and the proposal's own Non-Goals already scope out a general DSL; a closed, registrable set of condition kinds (D6) plus explicit `goto`-style step-id targets is sufficient for the "wait/branch" step types this milestone commits to, and is easy to extend later without an interpreter.
|
||||
|
||||
### D8: `WorkflowRunner` composes `TaskRunner` as a black box for planned-goal steps; never edits or subclasses it
|
||||
A `PlannedGoalStep` is executed by constructing a `core.models.Task(goal=step.goal, device_id=run.device_id)`, calling `TaskRunner(...).run(task)` (letting `agent-runtime`'s existing Planner/Executor/retry logic run unmodified), and reading back `task.status`/`task.failure_reason` to decide the step's `StepResult`. `WorkflowRunner` never imports or touches `runtime/task.py`'s internals beyond this public `run(task) -> Task` contract.
|
||||
**Alternative considered**: give `WorkflowRunner` its own lower-level loop that calls `Planner.plan()`/`Executor.execute()` directly per planned-goal step, skipping `TaskRunner`. Rejected — this would duplicate `TaskRunner`'s already-implemented max-steps/retry/failure-reason bookkeeping inside `workflow/`, doubling the surface area that has to stay behaviorally consistent with `agent-runtime`'s own spec; composing the existing public `run(task) -> Task` entry point is simpler and automatically inherits any future `agent-runtime` improvement (e.g., a smarter Planner) with zero change to `workflow/`.
|
||||
|
||||
### D9: `WorkflowStore` is a new, independently-owned SQLite database file, not a new table in `storage/task_metadata.py`'s existing database
|
||||
`workflow/store.py`'s `WorkflowStore` follows the same connect-per-call `sqlite3` pattern as `storage/task_metadata.py`, but opens its own file (default `workflows/workflows.sqlite3`) with its own schema (`workflow_definitions`, `workflow_runs`, `workflow_step_results` tables) rather than adding tables to `tasks/tasks.sqlite3`.
|
||||
**Alternative considered**: add `workflow_runs`/`workflow_step_results` tables directly into `storage/task_metadata.py`'s existing database and module. Rejected — `storage/` is owned by the pending, unapplied `task-memory` capability; adding tables/schema-migration code to it would be a de facto modification of another change's owned artifact with no corresponding spec delta to justify it, contradicting this session's explicit constraint not to touch other pending changes' files. A separate, `workflow/`-owned store with its own schema-creation code is fully additive and requires no coordination with `task-memory`'s eventual implementation.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- **[Risk]** Step-level checkpointing (D3/D4) does not provide a true exactly-once guarantee: if the process crashes *during* a mutating step's execution (after the tool call was issued but before the checkpoint write completes), resuming will re-execute that step, potentially double-sending a message or double-tapping an action → **Mitigation**: documented as an accepted limitation, not solved in this change (mirrors `task-memory`'s own per-tool-call durability limits); a future revision could require mutating skill/tool calls to carry a caller-supplied idempotency key threaded through `tools/`, but no such mechanism exists in `tools/` today to build on, so it is left as an Open Question rather than invented here.
|
||||
- **[Risk]** `WorkflowRunner`'s composition of `TaskRunner` (D8) depends on `agent-runtime`'s public `run(task) -> Task` contract remaining stable while `apex-agent-mvp` itself is still pending/unapplied; if that change's spec evolves before archiving, this composition point could silently drift → **Mitigation**: treat only `TaskRunner.run(task) -> Task` and the `Task.status`/`Task.failure_reason` fields as the relied-upon contract (not `TaskRunner`'s internals); cover this in an integration-style test that runs a real (not mocked) `TaskRunner` instance so any breaking drift fails a test, not silently.
|
||||
- **[Risk]** Skill-invocation execution (D5) drives `tools/` with parameter values resolved from step-supplied `args`, which could contain a malformed or unexpectedly-typed value that passes schema validation but produces an unintended device action (e.g., an empty string where a search query was expected) → **Mitigation**: parameter validation checks presence/type against `skill.parameters`' declared schema before any tool call is issued (fail the step, no partial execution), but does not attempt semantic validation of *values* (e.g., "is this a real search term") — that remains the same trust boundary `Executor.execute()` already has for planned-goal steps' tool-call arguments.
|
||||
- **[Trade-off]** Explicit `on_true`/`on_false` step-id branching (D7) instead of a general expression DSL means a workflow author must hand-wire every branch target and cannot express a condition beyond the registered kinds (D6) without a code change to `conditions.py` → **Acceptable**, matches this milestone's stated Non-Goal (no visual editor, no general DSL); a registrable-by-string-key extension point keeps adding a new kind cheap even though it requires a code change, not a data-only change.
|
||||
- **[Trade-off]** Requiring an explicit `skill_id` on `SkillInvocationStep` (D5's Non-Goal on retrieval-driven selection) means a workflow cannot dynamically pick "whichever skill best matches this sub-goal" at run time — only `skill-embedding-retrieval`'s `retrieve_candidate_skills()` at authoring time (outside this runtime) could inform which `skill_id` to hardcode into the step → **Acceptable** for this milestone; wiring retrieval into live step dispatch is future Planner-integration work, consistent with `world-model-runtime`'s and `skill-learning-runtime`'s shared precedent of exposing a capability without yet teaching a decision-maker to use it dynamically.
|
||||
- **[Trade-off]** A new, separate SQLite file (D9) means "all workflow runs" and "all tasks" are two different databases with no foreign-key enforcement between a `WorkflowRun`'s planned-goal step and the `Task`/`task_metadata` row it produced (only a `task_id` string reference stored in `WorkflowStepResult`) → **Acceptable**; this mirrors `skill-learning-runtime`'s D6 acceptance of "two stores, composed in prose, merge is a future concern" rather than forcing a schema dependency on another pending change's not-yet-applied storage.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
This is purely additive; no existing module is edited:
|
||||
1. Add the `workflow/` package: `models.py` (`WorkflowDefinition`, the four `WorkflowStep` variants, `WorkflowRun`, `WorkflowStepResult`, `ConditionSpec`), `conditions.py` (`ConditionEvaluator` port + registry + the four starting condition kinds), `skill_exec.py` (`run_flow_template_skill`), `runner.py` (`WorkflowRunner.run()`/`.resume()` and one private dispatch method per step kind), `store.py` (`WorkflowStore`, schema creation for its own `workflows.sqlite3`), `config.py` (default poll interval, default wait timeout, default db path).
|
||||
2. Wire `WorkflowRunner`'s planned-goal step handler to construct a `Task` and call the existing `runtime.task.TaskRunner(...).run(task)` — import only, zero edits to `runtime/`.
|
||||
3. Wire `WorkflowRunner`'s skill-invocation step handler to `skill_exec.run_flow_template_skill`, importing `Skill`/`FlowTemplateSkill` dataclass shapes from `skill-learning-runtime`'s planned `skills_learning/` package (or, if implemented first, `skill-catalog-subscription`'s canonical `skills/models.py`) by import, not redefinition; if neither is implemented yet when `workflow/` is built, define the minimal shared shape locally with a note to reconcile once one of those changes lands (same caveat `skill-learning-runtime`'s own migration plan already carries for the same dependency direction).
|
||||
4. Wire `WorkflowRunner`'s wait/branch step handlers to `conditions.py`'s registry; optionally read `TaskContext.world`/`WorldState.variables` (from `world-model-runtime`, if that change is applied and enabled) for `world_variable_equals` conditions — degrade to "condition never satisfied until timeout" if `WorldState` is unavailable, never raise.
|
||||
5. Add `workflow*` to `pyproject.toml`'s `[tool.setuptools.packages.find].include` list; no new third-party dependency.
|
||||
6. Add new tests (`tests/test_workflow_runner.py`-style): a linear multi-step workflow (planned-goal → skill-invocation → wait → planned-goal), a branch workflow taking each path, a wait step that times out, and a resume test that constructs a fresh `WorkflowRunner`/`WorkflowStore` pointed at the same db file mid-run to simulate a process restart and asserts already-completed steps are not re-executed.
|
||||
7. Run `pytest` — the full existing suite stays green with zero edits to existing test files, confirming this change touches no existing behavior.
|
||||
8. Rollback: net-new package plus a net-new SQLite file with its own schema; reverting is `git revert` of the commit(s), with no data migration of any existing store and no external system involved beyond whatever device the composed `TaskRunner`/`tools/` calls already target.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Whether mutating skill-invocation/planned-goal steps need a stronger idempotency mechanism (e.g., a caller-supplied idempotency key threaded through `tools/`) to close the crash-during-step-execution gap noted in Risks, once resumability is exercised against real, non-idempotent device flows — left unsolved here, no such mechanism exists in `tools/` to build on yet.
|
||||
- Whether `WorkflowRun.variables` should automatically bridge with a planned-goal step's underlying `TaskContext`/`WorldState.variables` (so a value a Planner "remembers" mid-task is visible to a later branch/wait step in the same workflow) or stay a separate, workflow-only variable scope populated only by explicit step outputs — this change keeps them separate (no automatic bridging) and leaves richer variable-flow as a follow-on decision once real multi-stage workflows using both are observed.
|
||||
- Whether `skill-embedding-retrieval`'s `retrieve_candidate_skills()` should eventually be wired into `SkillInvocationStep` resolution (author supplies a goal instead of a fixed `skill_id`, runner picks the top candidate at execution time) rather than requiring an explicit `skill_id` — left to a future revision once retrieval-based selection has been exercised standalone.
|
||||
- Whether Milestone 10 (Cloud Runtime)'s distributed/multi-device execution will require redesigning `WorkflowRun`'s persistence schema for multi-node coordination (e.g., leader election over which node resumes a given run, or sharding runs by device) — this milestone's schema assumes single-process, single-node execution only, and that assumption should be revisited when Cloud Runtime is scoped.
|
||||
@@ -0,0 +1,29 @@
|
||||
## Why
|
||||
|
||||
`agent-runtime` (`apex-agent-mvp`, code-complete, unapplied) gives the runtime exactly one shape of work: a single flat goal driven by one Planner/Executor Observe-Think-Act-Observe loop until it succeeds, fails, or exceeds `max_steps`. Real device-automation usage is rarely one flat goal — it is a sequence of distinct sub-goals with control flow between them: open an app, send a message, **wait** for a reply to arrive, take a screenshot, then end; or run a locally-learned Skill (`skill-learning-runtime`, Milestone 7) instead of re-planning from scratch, only re-planning if the skill's assumptions don't hold. Nothing in the current runtime can express "do A, then B, then wait until C is true, then D," track where a multi-stage run is, or resume it if the process restarts mid-way. Workflow Runtime introduces `Workflow` as a first-class, persisted, resumable object composed of planner-derived steps, skill-invocation steps, wait-for-condition steps, and simple branch steps — sitting one layer above the existing single-goal loop, not replacing it.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add a new `workflow/` package defining a `WorkflowDefinition` (an ordered, possibly-branching list of `WorkflowStep`s) as a discriminated union of four step kinds: a **planned-goal step** (delegates a sub-goal string to the existing `agent-runtime` Planner/Executor loop), a **skill-invocation step** (resolves parameters and executes a locally-synthesized `FlowTemplateSkill` from `skill-learning-runtime`, Milestone 7), a **wait-for-condition step** (polls a named condition — e.g. scene text present, a world variable equals a value, elapsed time — up to a timeout), and a **branch step** (evaluates a condition and jumps to a named step id instead of falling through sequentially).
|
||||
- Add a `WorkflowRun` — the persisted, mutable execution record for one execution of a `WorkflowDefinition`: status (`pending`/`running`/`waiting`/`completed`/`failed`/`cancelled`), `current_step_id`, workflow-scoped `variables`, and a per-step result log — checkpointed to storage after every completed step so a `WorkflowRunner.resume(run_id)` can continue from the last checkpoint instead of re-running from step 0, without re-executing already-completed mutating steps.
|
||||
- Add a `WorkflowRunner` orchestration component that composes (imports, never subclasses or edits) `runtime/task.py`'s existing `TaskRunner` for planned-goal steps, and drives skill-invocation/wait/branch steps itself.
|
||||
- Add a first, minimal skill-invocation execution path (`workflow/skill_exec.py`): resolve a `FlowTemplateSkill`'s declared parameters against a step's supplied argument values, validate against the skill's parameter schema, and drive the resolved tool calls through `tools/` — the "some future runner resolves parameters and drives `tools/`" gap `skill-learning-runtime` explicitly left open.
|
||||
- Add a `ConditionEvaluator` port + registry (`workflow/conditions.py`) for wait/branch conditions, mirroring the Driver Registry / `PerceptionProvider` extension-point pattern already established by `device-agent-runtime-foundation`, with a small starting set of condition kinds (`scene_contains_text`, `world_variable_equals`, `elapsed_seconds`, `step_result_success`).
|
||||
- Add a new, independently-owned SQLite-backed `WorkflowStore` (`workflow/store.py`) for `WorkflowDefinition`/`WorkflowRun` persistence, following the same connect-per-call `sqlite3` pattern as `storage/task_metadata.py` but in its own database file — not a schema change to `storage/`, which remains owned by the pending `task-memory` capability.
|
||||
- No changes to `runtime/task.py`, `runtime/planner.py`, `runtime/executor.py`, `runtime/context.py`, `storage/*`, `tools/*`, or any other existing module's public behavior — this change is purely additive composition on top of them.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- `workflow-orchestration`: A `Workflow` model (planned-goal / skill-invocation / wait-for-condition / branch step kinds), executed by a `WorkflowRunner` that composes the existing single-goal Planner/Executor loop and the not-yet-built skill-execution path, persisted via a task-memory-style store, and resumable from its last checkpoint if interrupted mid-workflow.
|
||||
|
||||
### Modified Capabilities
|
||||
(none — `openspec/specs/` has no applied baseline for `agent-runtime`, `task-memory`, `skill-authoring`, or `skill-embedding-retrieval` yet, so this change cannot and does not write a `MODIFIED Requirements` delta against any of them; it composes with their pending, unapplied designs in prose only, and does not alter their specified behavior.)
|
||||
|
||||
## Impact
|
||||
|
||||
- **New package**: `workflow/` — `models.py` (`WorkflowDefinition`, `WorkflowStep` variants, `WorkflowRun`, `WorkflowStepResult`), `runner.py` (`WorkflowRunner`), `conditions.py` (`ConditionEvaluator` port + registry), `skill_exec.py` (parameter resolution + tool dispatch for skill-invocation steps), `store.py` (`WorkflowStore`, SQLite-backed), `config.py` (enable flags, default poll interval, default wait timeout).
|
||||
- **New storage**: a new `workflows/workflows.sqlite3` database file (own schema: `workflow_definitions`, `workflow_runs`, `workflow_step_results` tables), independent of `storage/task_metadata.py`'s `tasks/tasks.sqlite3`.
|
||||
- **Composed, not modified, dependencies**: `runtime/task.py`'s `TaskRunner` (planned-goal steps construct a `Task` and call `TaskRunner(...).run(task)`, reading back `task.status`/`task.failure_reason`), `skill-learning-runtime`'s `FlowTemplateSkill`/`Skill` dataclass shape (imported, not redefined, matching that change's own D6 precedent for composing with `skill-catalog-subscription`), and optionally `world-model-runtime`'s `WorldState.variables` (read-only, for `world_variable_equals` conditions — degrades to "condition never satisfied until timeout" if `WorldState` is absent, never raises).
|
||||
- **Config**: add `workflow*` to `pyproject.toml`'s `[tool.setuptools.packages.find].include` list; no new third-party dependency beyond the standard library `sqlite3` already used by `storage/task_metadata.py`.
|
||||
- **Out of scope**: no visual workflow editor/UI (a future `web-console` concern, not touched here); no distributed/multi-device workflow execution (Milestone 10, Cloud Runtime); does not replace or remove the existing single-goal Planner/Executor loop, which remains the right tool for simple one-shot tasks; no generic expression/scripting language for branch conditions (a closed, registrable set of condition kinds only); no changes to `skill-catalog-subscription`, `web-console`, or any other pending change's files.
|
||||
@@ -0,0 +1,97 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Workflow definition as an ordered, branchable list of typed steps
|
||||
The system SHALL provide a `WorkflowDefinition` model representing an ordered, possibly-branching list of `WorkflowStep`s, where each step is exactly one of four kinds: a planned-goal step (a natural-language sub-goal delegated to the existing single-goal Planner/Executor loop), a skill-invocation step (a reference to a locally-synthesized flow-template skill plus argument values), a wait-for-condition step (a named condition, timeout, and poll interval), or a branch step (a named condition plus two target step ids). Each step SHALL have a unique `step_id` within its `WorkflowDefinition`.
|
||||
|
||||
#### Scenario: Workflow with heterogeneous step kinds is constructed
|
||||
- **WHEN** a `WorkflowDefinition` is built with a planned-goal step, a skill-invocation step, a wait-for-condition step, and a branch step in sequence
|
||||
- **THEN** the system accepts the definition and each step retains its declared kind and fields without requiring fields belonging to another step kind
|
||||
|
||||
#### Scenario: Duplicate step id is rejected
|
||||
- **WHEN** a `WorkflowDefinition` is constructed with two steps sharing the same `step_id`
|
||||
- **THEN** the system rejects the definition before any run is created from it
|
||||
|
||||
### Requirement: Persisted, checkpointed workflow run
|
||||
The system SHALL persist a `WorkflowRun` record for each execution of a `WorkflowDefinition`, containing the run's status, the currently active step id, workflow-scoped variables, and a per-step result log, and SHALL update this record after every completed step before advancing to the next one.
|
||||
|
||||
#### Scenario: Run status transitions as steps execute
|
||||
- **WHEN** a `WorkflowRun` is started for a `WorkflowDefinition`
|
||||
- **THEN** the system creates a persisted run record with status `running` and, as each step completes, updates the persisted `current_step_id` and per-step result log before the next step begins
|
||||
|
||||
#### Scenario: Run reaches terminal status
|
||||
- **WHEN** all steps in a `WorkflowDefinition` complete successfully, or a step fails without a defined recovery path
|
||||
- **THEN** the system updates the persisted `WorkflowRun` status to `completed` or `failed` respectively, and records a failure reason when failed
|
||||
|
||||
### Requirement: Resume from last checkpoint without re-executing completed steps
|
||||
The system SHALL support resuming an interrupted `WorkflowRun` from its last persisted checkpoint, continuing execution from the currently active step without re-executing any step already recorded as completed in that run's step result log.
|
||||
|
||||
#### Scenario: Resume after simulated process restart
|
||||
- **WHEN** a `WorkflowRun` has completed its first two steps and the process driving it stops before the third step completes, and a new `WorkflowRunner` instance is later pointed at the same persisted run id
|
||||
- **THEN** the system resumes execution starting at the third step and does not re-invoke the tool calls or sub-goal already recorded as completed for the first two steps
|
||||
|
||||
#### Scenario: Resume on an already-completed run is a no-op
|
||||
- **WHEN** `resume` is called with the id of a `WorkflowRun` whose status is already `completed`
|
||||
- **THEN** the system returns the run's existing final state without executing any further steps
|
||||
|
||||
### Requirement: Planned-goal step delegates to the existing single-goal loop
|
||||
The system SHALL execute a planned-goal step by delegating its sub-goal to the existing Planner/Executor Observe-Think-Act-Observe loop for a single task, and SHALL derive that step's success or failure from the resulting task's final status.
|
||||
|
||||
#### Scenario: Planned-goal step succeeds
|
||||
- **WHEN** a planned-goal step's delegated task reaches a completed status
|
||||
- **THEN** the workflow step is recorded as succeeded and the run advances to the next step
|
||||
|
||||
#### Scenario: Planned-goal step fails
|
||||
- **WHEN** a planned-goal step's delegated task reaches a failed status
|
||||
- **THEN** the workflow step is recorded as failed with the task's failure reason and the run's status becomes `failed` unless a branch step defines an alternate path
|
||||
|
||||
### Requirement: Skill-invocation step resolves parameters and executes a flow-template skill
|
||||
The system SHALL execute a skill-invocation step by validating its supplied argument values against the referenced flow-template skill's declared parameters, substituting the validated values into the skill's stored tool-call template, and executing the resolved tool calls in order.
|
||||
|
||||
#### Scenario: Skill invocation with valid parameters executes resolved tool calls
|
||||
- **WHEN** a skill-invocation step supplies argument values that satisfy the referenced skill's declared required parameters
|
||||
- **THEN** the system substitutes those values into the skill's stored steps and executes the resulting tool calls in the skill's recorded order
|
||||
|
||||
#### Scenario: Skill invocation with a missing required parameter fails without executing any tool call
|
||||
- **WHEN** a skill-invocation step omits a value for a parameter the referenced skill declares as required
|
||||
- **THEN** the system fails the step before issuing any tool call and records the missing-parameter reason
|
||||
|
||||
#### Scenario: Skill invocation referencing a non-flow-template skill is rejected
|
||||
- **WHEN** a skill-invocation step references a skill whose kind is not a flow-template
|
||||
- **THEN** the system fails the step as a step-definition error rather than attempting to execute it
|
||||
|
||||
### Requirement: Wait-for-condition step polls until satisfied or timed out
|
||||
The system SHALL execute a wait-for-condition step by repeatedly evaluating its named condition at the step's configured poll interval until the condition is satisfied or the step's configured timeout elapses.
|
||||
|
||||
#### Scenario: Condition becomes true before timeout
|
||||
- **WHEN** a wait-for-condition step's condition evaluates true within its configured timeout
|
||||
- **THEN** the system stops polling, records the step as succeeded, and advances the run to the next step
|
||||
|
||||
#### Scenario: Condition never becomes true before timeout
|
||||
- **WHEN** a wait-for-condition step's condition has not evaluated true by its configured timeout
|
||||
- **THEN** the system records the step as failed with a timeout reason and the run's status becomes `failed` unless a branch step defines an alternate path
|
||||
|
||||
### Requirement: Branch step selects the next step from a condition
|
||||
The system SHALL execute a branch step by evaluating its named condition and setting the run's next active step to the branch's configured true-target step id or false-target step id accordingly, instead of advancing to the next step in definition order.
|
||||
|
||||
#### Scenario: Branch condition true selects the true-target step
|
||||
- **WHEN** a branch step's condition evaluates true
|
||||
- **THEN** the system sets the run's current step to the branch's `on_true` target step id
|
||||
|
||||
#### Scenario: Branch condition false selects the false-target step
|
||||
- **WHEN** a branch step's condition evaluates false
|
||||
- **THEN** the system sets the run's current step to the branch's `on_false` target step id
|
||||
|
||||
### Requirement: Condition kinds are pluggable via a registry
|
||||
The system SHALL evaluate wait-for-condition and branch step conditions through a registry mapping a condition kind name to an evaluator, SHALL provide at least `scene_contains_text`, `world_variable_equals`, `elapsed_seconds`, and `step_result_success` as built-in kinds, and SHALL allow a new condition kind to be added without modifying the workflow runner's step-dispatch logic.
|
||||
|
||||
#### Scenario: Built-in condition kind evaluates against current state
|
||||
- **WHEN** a wait-for-condition or branch step specifies the `scene_contains_text` kind with a target text value
|
||||
- **THEN** the system evaluates the condition against the most recently observed scene and returns true only when the target text is present
|
||||
|
||||
#### Scenario: Unregistered condition kind fails the step
|
||||
- **WHEN** a step specifies a condition kind that is not present in the registry
|
||||
- **THEN** the system fails that step with an unrecognized-condition-kind reason instead of executing an undefined check
|
||||
|
||||
#### Scenario: World-state-dependent condition degrades safely when world state is absent
|
||||
- **WHEN** a `world_variable_equals` condition is evaluated for a run whose task context has no `WorldState` available
|
||||
- **THEN** the system treats the condition as not satisfied rather than raising an error, allowing the step to continue polling until its timeout
|
||||
@@ -0,0 +1,67 @@
|
||||
## 1. Package scaffolding
|
||||
|
||||
- [ ] 1.1 Create the `workflow/` package (`__init__.py`, `models.py`, `conditions.py`, `skill_exec.py`, `runner.py`, `store.py`, `config.py`)
|
||||
- [ ] 1.2 Add `workflow*` to `[tool.setuptools.packages.find].include` in `pyproject.toml` (no new third-party dependency; `sqlite3` is stdlib, already used by `storage/task_metadata.py`)
|
||||
- [ ] 1.3 Add Workflow Runtime configuration in `workflow/config.py`: default poll interval, default wait-step timeout, default `WorkflowStore` db path (`workflows/workflows.sqlite3`)
|
||||
- [ ] 1.4 Extend the project's smoke test (that imports every package) to import `workflow`
|
||||
|
||||
## 2. Workflow and step data model (capability: workflow-orchestration)
|
||||
|
||||
- [ ] 2.1 Implement `workflow/models.py`: `ConditionSpec{kind: str, params: dict}`, the four step dataclasses (`PlannedGoalStep{step_id, goal, next_step_id}`, `SkillInvocationStep{step_id, skill_id, args, next_step_id}`, `WaitForConditionStep{step_id, condition, timeout_seconds, poll_interval_seconds, next_step_id}`, `BranchStep{step_id, condition, on_true, on_false}`), and the `WorkflowStep` union type
|
||||
- [ ] 2.2 Implement `WorkflowDefinition{id, name, steps: list[WorkflowStep], entry_step_id}` with a constructor-time validation pass rejecting duplicate `step_id`s and any `next_step_id`/`on_true`/`on_false`/`entry_step_id` that does not reference an existing step in the same definition
|
||||
- [ ] 2.3 Implement `WorkflowRun{id, definition_id, status, current_step_id, variables: dict, step_results: list[WorkflowStepResult], created_at, updated_at}` and `WorkflowStepResult{step_id, kind, success, detail, task_id: str | None, timestamp}`, with `to_dict()`/`from_dict()` helpers mirroring `core/models.py`'s style
|
||||
- [ ] 2.4 Write unit tests for `WorkflowDefinition` construction: valid heterogeneous-step definition accepted; duplicate `step_id` rejected; dangling `next_step_id`/`on_true`/`on_false`/`entry_step_id` reference rejected
|
||||
|
||||
## 3. WorkflowStore persistence (capability: workflow-orchestration)
|
||||
|
||||
- [ ] 3.1 Implement `workflow/store.py`: `WorkflowStore(db_path)` with schema creation for `workflow_definitions`, `workflow_runs`, `workflow_step_results` tables in its own SQLite file, following `storage/task_metadata.py`'s connect-per-call pattern
|
||||
- [ ] 3.2 Implement `WorkflowStore.save_definition(definition)` / `get_definition(definition_id)`
|
||||
- [ ] 3.3 Implement `WorkflowStore.create_run(definition_id, initial_variables) -> WorkflowRun` (status `pending`/`running`, `current_step_id` set to the definition's `entry_step_id`)
|
||||
- [ ] 3.4 Implement `WorkflowStore.append_step_result(run_id, step_result)` and `WorkflowStore.update_run(run_id, *, status=None, current_step_id=None, variables=None)`, both persisting immediately (no in-memory-only buffering)
|
||||
- [ ] 3.5 Implement `WorkflowStore.get_run(run_id) -> WorkflowRun` reconstructing the full run (status, current step, variables, ordered step results) from persisted rows
|
||||
- [ ] 3.6 Write unit tests for `WorkflowStore`: create/get round-trip, step-result append ordering, run survives being re-opened via a fresh `WorkflowStore(same db_path)` instance (simulating a process restart)
|
||||
|
||||
## 4. ConditionEvaluator registry (capability: workflow-orchestration)
|
||||
|
||||
- [ ] 4.1 Implement `workflow/conditions.py`: `ConditionEvaluator` protocol/ABC with `evaluate(spec, *, scene, world_state) -> bool`, and a registry `dict[str, ConditionEvaluator]`
|
||||
- [ ] 4.2 Implement the `scene_contains_text` evaluator (checks the most recent `Scene`'s elements/text for a target substring from `spec.params`)
|
||||
- [ ] 4.3 Implement the `world_variable_equals` evaluator (compares `world_state.variables.get(spec.params["name"])` to `spec.params["value"]`; returns `False` — never raises — when `world_state` is `None` or the key is absent)
|
||||
- [ ] 4.4 Implement the `elapsed_seconds` evaluator (returns `True` once at least `spec.params["seconds"]` have elapsed since the wait step started polling)
|
||||
- [ ] 4.5 Implement the `step_result_success` evaluator (looks up a prior step's recorded `WorkflowStepResult.success` by `spec.params["step_id"]` from the run's step-result log)
|
||||
- [ ] 4.6 Implement `evaluate_condition(spec, *, scene, world_state) -> bool` as the registry lookup entry point, raising a distinguishable `UnknownConditionKindError` for an unregistered `kind` (caught by the runner and turned into a failed step, not an uncaught exception)
|
||||
- [ ] 4.7 Write unit tests for each built-in evaluator (true case, false case) and for the unregistered-kind error path
|
||||
|
||||
## 5. Skill-invocation execution path (capability: workflow-orchestration)
|
||||
|
||||
- [ ] 5.1 Implement `workflow/skill_exec.py`: `validate_skill_args(skill, args) -> None` raising a descriptive error listing missing required parameters or unrecognized argument keys, called before any tool dispatch
|
||||
- [ ] 5.2 Implement `resolve_skill_steps(skill, args) -> list[dict]` substituting `{param}` placeholders in the skill's stored tool-call template with validated `args` values
|
||||
- [ ] 5.3 Implement `run_flow_template_skill(skill, args) -> list[StepResult]` calling `validate_skill_args`, `resolve_skill_steps`, then dispatching each resolved tool call through the same `tools/*` functions `Executor.execute()` already uses, collecting one `StepResult` per resolved step
|
||||
- [ ] 5.4 Implement the `Skill.kind != "flow_template"` guard: reject with a step-definition error before attempting resolution
|
||||
- [ ] 5.5 Define (or import, if `skill-learning-runtime`/`skill-catalog-subscription` is already implemented) the minimal `Skill`/`FlowTemplateSkill` dataclass shape `skill_exec.py` depends on, with a code comment flagging reconciliation once one of those changes lands
|
||||
- [ ] 5.6 Write unit tests for `skill_exec`: valid-args resolution produces the expected resolved tool-call sequence; missing required parameter fails before any tool call; non-flow-template skill kind is rejected
|
||||
|
||||
## 6. WorkflowRunner orchestration (capability: workflow-orchestration)
|
||||
|
||||
- [ ] 6.1 Implement `workflow/runner.py`: `WorkflowRunner(store, task_runner_factory, ...)` with `run(definition, device_id, initial_variables=None) -> WorkflowRun` that creates a run via `WorkflowStore.create_run()` and drives steps until a terminal status
|
||||
- [ ] 6.2 Implement the planned-goal step handler: construct a `core.models.Task(goal=step.goal, device_id=...)`, call `TaskRunner(...).run(task)`, map `task.status`/`task.failure_reason` to a `WorkflowStepResult`
|
||||
- [ ] 6.3 Implement the skill-invocation step handler: look up the referenced skill by `skill_id`, call `skill_exec.run_flow_template_skill`, map the returned `StepResult`s to one aggregate `WorkflowStepResult`
|
||||
- [ ] 6.4 Implement the wait-for-condition step handler: poll `conditions.evaluate_condition()` at `poll_interval_seconds` until `True` or `timeout_seconds` elapses; record success or a timeout failure
|
||||
- [ ] 6.5 Implement the branch step handler: evaluate the condition once and set the run's next step to `on_true`/`on_false` accordingly, bypassing default sequential advancement
|
||||
- [ ] 6.6 Implement default sequential advancement (no explicit `next_step_id`/branch target) as "the next step in `WorkflowDefinition.steps` order" for non-branch step kinds
|
||||
- [ ] 6.7 After each step handler returns, call `WorkflowStore.append_step_result()` and `WorkflowStore.update_run()` (new `current_step_id`, and `status` if terminal) before advancing — no step is considered "done" until this checkpoint write completes
|
||||
- [ ] 6.8 Implement `WorkflowRunner.resume(run_id) -> WorkflowRun`: load the persisted run via `WorkflowStore.get_run()`, and continue driving from its persisted `current_step_id`, skipping any step already present in the loaded `step_results` log
|
||||
- [ ] 6.9 Implement `resume()` on an already-`completed`/`failed`/`cancelled` run as a no-op that returns the existing run state unchanged
|
||||
- [ ] 6.10 Write unit tests for `WorkflowRunner.run()` covering: a linear planned-goal-only workflow reaching `completed`; a workflow with a failing planned-goal step reaching `failed` with a recorded reason; a skill-invocation step executing resolved tool calls (mocked `tools/*`); a wait step succeeding before timeout and timing out after
|
||||
- [ ] 6.11 Write unit tests for branch step selection (true path and false path) and for default sequential advancement between non-branch steps
|
||||
|
||||
## 7. Resumability end-to-end validation (capability: workflow-orchestration)
|
||||
|
||||
- [ ] 7.1 Write an end-to-end test that runs a multi-step workflow through `WorkflowRunner`, stops after two steps (simulating a crash by discarding the in-memory `WorkflowRunner`/`TaskRunner` instances), constructs a fresh `WorkflowRunner`/`WorkflowStore` pointed at the same db file, and asserts `resume(run_id)` continues from the third step without re-invoking the first two steps' tool calls or delegated tasks
|
||||
- [ ] 7.2 Write an end-to-end test asserting a `WorkflowRun`'s persisted `variables` and `step_results` log are readable and correctly ordered after a full run via `WorkflowStore.get_run()`
|
||||
- [ ] 7.3 Write an end-to-end test combining a branch step with a wait step and a skill-invocation step in one workflow, asserting the run reaches `completed` via the expected branch path
|
||||
|
||||
## 8. Composition safety checks and full-suite validation
|
||||
|
||||
- [ ] 8.1 Confirm no existing file under `runtime/`, `storage/`, `tools/`, or `core/` is modified by this change (composition via import only, per design.md's D1/D8/D9)
|
||||
- [ ] 8.2 Write a test that runs a real (non-mocked) `runtime.task.TaskRunner` instance inside a `WorkflowRunner`-driven planned-goal step, guarding against silent drift in `agent-runtime`'s public `run(task) -> Task` contract this change composes over
|
||||
- [ ] 8.3 Run the full test suite (`pytest`) and confirm every existing test in `tests/` passes unmodified, with only new `tests/test_workflow_*.py`-style files added
|
||||
Reference in New Issue
Block a user