spec
This commit is contained in:
@@ -0,0 +1,49 @@
|
||||
# device-pool Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change cloud-runtime. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Host registration and heartbeat sync
|
||||
The system SHALL provide a `DevicePool` that tracks a `HostRegistration` (host id, address, last-seen timestamp) for each host process that registers itself, and SHALL update a host's last-seen timestamp whenever that host pushes a device snapshot via `sync_host_devices(host_id, snapshot)`.
|
||||
|
||||
#### Scenario: New host registers and syncs devices
|
||||
- **WHEN** a previously-unknown `host_id` calls `sync_host_devices` with a list of devices
|
||||
- **THEN** the pool creates a new `HostRegistration` for that host, records the current time as its last-seen timestamp, and stores each synced device as a `PooledDevice` owned by that host
|
||||
|
||||
#### Scenario: Known host re-syncs
|
||||
- **WHEN** an already-registered `host_id` calls `sync_host_devices` again with an updated device snapshot
|
||||
- **THEN** the pool updates that host's last-seen timestamp and replaces its previously-stored `PooledDevice` records with the new snapshot, without duplicating or losing devices from other hosts
|
||||
|
||||
### Requirement: Aggregated device listing across hosts
|
||||
The system SHALL provide a way to list all `PooledDevice` records across every registered host, including each device's owning `host_id`, `driver_type`, status, and capability tags.
|
||||
|
||||
#### Scenario: Listing devices across multiple hosts
|
||||
- **WHEN** two hosts have each synced a non-empty device snapshot
|
||||
- **THEN** a caller listing pool devices sees devices from both hosts in one combined result, each tagged with its correct `host_id`
|
||||
|
||||
#### Scenario: No hosts registered
|
||||
- **WHEN** a caller lists pool devices before any host has ever synced
|
||||
- **THEN** the pool returns an empty list rather than raising an error
|
||||
|
||||
### Requirement: Stale host devices degrade to unreachable
|
||||
The system SHALL mark all `PooledDevice`s belonging to a host `unreachable` once that host's last-seen timestamp exceeds a configured staleness threshold, computed at read time, without requiring any background process and without raising an error for the stale host's absence.
|
||||
|
||||
#### Scenario: Host misses its sync interval
|
||||
- **WHEN** a host's last-seen timestamp is older than `config.stale_after_seconds` at the time of a `list_devices()`/`get_device()` call
|
||||
- **THEN** every `PooledDevice` owned by that host is reported with status `unreachable`, regardless of the status value in its last-synced snapshot
|
||||
|
||||
#### Scenario: Host resumes syncing after being stale
|
||||
- **WHEN** a host previously marked stale calls `sync_host_devices` again
|
||||
- **THEN** its devices immediately stop being reported `unreachable` and reflect the statuses in the new snapshot
|
||||
|
||||
### Requirement: Device lookup by id across the pool
|
||||
The system SHALL allow looking up a single `PooledDevice` by `device_id` regardless of which host owns it, returning a clear not-found result when no host has ever reported that device id.
|
||||
|
||||
#### Scenario: Lookup finds device on any host
|
||||
- **WHEN** a caller requests a device by id that exists in some host's synced snapshot
|
||||
- **THEN** the pool returns that `PooledDevice` including its owning `host_id`
|
||||
|
||||
#### Scenario: Lookup for unknown device id
|
||||
- **WHEN** a caller requests a device by id that no host has ever synced
|
||||
- **THEN** the pool returns a not-found result (e.g. `None`) rather than raising an unhandled exception
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
# driver-registry Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change device-agent-runtime-foundation. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Driver type registry lives in the driver layer
|
||||
The system SHALL provide a registry, owned by the `driver` package, that maps a `driver_type` string (e.g. `"wda"`) to a builder function producing a `DriverFactory` for that type, so that any caller needing to construct a driver for a device does so without importing a concrete driver class directly.
|
||||
|
||||
#### Scenario: Building a factory for a known driver type
|
||||
- **WHEN** a caller requests a driver factory for `driver_type="wda"` with connection info (e.g. `server_url`, `udid`)
|
||||
- **THEN** the registry returns a `DriverFactory` that, when invoked, constructs a working `WDADriver` configured with that connection info
|
||||
|
||||
#### Scenario: Building a factory for an unknown driver type
|
||||
- **WHEN** a caller requests a driver factory for a `driver_type` that is not registered
|
||||
- **THEN** the registry raises a clear error naming the unsupported `driver_type`, instead of returning `None` or a factory that fails later at connect time
|
||||
|
||||
### Requirement: Adding a driver type requires no changes outside the driver layer
|
||||
The system SHALL allow a new driver type to be added by registering it in the `driver` package's registry alone; no other package (`api/`, `device/`, `tools/`, `runtime/`) SHALL need code changes to support constructing devices of the new type.
|
||||
|
||||
#### Scenario: API layer is agnostic to registered driver types
|
||||
- **WHEN** the console config API resolves a `driver_type` string into a driver factory to register a device
|
||||
- **THEN** it does so by calling into the driver registry rather than maintaining its own mapping of `driver_type` to concrete driver classes
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
# multi-agent-collaboration Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change multi-agent-runtime. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Handoff protocol data contracts
|
||||
The system SHALL define `Observation`, `VerificationVerdict`, `ReflectionOutcome`, and `ReflectionAction` as plain dataclasses with `to_dict()`/`from_dict()` methods, forming the stable, spec'd contract passed between the Observer, Verifier, and Reflector roles, independent of any single role's internal implementation.
|
||||
|
||||
#### Scenario: Handoff dataclasses round-trip through serialization
|
||||
- **WHEN** an `Observation`, `VerificationVerdict`, or `ReflectionOutcome` instance is serialized via `to_dict()` and then reconstructed via `from_dict()`
|
||||
- **THEN** the reconstructed instance is equal to the original instance
|
||||
|
||||
### Requirement: Observer produces an Observation from available device state
|
||||
The system SHALL provide an Observer role that produces an `Observation` summarizing current device state, using `SemanticScene` and `WorldState` when available and falling back to the raw `Scene`/`PlannedStep`/`StepResult` when either or both are absent.
|
||||
|
||||
#### Scenario: Observation succeeds with SemanticScene and WorldState present
|
||||
- **WHEN** `Observer.observe(...)` is called and both `SemanticScene` and `WorldState` are available
|
||||
- **THEN** it returns an `Observation` incorporating both as context
|
||||
|
||||
#### Scenario: Observation degrades gracefully when semantic state is absent
|
||||
- **WHEN** `Observer.observe(...)` is called and `SemanticScene` and/or `WorldState` is `None`
|
||||
- **THEN** it returns an `Observation` built from the raw `Scene`/`PlannedStep`/`StepResult` instead of raising an exception
|
||||
|
||||
### Requirement: Verifier checks whether an executed step achieved its intended effect
|
||||
The system SHALL provide a Verifier role that, after a `PlannedStep` has already been executed by the existing `Executor`, compares a pre-step and post-step `Observation` against the step's stated intent and produces a `VerificationVerdict` indicating whether the intended effect was actually achieved, distinct from and running after the Executor's own low-level tool-call retry.
|
||||
|
||||
#### Scenario: Verifier confirms an achieved effect
|
||||
- **WHEN** `Verifier.verify(...)` is called with a pre-step `Observation`, a post-step `Observation`, the executed `PlannedStep`, and its `StepResult`, and the post-step `Observation` reflects the step's stated intent
|
||||
- **THEN** it returns a `VerificationVerdict` marking the step as achieved
|
||||
|
||||
#### Scenario: Verifier flags a mechanically-successful but semantically-failed step
|
||||
- **WHEN** the `StepResult` reports mechanical success but the post-step `Observation` does not reflect the step's stated intent
|
||||
- **THEN** `Verifier.verify(...)` returns a `VerificationVerdict` marking the step as not achieved
|
||||
|
||||
### Requirement: Reflector proposes bounded recovery, never a blind re-issue
|
||||
The system SHALL invoke a Reflector role only when the Verifier produces a not-achieved `VerificationVerdict`, and the Reflector SHALL analyze the `Observation`/`PlannedStep`/`StepResult`/verdict to produce a `ReflectionOutcome` carrying either a bounded, distinct recovery `ReflectionAction` or a replan request back to the Planner, never an outcome that simply re-issues the identical failed step.
|
||||
|
||||
#### Scenario: Reflector proposes a distinct recovery action
|
||||
- **WHEN** `Reflector.reflect(...)` is invoked following a not-achieved `VerificationVerdict` and a distinct corrective action is identifiable
|
||||
- **THEN** it returns a `ReflectionOutcome` carrying a `ReflectionAction` that differs from the originally failed `PlannedStep`
|
||||
|
||||
#### Scenario: Reflector requests a replan when no bounded recovery action applies
|
||||
- **WHEN** `Reflector.reflect(...)` is invoked and no bounded corrective action is identifiable from the available `Observation`/`PlannedStep`/`StepResult`/verdict
|
||||
- **THEN** it returns a `ReflectionOutcome` carrying a replan request rather than re-issuing the failed step
|
||||
|
||||
### Requirement: Reflection-driven recovery is bounded by an explicit ceiling
|
||||
The system SHALL enforce a configurable maximum number of Reflector-triggered recovery attempts per task, tracked independently of and in addition to the Executor's own `max_retries`, so that a persistently-failing step cannot loop indefinitely between the Verifier and Reflector.
|
||||
|
||||
#### Scenario: Reflection loop stops once the ceiling is reached
|
||||
- **WHEN** a task's Verifier-Reflector loop reaches the configured maximum reflection-recovery attempts without a step being verified as achieved
|
||||
- **THEN** the `CollaborativeTaskRunner` stops attempting further reflection-driven recovery for that task and surfaces the failure instead of continuing the loop
|
||||
|
||||
### Requirement: Multi-agent collaboration composes existing Planner/Executor without modifying them
|
||||
The system SHALL provide a `CollaborativeTaskRunner` that composes the existing `Planner`, `Executor`, and `TaskRunner` (`agent-runtime`) by import, without modifying `runtime/planner.py`, `runtime/executor.py`, or `runtime/task.py`, and SHALL default to disabled so that applying this capability does not alter any existing task's behavior, latency, or cost unless explicitly enabled.
|
||||
|
||||
#### Scenario: Collaboration disabled by default leaves existing task behavior unchanged
|
||||
- **WHEN** a task is run without explicitly enabling multi-agent collaboration
|
||||
- **THEN** it executes exactly as `runtime/task.py`'s existing `TaskRunner.run()` today, with no Observer/Verifier/Reflector invoked
|
||||
|
||||
#### Scenario: Enabling collaboration does not require changes to Planner or Executor
|
||||
- **WHEN** a task is run with multi-agent collaboration explicitly enabled via `CollaborativeTaskRunner`
|
||||
- **THEN** the existing `Planner.plan()` and `Executor.execute()` are invoked unchanged, with the Observer/Verifier/Reflector roles composed around them
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
# perception-provider Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change device-agent-runtime-foundation. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Perception is exposed through a swappable provider port
|
||||
The system SHALL define a `PerceptionProvider` interface (`build_scene`) in the `perception` package that any concrete perception implementation (the existing OCR+tree fusion, a future cloud vision API, a future null/mock provider) must satisfy identically, so `tools/`, `runtime/`, and `api/` depend only on the port, never on a specific perception technique.
|
||||
|
||||
#### Scenario: Building a scene through the default provider
|
||||
- **WHEN** a caller requests a Scene for a screenshot and UI tree via the default (OCR+tree fusion) provider
|
||||
- **THEN** the provider returns a `Scene` identical in shape and content to what the existing fusion logic already produces — no change to `scene-perception`'s specified behavior
|
||||
|
||||
### Requirement: A null perception provider is available for testing and low-dependency development
|
||||
The system SHALL provide a `NullPerceptionProvider` that returns an empty `Scene` (correct screen width/height, zero elements) without requiring OCR/vision dependencies to be installed or invoked.
|
||||
|
||||
#### Scenario: Running without OCR dependencies installed
|
||||
- **WHEN** the runtime is configured to use `NullPerceptionProvider` (e.g. in a test or a minimal development environment)
|
||||
- **THEN** `describe_screen`/Agent Runtime calls succeed and return an empty Scene instead of failing due to missing OCR dependencies
|
||||
|
||||
### Requirement: Adding a perception technique requires no changes outside the perception layer
|
||||
The system SHALL allow a new perception provider (e.g. a cloud vision API) to be added by registering it within the `perception` package alone; `tools/`, `runtime/`, and `api/` SHALL NOT require code changes to use a newly registered provider.
|
||||
|
||||
#### Scenario: Runtime is agnostic to which provider is active
|
||||
- **WHEN** the Agent Runtime requests a Scene for the current screenshot and UI tree
|
||||
- **THEN** it does so through the `PerceptionProvider` port without importing or knowing about `scene_builder.py`, OCR, or any other concrete technique
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
# platform-sdk Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change cloud-runtime. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Versioned public API surface
|
||||
The system SHALL expose the platform SDK's REST endpoints under a versioned URL prefix (`/v1/...`), distinct from the `mcp-tool-server` and `console-status-api`/`console-config-api` surfaces, so external integrators have a stable base path that will not silently change shape.
|
||||
|
||||
#### Scenario: Routes are mounted under the version prefix
|
||||
- **WHEN** the platform SDK's router is mounted into an application
|
||||
- **THEN** every route it exposes (task submission, status queries, device/host listing, plugin listing/registration) is reachable only under the `/v1/` prefix
|
||||
|
||||
### Requirement: Task submission and status via the SDK
|
||||
The system SHALL allow an external integrator to submit a task (goal or workflow reference plus constraints) through the platform SDK's API, and to query that task's current status by id, backed by the `task-scheduler` capability.
|
||||
|
||||
#### Scenario: Submit a task via the API
|
||||
- **WHEN** an integrator calls the task-submission endpoint with a valid goal and optional constraints
|
||||
- **THEN** the API returns a task id that can be used to poll status, and the underlying `task-scheduler` records a new `queued` `ScheduledTask`
|
||||
|
||||
#### Scenario: Query status of a known task
|
||||
- **WHEN** an integrator requests status for a task id that exists
|
||||
- **THEN** the API returns that task's current status (`queued`, `assigned`, `dispatched`, `done`, or `failed`)
|
||||
|
||||
#### Scenario: Query status of an unknown task
|
||||
- **WHEN** an integrator requests status for a task id that does not exist
|
||||
- **THEN** the API returns a not-found response rather than an unhandled server error
|
||||
|
||||
### Requirement: Device and host visibility via the SDK
|
||||
The system SHALL allow an external integrator to list devices and hosts known to the `device-pool` capability through the platform SDK's API.
|
||||
|
||||
#### Scenario: List devices across the pool
|
||||
- **WHEN** an integrator calls the device-listing endpoint
|
||||
- **THEN** the API returns every `PooledDevice` known to the pool, including owning host id and current (possibly `unreachable`) status
|
||||
|
||||
#### Scenario: List registered hosts
|
||||
- **WHEN** an integrator calls the host-listing endpoint
|
||||
- **THEN** the API returns every `HostRegistration` known to the pool, including last-seen timestamp
|
||||
|
||||
### Requirement: Plugin listing and registration via the SDK
|
||||
The system SHALL allow an external integrator to list registered plugins and submit a new plugin manifest for registration through the platform SDK's API, backed by the `plugin-system` capability.
|
||||
|
||||
#### Scenario: List registered plugins
|
||||
- **WHEN** an integrator calls the plugin-listing endpoint
|
||||
- **THEN** the API returns every registered `PluginManifest`, including its `entry_point_kind` and whether it is wired to an execution path
|
||||
|
||||
#### Scenario: Register a new plugin manifest
|
||||
- **WHEN** an integrator submits a valid plugin manifest to the plugin-registration endpoint
|
||||
- **THEN** the API registers it via `plugin-system`'s `PluginRegistry` and returns the stored manifest, or a clear validation/conflict error if registration fails
|
||||
|
||||
### Requirement: Pluggable authentication hook with a safe default
|
||||
The system SHALL evaluate every platform SDK route through a configurable `AuthProvider` hook, defaulting to a no-op provider that treats every caller as an anonymous, authenticated principal, so real authentication can be added later without changing route signatures.
|
||||
|
||||
#### Scenario: Default configuration allows anonymous access
|
||||
- **WHEN** no `AuthProvider` is explicitly configured
|
||||
- **THEN** every route accepts requests without rejecting them for lack of credentials
|
||||
|
||||
#### Scenario: Custom AuthProvider is honored
|
||||
- **WHEN** a caller configures a custom `AuthProvider` that rejects a request
|
||||
- **THEN** the platform SDK's routes return an authorization error for that request instead of proceeding, without any route's own handler code needing to change
|
||||
|
||||
### Requirement: Python SDK client mirrors the REST API
|
||||
The system SHALL provide a Python client (`CloudClient`) exposing methods corresponding to each `/v1/...` route (submit task, get task status, list devices, list hosts, list plugins, register plugin), so integrators do not need to hand-construct HTTP requests.
|
||||
|
||||
#### Scenario: Client submits a task and retrieves status
|
||||
- **WHEN** a caller uses `CloudClient` to submit a task and then fetch its status by the returned id
|
||||
- **THEN** the client's methods produce the same result as calling the corresponding `/v1/...` endpoints directly over HTTP
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
# plugin-system Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change cloud-runtime. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Plugin manifest schema
|
||||
The system SHALL define a `PluginManifest` schema with a unique `name`, a `version`, an `entry_point_kind` restricted to `driver`, `tool`, or `skill`, and a `target` (a dotted module:attribute reference to the plugin's implementation), and SHALL reject a manifest missing any required field or using an unrecognized `entry_point_kind`.
|
||||
|
||||
#### Scenario: Valid manifest accepted
|
||||
- **WHEN** a manifest with all required fields and a recognized `entry_point_kind` is submitted for registration
|
||||
- **THEN** the registry accepts it and stores it as a known plugin
|
||||
|
||||
#### Scenario: Manifest with unrecognized entry_point_kind rejected
|
||||
- **WHEN** a manifest declares an `entry_point_kind` other than `driver`, `tool`, or `skill`
|
||||
- **THEN** the registry rejects it with a clear validation error and does not register it
|
||||
|
||||
#### Scenario: Duplicate plugin name rejected
|
||||
- **WHEN** a manifest is submitted whose `name` matches an already-registered plugin
|
||||
- **THEN** the registry rejects the new registration with a clear conflict error rather than silently overwriting the existing entry
|
||||
|
||||
### Requirement: Plugin discovery via entry points and manifest files
|
||||
The system SHALL discover plugin manifests both from installed Python packages declaring an entry point in the `device_agent_runtime.plugins` group and from local `plugin.json` files under a configured scan path, feeding both sources into the same validation-and-registration path.
|
||||
|
||||
#### Scenario: Discovery via installed entry point
|
||||
- **WHEN** an installed package declares an entry point in the `device_agent_runtime.plugins` group resolving to a valid manifest
|
||||
- **THEN** `PluginRegistry.discover()` finds and registers it
|
||||
|
||||
#### Scenario: Discovery via local manifest file
|
||||
- **WHEN** a `plugin.json` file exists under the configured plugin scan path and parses into a valid manifest
|
||||
- **THEN** `PluginRegistry.discover()` finds and registers it
|
||||
|
||||
#### Scenario: Malformed manifest file is skipped, not fatal
|
||||
- **WHEN** a `plugin.json` file under the scan path fails to parse or fails schema validation
|
||||
- **THEN** `PluginRegistry.discover()` skips that file, records it as a discovery error, and continues discovering remaining plugins rather than aborting the whole scan
|
||||
|
||||
### Requirement: Driver-kind plugins register into the driver registry extension point
|
||||
The system SHALL, for a manifest with `entry_point_kind == "driver"`, resolve its `target` to a driver-factory builder and register it under the manifest's `name` as a new `driver_type` in the existing driver-registry extension point, without requiring any edit to the `driver` package's own files.
|
||||
|
||||
#### Scenario: Driver plugin registered successfully
|
||||
- **WHEN** a valid `driver`-kind manifest is registered and its `target` resolves to a callable driver-factory builder
|
||||
- **THEN** the manifest's `name` becomes usable as a `driver_type` value by any caller building a driver factory, with no change to existing driver-registry code
|
||||
|
||||
#### Scenario: Driver registry extension point unavailable
|
||||
- **WHEN** a `driver`-kind manifest is registered but the driver-registry's registration function is not importable in the running environment
|
||||
- **THEN** the registry raises a clear, explicit error naming the missing integration point, rather than silently accepting the manifest without wiring it
|
||||
|
||||
### Requirement: Tool and skill plugin manifests are accepted but explicitly marked unwired
|
||||
The system SHALL accept and store `tool`- and `skill`-kind plugin manifests (listable like any other registered plugin) but SHALL report them as not wired to any execution path, rather than implying they are active.
|
||||
|
||||
#### Scenario: Tool-kind manifest registered
|
||||
- **WHEN** a valid `tool`-kind manifest is registered
|
||||
- **THEN** the registry stores it and it appears in a plugin listing with a `wired: false` indicator, and no tool dispatch path is modified as a result
|
||||
|
||||
#### Scenario: Skill-kind manifest registered
|
||||
- **WHEN** a valid `skill`-kind manifest is registered
|
||||
- **THEN** the registry stores it and it appears in a plugin listing with a `wired: false` indicator, and no skill store or execution path is modified as a result
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
# semantic-scene Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change semantic-scene-runtime. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Semantic scene enrichment from an existing Scene
|
||||
The system SHALL provide a function that, given an existing `Scene` (as produced by the `scene-perception` capability), produces a `SemanticScene` consisting of a page identity string, a list of plain-language supported intents, and a list of per-widget purpose labels referencing the `Scene`'s element IDs, using exactly one LLM call.
|
||||
|
||||
#### Scenario: Enrichment succeeds for a recognizable screen
|
||||
- **WHEN** `enrich_scene()` is called with a `Scene` describing a recognizable app screen (e.g. a chat screen with a text input and a send button)
|
||||
- **THEN** it returns a `SemanticScene` with a non-empty `page` string, a non-empty `intents` list of plain-language strings, and a `widgets` list where each entry's `element_id` matches an element ID present in the input `Scene`
|
||||
|
||||
#### Scenario: Widget purpose labels reference only known elements
|
||||
- **WHEN** the LLM response includes a widget purpose label whose `element_id` does not match any element ID in the input `Scene`
|
||||
- **THEN** `enrich_scene()` discards that widget entry from the returned `SemanticScene` rather than propagating a dangling element reference
|
||||
|
||||
### Requirement: Enrichment failure degrades to no semantic scene, never blocks the loop
|
||||
The system SHALL treat any enrichment failure (LLM call timeout, connection error, rate limit, malformed or schema-invalid response, or enrichment disabled by configuration) as a non-fatal condition, returning an absence of a semantic scene rather than raising an exception, so that callers always have a defined fallback of using the raw `Scene`.
|
||||
|
||||
#### Scenario: LLM call times out
|
||||
- **WHEN** the enrichment LLM call does not complete within the configured timeout
|
||||
- **THEN** `enrich_scene()` returns `None` and the caller proceeds using the raw `Scene` without the task step being marked as failed
|
||||
|
||||
#### Scenario: LLM response fails schema validation
|
||||
- **WHEN** the enrichment LLM call returns a response that does not conform to the expected `SemanticScene` JSON schema
|
||||
- **THEN** `enrich_scene()` returns `None` instead of raising, and no partially-parsed `SemanticScene` is returned
|
||||
|
||||
#### Scenario: Enrichment disabled by configuration
|
||||
- **WHEN** semantic enrichment is disabled in configuration
|
||||
- **THEN** `enrich_scene()` returns `None` immediately without making an LLM call
|
||||
|
||||
### Requirement: Structured, schema-constrained LLM output
|
||||
The system SHALL request the enrichment LLM call using a schema-constrained structured-output mechanism so that any successful response is guaranteed to be valid JSON matching the `SemanticScene` shape, rather than relying on free-text parsing of an unconstrained model reply.
|
||||
|
||||
#### Scenario: Successful call yields directly parseable output
|
||||
- **WHEN** the enrichment LLM call completes successfully
|
||||
- **THEN** the raw response body is valid JSON matching the declared `SemanticScene` schema without requiring text extraction, regex matching, or a JSON-repair step
|
||||
|
||||
### Requirement: Semantic enrichment is opt-in and does not alter existing tool behavior
|
||||
The system SHALL expose semantic enrichment through a new, separate tool entry point rather than modifying the existing `describe_screen` tool's signature or behavior, so that every existing caller of `describe_screen` continues to receive only a `Scene`, unchanged, unless it explicitly opts into the new semantic-enriched entry point.
|
||||
|
||||
#### Scenario: Existing describe_screen callers are unaffected
|
||||
- **WHEN** an existing caller invokes the `describe_screen` tool as it did before this change
|
||||
- **THEN** it receives the same `Scene` result as before, with no semantic enrichment attempted and no new LLM-related dependency invoked
|
||||
|
||||
#### Scenario: A caller opts into semantic enrichment
|
||||
- **WHEN** a caller invokes the new semantic-enrichment tool entry point for a given device
|
||||
- **THEN** it receives both the underlying `Scene` and, when enrichment succeeds, the corresponding `SemanticScene`; when enrichment fails or is disabled, it receives the `Scene` with an explicit absence of a `SemanticScene` rather than a partial or error result
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
# skill-authoring Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change skill-learning-runtime. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Synthesis triggers only on successful task completion
|
||||
The system SHALL synthesize a flow-template skill only when a task's final status is `succeeded`, and SHALL NOT attempt synthesis for a task that failed, was cancelled, or is still running.
|
||||
|
||||
#### Scenario: Successful task triggers synthesis
|
||||
- **WHEN** a task completes with status `succeeded` and Skill Authoring is enabled
|
||||
- **THEN** the system reads that task's timeline and goal and produces a flow-template skill candidate
|
||||
|
||||
#### Scenario: Failed task does not trigger synthesis
|
||||
- **WHEN** a task completes with status `failed` (or is cancelled/still running)
|
||||
- **THEN** the system does not synthesize any skill from that task's timeline
|
||||
|
||||
### Requirement: Skill Authoring defaults to disabled
|
||||
The system SHALL leave Skill Authoring disabled by default in configuration, so applying this capability does not change the cost or latency of any existing task run until a caller explicitly enables it.
|
||||
|
||||
#### Scenario: Default configuration performs no synthesis
|
||||
- **WHEN** a task completes successfully and Skill Authoring has not been explicitly enabled in configuration
|
||||
- **THEN** the system performs no synthesis work and the task's completion path behaves exactly as it would without this capability
|
||||
|
||||
#### Scenario: Explicit enable activates synthesis
|
||||
- **WHEN** an operator enables Skill Authoring in configuration
|
||||
- **THEN** subsequently completed successful tasks are eligible for synthesis
|
||||
|
||||
### Requirement: Flow-template skill synthesized from executed tool-call sequence
|
||||
The system SHALL derive a flow-template skill's ordered steps from the sequence of mutating tool calls (e.g. `tap`, `swipe`, `input_text`, `launch_app`) recorded in the completed task's timeline, in the order they were executed, and SHALL exclude read-only/observational tool calls (e.g. `describe_screen`, `screenshot`, `ui_tree`) from the synthesized step list.
|
||||
|
||||
#### Scenario: Mutating steps are included in order
|
||||
- **WHEN** a task's timeline contains a sequence of `launch_app`, `tap`, `input_text`, `tap` tool calls that all succeeded
|
||||
- **THEN** the synthesized skill's steps list contains those four steps in that same order
|
||||
|
||||
#### Scenario: Read-only observation calls are excluded
|
||||
- **WHEN** a task's timeline includes `describe_screen` or `screenshot` calls interleaved with mutating calls
|
||||
- **THEN** the synthesized skill's steps list omits those read-only calls and retains only the mutating steps
|
||||
|
||||
### Requirement: Parameter abstraction from cross-execution argument diffing
|
||||
The system SHALL abstract a synthesized skill's step arguments into named parameters by comparing the newly executed argument values against a previously stored skill with the same tool-name step sequence, promoting any argument value that differs between the two executions into a named placeholder, and SHALL leave a first-time synthesis (no prior matching skeleton) with zero parameters.
|
||||
|
||||
#### Scenario: First execution of a flow has no parameters
|
||||
- **WHEN** no previously stored skill shares the newly executed tool-name sequence
|
||||
- **THEN** the synthesized skill is stored with its literal argument values and an empty parameters list
|
||||
|
||||
#### Scenario: Second, divergent execution promotes a differing value to a parameter
|
||||
- **WHEN** a later successful task executes the same tool-name sequence as a stored skill but with a different literal value at one argument position
|
||||
- **THEN** the system promotes that argument position to a named parameter in the skill's `parameters` schema and replaces the literal in `steps` with a `{param}` placeholder referencing it
|
||||
|
||||
#### Scenario: Identical repeated execution does not spuriously add parameters
|
||||
- **WHEN** a later successful task executes the same tool-name sequence as a stored skill with identical argument values at every position
|
||||
- **THEN** the system does not introduce any new parameter for that skill
|
||||
|
||||
### Requirement: Locally-authored skills are stored separately from synced skills
|
||||
The system SHALL persist locally-synthesized flow-template skills tagged with a `source` of `local-synthesis`, in a store owned by this capability, and SHALL NOT write into or modify the externally-synced skill catalog store or its sync-only write contract.
|
||||
|
||||
#### Scenario: Synthesized skill is tagged as locally-authored
|
||||
- **WHEN** a flow-template skill is synthesized from a completed task
|
||||
- **THEN** its stored record has `source = "local-synthesis"` and no subscription identifier
|
||||
|
||||
#### Scenario: Synced skill catalog is untouched by synthesis
|
||||
- **WHEN** a skill is synthesized and stored by this capability
|
||||
- **THEN** no record in the externally-synced skill catalog store is created, modified, or removed as a result
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
# skill-embedding-retrieval Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change skill-learning-runtime. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Skill text embedded on synthesis and re-synthesis
|
||||
The system SHALL compute and persist an embedding vector for each locally-authored skill's name, description, and originating goal text whenever that skill is first synthesized or a new version is stored, associated with that skill's id and version.
|
||||
|
||||
#### Scenario: New skill gains an embedding
|
||||
- **WHEN** a flow-template skill is synthesized for the first time
|
||||
- **THEN** the system computes an embedding vector from its name, description, and originating goal, and stores it alongside the skill record
|
||||
|
||||
#### Scenario: New version gains its own embedding
|
||||
- **WHEN** a new version of an existing skill is created
|
||||
- **THEN** the system computes and stores an embedding for that version, independent of any embedding stored for prior versions
|
||||
|
||||
### Requirement: Embedding failure degrades to non-retrievable-by-similarity, never blocks synthesis
|
||||
The system SHALL NOT allow an embedding-provider failure (timeout, rate limit, disabled configuration, connection error) to prevent a skill from being synthesized or versioned; on such failure, the skill SHALL be stored without a similarity-searchable embedding.
|
||||
|
||||
#### Scenario: Embedding call fails but skill is still stored
|
||||
- **WHEN** the embedding provider call fails or times out during synthesis of an otherwise-successful skill
|
||||
- **THEN** the skill's flow-template record is still stored, and it is retrievable by exact name/id lookup but excluded from similarity-based retrieval results until a subsequent embedding attempt succeeds
|
||||
|
||||
### Requirement: Ranked retrieval of candidate skills by goal similarity
|
||||
The system SHALL provide a function that, given a new goal string and a requested result count, returns locally-authored skills that have a stored embedding, ranked by descending semantic similarity between the goal and each skill's stored embedding.
|
||||
|
||||
#### Scenario: Similar goal returns matching skill highest-ranked
|
||||
- **WHEN** a new goal is semantically similar to a previously-learned skill's originating goal
|
||||
- **THEN** that skill appears in the ranked candidate results, ordered ahead of less-similar skills
|
||||
|
||||
#### Scenario: Requested count limits results
|
||||
- **WHEN** a caller requests the top `k` candidate skills for a goal
|
||||
- **THEN** the system returns at most `k` ranked results, even if more embedded skills exist
|
||||
|
||||
#### Scenario: No embedded skills yields an empty result
|
||||
- **WHEN** no locally-authored skill currently has a stored embedding
|
||||
- **THEN** the retrieval function returns an empty ranked list rather than raising an error
|
||||
|
||||
### Requirement: Retrieval scoped to locally-authored skills unless explicitly extended
|
||||
The system SHALL restrict ranked candidate retrieval to skills stored by this capability's own local-synthesis store by default, and SHALL treat inclusion of externally-synced skills as a separate, explicit extension rather than an implicit default.
|
||||
|
||||
#### Scenario: Default retrieval excludes synced-only skills without embeddings
|
||||
- **WHEN** the skill catalog contains externally-synced skills that have never been embedded by this capability
|
||||
- **THEN** ranked candidate retrieval returns only locally-authored skills with stored embeddings, without erroring on the presence of unembedded synced skills
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
# skill-versioning Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change skill-learning-runtime. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Structural divergence triggers a new version
|
||||
The system SHALL compare a newly synthesized flow's tool-name step sequence against the currently-stored version of the matching skill, and SHALL create a new version (rather than overwriting the stored one) whenever the tool-name sequence differs by insertion, deletion, or reordering of a step.
|
||||
|
||||
#### Scenario: Extra step triggers a new version
|
||||
- **WHEN** a newly executed flow for a matching skill contains an additional tap step not present in the currently-stored version's sequence
|
||||
- **THEN** the system stores a new version of the skill rather than overwriting the existing stored version
|
||||
|
||||
#### Scenario: Reordered steps trigger a new version
|
||||
- **WHEN** a newly executed flow for a matching skill executes the same tool names as the stored version but in a different order
|
||||
- **THEN** the system stores a new version of the skill
|
||||
|
||||
#### Scenario: Argument-value-only differences do not trigger a version bump
|
||||
- **WHEN** a newly executed flow has the identical tool-name sequence as the stored version and differs only in argument values already covered by parameter abstraction
|
||||
- **THEN** the system does not create a new version, and instead updates the existing version's parameters per the skill-authoring capability
|
||||
|
||||
### Requirement: Version history is retained, never silently overwritten
|
||||
The system SHALL retain every version of a skill it creates, each carrying an incrementing `version` number and a reference to the version it diverged from, and SHALL NOT delete or overwrite a prior version's stored record when a new version is created.
|
||||
|
||||
#### Scenario: New version references its parent
|
||||
- **WHEN** a new version of a skill is created due to structural divergence
|
||||
- **THEN** the new version's record stores a reference to the prior version's id and an incremented version number
|
||||
|
||||
#### Scenario: Prior version remains fetchable
|
||||
- **WHEN** a new version of a skill has been created
|
||||
- **THEN** the prior version's record remains retrievable by its own id, unmodified
|
||||
|
||||
### Requirement: Default retrieval surfaces the newest version
|
||||
The system SHALL treat the highest-numbered version of a skill as the default result returned by a lookup-by-name/goal-family query, while still allowing an explicit lookup of any specific prior version by its id.
|
||||
|
||||
#### Scenario: Lookup by name returns newest version
|
||||
- **WHEN** a caller looks up a skill by its name or goal-family without specifying a version
|
||||
- **THEN** the system returns the highest-numbered stored version of that skill
|
||||
|
||||
#### Scenario: Explicit id lookup returns the requested version
|
||||
- **WHEN** a caller requests a skill by a specific prior version's id
|
||||
- **THEN** the system returns that exact version's record, not the newest version
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
# task-scheduler Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change cloud-runtime. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Task submission enqueues a scheduled task
|
||||
The system SHALL allow a caller to submit a task (a goal string, or a reference to a `WorkflowDefinition`, plus optional device constraints: `driver_type`, required capability tags) and SHALL enqueue it as a `ScheduledTask` with status `queued`, returning a stable task id the caller can poll.
|
||||
|
||||
#### Scenario: Successful submission
|
||||
- **WHEN** a caller submits a task with a goal and no constraints
|
||||
- **THEN** the scheduler creates a `ScheduledTask` with status `queued`, assigns it a unique id, and returns that id to the caller without blocking for a device to become available
|
||||
|
||||
#### Scenario: Queue depth limit reached
|
||||
- **WHEN** a caller submits a task while the queue already holds `config.max_queue_depth` queued tasks
|
||||
- **THEN** the scheduler rejects the submission with a clear error rather than accepting an unbounded backlog
|
||||
|
||||
### Requirement: Assignment matches a queued task to an idle, constraint-matching device
|
||||
The system SHALL assign a queued `ScheduledTask` to an idle `PooledDevice` (as reported by the `device-pool` capability) whose `driver_type` and capability tags satisfy the task's constraints, using a named, registrable `AssignmentStrategy`.
|
||||
|
||||
#### Scenario: Matching idle device available
|
||||
- **WHEN** `assign()` runs and at least one idle `PooledDevice` matches the head-of-queue task's constraints
|
||||
- **THEN** the scheduler selects one such device via the configured `AssignmentStrategy`, transitions the task to status `assigned`, and records the chosen `device_id`/`host_id`
|
||||
|
||||
#### Scenario: No matching device available
|
||||
- **WHEN** `assign()` runs and no idle `PooledDevice` matches the head-of-queue task's constraints
|
||||
- **THEN** the task remains `queued` (not failed), and `assign()` returns without error, ready to be retried on a later call
|
||||
|
||||
#### Scenario: Unknown assignment strategy configured
|
||||
- **WHEN** `TaskScheduler` is configured with an `AssignmentStrategy` name that is not registered
|
||||
- **THEN** the scheduler raises a clear configuration error at startup/first-assign rather than silently falling back to a default strategy
|
||||
|
||||
### Requirement: Assignment strategies are pluggable by name
|
||||
The system SHALL provide an `AssignmentStrategy` registry mapping a strategy name to an implementation, with a default `fifo_match` strategy (oldest-queued matching task first, first matching idle device), and SHALL allow a new strategy to be added by registering a name without modifying `TaskScheduler`'s control flow.
|
||||
|
||||
#### Scenario: Default FIFO strategy orders by submission time
|
||||
- **WHEN** two tasks with satisfiable, overlapping constraints are queued in order A then B, and one matching idle device exists
|
||||
- **THEN** the default `fifo_match` strategy assigns the device to task A, leaving task B queued
|
||||
|
||||
#### Scenario: Adding a new strategy requires no scheduler edit
|
||||
- **WHEN** a new `AssignmentStrategy` implementation is registered under a new name
|
||||
- **THEN** `TaskScheduler` can be configured to use it by name alone, with no change to `scheduler.py`'s assignment control flow
|
||||
|
||||
### Requirement: Local dispatch executes an assignment via existing runners
|
||||
The system SHALL provide a `TaskDispatcher` that, for an assignment whose device is owned by the local process's own host, executes the assigned task by composing the existing `agent-runtime` task-execution entry point (for a goal-based submission) or the `workflow-orchestration` workflow-execution entry point (for a workflow-based submission), without reimplementing planning/execution/retry logic.
|
||||
|
||||
#### Scenario: Dispatching a goal-based assignment
|
||||
- **WHEN** `TaskDispatcher.dispatch()` is called with an assignment for a goal-based `ScheduledTask` whose device is local
|
||||
- **THEN** the dispatcher constructs and runs a `Task` through the existing task-execution entry point, and updates the `ScheduledTask`'s status to `done` or `failed` based on the resulting task's outcome
|
||||
|
||||
#### Scenario: Dispatching a workflow-based assignment
|
||||
- **WHEN** `TaskDispatcher.dispatch()` is called with an assignment referencing a `WorkflowDefinition` whose device is local
|
||||
- **THEN** the dispatcher runs the definition through the existing workflow-execution entry point and updates the `ScheduledTask`'s status based on the resulting workflow run's outcome
|
||||
|
||||
### Requirement: Remote assignments are rejected explicitly, not silently ignored
|
||||
The system SHALL raise a distinct, typed error when `TaskDispatcher.dispatch()` is called for an assignment whose device is owned by a host other than the dispatching process's own host, rather than attempting execution or silently no-op'ing.
|
||||
|
||||
#### Scenario: Assignment targets a remote host's device
|
||||
- **WHEN** `TaskDispatcher.dispatch()` is called with an assignment whose `host_id` does not match the local process's own host id
|
||||
- **THEN** the dispatcher raises a `RemoteDispatchNotSupportedError` and leaves the `ScheduledTask`'s status unchanged from `assigned`
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
# workflow-orchestration Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change workflow-orchestration-runtime. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Workflow definition as an ordered, branchable list of typed steps
|
||||
The system SHALL provide a `WorkflowDefinition` model representing an ordered, possibly-branching list of `WorkflowStep`s, where each step is exactly one of four kinds: a planned-goal step (a natural-language sub-goal delegated to the existing single-goal Planner/Executor loop), a skill-invocation step (a reference to a locally-synthesized flow-template skill plus argument values), a wait-for-condition step (a named condition, timeout, and poll interval), or a branch step (a named condition plus two target step ids). Each step SHALL have a unique `step_id` within its `WorkflowDefinition`.
|
||||
|
||||
#### Scenario: Workflow with heterogeneous step kinds is constructed
|
||||
- **WHEN** a `WorkflowDefinition` is built with a planned-goal step, a skill-invocation step, a wait-for-condition step, and a branch step in sequence
|
||||
- **THEN** the system accepts the definition and each step retains its declared kind and fields without requiring fields belonging to another step kind
|
||||
|
||||
#### Scenario: Duplicate step id is rejected
|
||||
- **WHEN** a `WorkflowDefinition` is constructed with two steps sharing the same `step_id`
|
||||
- **THEN** the system rejects the definition before any run is created from it
|
||||
|
||||
### Requirement: Persisted, checkpointed workflow run
|
||||
The system SHALL persist a `WorkflowRun` record for each execution of a `WorkflowDefinition`, containing the run's status, the currently active step id, workflow-scoped variables, and a per-step result log, and SHALL update this record after every completed step before advancing to the next one.
|
||||
|
||||
#### Scenario: Run status transitions as steps execute
|
||||
- **WHEN** a `WorkflowRun` is started for a `WorkflowDefinition`
|
||||
- **THEN** the system creates a persisted run record with status `running` and, as each step completes, updates the persisted `current_step_id` and per-step result log before the next step begins
|
||||
|
||||
#### Scenario: Run reaches terminal status
|
||||
- **WHEN** all steps in a `WorkflowDefinition` complete successfully, or a step fails without a defined recovery path
|
||||
- **THEN** the system updates the persisted `WorkflowRun` status to `completed` or `failed` respectively, and records a failure reason when failed
|
||||
|
||||
### Requirement: Resume from last checkpoint without re-executing completed steps
|
||||
The system SHALL support resuming an interrupted `WorkflowRun` from its last persisted checkpoint, continuing execution from the currently active step without re-executing any step already recorded as completed in that run's step result log.
|
||||
|
||||
#### Scenario: Resume after simulated process restart
|
||||
- **WHEN** a `WorkflowRun` has completed its first two steps and the process driving it stops before the third step completes, and a new `WorkflowRunner` instance is later pointed at the same persisted run id
|
||||
- **THEN** the system resumes execution starting at the third step and does not re-invoke the tool calls or sub-goal already recorded as completed for the first two steps
|
||||
|
||||
#### Scenario: Resume on an already-completed run is a no-op
|
||||
- **WHEN** `resume` is called with the id of a `WorkflowRun` whose status is already `completed`
|
||||
- **THEN** the system returns the run's existing final state without executing any further steps
|
||||
|
||||
### Requirement: Planned-goal step delegates to the existing single-goal loop
|
||||
The system SHALL execute a planned-goal step by delegating its sub-goal to the existing Planner/Executor Observe-Think-Act-Observe loop for a single task, and SHALL derive that step's success or failure from the resulting task's final status.
|
||||
|
||||
#### Scenario: Planned-goal step succeeds
|
||||
- **WHEN** a planned-goal step's delegated task reaches a completed status
|
||||
- **THEN** the workflow step is recorded as succeeded and the run advances to the next step
|
||||
|
||||
#### Scenario: Planned-goal step fails
|
||||
- **WHEN** a planned-goal step's delegated task reaches a failed status
|
||||
- **THEN** the workflow step is recorded as failed with the task's failure reason and the run's status becomes `failed` unless a branch step defines an alternate path
|
||||
|
||||
### Requirement: Skill-invocation step resolves parameters and executes a flow-template skill
|
||||
The system SHALL execute a skill-invocation step by validating its supplied argument values against the referenced flow-template skill's declared parameters, substituting the validated values into the skill's stored tool-call template, and executing the resolved tool calls in order.
|
||||
|
||||
#### Scenario: Skill invocation with valid parameters executes resolved tool calls
|
||||
- **WHEN** a skill-invocation step supplies argument values that satisfy the referenced skill's declared required parameters
|
||||
- **THEN** the system substitutes those values into the skill's stored steps and executes the resulting tool calls in the skill's recorded order
|
||||
|
||||
#### Scenario: Skill invocation with a missing required parameter fails without executing any tool call
|
||||
- **WHEN** a skill-invocation step omits a value for a parameter the referenced skill declares as required
|
||||
- **THEN** the system fails the step before issuing any tool call and records the missing-parameter reason
|
||||
|
||||
#### Scenario: Skill invocation referencing a non-flow-template skill is rejected
|
||||
- **WHEN** a skill-invocation step references a skill whose kind is not a flow-template
|
||||
- **THEN** the system fails the step as a step-definition error rather than attempting to execute it
|
||||
|
||||
### Requirement: Wait-for-condition step polls until satisfied or timed out
|
||||
The system SHALL execute a wait-for-condition step by repeatedly evaluating its named condition at the step's configured poll interval until the condition is satisfied or the step's configured timeout elapses.
|
||||
|
||||
#### Scenario: Condition becomes true before timeout
|
||||
- **WHEN** a wait-for-condition step's condition evaluates true within its configured timeout
|
||||
- **THEN** the system stops polling, records the step as succeeded, and advances the run to the next step
|
||||
|
||||
#### Scenario: Condition never becomes true before timeout
|
||||
- **WHEN** a wait-for-condition step's condition has not evaluated true by its configured timeout
|
||||
- **THEN** the system records the step as failed with a timeout reason and the run's status becomes `failed` unless a branch step defines an alternate path
|
||||
|
||||
### Requirement: Branch step selects the next step from a condition
|
||||
The system SHALL execute a branch step by evaluating its named condition and setting the run's next active step to the branch's configured true-target step id or false-target step id accordingly, instead of advancing to the next step in definition order.
|
||||
|
||||
#### Scenario: Branch condition true selects the true-target step
|
||||
- **WHEN** a branch step's condition evaluates true
|
||||
- **THEN** the system sets the run's current step to the branch's `on_true` target step id
|
||||
|
||||
#### Scenario: Branch condition false selects the false-target step
|
||||
- **WHEN** a branch step's condition evaluates false
|
||||
- **THEN** the system sets the run's current step to the branch's `on_false` target step id
|
||||
|
||||
### Requirement: Condition kinds are pluggable via a registry
|
||||
The system SHALL evaluate wait-for-condition and branch step conditions through a registry mapping a condition kind name to an evaluator, SHALL provide at least `scene_contains_text`, `world_variable_equals`, `elapsed_seconds`, and `step_result_success` as built-in kinds, and SHALL allow a new condition kind to be added without modifying the workflow runner's step-dispatch logic.
|
||||
|
||||
#### Scenario: Built-in condition kind evaluates against current state
|
||||
- **WHEN** a wait-for-condition or branch step specifies the `scene_contains_text` kind with a target text value
|
||||
- **THEN** the system evaluates the condition against the most recently observed scene and returns true only when the target text is present
|
||||
|
||||
#### Scenario: Unregistered condition kind fails the step
|
||||
- **WHEN** a step specifies a condition kind that is not present in the registry
|
||||
- **THEN** the system fails that step with an unrecognized-condition-kind reason instead of executing an undefined check
|
||||
|
||||
#### Scenario: World-state-dependent condition degrades safely when world state is absent
|
||||
- **WHEN** a `world_variable_equals` condition is evaluated for a run whose task context has no `WorldState` available
|
||||
- **THEN** the system treats the condition as not satisfied rather than raising an error, allowing the step to continue polling until its timeout
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
# world-model Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change world-model-runtime. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Persistent per-task WorldState
|
||||
The system SHALL maintain one `WorldState` per task, consisting of a current app identifier, a current page identifier, a `variables` mapping, and a bounded history of recent semantic-scene/action pairs, that persists across the task's steps rather than being re-derived from scratch each step.
|
||||
|
||||
#### Scenario: WorldState survives across steps within a task
|
||||
- **WHEN** a task executes multiple steps in sequence
|
||||
- **THEN** the `WorldState` object associated with the task is the same object (or reflects continuously accumulated updates) across those steps, not reset between steps
|
||||
|
||||
#### Scenario: WorldState is scoped to a single task
|
||||
- **WHEN** two different tasks run (sequentially or concurrently) against the same or different devices
|
||||
- **THEN** each task has its own independent `WorldState`, and neither task's `WorldState` reflects the other task's app/page/variables/history
|
||||
|
||||
### Requirement: Incremental update after each executed step
|
||||
The system SHALL update a task's `WorldState` via a hook invoked once per executed step in the agent runtime's step loop, deriving the update from the step's observed `Scene`, optional `SemanticScene`, the `PlannedStep` that was executed, and its `StepResult`, without introducing a new LLM call or new network I/O.
|
||||
|
||||
#### Scenario: Update runs after a successful step
|
||||
- **WHEN** the agent runtime's step loop executes a step and records a successful `StepResult`
|
||||
- **THEN** the task's `WorldState` update hook is invoked with that step's `Scene`, `SemanticScene` (if any), the executed `PlannedStep`, and the `StepResult`, and updates the persisted `WorldState` accordingly
|
||||
|
||||
#### Scenario: Update runs after a failed step
|
||||
- **WHEN** the agent runtime's step loop executes a step and records a failed `StepResult`
|
||||
- **THEN** the task's `WorldState` update hook is still invoked with that step's data, and the update proceeds without raising an exception or blocking the loop's continuation/failure handling
|
||||
|
||||
#### Scenario: Update derivation makes no external calls
|
||||
- **WHEN** the `WorldState` update hook runs for any step
|
||||
- **THEN** the update completes using only the data already passed into the hook, without making any LLM call or other network request
|
||||
|
||||
### Requirement: Current app and page tracking
|
||||
The system SHALL derive and refresh `current_app` from successful app-lifecycle actions (e.g. `launch_app`, `terminate_app`) and SHALL derive and refresh `current_page` from the current step's `SemanticScene` page identity when a `SemanticScene` is available, leaving each field unchanged when no corresponding signal is present in a given step.
|
||||
|
||||
#### Scenario: Launching an app updates current_app
|
||||
- **WHEN** a step executes a successful `launch_app` action naming an app/bundle identifier
|
||||
- **THEN** the task's `WorldState.current_app` is updated to that identifier
|
||||
|
||||
#### Scenario: A page identity from SemanticScene updates current_page
|
||||
- **WHEN** a step's enrichment produces a `SemanticScene` with a non-empty `page` value
|
||||
- **THEN** the task's `WorldState.current_page` is updated to that `page` value
|
||||
|
||||
#### Scenario: No page signal leaves current_page unchanged
|
||||
- **WHEN** a step has no `SemanticScene` available (enrichment disabled, unavailable, or failed for that step)
|
||||
- **THEN** the task's `WorldState.current_page` retains its previous value rather than being cleared or set to an empty/placeholder value
|
||||
|
||||
### Requirement: Explicit variable memorization
|
||||
The system SHALL update `WorldState.variables` only when a step's `PlannedStep.args` contains an explicit memorization instruction, and SHALL NOT infer or write arbitrary variables from step content otherwise.
|
||||
|
||||
#### Scenario: A step explicitly remembers a value
|
||||
- **WHEN** a step's `PlannedStep.args` includes an explicit key/value pair designated for memorization
|
||||
- **THEN** the task's `WorldState.variables` is updated to include that key/value pair
|
||||
|
||||
#### Scenario: A step without a memorization instruction does not change variables
|
||||
- **WHEN** a step's `PlannedStep.args` contains no explicit memorization instruction
|
||||
- **THEN** the task's `WorldState.variables` is left unchanged by that step's update
|
||||
|
||||
### Requirement: Bounded history of recent scene/action pairs
|
||||
The system SHALL maintain `WorldState.history` as a fixed-size, bounded collection of the most recent semantic-scene-or-scene/action pairs, automatically evicting the oldest entry when a new entry is added past the configured bound, so that history size never grows unboundedly with task length.
|
||||
|
||||
#### Scenario: History accumulates recent entries up to the bound
|
||||
- **WHEN** a task executes a number of steps less than or equal to the configured history bound
|
||||
- **THEN** `WorldState.history` contains one entry per executed step, in order from oldest to newest
|
||||
|
||||
#### Scenario: History evicts the oldest entry once the bound is exceeded
|
||||
- **WHEN** a task executes more steps than the configured history bound
|
||||
- **THEN** `WorldState.history` retains only the most recent entries up to the bound, with earlier entries evicted, and never exceeds the configured bound in length
|
||||
|
||||
### Requirement: Read-only WorldState available to the Planner
|
||||
The system SHALL expose a task's current `WorldState` to the Planner as an additional, read-only input alongside the current `SemanticScene`/`Scene`, without requiring existing Planner implementations or call sites to change to keep working.
|
||||
|
||||
#### Scenario: Planner can read current WorldState
|
||||
- **WHEN** the agent runtime invokes the Planner to produce the next steps for a task
|
||||
- **THEN** the Planner is given access to the task's current `WorldState` (current app, current page, variables, bounded history) as of the most recently completed step
|
||||
|
||||
#### Scenario: Existing Planner call sites keep working unmodified
|
||||
- **WHEN** an existing caller invokes the Planner's planning entry point without passing any world-state argument
|
||||
- **THEN** the call succeeds exactly as it did before this capability existed, with the Planner treating the absence of world-state input as equivalent to "no world state available"
|
||||
|
||||
### Requirement: World Runtime tracking failure never blocks the task loop
|
||||
The system SHALL treat any failure or unavailable input during a `WorldState` update (e.g. missing `SemanticScene`, missing expected `PlannedStep` arguments, disabled configuration) as non-fatal, leaving the affected `WorldState` fields unchanged rather than raising an exception that would interrupt the agent runtime's step loop.
|
||||
|
||||
#### Scenario: Missing expected data during update does not raise
|
||||
- **WHEN** the `WorldState` update hook runs for a step whose data lacks a field an update rule expects (e.g. no app identifier on a `launch_app` step)
|
||||
- **THEN** the update hook completes without raising, leaving the corresponding `WorldState` field at its prior value
|
||||
|
||||
#### Scenario: World Runtime tracking disabled by configuration
|
||||
- **WHEN** World Runtime tracking is disabled in configuration for a task run
|
||||
- **THEN** the agent runtime's step loop proceeds normally without invoking the `WorldState` update hook, and the Planner receives an absence of world-state input rather than a partially-updated or stale `WorldState`
|
||||
|
||||
Reference in New Issue
Block a user