chore(openspec): archive task execution visibility
This commit is contained in:
+38
@@ -0,0 +1,38 @@
|
||||
## REMOVED Requirements
|
||||
|
||||
### Requirement: Planner-decision requests are not durably persisted
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Cloud Control Plane persists each resolved planner decision for later retrieval
|
||||
The Cloud Control Plane's planner-decision endpoint SHALL, for each request it successfully resolves to a tool-call decision, persist the request's system prompt, user prompt, the resolved tool name and arguments, and an assigned step index (scoped to the request's `task_id`/`attempt`) to a durable, bounded-retention store, in addition to returning the decision to the requesting Host Agent. The Cloud Control Plane SHALL NOT persist screenshot bytes from these requests.
|
||||
|
||||
#### Scenario: A planner-decision request resolves successfully
|
||||
- **WHEN** the Cloud Control Plane resolves a planner-decision request to a tool-call decision
|
||||
- **THEN** it persists the request's system prompt, user prompt, the resolved tool name and arguments, and a step index for that `task_id`/`attempt`, before returning the decision to the Host Agent
|
||||
|
||||
#### Scenario: A planner-decision request fails
|
||||
- **WHEN** the configured provider call fails and the endpoint returns a structured failure response
|
||||
- **THEN** no row is persisted for that request
|
||||
|
||||
#### Scenario: A screenshot was included in the request
|
||||
- **WHEN** a planner-decision request includes a screenshot
|
||||
- **THEN** the screenshot bytes are used only to call the LLM provider and are not written to the persisted decision log
|
||||
|
||||
### Requirement: Persisted planner decisions are retained within a bounded window
|
||||
The Cloud Control Plane SHALL prune persisted planner decisions once their owning task has been in a terminal state for longer than a configurable retention window, so that indefinite operation does not cause unbounded growth of the decision log.
|
||||
|
||||
#### Scenario: A task's retention window has elapsed since reaching a terminal state
|
||||
- **WHEN** a task reached a terminal state more than the configured retention window ago
|
||||
- **THEN** the Cloud Control Plane removes that task's persisted planner decisions
|
||||
|
||||
#### Scenario: A task is still active or within its retention window
|
||||
- **WHEN** a task is still active, or reached a terminal state less than the configured retention window ago
|
||||
- **THEN** its persisted planner decisions remain available for query
|
||||
|
||||
### Requirement: Cloud-side planner decision history is scoped to hosts using the cloud-proxy transport
|
||||
The Cloud Control Plane's persisted planner decision log SHALL only ever contain entries for hosts whose planner calls were routed through the cloud-proxy transport; it SHALL NOT contain entries, synthesized or otherwise, for hosts using the direct-to-provider transport.
|
||||
|
||||
#### Scenario: A host uses the direct-to-provider transport
|
||||
- **WHEN** a Host Agent configured for the direct-to-provider transport executes an assignment
|
||||
- **THEN** no planner decision entries for that assignment appear in the Cloud Control Plane's persisted decision log
|
||||
+38
@@ -0,0 +1,38 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Cloud Control Plane exposes the latest in-progress step status for an active assignment
|
||||
The Cloud Control Plane's existing task query surface SHALL include the latest reported step index, step status, and summary for any assignment that has an in-progress Host Agent execution, alongside the task's existing status fields.
|
||||
|
||||
#### Scenario: An assignment has reported progress
|
||||
- **WHEN** an operator queries a task that has an active, in-progress assignment with previously reported step progress
|
||||
- **THEN** the response includes that step's index, status, and summary alongside the task's existing fields
|
||||
|
||||
#### Scenario: An assignment has no reported progress yet
|
||||
- **WHEN** an operator queries a task whose active assignment has not yet reported any progress
|
||||
- **THEN** the response omits progress fields rather than showing stale or default values
|
||||
|
||||
#### Scenario: An assignment has reached a terminal state
|
||||
- **WHEN** an operator queries a task whose assignment has already completed (succeeded or failed)
|
||||
- **THEN** the response does not present the last in-progress step as current status; the task's terminal status and result take precedence
|
||||
|
||||
### Requirement: Cloud Console renders live step progress for in-progress tasks
|
||||
Cloud Console SHALL display the current step index, status, and summary for a task with an active, in-progress Host Agent execution, refreshed on its existing polling interval, without requiring a new push channel.
|
||||
|
||||
#### Scenario: Operator views an in-progress task
|
||||
- **WHEN** an operator opens a task detail view for a task with an active in-progress assignment
|
||||
- **THEN** Cloud Console shows the latest known step index, status, and summary, updating on subsequent polls as new progress is reported
|
||||
|
||||
#### Scenario: Operator views a task with no in-progress execution
|
||||
- **WHEN** an operator opens a task detail view for a queued, terminal, or otherwise not-currently-executing task
|
||||
- **THEN** Cloud Console does not display stale in-progress step information
|
||||
|
||||
### Requirement: Cloud Console displays a task's full LLM interaction history
|
||||
Cloud Console SHALL provide a view, for a given task, listing each persisted planner decision in step order, including its full prompt and resulting decision, sourced from the Cloud Control Plane's persisted planner decision log.
|
||||
|
||||
#### Scenario: Task has persisted planner decisions
|
||||
- **WHEN** an operator opens the LLM interaction history view for a task that has one or more persisted planner decisions
|
||||
- **THEN** Cloud Console shows each decision in step order with its prompt and resulting tool call
|
||||
|
||||
#### Scenario: Task's host used the direct-to-provider transport
|
||||
- **WHEN** an operator opens the LLM interaction history view for a task whose host used the direct-to-provider transport
|
||||
- **THEN** Cloud Console indicates that no LLM interaction history is available because the host does not report it, rather than showing an empty history with no explanation
|
||||
+67
@@ -0,0 +1,67 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Host Agent local console exposes read-only task history with per-step detail and screenshots
|
||||
The Host Agent's local console SHALL provide authenticated, read-only pages
|
||||
listing recently executed local tasks and, for a selected task, its full
|
||||
per-step history from the Host-local metadata store and Timeline. The task
|
||||
detail SHALL show available before and after screenshots, operation details and
|
||||
arguments, execution result, OCR observations, and normalized UI-tree results.
|
||||
It SHALL render legacy Timeline records that only have a single screenshot as a
|
||||
post-action image.
|
||||
|
||||
#### Scenario: Operator lists recent Host executions
|
||||
- **WHEN** an authenticated operator opens the Host Agent local console's task
|
||||
list page
|
||||
- **THEN** it shows local executions most recent first, including terminal
|
||||
tasks and any available Cloud task ID and attempt correlation
|
||||
|
||||
#### Scenario: Operator inspects a completed task's step history
|
||||
- **WHEN** an authenticated operator opens the detail page for a completed
|
||||
Host execution
|
||||
- **THEN** the page shows each recorded step in order with its tool call,
|
||||
result, and available before/after screenshots
|
||||
|
||||
#### Scenario: OCR was captured for a step
|
||||
- **WHEN** the selected Timeline record contains OCR observations
|
||||
- **THEN** the detail page shows each observation's text, confidence, and
|
||||
bounds
|
||||
|
||||
#### Scenario: A UI-tree tool returned normalized nodes
|
||||
- **WHEN** the selected Timeline record invoked `get_ui_tree` or `ui_tree`
|
||||
and its result contains normalized nodes
|
||||
- **THEN** the detail page exposes a structured, collapsible node view while
|
||||
retaining the persisted result JSON
|
||||
|
||||
#### Scenario: A legacy timeline record is displayed
|
||||
- **WHEN** a Timeline record has only `screenshot_path`
|
||||
- **THEN** the detail page renders it as the post-action image without failing
|
||||
|
||||
#### Scenario: Unauthenticated request
|
||||
- **WHEN** a request to the task list or task detail pages is made without a
|
||||
valid Host Agent console session
|
||||
- **THEN** the Host Agent rejects the request the same way it rejects
|
||||
unauthenticated requests to its other console pages
|
||||
|
||||
### Requirement: Host Agent local console task pages require no new cross-origin surface
|
||||
The Host Agent local console's task pages SHALL be served same-origin from the
|
||||
Host Agent's existing web application, without introducing new CORS allowances
|
||||
or a dependency on a separate Runtime frontend.
|
||||
|
||||
#### Scenario: Task pages are requested
|
||||
- **WHEN** an operator's browser requests the Host Agent local console's task
|
||||
pages
|
||||
- **THEN** the pages are served by the Host Agent's own application using its
|
||||
existing session/CSRF protections, with no additional cross-origin
|
||||
configuration required
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Host Agent console is the authority for actual execution evidence
|
||||
The Host Agent local console SHALL be the web authority for task evidence
|
||||
produced by that Host's in-process execution path. A standalone Runtime
|
||||
service/UI SHALL NOT be required or consulted to inspect a Host execution.
|
||||
|
||||
#### Scenario: A Cloud task is executed by a Host Agent
|
||||
- **WHEN** an operator opens that Host Agent's task page after execution starts
|
||||
- **THEN** the page reads the same Host-local metadata and Timeline that the
|
||||
executing `TaskRunner` writes
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Supervisor is opt-in and disabled by default
|
||||
The Host Agent SHALL NOT start, adopt-check, or supervise Appium unless
|
||||
`HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED` is explicitly set to true. Appium
|
||||
supervision SHALL additionally require `HOST_AGENT_APPIUM_SUPERVISED=true`
|
||||
and SHALL default to false.
|
||||
|
||||
#### Scenario: Default configuration behaves exactly as before
|
||||
- **WHEN** a Host Agent starts with no
|
||||
`HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED` or related Appium environment
|
||||
variable set
|
||||
- **THEN** the Host Agent does not attempt to connect to, probe, or spawn
|
||||
Appium, and its heartbeat/claim behavior is unchanged
|
||||
|
||||
#### Scenario: Top-level flag on and Appium flag off
|
||||
- **WHEN** `HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED=true` and
|
||||
`HOST_AGENT_APPIUM_SUPERVISED=false`
|
||||
- **THEN** the Host Agent does not probe, adopt, or spawn Appium
|
||||
|
||||
### Requirement: Spawn supervised dependencies that are not already running
|
||||
The Host Agent SHALL spawn Appium as a child process when Appium supervision is
|
||||
enabled and no healthy Appium instance is adopted, via
|
||||
`appium --address <host> --port <port>` and SHALL forward the child
|
||||
process's stdout/stderr into the Host Agent's own logging, tagged by dependency
|
||||
name.
|
||||
|
||||
#### Scenario: Appium is not running at Host Agent startup
|
||||
- **WHEN** Appium supervision is enabled and no healthy Appium instance is
|
||||
already listening
|
||||
- **THEN** the Host Agent spawns Appium before proceeding to its first
|
||||
device-connect attempt, and its output is visible in Host Agent logs
|
||||
|
||||
#### Scenario: Spawn fails because the executable is missing
|
||||
- **WHEN** the Host Agent attempts to spawn Appium but `appium` is not found
|
||||
on `PATH`
|
||||
- **THEN** the Host Agent logs a dependency-supervisor-specific startup error
|
||||
naming the missing dependency, distinct from a runtime crash of an
|
||||
already-started process
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Runtime supervision settings are retired
|
||||
The Host Agent SHALL reject `HOST_AGENT_RUNTIME_SUPERVISED`,
|
||||
`HOST_AGENT_RUNTIME_HOST`, and `HOST_AGENT_RUNTIME_PORT` because the
|
||||
standalone Runtime service no longer exists.
|
||||
|
||||
#### Scenario: A legacy Runtime supervision variable is set
|
||||
- **WHEN** startup configuration includes any removed Runtime supervision
|
||||
variable
|
||||
- **THEN** configuration fails with an actionable migration error
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Host Agent reports execution progress alongside lease renewal
|
||||
The Host Agent SHALL optionally include a bounded, screenshot-free progress summary (current step index, step status, and a short plain-text summary) in its periodic lease-renewal request for an active assignment, and the control plane SHALL accept and store only the most recent such summary per active assignment.
|
||||
|
||||
#### Scenario: Progress is available at renewal time
|
||||
- **WHEN** the Host Agent renews the lease for an in-progress assignment and has a current step index, status, and summary available
|
||||
- **THEN** the renewal request includes that progress summary and the control plane overwrites any previously stored progress for that assignment with it
|
||||
|
||||
#### Scenario: Progress is not available at renewal time
|
||||
- **WHEN** the Host Agent renews a lease without a progress summary available (e.g. before the first step completes)
|
||||
- **THEN** the renewal request omits the progress field and any previously stored progress for that assignment is left unchanged
|
||||
|
||||
#### Scenario: Assignment reaches a terminal state
|
||||
- **WHEN** an assignment's terminal result is recorded
|
||||
- **THEN** the control plane's stored progress for that assignment is no longer treated as current and is not exposed as an in-progress status
|
||||
|
||||
### Requirement: Progress reports exclude screenshot and scene payloads
|
||||
The control plane SHALL reject or ignore any progress field on a renewal request that includes screenshot, scene, or other bulk payload data beyond the bounded step index, status, and short text summary.
|
||||
|
||||
#### Scenario: Renewal request includes an oversized or non-text summary
|
||||
- **WHEN** a Host Agent submits a progress summary exceeding the configured length bound
|
||||
- **THEN** the control plane truncates or rejects the oversized field without failing the underlying lease renewal
|
||||
+74
@@ -0,0 +1,74 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Host Agent records step-level execution detail for its in-process TaskRunner
|
||||
The Host Agent SHALL construct its in-process `TaskRunner` with a durable
|
||||
metadata store and Timeline. `TaskRunner.run()` SHALL create the task's
|
||||
metadata row idempotently before its first status update, so every execution
|
||||
path persists its task status and evidence rather than discarding updates for a
|
||||
missing row. Every completed step SHALL retain its index, actual per-step LLM
|
||||
prompt and decision when available, tool call, result, distinct before/after
|
||||
screenshots when captured, raw OCR observations when available, and normalized
|
||||
UI-tree result when the invoked tool returned one.
|
||||
|
||||
#### Scenario: A goal assignment starts execution
|
||||
- **WHEN** the Host Agent's `AssignmentExecutor` invokes its `TaskRunner`
|
||||
- **THEN** the task metadata row exists before the runner records its running
|
||||
status
|
||||
|
||||
#### Scenario: A step completes during goal execution
|
||||
- **WHEN** the Host Agent's `TaskRunner` completes a step while executing an
|
||||
assigned goal
|
||||
- **THEN** the step's status, index, actual per-step LLM prompt and response,
|
||||
tool call, result, and available evidence are persisted before the next step
|
||||
begins
|
||||
|
||||
#### Scenario: A workflow creates a planned-goal task
|
||||
- **WHEN** a `WorkflowRunner` invokes a Host Agent-configured
|
||||
`TaskRunner` for a planned-goal step
|
||||
- **THEN** that task is persisted without requiring the workflow caller to
|
||||
create a metadata row separately
|
||||
|
||||
#### Scenario: An assignment finishes
|
||||
- **WHEN** an assignment reaches a terminal state (succeeded or failed)
|
||||
- **THEN** its full step history remains queryable from the Host Agent's local
|
||||
store after the in-memory `Task` object is discarded
|
||||
|
||||
### Requirement: Host-Agent-local task history is retained within a bounded window
|
||||
The Host Agent SHALL prune persisted task metadata, Timeline records, and
|
||||
associated screenshot artifacts once they exceed a configurable retention
|
||||
window or count, so that indefinite process uptime does not cause unbounded
|
||||
local disk growth.
|
||||
|
||||
#### Scenario: Retention window is exceeded
|
||||
- **WHEN** a persisted task's age or position exceeds the configured retention
|
||||
threshold
|
||||
- **THEN** the Host Agent removes that task's metadata row, Timeline records,
|
||||
and screenshot artifacts from local storage
|
||||
|
||||
#### Scenario: Retention has not been exceeded
|
||||
- **WHEN** a persisted task is within the configured retention threshold
|
||||
- **THEN** its metadata, Timeline records, and screenshot artifacts remain
|
||||
available for query
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Host Agent correlates local execution records with Cloud assignments
|
||||
For a Cloud-dispatched goal assignment, the Host Agent SHALL persist the Cloud
|
||||
task ID and attempt alongside its generated local Runtime task ID before
|
||||
execution starts. The correlation fields SHALL remain optional and generic in
|
||||
the shared storage layer.
|
||||
|
||||
#### Scenario: A Cloud goal assignment begins
|
||||
- **WHEN** the Host Agent begins executing a Cloud goal assignment
|
||||
- **THEN** the local task row records that assignment's Cloud task ID and
|
||||
attempt
|
||||
|
||||
#### Scenario: A task is not Cloud-dispatched
|
||||
- **WHEN** a shared Runtime caller executes a task without Host/Cloud
|
||||
assignment context
|
||||
- **THEN** the task metadata row is created and the optional source
|
||||
correlation fields remain empty
|
||||
|
||||
## REMOVED Requirements
|
||||
|
||||
### Requirement: Host Agent local task storage is isolated from an unrelated local Runtime
|
||||
+33
@@ -0,0 +1,33 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Runtime is not exposed as a standalone REST service or web console
|
||||
The repository SHALL not ship a standalone Runtime REST application, its
|
||||
unauthenticated web console, or JSON console routes. The shared Runtime and
|
||||
storage packages SHALL remain reusable execution libraries for the Host Agent
|
||||
and other in-process callers.
|
||||
|
||||
#### Scenario: An operator needs to inspect a Host-executed task
|
||||
- **WHEN** an operator needs task evidence for a Host Agent execution
|
||||
- **THEN** the operator uses the authenticated Host Agent console rather than
|
||||
starting or querying a separate Runtime service
|
||||
|
||||
#### Scenario: A package uses shared Runtime execution
|
||||
- **WHEN** the Host Agent or another in-process caller creates a
|
||||
`TaskRunner`
|
||||
- **THEN** it continues to use the shared Runtime and storage packages without
|
||||
importing a REST or UI adapter
|
||||
|
||||
### Requirement: Host Agent does not supervise a retired Runtime service
|
||||
The Host Agent SHALL not expose Runtime-supervision configuration or spawn a
|
||||
Runtime REST subprocess. It MAY continue to optionally supervise Appium.
|
||||
|
||||
#### Scenario: Host Agent dependency supervision is enabled
|
||||
- **WHEN** `HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED=true` and Appium
|
||||
supervision is enabled
|
||||
- **THEN** the Host Agent probes and supervises Appium only
|
||||
|
||||
#### Scenario: A removed Runtime-supervision variable is configured
|
||||
- **WHEN** a Host Agent configuration includes a removed
|
||||
`HOST_AGENT_RUNTIME_*` variable
|
||||
- **THEN** startup fails with a message directing the operator to the Host
|
||||
Agent console and Appium-only supervision
|
||||
+61
@@ -0,0 +1,61 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Runtime persists complete evidence for each executed action
|
||||
The shared Runtime SHALL persist, for each action it attempts, a screenshot
|
||||
captured immediately before the executor call, the action description and
|
||||
arguments, the execution result, and a screenshot captured immediately after
|
||||
the executor call. Existing Timeline records that contain only the legacy
|
||||
single screenshot SHALL remain readable, with that screenshot treated as the
|
||||
post-action image.
|
||||
|
||||
#### Scenario: An action succeeds
|
||||
- **WHEN** the Runtime executes an action for a task
|
||||
- **THEN** its Timeline record includes distinct before and after screenshots,
|
||||
action detail, and execution result
|
||||
|
||||
#### Scenario: An action fails
|
||||
- **WHEN** the Runtime executor exhausts its retries for an action
|
||||
- **THEN** the Timeline record still includes any captured screenshots and the
|
||||
failure result before the task is marked failed
|
||||
|
||||
#### Scenario: A legacy Timeline record is read
|
||||
- **WHEN** a Timeline record has only the prior `screenshot_path` field
|
||||
- **THEN** the Runtime exposes it as the post-action screenshot without
|
||||
failing to render the record
|
||||
|
||||
### Requirement: Runtime task evidence retains available OCR observations
|
||||
The shared Runtime SHALL persist raw OCR observations associated with the scene
|
||||
used to plan an action when available, without adding duplicate OCR data to the
|
||||
LLM-facing normalized Scene payload. The Host Agent task-detail UI SHALL render
|
||||
available OCR text, confidence, and bounds, and SHALL render normally when no
|
||||
OCR result exists.
|
||||
|
||||
#### Scenario: OCR found text while planning an action
|
||||
- **WHEN** perception produced one or more OCR observations for the action's
|
||||
planning scene
|
||||
- **THEN** the corresponding Timeline record includes those observations and
|
||||
the Host Agent task-detail page displays them
|
||||
|
||||
#### Scenario: OCR was unavailable or found no text
|
||||
- **WHEN** perception yields no OCR observations
|
||||
- **THEN** the Runtime records the action evidence and the Host Agent task
|
||||
detail renders without an OCR result list
|
||||
|
||||
### Requirement: Runtime task evidence retains UI-tree inspection results
|
||||
The Runtime SHALL retain a UI-tree inspection result when a step invokes the
|
||||
existing `get_ui_tree` or `ui_tree` tool and the result contains normalized
|
||||
nodes. The Host Agent task-detail UI SHALL render those nodes in a structured,
|
||||
collapsible view while retaining the recorded JSON result. The Runtime SHALL
|
||||
NOT change the tool response contract or duplicate the result in a separate
|
||||
persistence field.
|
||||
|
||||
#### Scenario: UI-tree inspection succeeds
|
||||
- **WHEN** a task step uses `get_ui_tree` or `ui_tree` and returns one or
|
||||
more normalized nodes
|
||||
- **THEN** the Host Agent task-detail page displays each node's type, visible
|
||||
text or identifier, bounds, and available confidence
|
||||
|
||||
#### Scenario: A non-UI-tree step is displayed
|
||||
- **WHEN** a task step did not invoke a UI-tree tool
|
||||
- **THEN** the Host Agent task-detail page does not render an empty UI-tree
|
||||
section
|
||||
Reference in New Issue
Block a user