This commit is contained in:
+56
-23
@@ -1,34 +1,67 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Host Agent local console exposes step-level status for the current assignment
|
||||
The Host Agent's local console SHALL display, for its currently executing assignment, the current step index, step status, and a short summary, sourced from the Host Agent's local task metadata store, refreshed on the console's existing polling interval.
|
||||
|
||||
#### Scenario: An assignment is currently executing
|
||||
- **WHEN** an operator views the Host Agent local console dashboard while an assignment is executing
|
||||
- **THEN** the dashboard shows the current step index, step status, and a short summary for that assignment, updating on subsequent polls
|
||||
|
||||
#### Scenario: No assignment is currently executing
|
||||
- **WHEN** an operator views the dashboard while the Host Agent is idle
|
||||
- **THEN** the dashboard shows no in-progress step information
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Host Agent local console exposes read-only task history with per-step detail and screenshots
|
||||
The Host Agent's local console SHALL provide authenticated, read-only pages listing recently executed tasks and, for a selected task, its full per-step history including any captured screenshots, sourced from the Host Agent's local task metadata store and timeline.
|
||||
The Host Agent's local console SHALL provide authenticated, read-only pages
|
||||
listing recently executed local tasks and, for a selected task, its full
|
||||
per-step history from the Host-local metadata store and Timeline. The task
|
||||
detail SHALL show available before and after screenshots, operation details and
|
||||
arguments, execution result, OCR observations, and normalized UI-tree results.
|
||||
It SHALL render legacy Timeline records that only have a single screenshot as a
|
||||
post-action image.
|
||||
|
||||
#### Scenario: Operator lists recent tasks
|
||||
- **WHEN** an authenticated operator opens the Host Agent local console's task list page
|
||||
- **THEN** it shows tasks from the local task metadata store, most recent first, including tasks that have already reached a terminal state
|
||||
#### Scenario: Operator lists recent Host executions
|
||||
- **WHEN** an authenticated operator opens the Host Agent local console's task
|
||||
list page
|
||||
- **THEN** it shows local executions most recent first, including terminal
|
||||
tasks and any available Cloud task ID and attempt correlation
|
||||
|
||||
#### Scenario: Operator inspects a completed task's step history
|
||||
- **WHEN** an authenticated operator opens the detail page for a specific completed task
|
||||
- **THEN** the page shows each recorded step in order, including its tool call, result, and any captured screenshot
|
||||
- **WHEN** an authenticated operator opens the detail page for a completed
|
||||
Host execution
|
||||
- **THEN** the page shows each recorded step in order with its tool call,
|
||||
result, and available before/after screenshots
|
||||
|
||||
#### Scenario: OCR was captured for a step
|
||||
- **WHEN** the selected Timeline record contains OCR observations
|
||||
- **THEN** the detail page shows each observation's text, confidence, and
|
||||
bounds
|
||||
|
||||
#### Scenario: A UI-tree tool returned normalized nodes
|
||||
- **WHEN** the selected Timeline record invoked `get_ui_tree` or `ui_tree`
|
||||
and its result contains normalized nodes
|
||||
- **THEN** the detail page exposes a structured, collapsible node view while
|
||||
retaining the persisted result JSON
|
||||
|
||||
#### Scenario: A legacy timeline record is displayed
|
||||
- **WHEN** a Timeline record has only `screenshot_path`
|
||||
- **THEN** the detail page renders it as the post-action image without failing
|
||||
|
||||
#### Scenario: Unauthenticated request
|
||||
- **WHEN** a request to the task list or task detail pages is made without a valid Host Agent console session
|
||||
- **THEN** the Host Agent rejects the request the same way it rejects unauthenticated requests to its other console pages
|
||||
- **WHEN** a request to the task list or task detail pages is made without a
|
||||
valid Host Agent console session
|
||||
- **THEN** the Host Agent rejects the request the same way it rejects
|
||||
unauthenticated requests to its other console pages
|
||||
|
||||
### Requirement: Host Agent local console task pages require no new cross-origin surface
|
||||
The Host Agent local console's task pages SHALL be served same-origin from the Host Agent's existing web application, without introducing new CORS allowances or a dependency on the separate Runtime `console/` frontend.
|
||||
The Host Agent local console's task pages SHALL be served same-origin from the
|
||||
Host Agent's existing web application, without introducing new CORS allowances
|
||||
or a dependency on a separate Runtime frontend.
|
||||
|
||||
#### Scenario: Task pages are requested
|
||||
- **WHEN** an operator's browser requests the Host Agent local console's task pages
|
||||
- **THEN** the pages are served by the Host Agent's own application using its existing session/CSRF protections, with no additional cross-origin configuration required
|
||||
- **WHEN** an operator's browser requests the Host Agent local console's task
|
||||
pages
|
||||
- **THEN** the pages are served by the Host Agent's own application using its
|
||||
existing session/CSRF protections, with no additional cross-origin
|
||||
configuration required
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Host Agent console is the authority for actual execution evidence
|
||||
The Host Agent local console SHALL be the web authority for task evidence
|
||||
produced by that Host's in-process execution path. A standalone Runtime
|
||||
service/UI SHALL NOT be required or consulted to inspect a Host execution.
|
||||
|
||||
#### Scenario: A Cloud task is executed by a Host Agent
|
||||
- **WHEN** an operator opens that Host Agent's task page after execution starts
|
||||
- **THEN** the page reads the same Host-local metadata and Timeline that the
|
||||
executing `TaskRunner` writes
|
||||
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Supervisor is opt-in and disabled by default
|
||||
The Host Agent SHALL NOT start, adopt-check, or supervise Appium unless
|
||||
`HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED` is explicitly set to true. Appium
|
||||
supervision SHALL additionally require `HOST_AGENT_APPIUM_SUPERVISED=true`
|
||||
and SHALL default to false.
|
||||
|
||||
#### Scenario: Default configuration behaves exactly as before
|
||||
- **WHEN** a Host Agent starts with no
|
||||
`HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED` or related Appium environment
|
||||
variable set
|
||||
- **THEN** the Host Agent does not attempt to connect to, probe, or spawn
|
||||
Appium, and its heartbeat/claim behavior is unchanged
|
||||
|
||||
#### Scenario: Top-level flag on and Appium flag off
|
||||
- **WHEN** `HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED=true` and
|
||||
`HOST_AGENT_APPIUM_SUPERVISED=false`
|
||||
- **THEN** the Host Agent does not probe, adopt, or spawn Appium
|
||||
|
||||
### Requirement: Spawn supervised dependencies that are not already running
|
||||
The Host Agent SHALL spawn Appium as a child process when Appium supervision is
|
||||
enabled and no healthy Appium instance is adopted, via
|
||||
`appium --address <host> --port <port>` and SHALL forward the child
|
||||
process's stdout/stderr into the Host Agent's own logging, tagged by dependency
|
||||
name.
|
||||
|
||||
#### Scenario: Appium is not running at Host Agent startup
|
||||
- **WHEN** Appium supervision is enabled and no healthy Appium instance is
|
||||
already listening
|
||||
- **THEN** the Host Agent spawns Appium before proceeding to its first
|
||||
device-connect attempt, and its output is visible in Host Agent logs
|
||||
|
||||
#### Scenario: Spawn fails because the executable is missing
|
||||
- **WHEN** the Host Agent attempts to spawn Appium but `appium` is not found
|
||||
on `PATH`
|
||||
- **THEN** the Host Agent logs a dependency-supervisor-specific startup error
|
||||
naming the missing dependency, distinct from a runtime crash of an
|
||||
already-started process
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Runtime supervision settings are retired
|
||||
The Host Agent SHALL reject `HOST_AGENT_RUNTIME_SUPERVISED`,
|
||||
`HOST_AGENT_RUNTIME_HOST`, and `HOST_AGENT_RUNTIME_PORT` because the
|
||||
standalone Runtime service no longer exists.
|
||||
|
||||
#### Scenario: A legacy Runtime supervision variable is set
|
||||
- **WHEN** startup configuration includes any removed Runtime supervision
|
||||
variable
|
||||
- **THEN** configuration fails with an actionable migration error
|
||||
+58
-14
@@ -1,30 +1,74 @@
|
||||
## ADDED Requirements
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Host Agent records step-level execution detail for its in-process TaskRunner
|
||||
The Host Agent SHALL construct its in-process `TaskRunner` with a durable metadata store and timeline so that every step transition (status, index, the actual prompt submitted to the LLM for that step, the model's resulting decision, result, and screenshot when captured) is persisted as it happens, rather than discarded when the assignment completes. The persisted prompt SHALL be the prompt actually sent to the LLM for that specific step, not the task's overall goal.
|
||||
The Host Agent SHALL construct its in-process `TaskRunner` with a durable
|
||||
metadata store and Timeline. `TaskRunner.run()` SHALL create the task's
|
||||
metadata row idempotently before its first status update, so every execution
|
||||
path persists its task status and evidence rather than discarding updates for a
|
||||
missing row. Every completed step SHALL retain its index, actual per-step LLM
|
||||
prompt and decision when available, tool call, result, distinct before/after
|
||||
screenshots when captured, raw OCR observations when available, and normalized
|
||||
UI-tree result when the invoked tool returned one.
|
||||
|
||||
#### Scenario: A goal assignment starts execution
|
||||
- **WHEN** the Host Agent's `AssignmentExecutor` invokes its `TaskRunner`
|
||||
- **THEN** the task metadata row exists before the runner records its running
|
||||
status
|
||||
|
||||
#### Scenario: A step completes during goal execution
|
||||
- **WHEN** the Host Agent's `TaskRunner` completes a step while executing an assigned goal
|
||||
- **THEN** the step's status, index, the actual per-step LLM prompt and response, tool call, result, and any captured screenshot are persisted to the Host Agent's local task metadata store and timeline before the next step begins
|
||||
- **WHEN** the Host Agent's `TaskRunner` completes a step while executing an
|
||||
assigned goal
|
||||
- **THEN** the step's status, index, actual per-step LLM prompt and response,
|
||||
tool call, result, and available evidence are persisted before the next step
|
||||
begins
|
||||
|
||||
#### Scenario: A workflow creates a planned-goal task
|
||||
- **WHEN** a `WorkflowRunner` invokes a Host Agent-configured
|
||||
`TaskRunner` for a planned-goal step
|
||||
- **THEN** that task is persisted without requiring the workflow caller to
|
||||
create a metadata row separately
|
||||
|
||||
#### Scenario: An assignment finishes
|
||||
- **WHEN** an assignment reaches a terminal state (succeeded or failed)
|
||||
- **THEN** its full step history remains queryable from the Host Agent's local store after the in-memory `Task` object is discarded
|
||||
- **THEN** its full step history remains queryable from the Host Agent's local
|
||||
store after the in-memory `Task` object is discarded
|
||||
|
||||
### Requirement: Host-Agent-local task history is retained within a bounded window
|
||||
The Host Agent SHALL prune persisted task metadata, timeline records, and associated screenshot artifacts once they exceed a configurable retention window or count, so that indefinite process uptime does not cause unbounded local disk growth.
|
||||
The Host Agent SHALL prune persisted task metadata, Timeline records, and
|
||||
associated screenshot artifacts once they exceed a configurable retention
|
||||
window or count, so that indefinite process uptime does not cause unbounded
|
||||
local disk growth.
|
||||
|
||||
#### Scenario: Retention window is exceeded
|
||||
- **WHEN** a persisted task's age or position exceeds the configured retention threshold
|
||||
- **THEN** the Host Agent removes that task's metadata row, timeline records, and screenshot artifacts from local storage
|
||||
- **WHEN** a persisted task's age or position exceeds the configured retention
|
||||
threshold
|
||||
- **THEN** the Host Agent removes that task's metadata row, Timeline records,
|
||||
and screenshot artifacts from local storage
|
||||
|
||||
#### Scenario: Retention has not been exceeded
|
||||
- **WHEN** a persisted task is within the configured retention threshold
|
||||
- **THEN** its metadata, timeline records, and screenshot artifacts remain available for query
|
||||
- **THEN** its metadata, Timeline records, and screenshot artifacts remain
|
||||
available for query
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Host Agent correlates local execution records with Cloud assignments
|
||||
For a Cloud-dispatched goal assignment, the Host Agent SHALL persist the Cloud
|
||||
task ID and attempt alongside its generated local Runtime task ID before
|
||||
execution starts. The correlation fields SHALL remain optional and generic in
|
||||
the shared storage layer.
|
||||
|
||||
#### Scenario: A Cloud goal assignment begins
|
||||
- **WHEN** the Host Agent begins executing a Cloud goal assignment
|
||||
- **THEN** the local task row records that assignment's Cloud task ID and
|
||||
attempt
|
||||
|
||||
#### Scenario: A task is not Cloud-dispatched
|
||||
- **WHEN** a shared Runtime caller executes a task without Host/Cloud
|
||||
assignment context
|
||||
- **THEN** the task metadata row is created and the optional source
|
||||
correlation fields remain empty
|
||||
|
||||
## REMOVED Requirements
|
||||
|
||||
### Requirement: Host Agent local task storage is isolated from an unrelated local Runtime
|
||||
The Host Agent SHALL use a configurable, Host-Agent-specific database and artifact path for its task metadata store and timeline, distinct from any local Runtime API's own task storage path, so that the two processes cannot silently collide or share state when run on the same machine.
|
||||
|
||||
#### Scenario: Host Agent and local Runtime run on the same machine
|
||||
- **WHEN** both a Host Agent process and a local Runtime API process run on the same machine with their default configurations
|
||||
- **THEN** each process reads and writes its own task metadata store and timeline without observing or modifying the other's data
|
||||
|
||||
+33
@@ -0,0 +1,33 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Runtime is not exposed as a standalone REST service or web console
|
||||
The repository SHALL not ship a standalone Runtime REST application, its
|
||||
unauthenticated web console, or JSON console routes. The shared Runtime and
|
||||
storage packages SHALL remain reusable execution libraries for the Host Agent
|
||||
and other in-process callers.
|
||||
|
||||
#### Scenario: An operator needs to inspect a Host-executed task
|
||||
- **WHEN** an operator needs task evidence for a Host Agent execution
|
||||
- **THEN** the operator uses the authenticated Host Agent console rather than
|
||||
starting or querying a separate Runtime service
|
||||
|
||||
#### Scenario: A package uses shared Runtime execution
|
||||
- **WHEN** the Host Agent or another in-process caller creates a
|
||||
`TaskRunner`
|
||||
- **THEN** it continues to use the shared Runtime and storage packages without
|
||||
importing a REST or UI adapter
|
||||
|
||||
### Requirement: Host Agent does not supervise a retired Runtime service
|
||||
The Host Agent SHALL not expose Runtime-supervision configuration or spawn a
|
||||
Runtime REST subprocess. It MAY continue to optionally supervise Appium.
|
||||
|
||||
#### Scenario: Host Agent dependency supervision is enabled
|
||||
- **WHEN** `HOST_AGENT_DEPENDENCY_SUPERVISOR_ENABLED=true` and Appium
|
||||
supervision is enabled
|
||||
- **THEN** the Host Agent probes and supervises Appium only
|
||||
|
||||
#### Scenario: A removed Runtime-supervision variable is configured
|
||||
- **WHEN** a Host Agent configuration includes a removed
|
||||
`HOST_AGENT_RUNTIME_*` variable
|
||||
- **THEN** startup fails with a message directing the operator to the Host
|
||||
Agent console and Appium-only supervision
|
||||
+39
-16
@@ -1,38 +1,61 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Runtime persists complete evidence for each executed action
|
||||
The Runtime SHALL persist, for each action it attempts, a screenshot captured immediately before the executor call, the action description and arguments, the execution result, and a screenshot captured immediately after the executor call. Existing timeline records that contain only the legacy single screenshot SHALL remain readable, with that screenshot treated as the post-action image.
|
||||
The shared Runtime SHALL persist, for each action it attempts, a screenshot
|
||||
captured immediately before the executor call, the action description and
|
||||
arguments, the execution result, and a screenshot captured immediately after
|
||||
the executor call. Existing Timeline records that contain only the legacy
|
||||
single screenshot SHALL remain readable, with that screenshot treated as the
|
||||
post-action image.
|
||||
|
||||
#### Scenario: An action succeeds
|
||||
- **WHEN** the Runtime executes an action for a task
|
||||
- **THEN** its timeline record includes distinct before and after screenshots, the action detail, and the execution result
|
||||
- **THEN** its Timeline record includes distinct before and after screenshots,
|
||||
action detail, and execution result
|
||||
|
||||
#### Scenario: An action fails
|
||||
- **WHEN** the Runtime executor exhausts its retries for an action
|
||||
- **THEN** the action's timeline record still includes any screenshots that were captured and the failure result before the task is marked failed
|
||||
- **THEN** the Timeline record still includes any captured screenshots and the
|
||||
failure result before the task is marked failed
|
||||
|
||||
#### Scenario: A legacy timeline record is read
|
||||
- **WHEN** a timeline record has only the prior `screenshot_path` field
|
||||
- **THEN** the Runtime exposes it as the post-action screenshot without failing to render the record
|
||||
#### Scenario: A legacy Timeline record is read
|
||||
- **WHEN** a Timeline record has only the prior `screenshot_path` field
|
||||
- **THEN** the Runtime exposes it as the post-action screenshot without
|
||||
failing to render the record
|
||||
|
||||
### Requirement: Runtime task evidence exposes available OCR observations
|
||||
The Runtime SHALL persist raw OCR observations associated with the scene used to plan an action when available, without adding duplicate OCR data to the LLM-facing normalized Scene payload. The Runtime task-detail UI SHALL render available OCR text, confidence, and bounds, and SHALL render normally when no OCR result exists.
|
||||
### Requirement: Runtime task evidence retains available OCR observations
|
||||
The shared Runtime SHALL persist raw OCR observations associated with the scene
|
||||
used to plan an action when available, without adding duplicate OCR data to the
|
||||
LLM-facing normalized Scene payload. The Host Agent task-detail UI SHALL render
|
||||
available OCR text, confidence, and bounds, and SHALL render normally when no
|
||||
OCR result exists.
|
||||
|
||||
#### Scenario: OCR found text while planning an action
|
||||
- **WHEN** perception produced one or more OCR observations for the action's planning scene
|
||||
- **THEN** the corresponding timeline record includes those observations and the Runtime task-detail UI displays them
|
||||
- **WHEN** perception produced one or more OCR observations for the action's
|
||||
planning scene
|
||||
- **THEN** the corresponding Timeline record includes those observations and
|
||||
the Host Agent task-detail page displays them
|
||||
|
||||
#### Scenario: OCR was unavailable or found no text
|
||||
- **WHEN** perception yields no OCR observations
|
||||
- **THEN** the Runtime records the action evidence and renders the task detail without an OCR result list
|
||||
- **THEN** the Runtime records the action evidence and the Host Agent task
|
||||
detail renders without an OCR result list
|
||||
|
||||
### Requirement: Runtime task evidence renders UI-tree inspection results
|
||||
When a Runtime step invokes the existing `get_ui_tree` or `ui_tree` tool and the persisted result contains normalized UI nodes, the Runtime task-detail UI SHALL render those nodes in a structured, collapsible view while retaining the recorded JSON result. The Runtime SHALL NOT change the tool's response contract or duplicate the result in a separate persistence field.
|
||||
### Requirement: Runtime task evidence retains UI-tree inspection results
|
||||
The Runtime SHALL retain a UI-tree inspection result when a step invokes the
|
||||
existing `get_ui_tree` or `ui_tree` tool and the result contains normalized
|
||||
nodes. The Host Agent task-detail UI SHALL render those nodes in a structured,
|
||||
collapsible view while retaining the recorded JSON result. The Runtime SHALL
|
||||
NOT change the tool response contract or duplicate the result in a separate
|
||||
persistence field.
|
||||
|
||||
#### Scenario: UI-tree inspection succeeds
|
||||
- **WHEN** a task step uses `get_ui_tree` or `ui_tree` and returns one or more normalized nodes
|
||||
- **THEN** the task-detail UI displays each node's type, visible text or identifier, bounds, and available confidence
|
||||
- **WHEN** a task step uses `get_ui_tree` or `ui_tree` and returns one or
|
||||
more normalized nodes
|
||||
- **THEN** the Host Agent task-detail page displays each node's type, visible
|
||||
text or identifier, bounds, and available confidence
|
||||
|
||||
#### Scenario: A non-UI-tree step is displayed
|
||||
- **WHEN** a task step did not invoke a UI-tree tool
|
||||
- **THEN** the task-detail UI does not render an empty UI-tree section
|
||||
- **THEN** the Host Agent task-detail page does not render an empty UI-tree
|
||||
section
|
||||
|
||||
Reference in New Issue
Block a user