chore(openspec): archive task execution visibility

2026-07-15 12:10:14 +08:00
parent 77d4813bb2
commit 96e403ee47
17 changed files with 372 additions and 64 deletions
@@ -0,0 +1,78 @@
+# runtime-task-evidence Specification
+
+## Purpose
+
+Define the per-action evidence retained by shared Runtime Timeline records and
+rendered by the Host Agent execution-history console.
+
+## Requirements
+
+### Requirement: Runtime persists complete evidence for each executed action
+
+The shared Runtime SHALL persist, for each action it attempts, a screenshot
+captured immediately before the executor call, the action description and
+arguments, the execution result, and a screenshot captured immediately after
+the executor call. Existing Timeline records that contain only the legacy
+single screenshot SHALL remain readable, with that screenshot treated as the
+post-action image.
+
+#### Scenario: An action succeeds
+
+- **WHEN** the Runtime executes an action for a task
+- **THEN** its Timeline record includes distinct before and after screenshots,
+  action detail, and execution result
+
+#### Scenario: An action fails
+
+- **WHEN** the Runtime executor exhausts its retries for an action
+- **THEN** the Timeline record still includes any captured screenshots and the
+  failure result before the task is marked failed
+
+#### Scenario: A legacy Timeline record is read
+
+- **WHEN** a Timeline record has only the prior `screenshot_path` field
+- **THEN** the Runtime exposes it as the post-action screenshot without
+  failing to render the record
+
+### Requirement: Runtime task evidence retains available OCR observations
+
+The shared Runtime SHALL persist raw OCR observations associated with the scene
+used to plan an action when available, without adding duplicate OCR data to the
+LLM-facing normalized Scene payload. The Host Agent task-detail UI SHALL render
+available OCR text, confidence, and bounds, and SHALL render normally when no
+OCR result exists.
+
+#### Scenario: OCR found text while planning an action
+
+- **WHEN** perception produced one or more OCR observations for the action's
+  planning scene
+- **THEN** the corresponding Timeline record includes those observations and
+  the Host Agent task-detail page displays them
+
+#### Scenario: OCR was unavailable or found no text
+
+- **WHEN** perception yields no OCR observations
+- **THEN** the Runtime records the action evidence and the Host Agent task
+  detail renders without an OCR result list
+
+### Requirement: Runtime task evidence retains UI-tree inspection results
+
+The Runtime SHALL retain a UI-tree inspection result when a step invokes the
+existing `get_ui_tree` or `ui_tree` tool and the result contains normalized
+nodes. The Host Agent task-detail UI SHALL render those nodes in a structured,
+collapsible view while retaining the recorded JSON result. The Runtime SHALL
+NOT change the tool response contract or duplicate the result in a separate
+persistence field.
+
+#### Scenario: UI-tree inspection succeeds
+
+- **WHEN** a task step uses `get_ui_tree` or `ui_tree` and returns one or
+  more normalized nodes
+- **THEN** the Host Agent task-detail page displays each node's type, visible
+  text or identifier, bounds, and available confidence
+
+#### Scenario: A non-UI-tree step is displayed
+
+- **WHEN** a task step did not invoke a UI-tree tool
+- **THEN** the Host Agent task-detail page does not render an empty UI-tree
+  section