# semantic-scene Specification

## Purpose
TBD - created by archiving change semantic-scene-runtime. Update Purpose after archive.
## Requirements
### Requirement: Semantic scene enrichment from an existing Scene
The system SHALL provide a function that, given an existing `Scene` (as produced by the `scene-perception` capability), produces a `SemanticScene` consisting of a page identity string, a list of plain-language supported intents, and a list of per-widget purpose labels referencing the `Scene`'s element IDs, using exactly one LLM call.

#### Scenario: Enrichment succeeds for a recognizable screen
- **WHEN** `enrich_scene()` is called with a `Scene` describing a recognizable app screen (e.g. a chat screen with a text input and a send button)
- **THEN** it returns a `SemanticScene` with a non-empty `page` string, a non-empty `intents` list of plain-language strings, and a `widgets` list where each entry's `element_id` matches an element ID present in the input `Scene`

#### Scenario: Widget purpose labels reference only known elements
- **WHEN** the LLM response includes a widget purpose label whose `element_id` does not match any element ID in the input `Scene`
- **THEN** `enrich_scene()` discards that widget entry from the returned `SemanticScene` rather than propagating a dangling element reference

### Requirement: Enrichment failure degrades to no semantic scene, never blocks the loop
The system SHALL treat any enrichment failure (LLM call timeout, connection error, rate limit, malformed or schema-invalid response, or enrichment disabled by configuration) as a non-fatal condition, returning an absence of a semantic scene rather than raising an exception, so that callers always have a defined fallback of using the raw `Scene`.

#### Scenario: LLM call times out
- **WHEN** the enrichment LLM call does not complete within the configured timeout
- **THEN** `enrich_scene()` returns `None` and the caller proceeds using the raw `Scene` without the task step being marked as failed

#### Scenario: LLM response fails schema validation
- **WHEN** the enrichment LLM call returns a response that does not conform to the expected `SemanticScene` JSON schema
- **THEN** `enrich_scene()` returns `None` instead of raising, and no partially-parsed `SemanticScene` is returned

#### Scenario: Enrichment disabled by configuration
- **WHEN** semantic enrichment is disabled in configuration
- **THEN** `enrich_scene()` returns `None` immediately without making an LLM call

### Requirement: Structured, schema-constrained LLM output
The system SHALL request the enrichment LLM call using a schema-constrained structured-output mechanism so that any successful response is guaranteed to be valid JSON matching the `SemanticScene` shape, rather than relying on free-text parsing of an unconstrained model reply.

#### Scenario: Successful call yields directly parseable output
- **WHEN** the enrichment LLM call completes successfully
- **THEN** the raw response body is valid JSON matching the declared `SemanticScene` schema without requiring text extraction, regex matching, or a JSON-repair step

### Requirement: Semantic enrichment is opt-in and does not alter existing tool behavior
The system SHALL expose semantic enrichment through a new, separate tool entry point rather than modifying the existing `describe_screen` tool's signature or behavior, so that every existing caller of `describe_screen` continues to receive only a `Scene`, unchanged, unless it explicitly opts into the new semantic-enriched entry point.

#### Scenario: Existing describe_screen callers are unaffected
- **WHEN** an existing caller invokes the `describe_screen` tool as it did before this change
- **THEN** it receives the same `Scene` result as before, with no semantic enrichment attempted and no new LLM-related dependency invoked

#### Scenario: A caller opts into semantic enrichment
- **WHEN** a caller invokes the new semantic-enrichment tool entry point for a given device
- **THEN** it receives both the underlying `Scene` and, when enrichment succeeds, the corresponding `SemanticScene`; when enrichment fails or is disabled, it receives the `Scene` with an explicit absence of a `SemanticScene` rather than a partial or error result