2.0 KiB
ADDED Requirements
Requirement: Perception is exposed through a swappable provider port
The system SHALL define a PerceptionProvider interface (build_scene) in the perception package that any concrete perception implementation (the existing OCR+tree fusion, a future cloud vision API, a future null/mock provider) must satisfy identically, so tools/, runtime/, and api/ depend only on the port, never on a specific perception technique.
Scenario: Building a scene through the default provider
- WHEN a caller requests a Scene for a screenshot and UI tree via the default (OCR+tree fusion) provider
- THEN the provider returns a
Sceneidentical in shape and content to what the existing fusion logic already produces — no change toscene-perception's specified behavior
Requirement: A null perception provider is available for testing and low-dependency development
The system SHALL provide a NullPerceptionProvider that returns an empty Scene (correct screen width/height, zero elements) without requiring OCR/vision dependencies to be installed or invoked.
Scenario: Running without OCR dependencies installed
- WHEN the runtime is configured to use
NullPerceptionProvider(e.g. in a test or a minimal development environment) - THEN
describe_screen/Agent Runtime calls succeed and return an empty Scene instead of failing due to missing OCR dependencies
Requirement: Adding a perception technique requires no changes outside the perception layer
The system SHALL allow a new perception provider (e.g. a cloud vision API) to be added by registering it within the perception package alone; tools/, runtime/, and api/ SHALL NOT require code changes to use a newly registered provider.
Scenario: Runtime is agnostic to which provider is active
- WHEN the Agent Runtime requests a Scene for the current screenshot and UI tree
- THEN it does so through the
PerceptionProviderport without importing or knowing aboutscene_builder.py, OCR, or any other concrete technique