50 lines
7.2 KiB
Markdown
50 lines
7.2 KiB
Markdown
## 1. Package scaffolding
|
|
|
|
- [x] 1.1 Create `skills_learning/` package with `__init__.py`, `models.py`, `synthesis.py`, `versioning.py`, `embeddings.py`, `retrieval.py`, `config.py`, `store.py`
|
|
- [x] 1.2 Add `skills_learning*` to `pyproject.toml`'s `[tool.setuptools.packages.find].include` list and add the embedding-provider SDK dependency
|
|
- [x] 1.3 Add `skills_learning/config.py` with `SkillAuthoringConfig` (enable flag default `False`, divergence tolerance, embedding model name, default `top_k`) and a module-level accessor mirroring `semantic-scene-runtime`'s/`world-model-runtime`'s config-module pattern
|
|
- [x] 1.4 Add `skills_learning/models.py` defining (or importing, if `skill-catalog-subscription` is already implemented) the shared `Skill`/`FlowTemplateSkill`/`SkillMetadata` dataclass shapes, plus this capability's own `source`, `version`, `parent_version_id` fields
|
|
|
|
## 2. Local skill store (skill-authoring, skill-versioning)
|
|
|
|
- [x] 2.1 Implement `skills_learning/store.py`: a local store for locally-synthesized `FlowTemplateSkill` records, separate from `skill-catalog-subscription`'s synced catalog, with `create_version()`, `get_by_id()`, `get_latest_by_name()`, `list_versions(name)`
|
|
- [x] 2.2 Enforce `source = "local-synthesis"` tagging on every record written by this store; add a guard/test that this store never writes into or imports a write-path of `skill-catalog-subscription`'s catalog module
|
|
- [x] 2.3 Write unit tests for the store's version-chain semantics: creating a new version does not delete/modify prior versions, and `get_latest_by_name()` returns the highest `version`
|
|
|
|
## 3. Timeline extraction and parameter abstraction (skill-authoring)
|
|
|
|
- [x] 3.1 Implement `skills_learning/synthesis.py`'s tool-call extraction: given a `task_id`, read `storage.timeline.Timeline.read(task_id)` and produce an ordered list of `(tool_name, args)` pairs, filtering out read-only tool names (`describe_screen`, `screenshot`, `ui_tree`, `find_text`, `find_icon`)
|
|
- [x] 3.2 Implement skeleton matching: given an extracted tool-name sequence, look up any stored skill (via `store.py`) whose latest version has the identical tool-name sequence
|
|
- [x] 3.3 Implement cross-execution argument diffing: compare extracted argument values position-by-position against a matched stored version's steps, and promote any differing value into a named `{param}` placeholder plus a corresponding entry in the skill's `parameters` schema
|
|
- [x] 3.4 Implement parameter naming: prefer a name derived from the corresponding `SemanticScene.widgets[].purpose` label when available (optional dependency on `semantic/`'s output, degrading gracefully when absent), else fall back to a positional name (e.g. `param_2`)
|
|
- [x] 3.5 Implement `synthesize_flow_skill(goal, timeline) -> FlowTemplateSkill`: orchestrates extraction → skeleton match → diffing → parameter promotion → returns a candidate skill record (not yet persisted)
|
|
- [x] 3.6 Write unit tests for first-time synthesis (no prior match, zero parameters), second-execution parameter promotion, and identical-repeat synthesis (no spurious new parameters), using canned `TimelineRecord` fixtures
|
|
|
|
## 4. Version divergence detection (skill-versioning)
|
|
|
|
- [x] 4.1 Implement `skills_learning/versioning.py`'s `diff_flow_versions(stored_steps, executed_steps) -> VersionDiff`: detect tool-name-sequence insertion/deletion/reorder (structural divergence) versus argument-value-only differences
|
|
- [x] 4.2 Implement version-bump logic: on structural divergence, construct a new `FlowTemplateSkill` version with incremented `version` and `parent_version_id` set to the prior version's id; on argument-only divergence, update the existing version's parameters in place (no bump)
|
|
- [x] 4.3 Write unit tests: extra/missing/reordered step triggers a version bump; identical-sequence-different-values does not bump but does update parameters; assert prior version records remain retrievable and unmodified after a bump
|
|
|
|
## 5. Post-task synthesis hook wiring
|
|
|
|
- [x] 5.1 Add an optional `on_task_succeeded: Callable[[str, str, Timeline], None] | None = None` constructor argument to `TaskRunner` in `runtime/task.py`, invoked exactly once at the end of `run()` when the final status is `succeeded`
|
|
- [x] 5.2 Wire a default hook (when `on_task_succeeded` is not explicitly passed and Skill Authoring is enabled in `skills_learning/config.py`) that calls `synthesis.synthesize_flow_skill()`, runs versioning via `versioning.py`, and persists the result via `store.py`
|
|
- [x] 5.3 Verify that when Skill Authoring is disabled (default) or `on_task_succeeded` is left `None` and disabled, `TaskRunner.run()`'s behavior and return value are byte-for-byte identical to before this change
|
|
- [x] 5.4 Write a unit test that runs a fake successful `TaskRunner` loop with Skill Authoring enabled and asserts a skill record is stored after completion, and a test that asserts no store write occurs when disabled
|
|
|
|
## 6. Embedding and retrieval (skill-embedding-retrieval)
|
|
|
|
- [x] 6.1 Implement `skills_learning/embeddings.py`'s embedding client interface: `embed_skill_text(text) -> list[float] | None`, catching timeout/rate-limit/disabled-config/connection-error internally and returning `None` rather than raising, mirroring `semantic/llm_client.py`'s degrade-safe contract
|
|
- [x] 6.2 Implement a local `skill_embeddings` index (skill id + version → vector, model name, `updated_at`) in `skills_learning/store.py` or a dedicated `skills_learning/embeddings_store.py`
|
|
- [x] 6.3 Wire embedding computation into the post-synthesis/versioning path: call `embed_skill_text()` on `name + description + goal` for every newly stored skill version, storing the resulting vector (or leaving the skill un-embedded if the call returns `None`)
|
|
- [x] 6.4 Implement `skills_learning/retrieval.py`'s `retrieve_candidate_skills(goal, top_k) -> list[ScoredSkill]`: embed the incoming goal, compute cosine similarity against every stored skill embedding, and return the top `top_k` ranked results, skipping skills with no stored embedding
|
|
- [x] 6.5 Write unit tests: ranked ordering for a goal similar to a stored skill's originating goal (using a fake/deterministic embedding function), `top_k` truncation, empty-result case when no skill has an embedding, and a case where an embedding call returns `None` and the skill is stored but excluded from retrieval results
|
|
|
|
## 7. Integration tests and validation
|
|
|
|
- [x] 7.1 Write an end-to-end test: run a fake successful task twice with slightly different goal text/argument values through `TaskRunner` (Skill Authoring enabled, embedding client mocked), asserting the second run produces a new skill version with a promoted parameter and its own embedding
|
|
- [x] 7.2 Write an end-to-end test: run a fake successful task, then call `retrieve_candidate_skills()` with a new, semantically similar goal string, asserting the synthesized skill is returned
|
|
- [x] 7.3 Run the full existing `pytest` suite and confirm zero existing test files require content changes (only new `tests/test_skill_*.py`-style files are added)
|
|
- [x] 7.4 Add a smoke test importing `skills_learning` alongside existing `tests/` smoke coverage, confirming the package has no import-time dependency on `skill-catalog-subscription`'s sync client (only, optionally, its shared model shapes if already implemented)
|