DataMate/__init__.py at lsf - DataMate - Jerry's Git

q792602257/DataMate

Files

Jerry Yan 49f99527cc feat(auto-annotation): add LLM-based annotation operators

Add three new LLM-powered auto-annotation operators:
- LLMTextClassification: Text classification using LLM
- LLMNamedEntityRecognition: Named entity recognition with type validation
- LLMRelationExtraction: Relation extraction with entity and relation type validation

Key features:
- Load LLM config from t_model_config table via modelId parameter
- Lazy loading of LLM configuration on first execute()
- Result validation with whitelist checking for entity/relation types
- Fault-tolerant: returns empty results on LLM failure instead of throwing
- Fully compatible with existing Worker pipeline

Files added:
- runtime/ops/annotation/_llm_utils.py: Shared LLM utilities
- runtime/ops/annotation/llm_text_classification/: Text classification operator
- runtime/ops/annotation/llm_named_entity_recognition/: NER operator
- runtime/ops/annotation/llm_relation_extraction/: Relation extraction operator

Files modified:
- runtime/ops/annotation/__init__.py: Register 3 new operators
- runtime/python-executor/datamate/auto_annotation_worker.py: Add to Worker whitelist
- frontend/src/pages/DataAnnotation/OperatorCreate/hooks/useOperatorOperations.ts: Add to frontend whitelist

2026-02-10 15:22:23 +08:00

11 lines

285 B

Python

Raw Permalink Blame History

 # -*- coding: utf-8 -*-
 from datamate.core.base_op import OPERATORS
 from .process import LLMRelationExtraction
 OPERATORS.register_module(
     module_name="LLMRelationExtraction",
     module_path="ops.annotation.llm_relation_extraction.process",
 )
 __all__ = ["LLMRelationExtraction"]