You've already forked DataMate
feat(auto-annotation): add LLM-based annotation operators
Add three new LLM-powered auto-annotation operators: - LLMTextClassification: Text classification using LLM - LLMNamedEntityRecognition: Named entity recognition with type validation - LLMRelationExtraction: Relation extraction with entity and relation type validation Key features: - Load LLM config from t_model_config table via modelId parameter - Lazy loading of LLM configuration on first execute() - Result validation with whitelist checking for entity/relation types - Fault-tolerant: returns empty results on LLM failure instead of throwing - Fully compatible with existing Worker pipeline Files added: - runtime/ops/annotation/_llm_utils.py: Shared LLM utilities - runtime/ops/annotation/llm_text_classification/: Text classification operator - runtime/ops/annotation/llm_named_entity_recognition/: NER operator - runtime/ops/annotation/llm_relation_extraction/: Relation extraction operator Files modified: - runtime/ops/annotation/__init__.py: Register 3 new operators - runtime/python-executor/datamate/auto_annotation_worker.py: Add to Worker whitelist - frontend/src/pages/DataAnnotation/OperatorCreate/hooks/useOperatorOperations.ts: Add to frontend whitelist
This commit is contained in:
@@ -22,6 +22,9 @@ type CategoryGroup = {
|
||||
const ANNOTATION_OPERATOR_ID_WHITELIST = new Set([
|
||||
"ImageObjectDetectionBoundingBox",
|
||||
"test_annotation_marker",
|
||||
"LLMTextClassification",
|
||||
"LLMNamedEntityRecognition",
|
||||
"LLMRelationExtraction",
|
||||
]);
|
||||
|
||||
const ensureArray = (value: unknown): string[] => {
|
||||
|
||||
Reference in New Issue
Block a user