Commit Graph

20 Commits

Author SHA1 Message Date
0bb9abb200 feat(annotation): 添加标注类型显示功能
- 在前端页面中新增标注类型列并使用Tag组件展示
- 添加AnnotationTypeMap常量用于标注类型的映射
- 修改接口定义支持labelingType字段的传递
- 更新后端项目创建和更新逻辑以存储标注类型
- 添加标注类型配置键常量统一管理
- 扩展数据传输对象支持标注类型属性
- 实现模板标注类型的继承逻辑
2026-02-01 19:08:11 +08:00
150af1a741 fix(annotation): 修复项目映射查询逻辑错误
- 移除旧的映射服务查询方式,改为直接查询 ORM 模型获取原始数据
- 更新配置字段读取逻辑以使用新的 ORM 对象
- 修复更新无变化时的响应数据返回问题
- 添加软删除过滤条件确保只返回未删除的项目记录
- 统一数据访问方式提高查询效率和代码一致性
2026-01-31 18:57:08 +08:00
e28f680abb feat(annotation): 添加标注项目信息更新功能
- 引入 DatasetMappingUpdateRequest 请求模型支持 name、description、template_id 和 label_config 字段更新
- 在项目接口中添加 PUT /{project_id} 端点用于更新标注项目信息
- 实现更新逻辑包括映射记录查询、配置信息处理和数据库更新操作
- 集成标准响应格式返回更新结果
- 添加异常处理和日志记录确保操作可追溯性
2026-01-31 18:54:05 +08:00
b5d7c66240 feat(data-management): 扩展源文档排除功能支持Excel文件类型
- 在后端服务中扩展源文档类型检查,新增对XLS和XLSX文件的支持
- 修改DatasetFileApplicationService中的过滤逻辑,统一处理所有源文档类型
- 新增isSourceDocument和isDerivedFile辅助方法进行文件类型判断
- 更新前端DatasetFileTransfer组件中的注释说明
- 在Python运行时依赖中添加openpyxl和xlrd库以支持Excel文件处理
- 修改标注项目接口中源文档类型的集合定义
- 更新文件操作钩子中的派生文件排除逻辑
2026-01-31 11:30:55 +08:00
9a205919d7 refactor(data-import): 优化数据源文件扫描和复制逻辑
- 修改数据源文件扫描方法,直接在主流程中获取任务详情和路径
- 移除独立的getFilePaths方法,将路径扫描逻辑整合到scanFilePaths方法中
- 新增copyFilesToDatasetDirWithSourceRoot方法支持保留相对路径的文件复制
- 更新数据集文件应用服务中的文件复制逻辑,支持相对路径处理
- 修改Python后端项目接口中的文件查询逻辑,移除注释掉的编辑器服务引用
- 调整文件过滤逻辑,基于元数据中的派生源ID进行文件筛选
- 移除编辑器服务中已废弃的源文档过滤条件
2026-01-30 18:58:34 +08:00
8b2a19f09a feat(annotation): 添加标注项目文件快照功能
- 新增 LabelingProjectFile 模型用于存储标注项目的文件快照
- 在创建标注项目时记录关联的文件快照数据
- 更新查询逻辑以基于项目快照过滤文件列表
- 优化导出统计功能使用快照数据进行计算
- 添加数据库表结构支持项目文件快照关系
2026-01-30 18:10:13 +08:00
371df12a96 feat(annotation): 添加文本数据集段落切片功能
- 在前端组件中新增 segmentationEnabled 字段控制切片开关
- 为文本数据集添加段落切片配置选项,默认启用切片功能
- 在后端接口中新增 segmentation_enabled 参数传递给标注项目
- 实现切片逻辑控制,支持文本数据的自动段落分割
- 添加数据集类型判断,仅文本数据集支持切片配置
- 更新标注任务创建和编辑表单中的切片相关字段处理
2026-01-26 12:14:27 +08:00
d890a5679d refactor(annotation): 统一查询参数命名规范
- 将分页查询参数 pageSize 替换为 size
- 更新所有相关函数中的参数引用
- 修改日志输出中的参数名称显示
- 保持原有的分页逻辑不变
2026-01-19 23:56:40 +08:00
649ab2f6bb refactor(annotation): 移除调试日志和异常堆栈跟踪
- 移除了项目映射获取接口中的traceback打印
- 简化了内部服务器错误响应消息
- 删除了映射服务中的多个调试日志输出
- 清理了响应数据构建过程中的调试信息
2026-01-19 21:58:00 +08:00
496161b1f1 ```
chore(annotation): 添加调试日志到映射服务

- 在 _to_response_from_row 方法中添加配置和标签配置的调试日志
- 在 _to_response 方法中添加映射ID和配置信息的调试日志
- 添加响应数据键名的调试日志
- 优化配置解析逻辑以确保字典类型的正确检查
```
2026-01-19 21:52:01 +08:00
01dcd16a98 feat(annotation): 添加标注任务自定义配置功能
- 新增 LabelStudioEmbed 组件用于嵌入式标注界面预览
- 在创建标注任务对话框中添加 XML 配置编辑器
- 支持从现有模板加载配置并进行自定义修改
- 实现标注界面实时预览功能
- 后端支持直接传递 label_config 覆盖模板配置
- 更新 CreateAnnotationTaskRequest 模型添加 labelConfig 字段
2026-01-18 14:12:12 +08:00
3aa7f6e3a1 refactor(annotation): 移除对 Label Studio Server 的依赖并切换到内嵌编辑器模式
- 移除 LabelStudioClient 和 SyncService 的导入及使用
- 删除与 Label Studio 项目的创建、删除和同步相关代码
- 修改创建数据集映射功能,改为创建 DataMate 标注项目
- 更新删除映射接口,仅进行软删除不再删除 Label Studio 项目
- 修改同步接口为兼容性保留,实际操作为空操作
- 移除 Label Studio 连接诊断功能
- 更新文档说明以反映内嵌编辑器模式的变化
2026-01-09 12:01:20 +08:00
hefanli
29e4a333a9 feature: LabelStudio jumps without login (#201) 2025-12-25 16:49:06 +08:00
Jason Wang
df853a5177 feat: Enhance file tag update functionality with automatic format conversion (#84)
- Updated `update_file_tags` to support both simplified and full tag formats.
- Introduced `TagFormatConverter` to handle conversion from simplified external tags to internal storage format.
- Added logic to fetch and utilize the appropriate annotation template for conversion.
- Improved error handling for missing templates and unknown controls during tag updates.
- Created example script demonstrating the usage of the new tag format conversion feature.
- Added unit tests for `TagFormatConverter` to ensure correct functionality and edge case handling.
2025-11-14 12:42:39 +08:00
Jason Wang
45743f39f5 feat: add labeling template. refactor: switch to Poetry, build and deploy of backend Python (#79)
* feat: Enhance annotation module with template management and validation

- Added DatasetMappingCreateRequest and DatasetMappingUpdateRequest schemas to handle dataset mapping requests with camelCase and snake_case support.
- Introduced Annotation Template schemas including CreateAnnotationTemplateRequest, UpdateAnnotationTemplateRequest, and AnnotationTemplateResponse for managing annotation templates.
- Implemented AnnotationTemplateService for creating, updating, retrieving, and deleting annotation templates, including validation of configurations and XML generation.
- Added utility class LabelStudioConfigValidator for validating Label Studio configurations and XML formats.
- Updated database schema for annotation templates and labeling projects to include new fields and constraints.
- Seeded initial annotation templates for various use cases including image classification, object detection, and text classification.

* feat: Enhance TemplateForm with improved validation and dynamic field rendering; update LabelStudio config validation for camelCase support

* feat: Update docker-compose.yml to mark datamate dataset volume and network as external

* feat: Add tag configuration management and related components

- Introduced new components for tag selection and browsing in the frontend.
- Added API endpoint to fetch tag configuration from the backend.
- Implemented tag configuration management in the backend, including loading from YAML.
- Enhanced template service to support dynamic tag rendering based on configuration.
- Updated validation utilities to incorporate tag configuration checks.
- Refactored existing code to utilize the new tag configuration structure.

* feat: Refactor LabelStudioTagConfig for improved configuration loading and validation

* feat: Update Makefile to include backend-python-docker-build in the build process

* feat: Migrate to poetry for better deps management

* Add pyyaml dependency and update Dockerfile to use Poetry for dependency management

- Added pyyaml (>=6.0.3,<7.0.0) to pyproject.toml dependencies.
- Updated Dockerfile to install Poetry and manage dependencies using it.
- Improved layer caching by copying only dependency files before the application code.
- Removed unnecessary installation of build dependencies to keep the final image size small.

* feat: Remove duplicated backend-python-docker-build target from Makefile

* fix: airflow is not ready for adding yet

* feat: update Python version to 3.12 and remove project installation step in Dockerfile
2025-11-13 15:32:30 +08:00
Jason Wang
c5ccc56cca feat: Add labeling template (#72)
* feat: Enhance annotation module with template management and validation

- Added DatasetMappingCreateRequest and DatasetMappingUpdateRequest schemas to handle dataset mapping requests with camelCase and snake_case support.
- Introduced Annotation Template schemas including CreateAnnotationTemplateRequest, UpdateAnnotationTemplateRequest, and AnnotationTemplateResponse for managing annotation templates.
- Implemented AnnotationTemplateService for creating, updating, retrieving, and deleting annotation templates, including validation of configurations and XML generation.
- Added utility class LabelStudioConfigValidator for validating Label Studio configurations and XML formats.
- Updated database schema for annotation templates and labeling projects to include new fields and constraints.
- Seeded initial annotation templates for various use cases including image classification, object detection, and text classification.

* feat: Enhance TemplateForm with improved validation and dynamic field rendering; update LabelStudio config validation for camelCase support

* feat: Update docker-compose.yml to mark datamate dataset volume and network as external
2025-11-11 09:14:14 +08:00
Jason Wang
78f50ea520 feat: File and Annotation 2-way sync implementation (#63)
* feat: Refactor configuration and sync logic for improved dataset handling and logging

* feat: Enhance annotation synchronization and dataset file management

- Added new fields `tags_updated_at` to `DatasetFiles` model for tracking the last update time of tags.
- Implemented new asynchronous methods in the Label Studio client for fetching, creating, updating, and deleting task annotations.
- Introduced bidirectional synchronization for annotations between DataMate and Label Studio, allowing for flexible data management.
- Updated sync service to handle annotation conflicts based on timestamps, ensuring data integrity during synchronization.
- Enhanced dataset file response model to include tags and their update timestamps.
- Modified database initialization script to create a new column for `tags_updated_at` in the dataset files table.
- Updated requirements to ensure compatibility with the latest dependencies.
2025-11-07 15:03:07 +08:00
Jason Wang
ba0c69086a feat: data annotation page adaptation to backend API. Improve labeling project creation module.
* feat: data annotation page adaptation to the backend API.

* feat: Implement labeling configuration editor and enhance annotation task creation form
2025-10-31 15:56:29 +08:00
Jason Wang
e0884ab048 Develop py update schema (#37)
* feature: implement endpoints with multi-level response models
* refactor: move `/health` and `/config` endpoints to system module, remove example from base schemas
* refactor: remove unused get_standard_response_model()
2025-10-30 16:24:37 +08:00
Jason Wang
2f7341dc1f refactor: Reorganize datamate-python (#34)
refactor: Reorganize datamate-python (previously label-studio-adapter) into a DDD style structure.
2025-10-30 01:32:59 +08:00