Commit Graph

178 Commits

Author SHA1 Message Date
a2b0fc3674 feat(annotation): 添加模板示例数据配置功能
- 在模板配置表单中新增示例数据输入区域
- 实现不同数据类型的示例输入框(文本、图片、音频、视频等)
- 添加图片类型示例的实时预览功能
- 在模板详情页增加示例数据预览卡片
- 支持多种媒体类型的示例展示(图片、音频、视频、文本)
- 更新前后端数据模型以支持exampleData字段
- 添加示例数据的placeholder提示文案
2026-01-18 21:59:41 +08:00
5057457329 feat(annotation): 添加自定义标注模板配置功能
- 新增 TemplateConfigurationForm 组件用于自定义配置
- 实现模板模式和自定义模式的切换功能
- 添加 generateXmlFromConfig 函数动态生成 XML 配置
- 支持通过表单方式配置数据对象和标签控件
- 移除模板选择时多余的 XML 清空逻辑
- 优化配置预览按钮显示逻辑
2026-01-18 21:32:01 +08:00
fc978620a7 refactor(annotation): 调整模板配置表单组件导入路径
- 更新 CreateTask.tsx 中 TemplateConfigurationForm 的导入路径
- 更新 TemplateForm.tsx 中 TemplateConfigurationForm 的导入路径
- 统一模板配置表单组件的引用方式
- 优化组件文件结构和依赖关系
2026-01-18 21:15:30 +08:00
6e08255820 feat(DataAnnotation): 新增模板配置表单组件
- 实现了数据对象配置区域,支持添加、删除数据对象字段
- 添加了标签控件配置区域,支持多种控件类型的动态配置
- 集成了TagSelector组件用于对象类型和控件类型的选择
- 实现了表单验证规则,包括必填项和值格式校验
- 添加了动态选项渲染功能,根据控件类型显示相应配置项
- 实现了表单联动逻辑,支持对象选择和控件配置的关联
- 添加了用户友好的界面布局和交互提示功能
2026-01-18 21:08:47 +08:00
0e19178a5e feat(annotation): 扩展标注模板的数据类型和标注类型选项
- 新增多种数据类型包括PDF、时间序列、聊天数据、HTML和表格数据
- 扩展分类标签体系增加音频语音、聊天评估、对话AI、生成式AI等分类
- 实现动态下拉选项渲染替代硬编码选项列表
- 添加超过100种新的标注类型涵盖音频语音、聊天评估、计算机视觉等领域
- 更新列表组件显示映射后的标签文本而非原始值
- 优化表单布局添加wrap属性支持选项换行显示
2026-01-18 20:10:57 +08:00
9356ee51ad feat(menu): 移除知识库功能入口和Webhook配置
- 从菜单中移除知识生成相关的BookOpen图标
- 删除知识库功能的导航项及其相关配置
- 移除Webhook配置页面和对应的菜单选项
- 简化设置页面的导入依赖和组件结构
2026-01-18 18:01:19 +08:00
196cb0b037 feat(data-annotation): 移除标注任务同步功能
- 删除 SyncOutlined 图标导入
- 移除 syncAnnotationTaskUsingPost API 调用
- 简化删除确认对话框内容,移除同步提示信息
- 删除 handleSync 和 handleBatchSync 同步处理函数
- 从操作菜单中移除同步选项
- 从批量操作按钮中移除批量同步功能
- 更新批量删除确认对话框内容
2026-01-18 17:53:31 +08:00
3a3a28d5ea fix(export): 解决统计数据字段兼容性问题
- 添加对 snake_case 格式字段的兼容支持
- 使用空值合并运算符处理字段缺失情况
- 保持原有 camelCase 字段的向后兼容性
- 确保统计数据正确显示无论后端返回哪种命名格式
2026-01-18 17:43:55 +08:00
c48d2fdeb8 feat(annotation): 添加标注数据导出功能
- 新增导出对话框组件,支持多种格式选择
- 实现 JSON、JSONL、CSV、COCO、YOLO 五种导出格式
- 添加导出统计信息显示,包括总文件数和已标注数
- 集成前端导出按钮和后端 API 接口
- 支持仅导出已标注数据和包含原始数据选项
- 实现文件下载和命名功能
2026-01-18 16:54:02 +08:00
6fbf7cc84d feat(label-studio): 添加预加载组件以优化加载性能
- 实现 LabelStudioPreloader 组件用于预加载资源
- 使用 iframe 在后台加载 /lsf/lsf.html 页面
- 设置 2 秒延迟避免与初始页面加载竞争资源
- 配置隐藏样式确保预加载过程不可见
- 添加清理函数防止内存泄漏
- 实现条件渲染控制预加载时机
2026-01-18 14:51:37 +08:00
453aa679c2 feat(app): 添加标签工作室预加载组件
- 引入 LabelStudioPreloader 组件
- 在应用根组件中添加预加载器渲染
- 确保标签工作室资源预先加载完成
2026-01-18 14:39:35 +08:00
01dcd16a98 feat(annotation): 添加标注任务自定义配置功能
- 新增 LabelStudioEmbed 组件用于嵌入式标注界面预览
- 在创建标注任务对话框中添加 XML 配置编辑器
- 支持从现有模板加载配置并进行自定义修改
- 实现标注界面实时预览功能
- 后端支持直接传递 label_config 覆盖模板配置
- 更新 CreateAnnotationTaskRequest 模型添加 labelConfig 字段
2026-01-18 14:12:12 +08:00
87c2ef8a58 首页 2026-01-16 01:50:36 +08:00
afa7fee59e style 2026-01-14 16:35:33 +08:00
70b2ddd4a2 feat(label-studio): 添加编辑器就绪状态检查和任务ID验证
- 添加 expectedTaskIdRef 用于跟踪预期任务ID
- 添加 lsReady 状态管理编辑器就绪状态
- 在任务加载时重置编辑器就绪状态和预期任务ID
- 实现 LS_READY 消息处理和任务ID验证逻辑
- 添加错误处理时设置编辑器为未就绪状态
- 更新加载提示显示不同状态信息
- 改进条件渲染确保编辑器完全准备就绪后显示
2026-01-12 22:44:13 +08:00
0f68146283 feat(annotation): 更新标签工作室文本编辑器界面配置
- 修改标注接口配置,启用当前标注和历史记录功能
- 替换提交按钮为标注标签页和菜单功能
- 优化文件卡片布局样式,调整高度和溢出处理
- 更新编辑器卡片样式,改进响应式布局结构
2026-01-09 18:38:55 +08:00
95c480360c build(lsf): 添加 Label Studio Frontend 构建文件和版本信息
- 新增 478.chunk.js 构建文件,包含 Label Studio 核心功能
- 添加 version.json 文件记录构建版本信息
- 版本信息包含提交消息、提交哈希、分支名称和构建日期
- 修复 LSDV-4887 任务视图分组 bug
2026-01-09 18:19:36 +08:00
4bfe96048c feat(lsf): 将标签工作室前端从CDN迁移到本地资源
- 更新页面标题为更简洁的数据标注描述
- 移除CDN链接改为使用本地静态资源路径
- 删除冗余的注释说明内容
- 优化资源加载方式提高离线可用性
2026-01-09 17:14:24 +08:00
08336e2a13 feat(annotation): 添加标注模板配置功能
- 在schema中新增choice和show_inline字段支持选择模式配置
- 为编辑器服务添加空标注创建逻辑避免前端异常
- 实现标签类型的标准化处理和大小写兼容
- 支持Choices标签的单选/多选和行内显示配置
- 优化前端界面滚动条显示控制样式
2026-01-09 13:05:09 +08:00
a82f4f1bc3 refactor(annotation): 移除对 Label Studio Server 的依赖并切换到内嵌编辑器模式
- 移除 LabelStudioClient 和 SyncService 的导入及使用
- 删除与 Label Studio 项目的创建、删除和同步相关代码
- 修改创建数据集映射功能,改为创建 DataMate 标注项目
- 更新删除映射接口,仅进行软删除不再删除 Label Studio 项目
- 修改同步接口为兼容性保留,实际操作为空操作
- 移除 Label Studio 连接诊断功能
- 更新文档说明以反映内嵌编辑器模式的变化
2026-01-09 12:31:03 +08:00
f3f1609455 mirror 2026-01-09 08:46:01 +08:00
d5b75fee0d LSF 2026-01-07 00:00:16 +08:00
hhhhsc701
7d4dcb756b fix: 修复入库可能重复;筛选逻辑优化 (#226)
* 修改数据清洗筛选逻辑-筛选修改为多选

* 修改数据清洗筛选逻辑-筛选修改为多选

* antd 组件库样式定制修改

* fix: 修复入库可能重复

* fix: 算子市场筛选逻辑优化

* fix: 清洗任务创建筛选逻辑优化

* fix: 清洗任务创建筛选逻辑优化

---------

Co-authored-by: chase <byzhangxin11@126.com>
2026-01-06 17:57:25 +08:00
Kecheng Sha
49cc98941f 优化合成任务数据集选择交互体验 (#225) 2026-01-06 15:02:06 +08:00
o0Shark0o
843bfef07b fix: correct dataset file pagination for synthesis task 2026-01-06 11:28:37 +08:00
hefanli
a15a6134ff fix the ratio task config (#224)
* fix: fix the dataset card icon

* fix: fix the dataset file tag distribution and ratio task

* refactor: change dateRange config from latest to start-end
2026-01-05 17:02:28 +08:00
Kecheng Sha
3f1ad6a872 feat(auto-annotation): integrate YOLO auto-labeling and enhance data management (#223)
* feat(auto-annotation): initial setup

* chore: remove package-lock.json

* chore: 清理本地测试脚本与 Maven 设置

* chore: change package-lock.json
2026-01-05 14:22:44 +08:00
hefanli
ccfb84c034 feature: add mysql collection and starrocks collection (#222)
* fix: fix the path for backend-python imaage building

* feature: add mysql collection and starrocks collection

* feature: add mysql collection and starrocks collection

* fix: change the permission of those files which collected from nfs to 754

* fix: delete collected files, config files and log files while deleting collection task

* fix: add the collection task detail api

* fix: change the log of collecting for dataset

* fix: add collection task selecting while creating and updating dataset

* fix: set the umask value to 0022 for java process
2026-01-04 19:05:08 +08:00
o0Shark0o
cbed6fbcd7 Revert "Merge branch 'main' of https://github.com/ModelEngine-Group/DataMate"
This reverts commit a12f4c90a5, reversing
changes made to 34f08df86b.
2025-12-31 16:19:19 +08:00
o0Shark0o
a12f4c90a5 Merge branch 'main' of https://github.com/ModelEngine-Group/DataMate 2025-12-31 16:04:05 +08:00
o0Shark0o
34f08df86b Add lucide icons and colors for video, audio and multimodal operators. 2025-12-31 16:03:57 +08:00
hefanli
3a874fe699 fix: fix the collection for nfs (#218)
* fix: remove the datax-builder for the Backend Image

* fix: fix the collection for nfs
2025-12-31 15:56:01 +08:00
Kecheng Sha
01e1c6c7e9 polish(operator cards): improve icon color distinction and subtle UI … (#217)
polish(operator cards): improve icon color distinction and subtle UI details
2025-12-31 10:54:34 +08:00
hhhhsc701
6a1eb85e8e feat: 支持运行data-juicer算子 (#215)
* feature: 增加data-juicer算子

* feat: 支持运行data-juicer算子

* feat: 支持data-juicer任务下发

* feat: 支持data-juicer结果数据集归档

* feat: 支持data-juicer结果数据集归档
2025-12-31 09:20:41 +08:00
hefanli
63f4e3e447 refactor: modify data collection to python implementation (#214)
* feature: LabelStudio jumps without login

* refactor: modify data collection to python implementation

* refactor: modify data collection to python implementation

* refactor: modify data collection to python implementation

* refactor: modify data collection to python implementation

* refactor: modify data collection to python implementation

* refactor: modify data collection to python implementation

* fix: remove terrabase dependency

* feature: add the collection task executions page and the collection template page

* fix: fix the collection task creation

* fix: fix the collection task creation
2025-12-30 18:48:43 +08:00
Kecheng Sha
e22f16166c fix: reset pagination when switching operator market category filters (#205) 2025-12-29 15:16:33 +08:00
Kecheng Sha
0df7a872e4 Revert "feat: fix the problem in the Operator Market frontend pages" 2025-12-29 12:00:37 +08:00
root
844add27ea feat: fix the problem in the Operator Market frontend pages 2025-12-29 11:38:47 +08:00
hefanli
29e4a333a9 feature: LabelStudio jumps without login (#201) 2025-12-25 16:49:06 +08:00
hhhhsc701
6d61348388 feat: deer-flow通过gateway转发 (#193) 2025-12-23 11:35:45 +08:00
hhhhsc701
d82bff441a fix: prevent deletion of predefined operators and improve error handling (#192)
* fix: prevent deletion of predefined operators and improve error handling

* fix: prevent deletion of predefined operators and improve error handling
2025-12-22 19:30:41 +08:00
hhhhsc701
46f4a8c219 feat: add download functionality for example operator and update Dock… (#188)
* feat: add download functionality for example operator and update Dockerfile

* feat: enhance download response by exposing content disposition header

* feat: update download function to accept filename parameter for example operator
2025-12-22 15:39:32 +08:00
Dallas98
e0e9b1d94d feat:问题生成过程优化及COT数据生成优化 (#169)
* fix(chart): update Helm chart helpers and values for improved configuration

* feat(SynthesisTaskTab): enhance task table with tooltip support and improved column widths

* feat(CreateTask, SynthFileTask): improve task creation and detail view with enhanced payload handling and UI updates

* feat(SynthFileTask): enhance file display with progress tracking and delete action

* feat(SynthFileTask): enhance file display with progress tracking and delete action

* feat(SynthDataDetail): add delete action for chunks with confirmation prompt

* feat(SynthDataDetail): update edit and delete buttons to icon-only format

* feat(SynthDataDetail): add confirmation modals for chunk and synthesis data deletion

* feat(DocumentSplitter): add enhanced document splitting functionality with CJK support and metadata detection

* feat(DataSynthesis): refactor data synthesis models and update task handling logic

* feat(DataSynthesis): streamline synthesis task handling and enhance chunk processing logic

* feat(DataSynthesis): refactor data synthesis models and update task handling logic

* fix(generation_service): ensure processed chunks are incremented regardless of question generation success

* feat(CreateTask): enhance task creation with new synthesis templates and improved configuration options

* feat(CreateTask): enhance task creation with new synthesis templates and improved configuration options

* feat(CreateTask): enhance task creation with new synthesis templates and improved configuration options

* feat(CreateTask): enhance task creation with new synthesis templates and improved configuration options
2025-12-18 16:51:18 +08:00
hefanli
082aca1597 fix: the interface for querying data set files is compatible with ret… (#171)
fix: the interface for querying data set files is compatible with returns in file system format and list returns.
2025-12-16 11:31:52 +08:00
Dallas98
ec87e4f204 feat(frontend): 增强Synthesis Data Detail页面UX体验 (#163)
* fix(chart): update Helm chart helpers and values for improved configuration

* feat(SynthesisTaskTab): enhance task table with tooltip support and improved column widths

* feat(CreateTask, SynthFileTask): improve task creation and detail view with enhanced payload handling and UI updates

* feat(SynthFileTask): enhance file display with progress tracking and delete action

* feat(SynthFileTask): enhance file display with progress tracking and delete action

* feat(SynthDataDetail): add delete action for chunks with confirmation prompt

* feat(SynthDataDetail): update edit and delete buttons to icon-only format

* feat(SynthDataDetail): add confirmation modals for chunk and synthesis data deletion
2025-12-11 21:02:44 +08:00
o0Shark0o
12529276ee fix(settings): improve ModelAccess table responsiveness during browser zoom 2025-12-11 14:53:01 +08:00
Dallas98
2f3ae21f8a feat: enhance dataset file fetching with improved pagination and document loading support (#156) 2025-12-10 22:39:24 +08:00
Dallas98
e9fd6a3ae1 fix: adjust pagination logic in dataset fetching to start from the current page 2025-12-10 19:52:06 +08:00
hefanli
99fd46cb70 fix: fix the Data Evaluation Detail page (#154)
* fix: the Data Evaluation Detail Page should show the model used

* fix: fix the time format displayed

* fix: fix the Data Evaluation Detail page
2025-12-10 18:35:29 +08:00
hefanli
f87060490c feature: data management supports nested folders (#150)
* fix: k8s部署场景下,backend-python服务挂载需要存储

* fix: 增加数据集文件免拷贝的接口定义

* fix: 评估时评估结果赋予初始空值,防止未评估完成时接口报错

* feature: 数据管理支持嵌套文件夹(展示时按照文件系统展示;批量下载时带上相对路径)

* fix: 去除多余的文件重命名逻辑

* refactor: remove unused imports
2025-12-10 16:42:45 +08:00