7d4dcb756b
fix: 修复入库可能重复;筛选逻辑优化 ( #226 )
...
* 修改数据清洗筛选逻辑-筛选修改为多选
* 修改数据清洗筛选逻辑-筛选修改为多选
* antd 组件库样式定制修改
* fix: 修复入库可能重复
* fix: 算子市场筛选逻辑优化
* fix: 清洗任务创建筛选逻辑优化
* fix: 清洗任务创建筛选逻辑优化
---------
Co-authored-by: chase <byzhangxin11@126.com >
2026-01-06 17:57:25 +08:00
hhhhsc701 and GitHub
8d61eb28c3
fix: ray部署修改 ( #221 )
2026-01-04 09:42:25 +08:00
hhhhsc701 and GitHub
f183b9f2f3
feat: 算子上传适配 ( #216 )
2025-12-31 10:30:32 +08:00
hhhhsc701 and GitHub
6a1eb85e8e
feat: 支持运行data-juicer算子 ( #215 )
...
* feature: 增加data-juicer算子
* feat: 支持运行data-juicer算子
* feat: 支持data-juicer任务下发
* feat: 支持data-juicer结果数据集归档
* feat: 支持data-juicer结果数据集归档
2025-12-31 09:20:41 +08:00
hhhhsc701 and GitHub
80d4dfd285
feat: 修复清洗任务创建 ( #207 )
2025-12-30 14:41:39 +08:00
hhhhsc701 and GitHub
87e73d3bf7
feat: label-studio支持指定sc ( #200 )
...
* feat: label-studio构建脚本
* feat: label-studio构建脚本
* feat: label-studio构建脚本
* feat: label-studio安装脚本
* feat: label-studio支持指定sc
2025-12-25 16:13:38 +08:00
hhhhsc701 and GitHub
7e842c7cd5
feat: label-studio构建脚本 ( #198 )
...
* feat: label-studio构建脚本
* feat: label-studio构建脚本
* feat: label-studio构建脚本
* feat: label-studio安装脚本
2025-12-25 11:44:05 +08:00
hhhhsc701 and GitHub
1c507ac98a
feat: 支持npu自动扩缩容 ( #197 )
...
* feat: npu动态调度
* feat: 数据集分页优化
* feat: 支持npu自动扩缩容
* feat: 支持npu自动扩缩容
* feat: 支持npu自动扩缩容
* feat: clean code
2025-12-24 18:03:30 +08:00
hhhhsc701 and GitHub
6d61348388
feat: deer-flow通过gateway转发 ( #193 )
2025-12-23 11:35:45 +08:00
hhhhsc701 and GitHub
d82bff441a
fix: prevent deletion of predefined operators and improve error handling ( #192 )
...
* fix: prevent deletion of predefined operators and improve error handling
* fix: prevent deletion of predefined operators and improve error handling
2025-12-22 19:30:41 +08:00
hhhhsc701 and GitHub
46f4a8c219
feat: add download functionality for example operator and update Dock… ( #188 )
...
* feat: add download functionality for example operator and update Dockerfile
* feat: enhance download response by exposing content disposition header
* feat: update download function to accept filename parameter for example operator
2025-12-22 15:39:32 +08:00
hhhhsc701 and GitHub
ab4523b556
add export type settings and enhance metadata structure ( #181 )
...
* fix(session): enhance database connection settings with pool pre-ping and recycle options
* feat(metadata): add export type settings and enhance metadata structure
* fix(base_op): improve sample handling by introducing target_type key and consolidating text/data retrieval logic
* feat(metadata): add export type settings and enhance metadata structure
* feat(metadata): add export type settings and enhance metadata structure
2025-12-19 11:54:08 +08:00
hhhhsc701 and GitHub
be875086db
feat: add operator-packages-volume to docker-compose and update Docke… ( #179 )
...
* feat: add operator-packages-volume to docker-compose and update Dockerfile for site-packages path
* feat: add retry
2025-12-18 20:32:42 +08:00
hhhhsc701 and GitHub
761f7f6a51
fix: optimize PDF parsing by implementing concurrent processing with … ( #177 )
...
* fix: optimize PDF parsing by implementing concurrent processing with ThreadPoolExecutor
* Refactor to async processing for file extraction
Refactor the file processing to use asyncio for improved performance and concurrency.
2025-12-18 15:28:30 +08:00
hhhhsc701 and GitHub
12ade8bc7b
fix: streamline Dockerfile by removing redundant mirror configuration… ( #175 )
...
fix: streamline Dockerfile by removing redundant mirror configurations and simplifying package installation
2025-12-17 16:34:41 +08:00
hhhhsc701 and GitHub
924d977d6f
支持mineru npu处理 ( #174 )
...
* feature: unstructured支持简单pdf处理
* feature: update values.yaml to enhance ray-cluster configuration with security context, environment variables, and resource limits
* feature: update deploy.yaml and process.py for mineru server configuration and PDF processing enhancements
* feature: update deploy.yaml and process.py for mineru server configuration and PDF processing enhancements
* feature: improve PDF processing logic and update dependencies in process.py and pyproject.toml
* feature: improve PDF processing logic and update dependencies in process.py and pyproject.toml
* feature: update Dockerfile for improved package source mirrors and add mineru-npu to build targets
2025-12-17 16:31:06 +08:00
hhhhsc701 and GitHub
3b4f8488e8
fix: update Dockerfile to improve pip installation process and remove unnecessary uninstalls ( #173 )
...
* feature: unstructured支持简单pdf处理
* feature: update values.yaml to enhance ray-cluster configuration with security context, environment variables, and resource limits
* fix: update Dockerfile to improve pip installation process and remove unnecessary uninstalls
2025-12-17 11:49:47 +08:00
hhhhsc701 and GitHub
62b91b6deb
bugfix: update values.yaml to enhance ray-cluster configuration with security context, environment variables, and resource limits ( #172 )
...
* feature: unstructured支持简单pdf处理
* feature: update values.yaml to enhance ray-cluster configuration with security context, environment variables, and resource limits
2025-12-17 10:41:13 +08:00
hhhhsc701 and GitHub
c51058a867
feature: 商业版构建 ( #166 )
2025-12-12 16:19:46 +08:00
hhhhsc701 and GitHub
d8c0b0ed73
补充modal范围 ( #165 )
2025-12-12 13:34:03 +08:00
hhhhsc701 and GitHub
fc9fb07e77
bugfix ( #164 )
2025-12-11 23:17:01 +08:00
hhhhsc701 and GitHub
bb8641bea2
docs: update README files to include instructions for accessing the f… ( #161 )
...
docs: update README files to include instructions for accessing the front-end interface
2025-12-11 16:15:23 +08:00
hhhhsc701 and GitHub
72669d1293
feat: add .env and conf.yaml for deer-flow configuration ( #160 )
...
* fix: update MILVUS_URI in .env.example for correct service endpoint
* feat: add .env and conf.yaml for deer-flow configuration
2025-12-11 14:33:06 +08:00
hhhhsc701 and GitHub
a6e82ce68b
fix: update MILVUS_URI in .env.example for correct service endpoint ( #159 )
2025-12-11 14:17:02 +08:00
hhhhsc701 and GitHub
f69ed6b8aa
Revert "feature: 增加data-juicer算子" ( #158 )
...
Revert "feature: 增加data-juicer算子 (#157 )"
This reverts commit 786f13f9c3
2025-12-11 10:32:53 +08:00
hhhhsc701 and GitHub
786f13f9c3
feature: 增加data-juicer算子 ( #157 )
2025-12-11 10:32:19 +08:00
hhhhsc701 and GitHub
19a04df276
feature: 增加水印去除/高级匿名化算子 ( #151 )
...
* feature: 增加水印去除算子
* feature: clean code
* feature: clean code
* feature: 增加高级匿名化算子
2025-12-10 18:12:47 +08:00
hhhhsc701 and GitHub
103c21945d
修复部分功能 ( #138 )
...
* feature: 版本统一
* feature: 定时同步时默认值展示异常，增加提示
* feature: 修复数据归集搜索
* feature: 优化标注模板查询
* feature: 屏蔽webhook功能
2025-12-10 14:31:05 +08:00
hhhhsc701 and GitHub
d59c167da4
算子将抽取与落盘固定到流程中 ( #134 )
...
* feature: 将抽取动作移到每一个算子中
* feature: 落盘算子改为默认执行
* feature: 优化前端展示
* feature: 使用pyproject管理依赖
2025-12-05 17:26:29 +08:00
hhhhsc701 and GitHub
7a9530c1e3
feature: 增加对redis未部署时异常捕获 ( #131 )
...
* feature: 增加download-deer-flow
* feature: 增加对redis未部署时异常捕获
* feature: clean code
2025-12-04 16:09:29 +08:00
hhhhsc701 and GitHub
265e284fb8
feature: 修改算子开发指南 ( #127 )
2025-12-03 17:45:08 +08:00
hhhhsc701 and GitHub
c22683d635
优化部分问题 ( #126 )
...
* feature: 支持相对路径引用
* feature: 优化本地部署命令
* feature: 优化算子编排展示
* feature: 优化清洗任务失败后重试
2025-12-03 16:41:48 +08:00
hhhhsc701 and GitHub
b5fa8af900
bugfix: 修复deer-flow部署 ( #124 )
2025-12-02 19:23:30 +08:00
hhhhsc701 and GitHub
f730bd5b0c
bugfix: 支持使用runtime单实例 ( #118 )
...
* bugfix: 支持使用runtime单实例
2025-12-01 19:05:50 +08:00
hhhhsc701 and GitHub
bb3345268e
bugfix: 清洗/算子支持名称/描述搜索 ( #116 )
...
* bugfix: milvus适配etcd deploy部署
* bugfix: 可以在知识库界面跳转到创建模型
2025-11-29 18:15:43 +08:00
hhhhsc701 and GitHub
fe42b03548
bugfix: milvus部分组件支持镜像仓库 ( #114 )
2025-11-28 17:39:56 +08:00
hhhhsc701 and GitHub
07029d07ff
优化清洗重试机制，优化清洗进度展示，修复模板无法展示参数 ( #113 )
...
* bugfix: 模板无法展示参数
* bugfix: 优化清洗进度展示
* bugfix: 优化清洗重试机制
2025-11-28 15:28:10 +08:00
hhhhsc701 and GitHub
f1bffdcd61
bugfix: 创建清洗任务时修改数据集状态；无法删除已在模板/运行任务的算子
...
* bugfix: 创建清洗任务时修改数据集状态；无法删除已在模板/运行任务的算子
2025-11-27 17:34:53 +08:00
hhhhsc701 and GitHub
91390cace0
feature: 北向接口：支持通过模板创建清洗任务 ( #111 )
...
feature: 北向接口：支持通过模板创建清洗任务
2025-11-26 17:30:54 +08:00
hhhhsc701 and GitHub
bc2f57f2c0
feature: 修改缩进 ( #109 )
2025-11-26 11:24:50 +08:00
hhhhsc701 and GitHub
b50c12d135
feature: 删除kuberay镜像推送 ( #107 )
...
feature: 删除kuberay镜像推送
2025-11-25 17:46:50 +08:00
hhhhsc701 and GitHub
af2a01e52d
feature: milvus pvc支持本地目录 ( #105 )
...
feature: milvus pvc支持本地目录
2025-11-25 16:54:24 +08:00
hhhhsc701 and GitHub
fb399b74cf
feature: pvc支持本地盘+配置sc ( #104 )
2025-11-24 17:29:32 +08:00
hhhhsc701 and GitHub
497a5688e9
feature: df构建双架构镜像 ( #102 )
2025-11-24 11:52:08 +08:00
hhhhsc701 and GitHub
a53f6776b8
feature: 构建双架构镜像 ( #101 )
...
feature: 构建双架构镜像
2025-11-24 11:34:53 +08:00
hhhhsc701 and GitHub
536ef9f556
feature: milvus service名称变更 兼容k8s ( #97 )
...
feature: milvus service名称变更 兼容k8s (#97 )
2025-11-21 12:06:53 +08:00
hhhhsc701 and GitHub
d9e163c163
Develop deer flow ( #85 )
...
* fix: deer-flow支持从datamate获取搜索引擎
2025-11-14 17:36:55 +08:00
hhhhsc701 and GitHub
5cef9cb273
feature: deer-flow支持从datamate获取外部接入模型 ( #83 )
...
* feature: deer-flow支持从datamate获取外部接入模型
2025-11-13 20:13:16 +08:00
hhhhsc701 and GitHub
6bbde0ec56
feature: 清洗任务详情页 ( #73 )
...
* feature: 清洗任务详情
* fix: 取消构建镜像，改为直接拉取
* fix: 增加清洗任务详情页
* fix: 增加清洗任务详情页
* fix: 算子列表可点击
* fix: 模板详情和更新
2025-11-12 18:00:19 +08:00
hhhhsc701 and GitHub
451d3c8207
fix: 归档包流水线 ( #71 )
2025-11-10 20:34:35 +08:00
hhhhsc701 and GitHub
9dd26d622f
feature: 数据库镜像制作 ( #70 )
...
* feature: 数据库镜像制作
* feature: 增加归档包流水线
2025-11-10 19:06:53 +08:00
hhhhsc701 and GitHub
28b7c631a4
feature: 修改部署文档 ( #65 )
2025-11-07 17:19:00 +08:00
hhhhsc701 and GitHub
2138ba23c7
feature: 增加算子详情页；优化算子上传更新逻辑 ( #64 )
...
* feature: 增加算子详情页；优化算子上传更新逻辑
2025-11-07 16:54:00 +08:00
hhhhsc701 and GitHub
f78475e29f
Develop hsc ( #58 )
...
feature: 优化镜像构建/部署
2025-11-06 17:14:54 +08:00
hhhhsc701 and GitHub
05b26a2981
feature: 更新算子名称；增加创建任务、模板校验 ( #57 )
...
* feature: 更新算子名称；增加创建任务、模板校验
* feature: 镜像构建增加缓存
2025-11-05 17:38:03 +08:00
hhhhsc701 and GitHub
6917ae5b30
feature: 修复deer-flow流水线 ( #56 )
...
feature: 修复deer-flow流水线
2025-11-05 10:42:43 +08:00
hhhhsc701 and GitHub
f3958f08d9
feature: 对接deer-flow ( #54 )
...
feature: 对接deer-flow
2025-11-04 20:30:40 +08:00
hhhhsc701 and GitHub
d0bac68d3f
bugfix: sql指定数据库 ( #45 )
2025-10-31 10:41:19 +08:00
hhhhsc701 and GitHub
ba6caabbc6
前端增加对deer-flow服务的检测 ( #42 )
...
前端增加对deer-flow服务的检测 (#42 )
2025-10-30 20:21:25 +08:00
hhhhsc701 and GitHub
b9b97c1ac2
Develop op ( #35 )
...
* refactor: enhance CleaningTaskService and related components with validation and repository updates
* feature: 支持算子上传创建
2025-10-30 17:17:00 +08:00
hhhhsc701 and GitHub
67eb571d8d
feature: 对接deer-flow ( #27 )
...
feature: 对接deer-flow
2025-10-28 16:28:26 +08:00
hhhhsc701 and GitHub
871ba5758d
Merge pull request #23 from ModelEngine-Group/develop_op
...
refactor: update service and repository structure to use DTOs and imp…
2025-10-24 17:56:32 +08:00
hhhhsc701 and GitHub
f9dbefd737
Merge pull request #21 from ModelEngine-Group/develop_db
...
refactor: rename and reorganize data models and repositories for clarity
2025-10-24 15:46:32 +08:00
hhhhsc701 and GitHub
d58c2a0ac7
Merge pull request #18 from ModelEngine-Group/develop_doc
...
refactor: reorganize Helm chart structure and update service configur…
2025-10-23 17:17:05 +08:00
hhhhsc701 and GitHub
31ef8bc265
[Feature] Refactor project to use 'datamate' naming convention for services and configurations ( #14 )
...
* Enhance CleaningTaskService to track cleaning process progress and update ExecutorType to DATAMATE
* Refactor project to use 'datamate' naming convention for services and configurations
2025-10-22 17:53:16 +08:00
