8fdc7d99b8
feat(docker): 优化 Dockerfile 支持弱网环境缓存
...
- 使用缓存挂载 DataX 源码，避免重复克隆提高构建效率
- 添加 NLTK 数据缓存挂载并增加失败检查机制
- 实现 PaddleOCR 模型下载缓存，支持离线重用
- 集成 spaCy 模型缓存机制，提升安装稳定性
- 优化构建流程适配弱网环境下的依赖下载
2026-01-31 14:31:47 +08:00
283f4af094
chore(gateway): 移除Dockerfile中的离线模式参数
...
- 移除了mvn命令中的-o参数，允许在线下载依赖包
- 优化了编译过程以支持网络连接场景下的依赖管理
2026-01-30 14:13:16 +08:00
adb90bbf10
mirror
2026-01-09 09:28:02 +08:00
a98cdae209
mirror
2026-01-09 08:51:58 +08:00
f3f1609455
mirror
2026-01-09 08:46:01 +08:00
Kecheng Sha
3f1ad6a872
feat(auto-annotation): integrate YOLO auto-labeling and enhance data management ( #223 )
...
* feat(auto-annotation): initial setup
* chore: remove package-lock.json
* chore: 清理本地测试脚本与 Maven 设置
* chore: change package-lock.json
2026-01-05 14:22:44 +08:00
hhhhsc701
6a1eb85e8e
feat: 支持运行data-juicer算子 ( #215 )
...
* feature: 增加data-juicer算子
* feat: 支持运行data-juicer算子
* feat: 支持data-juicer任务下发
* feat: 支持data-juicer结果数据集归档
* feat: 支持data-juicer结果数据集归档
2025-12-31 09:20:41 +08:00
hhhhsc701
ab4523b556
add export type settings and enhance metadata structure ( #181 )
...
* fix(session): enhance database connection settings with pool pre-ping and recycle options
* feat(metadata): add export type settings and enhance metadata structure
* fix(base_op): improve sample handling by introducing target_type key and consolidating text/data retrieval logic
* feat(metadata): add export type settings and enhance metadata structure
* feat(metadata): add export type settings and enhance metadata structure
2025-12-19 11:54:08 +08:00
hhhhsc701
be875086db
feat: add operator-packages-volume to docker-compose and update Docke… ( #179 )
...
* feat: add operator-packages-volume to docker-compose and update Dockerfile for site-packages path
* feat: add retry
2025-12-18 20:32:42 +08:00
hhhhsc701
3b4f8488e8
fix: update Dockerfile to improve pip installation process and remove unnecessary uninstalls ( #173 )
...
* feature: unstructured支持简单pdf处理
* feature: update values.yaml to enhance ray-cluster configuration with security context, environment variables, and resource limits
* fix: update Dockerfile to improve pip installation process and remove unnecessary uninstalls
2025-12-17 11:49:47 +08:00
hhhhsc701
62b91b6deb
bugfix: update values.yaml to enhance ray-cluster configuration with security context, environment variables, and resource limits ( #172 )
...
* feature: unstructured支持简单pdf处理
* feature: update values.yaml to enhance ray-cluster configuration with security context, environment variables, and resource limits
2025-12-17 10:41:13 +08:00
hhhhsc701
19a04df276
feature: 增加水印去除/高级匿名化算子 ( #151 )
...
* feature: 增加水印去除算子
* feature: clean code
* feature: clean code
* feature: 增加高级匿名化算子
2025-12-10 18:12:47 +08:00
hhhhsc701
d59c167da4
算子将抽取与落盘固定到流程中 ( #134 )
...
* feature: 将抽取动作移到每一个算子中
* feature: 落盘算子改为默认执行
* feature: 优化前端展示
* feature: 使用pyproject管理依赖
2025-12-05 17:26:29 +08:00
hhhhsc701
6bbde0ec56
feature: 清洗任务详情页 ( #73 )
...
* feature: 清洗任务详情
* fix: 取消构建镜像，改为直接拉取
* fix: 增加清洗任务详情页
* fix: 增加清洗任务详情页
* fix: 算子列表可点击
* fix: 模板详情和更新
2025-11-12 18:00:19 +08:00
hhhhsc701
05b26a2981
feature: 更新算子名称；增加创建任务、模板校验 ( #57 )
...
* feature: 更新算子名称；增加创建任务、模板校验
* feature: 镜像构建增加缓存
2025-11-05 17:38:03 +08:00
hhhhsc701
f3958f08d9
feature: 对接deer-flow ( #54 )
...
feature: 对接deer-flow
2025-11-04 20:30:40 +08:00
Startalker
06b05a65a9
feature: add unstructured xlsx/xls/csv/pptx/ppt ( #41 )
...
* feature: add UnstructuredFormatter
* feature: add UnstructuredFormatter in db
* feature: add unstructured[docx]==0.18.15
* feature: support doc
* feature: add mineru
* feature: add external pdf extract operator by using mineru
* feature: mineru docker install bugfix
* feature: add unstructured xlsx/xls/csv/pptx/ppt
---------
Co-authored-by: Startalker <438747480@qq.com >
2025-10-30 20:21:12 +08:00
hhhhsc701
b9b97c1ac2
Develop op ( #35 )
...
* refactor: enhance CleaningTaskService and related components with validation and repository updates
* feature: 支持算子上传创建
2025-10-30 17:17:00 +08:00
hhhhsc
4f5a9a9a83
refactor: simplify Dockerfile by removing redundant mirror configurations and cleaning up package installation commands
2025-10-28 16:24:40 +08:00
hhhhsc
a69b9f4921
feature: 对接deer-flow
2025-10-28 10:54:29 +08:00
Startalker
f86d4fae25
feature: add unstructured formatter operator for doc/docx ( #17 )
...
* feature: add UnstructuredFormatter
* feature: add UnstructuredFormatter in db
* feature: add unstructured[docx]==0.18.15
* feature: support doc
---------
Co-authored-by: Startalker <438747480@qq.com >
2025-10-23 16:49:03 +08:00
hhhhsc701
31ef8bc265
[Feature] Refactor project to use 'datamate' naming convention for services and configurations ( #14 )
...
* Enhance CleaningTaskService to track cleaning process progress and update ExecutorType to DATAMATE
* Refactor project to use 'datamate' naming convention for services and configurations
2025-10-22 17:53:16 +08:00
Dallas98
1c97afed7d
init datamate
2025-10-21 23:00:48 +08:00
