Jerry Yan
3a9afe3480
核心功能： - 实体/关系编辑表单（创建/更新/删除） - 批量操作（批量删除节点/边） - 审核流程（提交审核 → 待审核列表 → 通过/拒绝） - 编辑模式切换（查看/编辑模式） - 权限控制（knowledgeGraphWrite 权限） 新增文件（后端，9 个）： - EditReview.java - 审核记录领域模型（Neo4j 节点） - EditReviewRepository.java - 审核记录仓储（CRUD + 分页查询） - EditReviewService.java - 审核业务服务（提交/通过/拒绝，通过时自动执行变更） - EditReviewController.java - REST API（POST submit, POST approve/reject, GET pending） - DTOs: SubmitReviewRequest, EditReviewVO, ReviewActionRequest, BatchDeleteRequest - EditReviewServiceTest.java - 单元测试（21 tests） - EditReviewControllerTest.java - 集成测试（10 tests） 新增文件（前端，3 个）： - EntityEditForm.tsx - 实体创建/编辑表单（Modal，支持名称/类型/描述/别名/置信度） - RelationEditForm.tsx - 关系创建/编辑表单（Modal，支持源/目标实体搜索、关系类型、权重/置信度） - ReviewPanel.tsx - 审核面板（待审核列表，通过/拒绝操作，拒绝带备注） 修改文件（后端，7 个）： - GraphEntityService.java - 新增 batchDeleteEntities()，updateEntity 支持 confidence - GraphRelationService.java - 新增 batchDeleteRelations() - GraphEntityController.java - 删除批量删除端点（改为审核流程） - GraphRelationController.java - 删除批量删除端点（改为审核流程） - UpdateEntityRequest.java - 添加 confidence 字段 - KnowledgeGraphErrorCode.java - 新增 REVIEW_NOT_FOUND、REVIEW_ALREADY_PROCESSED - PermissionRuleMatcher.java - 添加 /api/knowledge-graph/** 写操作权限规则 修改文件（前端，8 个）： - knowledge-graph.model.ts - 新增 EditReviewVO、ReviewOperationType、ReviewStatus 类型 - knowledge-graph.api.ts - BASE 改为 /api/knowledge-graph（走网关权限链），新增审核相关 API，删除批量删除直删方法 - vite.config.ts - 更新 dev proxy 路径 - NodeDetail.tsx - 新增 editMode 属性，编辑模式下显示编辑/删除按钮 - RelationDetail.tsx - 新增 editMode 属性，编辑模式下显示编辑/删除按钮 - KnowledgeGraphPage.tsx - 新增编辑模式开关（需要 knowledgeGraphWrite 权限）、创建实体/关系工具栏按钮、审核 Tab、批量操作 - GraphCanvas.tsx - 支持多选（editMode 时）、onSelectionChange 回调 - graphConfig.ts - 支持 multiSelect 参数 审核流程： - 所有编辑操作（创建/更新/删除/批量删除）都通过 submitReview 提交审核 - 审核通过后，EditReviewService.applyChange() 自动执行变更 - 批量删除端点已删除，只能通过审核流程 权限控制： - API 路径从 /knowledge-graph 改为 /api/knowledge-graph，走网关权限链 - 编辑模式开关需要 knowledgeGraphWrite 权限 - PermissionRuleMatcher 添加 /api/knowledge-graph/** 写操作规则 Bug 修复（Codex 审查后修复）： - P0: 权限绕过（API 路径改为 /api/knowledge-graph） - P1: 审核流程未接入（所有编辑操作改为 submitReview） - P1: 批量删除绕过审核（删除直删端点，改为审核流程） - P1: confidence 字段丢失（UpdateEntityRequest 添加 confidence） - P2: 审核提交校验不足（添加跨字段校验器） - P2: 批量删除安全（添加 @Size(max=100) 限制，收集失败 ID） - P2: 前端错误处理（分开处理表单校验和 API 失败） 测试结果： - 后端: 311 tests pass ✅ (280 → 311, +31 new) - 前端: eslint clean ✅, tsc clean ✅, vite build success ✅
DataMate All-in-One Data Work Platform
DataMate is an enterprise-level data processing platform for model fine-tuning and RAG retrieval, supporting core functions such as data collection, data management, operator marketplace, data cleaning, data synthesis, data annotation, data evaluation, and knowledge generation.
If you like this project, please give it a Star⭐️!
🌟 Core Features
- Core Modules: Data Collection, Data Management, Operator Marketplace, Data Cleaning, Data Synthesis, Data Annotation, Data Evaluation, Knowledge Generation.
- Visual Orchestration: Drag-and-drop data processing workflow design.
- Operator Ecosystem: Rich built-in operators and support for custom operators.
🚀 Quick Start
Prerequisites
- Git (for pulling source code)
- Make (for building and installing)
- Docker (for building images and deploying services)
- Docker-Compose (for service deployment - Docker method)
- Kubernetes (for service deployment - k8s method)
- Helm (for service deployment - k8s method)
This project supports deployment via two methods: docker-compose and helm. After executing the command, please enter the corresponding number for the deployment method. The command echo is as follows:
Choose a deployment method:
1. Docker/Docker-Compose
2. Kubernetes/Helm
Enter choice:
Clone the Code
git clone git@github.com:ModelEngine-Group/DataMate.git
cd DataMate
Deploy the basic services
make install
If the machine you are using does not have make installed, please run the following command to deploy it:
# Windows
set REGISTRY=ghcr.io/modelengine-group/
docker compose -f ./deployment/docker/datamate/docker-compose.yml up -d
docker compose -f ./deployment/docker/milvus/docker-compose.yml up -d
# Linux/Mac
export REGISTRY=ghcr.io/modelengine-group/
docker compose -f ./deployment/docker/datamate/docker-compose.yml up -d
docker compose -f ./deployment/docker/milvus/docker-compose.yml up -d
Once the container is running, access http://localhost:30000 in a browser to view the front-end interface.
To list all available Make targets, flags and help text, run:
make help
Build and deploy Mineru Enhanced PDF Processing
make build-mineru
make install-mineru
Deploy the DeerFlow service
make install-deer-flow
Local Development and Deployment
After modifying the local code, please execute the following commands to build the image and deploy using the local image.
make build
make install dev=true
Uninstall
make uninstall
When running make uninstall, the installer will prompt once whether to delete volumes; that single choice is applied to all components. The uninstall order is: milvus -> label-studio -> datamate, which ensures the datamate network is removed cleanly after services that use it have stopped.
🤝 Contribution Guidelines
Thank you for your interest in this project! We warmly welcome contributions from the community. Whether it's submitting bug reports, suggesting new features, or directly participating in code development, all forms of help make the project better.
• 📮 GitHub Issues: Submit bugs or feature suggestions.
• 🔧 GitHub Pull Requests: Contribute code improvements.
📄 License
DataMate is open source under the MIT license. You are free to use, modify, and distribute the code of this project in compliance with the license terms.