Commit Graph

16 Commits

Author SHA1 Message Date
Jason Wang
c097a06fbe feat: Improve makefile readability, Add user control on volume keep at uninstallation, Add Label Studio install and uninstall via Make. (#106)
* feature: Add label studio installation and uninstallation commands to Makefile

* feature: Enhance Makefile with detailed help commands and improve install/uninstall targets for services

* feature: Update Makefile help commands to clarify usage of local images

* feature: Improve error handling in Makefile for build, install, and uninstall targets

* feature: Enhance uninstall process in Makefile to prompt for volume deletion and update README with usage details

---------

Co-authored-by: Jason Wang <wjl_jason@qq.com>
2025-11-25 17:37:28 +08:00
hhhhsc701
a53f6776b8 feature: 构建双架构镜像 (#101)
feature: 构建双架构镜像
2025-11-24 11:34:53 +08:00
hhhhsc701
5cef9cb273 feature: deer-flow支持从datamate获取外部接入模型 (#83)
* feature: deer-flow支持从datamate获取外部接入模型
2025-11-13 20:13:16 +08:00
Jason Wang
960323f41c fix: Restore database-docker-build to build target (#81)
Re-adds the 'database-docker-build' step to the build process. This target was accidentally removed in PR #79.
2025-11-13 16:25:42 +08:00
Jason Wang
45743f39f5 feat: add labeling template. refactor: switch to Poetry, build and deploy of backend Python (#79)
* feat: Enhance annotation module with template management and validation

- Added DatasetMappingCreateRequest and DatasetMappingUpdateRequest schemas to handle dataset mapping requests with camelCase and snake_case support.
- Introduced Annotation Template schemas including CreateAnnotationTemplateRequest, UpdateAnnotationTemplateRequest, and AnnotationTemplateResponse for managing annotation templates.
- Implemented AnnotationTemplateService for creating, updating, retrieving, and deleting annotation templates, including validation of configurations and XML generation.
- Added utility class LabelStudioConfigValidator for validating Label Studio configurations and XML formats.
- Updated database schema for annotation templates and labeling projects to include new fields and constraints.
- Seeded initial annotation templates for various use cases including image classification, object detection, and text classification.

* feat: Enhance TemplateForm with improved validation and dynamic field rendering; update LabelStudio config validation for camelCase support

* feat: Update docker-compose.yml to mark datamate dataset volume and network as external

* feat: Add tag configuration management and related components

- Introduced new components for tag selection and browsing in the frontend.
- Added API endpoint to fetch tag configuration from the backend.
- Implemented tag configuration management in the backend, including loading from YAML.
- Enhanced template service to support dynamic tag rendering based on configuration.
- Updated validation utilities to incorporate tag configuration checks.
- Refactored existing code to utilize the new tag configuration structure.

* feat: Refactor LabelStudioTagConfig for improved configuration loading and validation

* feat: Update Makefile to include backend-python-docker-build in the build process

* feat: Migrate to poetry for better deps management

* Add pyyaml dependency and update Dockerfile to use Poetry for dependency management

- Added pyyaml (>=6.0.3,<7.0.0) to pyproject.toml dependencies.
- Updated Dockerfile to install Poetry and manage dependencies using it.
- Improved layer caching by copying only dependency files before the application code.
- Removed unnecessary installation of build dependencies to keep the final image size small.

* feat: Remove duplicated backend-python-docker-build target from Makefile

* fix: airflow is not ready for adding yet

* feat: update Python version to 3.12 and remove project installation step in Dockerfile
2025-11-13 15:32:30 +08:00
hhhhsc701
9dd26d622f feature: 数据库镜像制作 (#70)
* feature: 数据库镜像制作

* feature: 增加归档包流水线
2025-11-10 19:06:53 +08:00
hhhhsc701
f78475e29f Develop hsc (#58)
feature: 优化镜像构建/部署
2025-11-06 17:14:54 +08:00
hhhhsc701
f3958f08d9 feature: 对接deer-flow (#54)
feature: 对接deer-flow
2025-11-04 20:30:40 +08:00
hhhhsc701
b9b97c1ac2 Develop op (#35)
* refactor: enhance CleaningTaskService and related components with validation and repository updates
* feature: 支持算子上传创建
2025-10-30 17:17:00 +08:00
Startalker
155603b1ca feature: add external pdf extract operator by using mineru (#36)
* feature: add UnstructuredFormatter

* feature: add UnstructuredFormatter in db

* feature: add unstructured[docx]==0.18.15

* feature: support doc

* feature: add mineru

* feature: add external pdf extract operator by using mineru

* feature: mineru docker install bugfix

---------

Co-authored-by: Startalker <438747480@qq.com>
2025-10-30 15:55:10 +08:00
hhhhsc
a69b9f4921 feature: 对接deer-flow 2025-10-28 10:54:29 +08:00
hhhhsc
2d2419205a refactor: rename and reorganize data models and repositories for clarity 2025-10-24 15:33:46 +08:00
hhhhsc
17e6cea1d9 refactor: reorganize Helm chart structure and update service configurations 2025-10-23 16:57:12 +08:00
hhhhsc701
31ef8bc265 [Feature] Refactor project to use 'datamate' naming convention for services and configurations (#14)
* Enhance CleaningTaskService to track cleaning process progress and update ExecutorType to DATAMATE

* Refactor project to use 'datamate' naming convention for services and configurations
2025-10-22 17:53:16 +08:00
Jason Wang
c640105333 Add Label Studio adapter module and its build scipts. 2025-10-22 15:14:01 +08:00
Dallas98
1c97afed7d init datamate 2025-10-21 23:00:48 +08:00