This commit is contained in:
hhhhsc701
2025-12-11 23:17:01 +08:00
committed by GitHub
parent ec87e4f204
commit fc9fb07e77
4 changed files with 54 additions and 11 deletions

View File

@@ -42,8 +42,6 @@ Choose a deployment method:
Enter choice: Enter choice:
``` ```
在运行 `make uninstall` 时,卸载流程会只询问一次是否删除卷(数据),该选择会应用到所有组件。卸载顺序为:milvus -> label-studio -> datamate,确保在移除 datamate 网络前,所有使用该网络的服务已先停止。
### 拉取代码 ### 拉取代码
```bash ```bash
@@ -56,6 +54,20 @@ cd DataMate
```bash ```bash
make install make install
``` ```
若您使用的机器没有make,请执行如下命令部署:
```bash
# Windows
set REGISTRY=ghcr.io/modelengine-group/
docker compose -f ./deployment/docker/datamate/docker-compose.yml up -d
docker compose -f ./deployment/docker/milvus/docker-compose.yml up -d
# Linux/Mac
export REGISTRY=ghcr.io/modelengine-group/
docker compose -f ./deployment/docker/datamate/docker-compose.yml up -d
docker compose -f ./deployment/docker/milvus/docker-compose.yml up -d
```
当容器运行后,请在浏览器打开 http://localhost:30000 查看前端界面。 当容器运行后,请在浏览器打开 http://localhost:30000 查看前端界面。
要查看所有可用的 Make 目标、选项和帮助信息,请运行: 要查看所有可用的 Make 目标、选项和帮助信息,请运行:
@@ -82,6 +94,13 @@ make build
make install dev=true make install dev=true
``` ```
### 卸载服务
```bash
make uninstall
```
在运行 `make uninstall` 时,卸载流程会只询问一次是否删除卷(数据),该选择会应用到所有组件。卸载顺序为:milvus -> label-studio -> datamate,确保在移除 datamate 网络前,所有使用该网络的服务已先停止。
## 🤝 贡献指南 ## 🤝 贡献指南
感谢您对本项目的关注!我们非常欢迎社区的贡献,无论是提交 Bug 报告、提出功能建议,还是直接参与代码开发,都能帮助项目变得更好。 感谢您对本项目的关注!我们非常欢迎社区的贡献,无论是提交 Bug 报告、提出功能建议,还是直接参与代码开发,都能帮助项目变得更好。

View File

@@ -45,8 +45,6 @@ Choose a deployment method:
Enter choice: Enter choice:
``` ```
When running make uninstall, the installer will prompt once whether to delete volumes; that single choice is applied to all components. The uninstall order is: milvus -> label-studio -> datamate, which ensures the datamate network is removed cleanly after services that use it have stopped.
### Clone the Code ### Clone the Code
```bash ```bash
@@ -59,6 +57,20 @@ cd DataMate
```bash ```bash
make install make install
``` ```
If the machine you are using does not have make installed, please run the following command to deploy it:
```bash
# Windows
set REGISTRY=ghcr.io/modelengine-group/
docker compose -f ./deployment/docker/datamate/docker-compose.yml up -d
docker compose -f ./deployment/docker/milvus/docker-compose.yml up -d
# Linux/Mac
export REGISTRY=ghcr.io/modelengine-group/
docker compose -f ./deployment/docker/datamate/docker-compose.yml up -d
docker compose -f ./deployment/docker/milvus/docker-compose.yml up -d
```
Once the container is running, access http://localhost:30000 in a browser to view the front-end interface. Once the container is running, access http://localhost:30000 in a browser to view the front-end interface.
To list all available Make targets, flags and help text, run: To list all available Make targets, flags and help text, run:
@@ -85,6 +97,13 @@ make build
make install dev=true make install dev=true
``` ```
### Uninstall
```bash
make uninstall
```
When running make uninstall, the installer will prompt once whether to delete volumes; that single choice is applied to all components. The uninstall order is: milvus -> label-studio -> datamate, which ensures the datamate network is removed cleanly after services that use it have stopped.
## 🤝 Contribution Guidelines ## 🤝 Contribution Guidelines
Thank you for your interest in this project! We warmly welcome contributions from the community. Whether it's submitting Thank you for your interest in this project! We warmly welcome contributions from the community. Whether it's submitting

View File

@@ -23,6 +23,13 @@ server {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
} }
location /api/evaluation/ {
proxy_pass http://datamate-backend-python:18000/api/evaluation/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /api/ { location /api/ {
proxy_pass http://datamate-backend:8080/api/; proxy_pass http://datamate-backend:8080/api/;
proxy_set_header Host $host; proxy_set_header Host $host;

View File

@@ -456,9 +456,8 @@ class FileExporter(BaseOp):
new_file_name = base_name + '.' + file_type new_file_name = base_name + '.' + file_type
sample[self.filename_key] = new_file_name sample[self.filename_key] = new_file_name
base_name, _ = os.path.splitext(save_path) sample[self.filepath_key] = save_path
sample[self.filepath_key] = base_name file_size = os.path.getsize(save_path)
file_size = os.path.getsize(base_name)
sample[self.filesize_key] = f"{file_size}" sample[self.filesize_key] = f"{file_size}"
logger.info(f"origin file named {file_name} has been save to {save_path}") logger.info(f"origin file named {file_name} has been save to {save_path}")
@@ -514,16 +513,15 @@ class FileExporter(BaseOp):
return sample, save_path return sample, save_path
def save_file(self, sample, save_path): def save_file(self, sample, save_path):
file_name, _ = os.path.splitext(save_path)
# 以二进制格式保存文件 # 以二进制格式保存文件
file_sample = sample[self.text_key].encode('utf-8') if sample[self.text_key] else sample[self.data_key] file_sample = sample[self.text_key].encode('utf-8') if sample[self.text_key] else sample[self.data_key]
with open(file_name, 'wb') as f: with open(save_path, 'wb') as f:
f.write(file_sample) f.write(file_sample)
# 获取父目录路径 # 获取父目录路径
parent_dir = os.path.dirname(file_name) parent_dir = os.path.dirname(save_path)
os.chmod(parent_dir, 0o770) os.chmod(parent_dir, 0o770)
os.chmod(file_name, 0o640) os.chmod(save_path, 0o640)
def _get_from_data(self, sample: Dict[str, Any]) -> Dict[str, Any]: def _get_from_data(self, sample: Dict[str, Any]) -> Dict[str, Any]:
sample[self.data_key] = bytes(sample[self.data_key]) sample[self.data_key] = bytes(sample[self.data_key])