You've already forked DataMate
bugfix (#164)
This commit is contained in:
23
README-zh.md
23
README-zh.md
@@ -42,8 +42,6 @@ Choose a deployment method:
|
|||||||
Enter choice:
|
Enter choice:
|
||||||
```
|
```
|
||||||
|
|
||||||
在运行 `make uninstall` 时,卸载流程会只询问一次是否删除卷(数据),该选择会应用到所有组件。卸载顺序为:milvus -> label-studio -> datamate,确保在移除 datamate 网络前,所有使用该网络的服务已先停止。
|
|
||||||
|
|
||||||
### 拉取代码
|
### 拉取代码
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -56,6 +54,20 @@ cd DataMate
|
|||||||
```bash
|
```bash
|
||||||
make install
|
make install
|
||||||
```
|
```
|
||||||
|
|
||||||
|
若您使用的机器没有make,请执行如下命令部署:
|
||||||
|
```bash
|
||||||
|
# Windows
|
||||||
|
set REGISTRY=ghcr.io/modelengine-group/
|
||||||
|
docker compose -f ./deployment/docker/datamate/docker-compose.yml up -d
|
||||||
|
docker compose -f ./deployment/docker/milvus/docker-compose.yml up -d
|
||||||
|
|
||||||
|
# Linux/Mac
|
||||||
|
export REGISTRY=ghcr.io/modelengine-group/
|
||||||
|
docker compose -f ./deployment/docker/datamate/docker-compose.yml up -d
|
||||||
|
docker compose -f ./deployment/docker/milvus/docker-compose.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
当容器运行后,请在浏览器打开 http://localhost:30000 查看前端界面。
|
当容器运行后,请在浏览器打开 http://localhost:30000 查看前端界面。
|
||||||
|
|
||||||
要查看所有可用的 Make 目标、选项和帮助信息,请运行:
|
要查看所有可用的 Make 目标、选项和帮助信息,请运行:
|
||||||
@@ -82,6 +94,13 @@ make build
|
|||||||
make install dev=true
|
make install dev=true
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### 卸载服务
|
||||||
|
```bash
|
||||||
|
make uninstall
|
||||||
|
```
|
||||||
|
|
||||||
|
在运行 `make uninstall` 时,卸载流程会只询问一次是否删除卷(数据),该选择会应用到所有组件。卸载顺序为:milvus -> label-studio -> datamate,确保在移除 datamate 网络前,所有使用该网络的服务已先停止。
|
||||||
|
|
||||||
## 🤝 贡献指南
|
## 🤝 贡献指南
|
||||||
|
|
||||||
感谢您对本项目的关注!我们非常欢迎社区的贡献,无论是提交 Bug 报告、提出功能建议,还是直接参与代码开发,都能帮助项目变得更好。
|
感谢您对本项目的关注!我们非常欢迎社区的贡献,无论是提交 Bug 报告、提出功能建议,还是直接参与代码开发,都能帮助项目变得更好。
|
||||||
|
|||||||
23
README.md
23
README.md
@@ -45,8 +45,6 @@ Choose a deployment method:
|
|||||||
Enter choice:
|
Enter choice:
|
||||||
```
|
```
|
||||||
|
|
||||||
When running make uninstall, the installer will prompt once whether to delete volumes; that single choice is applied to all components. The uninstall order is: milvus -> label-studio -> datamate, which ensures the datamate network is removed cleanly after services that use it have stopped.
|
|
||||||
|
|
||||||
### Clone the Code
|
### Clone the Code
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -59,6 +57,20 @@ cd DataMate
|
|||||||
```bash
|
```bash
|
||||||
make install
|
make install
|
||||||
```
|
```
|
||||||
|
|
||||||
|
If the machine you are using does not have make installed, please run the following command to deploy it:
|
||||||
|
```bash
|
||||||
|
# Windows
|
||||||
|
set REGISTRY=ghcr.io/modelengine-group/
|
||||||
|
docker compose -f ./deployment/docker/datamate/docker-compose.yml up -d
|
||||||
|
docker compose -f ./deployment/docker/milvus/docker-compose.yml up -d
|
||||||
|
|
||||||
|
# Linux/Mac
|
||||||
|
export REGISTRY=ghcr.io/modelengine-group/
|
||||||
|
docker compose -f ./deployment/docker/datamate/docker-compose.yml up -d
|
||||||
|
docker compose -f ./deployment/docker/milvus/docker-compose.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
Once the container is running, access http://localhost:30000 in a browser to view the front-end interface.
|
Once the container is running, access http://localhost:30000 in a browser to view the front-end interface.
|
||||||
|
|
||||||
To list all available Make targets, flags and help text, run:
|
To list all available Make targets, flags and help text, run:
|
||||||
@@ -85,6 +97,13 @@ make build
|
|||||||
make install dev=true
|
make install dev=true
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Uninstall
|
||||||
|
```bash
|
||||||
|
make uninstall
|
||||||
|
```
|
||||||
|
|
||||||
|
When running make uninstall, the installer will prompt once whether to delete volumes; that single choice is applied to all components. The uninstall order is: milvus -> label-studio -> datamate, which ensures the datamate network is removed cleanly after services that use it have stopped.
|
||||||
|
|
||||||
## 🤝 Contribution Guidelines
|
## 🤝 Contribution Guidelines
|
||||||
|
|
||||||
Thank you for your interest in this project! We warmly welcome contributions from the community. Whether it's submitting
|
Thank you for your interest in this project! We warmly welcome contributions from the community. Whether it's submitting
|
||||||
|
|||||||
@@ -23,6 +23,13 @@ server {
|
|||||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
location /api/evaluation/ {
|
||||||
|
proxy_pass http://datamate-backend-python:18000/api/evaluation/;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
}
|
||||||
|
|
||||||
location /api/ {
|
location /api/ {
|
||||||
proxy_pass http://datamate-backend:8080/api/;
|
proxy_pass http://datamate-backend:8080/api/;
|
||||||
proxy_set_header Host $host;
|
proxy_set_header Host $host;
|
||||||
|
|||||||
@@ -456,9 +456,8 @@ class FileExporter(BaseOp):
|
|||||||
new_file_name = base_name + '.' + file_type
|
new_file_name = base_name + '.' + file_type
|
||||||
sample[self.filename_key] = new_file_name
|
sample[self.filename_key] = new_file_name
|
||||||
|
|
||||||
base_name, _ = os.path.splitext(save_path)
|
sample[self.filepath_key] = save_path
|
||||||
sample[self.filepath_key] = base_name
|
file_size = os.path.getsize(save_path)
|
||||||
file_size = os.path.getsize(base_name)
|
|
||||||
sample[self.filesize_key] = f"{file_size}"
|
sample[self.filesize_key] = f"{file_size}"
|
||||||
|
|
||||||
logger.info(f"origin file named {file_name} has been save to {save_path}")
|
logger.info(f"origin file named {file_name} has been save to {save_path}")
|
||||||
@@ -514,16 +513,15 @@ class FileExporter(BaseOp):
|
|||||||
return sample, save_path
|
return sample, save_path
|
||||||
|
|
||||||
def save_file(self, sample, save_path):
|
def save_file(self, sample, save_path):
|
||||||
file_name, _ = os.path.splitext(save_path)
|
|
||||||
# 以二进制格式保存文件
|
# 以二进制格式保存文件
|
||||||
file_sample = sample[self.text_key].encode('utf-8') if sample[self.text_key] else sample[self.data_key]
|
file_sample = sample[self.text_key].encode('utf-8') if sample[self.text_key] else sample[self.data_key]
|
||||||
with open(file_name, 'wb') as f:
|
with open(save_path, 'wb') as f:
|
||||||
f.write(file_sample)
|
f.write(file_sample)
|
||||||
# 获取父目录路径
|
# 获取父目录路径
|
||||||
|
|
||||||
parent_dir = os.path.dirname(file_name)
|
parent_dir = os.path.dirname(save_path)
|
||||||
os.chmod(parent_dir, 0o770)
|
os.chmod(parent_dir, 0o770)
|
||||||
os.chmod(file_name, 0o640)
|
os.chmod(save_path, 0o640)
|
||||||
|
|
||||||
def _get_from_data(self, sample: Dict[str, Any]) -> Dict[str, Any]:
|
def _get_from_data(self, sample: Dict[str, Any]) -> Dict[str, Any]:
|
||||||
sample[self.data_key] = bytes(sample[self.data_key])
|
sample[self.data_key] = bytes(sample[self.data_key])
|
||||||
|
|||||||
Reference in New Issue
Block a user