q792602257/DataMate
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feature: add mysql collection and starrocks collection (#222)

Browse Source 
* fix: fix the path for backend-python imaage building

* feature: add mysql collection and starrocks collection

* feature: add mysql collection and starrocks collection

* fix: change the permission of those files which collected from nfs to 754

* fix: delete collected files, config files and log files while deleting collection task

* fix: add the collection task detail api

* fix: change the log of collecting for dataset

* fix: add collection task selecting while creating and updating dataset

* fix: set the umask value to 0022 for java process
This commit is contained in:
hefanli
2026-01-04 19:05:08 +08:00
committed by GitHub
parent 8d61eb28c3
commit ccfb84c034
13 changed files with 208 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
runtime/datamate-python/app/module/collection/service/collection.py
View File

@@ -58,6 +58,7 @@ class CollectionTaskService:
logger.error(f"task {task_id} not exist")
return
template = await session.execute(select(CollectionTemplate).where(CollectionTemplate.id == task.template_id))
template = template.scalar_one_or_none()
if not template:
logger.error(f"template {task.template_name} not exist")
return
@@ -65,6 +66,6 @@ class CollectionTaskService:
session.add(task_execution)
await session.commit()
await asyncio.to_thread(
DataxClient(execution=task_execution, task=task).run_datax_job
DataxClient(execution=task_execution, task=task, template=template).run_datax_job
)
await session.commit()