You've already forked DataMate
refactor(data-import): 优化数据源文件扫描和复制逻辑
- 修改数据源文件扫描方法,直接在主流程中获取任务详情和路径 - 移除独立的getFilePaths方法,将路径扫描逻辑整合到scanFilePaths方法中 - 新增copyFilesToDatasetDirWithSourceRoot方法支持保留相对路径的文件复制 - 更新数据集文件应用服务中的文件复制逻辑,支持相对路径处理 - 修改Python后端项目接口中的文件查询逻辑,移除注释掉的编辑器服务引用 - 调整文件过滤逻辑,基于元数据中的派生源ID进行文件筛选 - 移除编辑器服务中已废弃的源文档过滤条件
This commit is contained in:
@@ -429,19 +429,10 @@ class AnnotationEditorService:
|
||||
exclude_source_documents: Optional[bool] = None,
|
||||
) -> EditorTaskListResponse:
|
||||
project = await self._get_project_or_404(project_id)
|
||||
dataset_type = self._normalize_dataset_type(await self._get_dataset_type(project.dataset_id))
|
||||
should_exclude_source_documents = False
|
||||
if dataset_type == DATASET_TYPE_TEXT:
|
||||
should_exclude_source_documents = (
|
||||
exclude_source_documents if exclude_source_documents is not None else True
|
||||
)
|
||||
|
||||
base_conditions = [
|
||||
LabelingProjectFile.project_id == project_id,
|
||||
DatasetFiles.dataset_id == project.dataset_id,
|
||||
]
|
||||
if should_exclude_source_documents:
|
||||
base_conditions.append(~self._build_source_document_filter())
|
||||
|
||||
count_result = await self.db.execute(
|
||||
select(func.count())
|
||||
|
||||
Reference in New Issue
Block a user