You've already forked DataMate
算子将抽取与落盘固定到流程中 (#134)
* feature: 将抽取动作移到每一个算子中 * feature: 落盘算子改为默认执行 * feature: 优化前端展示 * feature: 使用pyproject管理依赖
This commit is contained in:
@@ -36,6 +36,7 @@ class LegendCleaner(Mapper):
|
||||
|
||||
def execute(self, sample: Dict[str, Any]) -> Dict[str, Any]:
|
||||
start = time.time()
|
||||
self.read_file_first(sample)
|
||||
sample[self.text_key] = self._clean_html_tag(sample[self.text_key])
|
||||
logger.info(f"fileName: {sample[self.filename_key]}, method: LegendCleaner costs {time.time() - start:6f} s")
|
||||
return sample
|
||||
|
||||
Reference in New Issue
Block a user