feat(kg): 实现实体对齐功能(aligner.py)

- 实现三层对齐策略:规则层 + 向量相似度层 + LLM 仲裁层
- 规则层:名称规范化(NFKC、小写、去标点/空格)+ 规则评分
- 向量层:OpenAI Embeddings + cosine 相似度计算
- LLM 层:仅对边界样本调用,严格 JSON schema 校验
- 使用 Union-Find 实现传递合并
- 支持批内对齐(库内对齐待 KG 服务 API 支持)

核心组件:
- EntityAligner 类:align() (async)、align_rules_only() (sync)
- 配置项:kg_alignment_enabled(默认 false)、embedding_model、阈值
- 失败策略:fail-open(对齐失败不中断请求)

集成:
- 已集成到抽取主链路(extract → align → return)
- extract() 调用 async align()
- extract_sync() 调用 sync align_rules_only()

修复:
- P1-1:使用 (name, type) 作为 key,避免同名跨类型误合并
- P1-2:LLM 计数在 finally 块中增加,异常也计数
- P1-3:添加库内对齐说明(待后续实现)

新增 41 个测试用例,全部通过
测试结果:41 tests pass
This commit is contained in:
2026-02-19 18:26:54 +08:00
parent 7abdafc338
commit 0ed7dcbee7
5 changed files with 969 additions and 0 deletions

View File

@@ -15,6 +15,7 @@ from langchain_experimental.graph_transformers import LLMGraphTransformer
from pydantic import SecretStr
from app.core.logging import get_logger
from app.module.kg_extraction.aligner import EntityAligner
from app.module.kg_extraction.models import (
ExtractionRequest,
ExtractionResult,
@@ -47,6 +48,7 @@ class KnowledgeGraphExtractor:
temperature: float = 0.0,
timeout: int = 60,
max_retries: int = 2,
aligner: EntityAligner | None = None,
) -> None:
logger.info(
"Initializing KnowledgeGraphExtractor (model=%s, base_url=%s, timeout=%ds, max_retries=%d)",
@@ -63,6 +65,7 @@ class KnowledgeGraphExtractor:
timeout=timeout,
max_retries=max_retries,
)
self._aligner = aligner or EntityAligner()
@classmethod
def from_settings(cls) -> KnowledgeGraphExtractor:
@@ -76,6 +79,7 @@ class KnowledgeGraphExtractor:
temperature=settings.kg_llm_temperature,
timeout=settings.kg_llm_timeout_seconds,
max_retries=settings.kg_llm_max_retries,
aligner=EntityAligner.from_settings(),
)
def _build_transformer(
@@ -119,6 +123,7 @@ class KnowledgeGraphExtractor:
raise
result = self._convert_result(graph_documents, request)
result = await self._aligner.align(result)
logger.info(
"Extraction complete: graph_id=%s, nodes=%d, edges=%d, triples=%d",
request.graph_id,
@@ -154,6 +159,7 @@ class KnowledgeGraphExtractor:
raise
result = self._convert_result(graph_documents, request)
result = self._aligner.align_rules_only(result)
logger.info(
"Sync extraction complete: graph_id=%s, nodes=%d, edges=%d",
request.graph_id,