You've already forked DataMate
feat(knowledge-graph): 实现知识图谱基础设施搭建
实现功能: - Neo4j Docker Compose 配置(社区版,端口 7474/7687,数据持久化) - Makefile 新增 Neo4j 命令(neo4j-up/down/logs/shell) - knowledge-graph-service Spring Boot 服务(完整的 DDD 分层架构) - kg_extraction Python 模块(基于 LangChain LLMGraphTransformer) 技术实现: - Neo4j 配置:环境变量化密码,统一默认值 datamate123 - Java 服务: - Domain: GraphEntity, GraphRelation 实体模型 - Repository: Spring Data Neo4j,支持 graphId 范围查询 - Service: 业务逻辑,graphId 双重校验,查询限流 - Controller: REST API,UUID 格式校验 - Exception: 实现 ErrorCode 接口,统一异常体系 - Python 模块: - KnowledgeGraphExtractor 类 - 支持异步/同步/批量抽取 - 支持 schema-guided 模式 - 兼容 OpenAI 及自部署模型 关键设计: - graphId 权限边界:所有实体操作都在正确的 graphId 范围内 - 查询限流:depth 和 limit 参数受配置约束 - 异常处理:统一使用 BusinessException + ErrorCode - 凭据管理:环境变量化,避免硬编码 - 双重防御:Controller 格式校验 + Service 业务校验 代码审查: - 经过 3 轮 Codex 审查和 2 轮 Claude 修复 - 所有 P0 和 P1 问题已解决 - 编译通过,无阻塞性问题 文件变更: - 新增:Neo4j 配置、knowledge-graph-service(11 个 Java 文件)、kg_extraction(3 个 Python 文件) - 修改:Makefile、pom.xml、application.yml、pyproject.toml
This commit is contained in:
75
runtime/datamate-python/app/module/kg_extraction/models.py
Normal file
75
runtime/datamate-python/app/module/kg_extraction/models.py
Normal file
@@ -0,0 +1,75 @@
|
||||
"""知识图谱三元组抽取数据模型。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class GraphNode(BaseModel):
|
||||
"""图谱节点(实体)。"""
|
||||
|
||||
name: str = Field(..., description="实体名称")
|
||||
type: str = Field(..., description="实体类型, 如 Person, Organization, Location")
|
||||
properties: dict[str, object] = Field(default_factory=dict, description="扩展属性")
|
||||
|
||||
|
||||
class GraphEdge(BaseModel):
|
||||
"""图谱边(关系)。"""
|
||||
|
||||
source: str = Field(..., description="源实体名称")
|
||||
target: str = Field(..., description="目标实体名称")
|
||||
relation_type: str = Field(..., description="关系类型, 如 works_at, located_in")
|
||||
properties: dict[str, object] = Field(default_factory=dict, description="关系属性")
|
||||
|
||||
|
||||
class Triple(BaseModel):
|
||||
"""知识三元组: (主体, 关系, 客体)。"""
|
||||
|
||||
subject: GraphNode
|
||||
predicate: str = Field(..., description="关系类型")
|
||||
object: GraphNode
|
||||
|
||||
|
||||
class EntityTypeConstraint(BaseModel):
|
||||
"""实体类型约束,用于 Schema-guided 抽取。"""
|
||||
|
||||
name: str = Field(..., description="类型名称")
|
||||
description: str = Field(default="", description="类型说明")
|
||||
|
||||
|
||||
class RelationTypeConstraint(BaseModel):
|
||||
"""关系类型约束。"""
|
||||
|
||||
name: str = Field(..., description="关系类型名称")
|
||||
source_types: list[str] = Field(default_factory=list, description="允许的源实体类型")
|
||||
target_types: list[str] = Field(default_factory=list, description="允许的目标实体类型")
|
||||
description: str = Field(default="", description="关系说明")
|
||||
|
||||
|
||||
class ExtractionSchema(BaseModel):
|
||||
"""抽取 schema 约束,约束 LLM 输出的实体和关系类型范围。"""
|
||||
|
||||
entity_types: list[EntityTypeConstraint] = Field(default_factory=list)
|
||||
relation_types: list[RelationTypeConstraint] = Field(default_factory=list)
|
||||
|
||||
|
||||
class ExtractionRequest(BaseModel):
|
||||
"""三元组抽取请求。"""
|
||||
|
||||
text: str = Field(..., description="待抽取的文本")
|
||||
graph_id: str = Field(..., description="目标图谱 ID")
|
||||
schema: ExtractionSchema | None = Field(
|
||||
default=None, description="可选的 schema 约束, 提供后做 schema-guided 抽取"
|
||||
)
|
||||
source_id: str | None = Field(default=None, description="来源 ID(数据集/知识库条目)")
|
||||
source_type: str = Field(default="KNOWLEDGE_BASE", description="来源类型")
|
||||
|
||||
|
||||
class ExtractionResult(BaseModel):
|
||||
"""三元组抽取结果。"""
|
||||
|
||||
nodes: list[GraphNode] = Field(default_factory=list)
|
||||
edges: list[GraphEdge] = Field(default_factory=list)
|
||||
triples: list[Triple] = Field(default_factory=list)
|
||||
raw_text: str = Field(default="", description="原始文本")
|
||||
source_id: str | None = None
|
||||
Reference in New Issue
Block a user