You've already forked DataMate
feat(kg): 实现 Phase 3.1 前端图谱浏览器
核心功能: - G6 v5 力导向图,支持交互式缩放、平移、拖拽 - 5 种布局模式:force, circular, grid, radial, concentric - 双击展开节点邻居到图中(增量探索) - 全文搜索,类型过滤,结果高亮(变暗/高亮状态) - 节点详情抽屉:实体属性、别名、置信度、关系列表(可导航) - 关系详情抽屉:类型、源/目标、权重、置信度、属性 - 查询构建器:最短路径/全路径查询,可配置 maxDepth/maxPaths - 基于 UUID 的图加载(输入或 URL 参数 ?graphId=...) - 大图性能优化(200 节点阈值,超过时禁用动画) 新增文件(13 个): - knowledge-graph.model.ts - TypeScript 接口,匹配 Java DTOs - knowledge-graph.api.ts - API 服务,包含所有 KG REST 端点 - knowledge-graph.const.ts - 实体类型颜色、关系类型标签、中文显示名称 - graphTransform.ts - 后端数据 → G6 节点/边格式转换 + 合并工具 - graphConfig.ts - G6 v5 图配置(节点/边样式、行为、布局) - hooks/useGraphData.ts - 数据钩子:加载子图、展开节点、搜索、合并 - hooks/useGraphLayout.ts - 布局钩子:5 种布局类型 - components/GraphCanvas.tsx - G6 v5 画布,力导向布局,缩放/平移/拖拽 - components/SearchPanel.tsx - 全文实体搜索,类型过滤 - components/NodeDetail.tsx - 实体详情抽屉 - components/RelationDetail.tsx - 关系详情抽屉 - components/QueryBuilder.tsx - 路径查询构建器 - Home/KnowledgeGraphPage.tsx - 主页面,整合所有组件 修改文件(5 个): - package.json - 添加 @antv/g6 v5 依赖 - vite.config.ts - 添加 /knowledge-graph 代理规则 - auth/permissions.ts - 添加 knowledgeGraphRead/knowledgeGraphWrite - pages/Layout/menu.tsx - 添加知识图谱菜单项(Network 图标) - routes/routes.ts - 添加 /data/knowledge-graph 路由 新增文档(10 个): - docs/knowledge-graph/ - 完整的知识图谱设计文档 Bug 修复(Codex 审查后修复): - P1: 详情抽屉状态与选中状态不一致(显示旧数据) - P1: 查询构建器未实现(最短路径/多路径查询) - P2: 实体类型映射 Organization → Org(匹配后端) - P2: getSubgraph depth 参数无效(改用正确端点) - P2: AllPathsVO 字段名不一致(totalPaths → pathCount) - P2: 搜索取消逻辑无效(传递 AbortController.signal) - P2: 大图性能优化(动画降级) - P3: 移除未使用的类型导入 构建验证: - tsc --noEmit ✅ clean - eslint ✅ 0 errors/warnings - vite build ✅ successful
This commit is contained in:
329
docs/knowledge-graph/schema/entities.md
Normal file
329
docs/knowledge-graph/schema/entities.md
Normal file
@@ -0,0 +1,329 @@
|
||||
# DataMate 知识图谱 - 核心实体定义
|
||||
|
||||
> Schema 版本:1.0.0
|
||||
> 更新日期:2026-02-17
|
||||
|
||||
## 概述
|
||||
|
||||
DataMate 知识图谱定义了 **8 类核心实体**,覆盖数据资产管理、任务追踪、组织归属和知识管理四大领域。
|
||||
|
||||
所有实体在 Neo4j 中统一使用 `Entity` 标签,通过 `type` 属性区分语义类型。每个实体都包含以下公共属性:
|
||||
|
||||
| 公共属性 | 类型 | 必填 | 说明 |
|
||||
|---------|------|------|------|
|
||||
| `id` | String (UUID) | 是 | 全局唯一标识符 |
|
||||
| `name` | String | 是 | 实体名称 |
|
||||
| `type` | String | 是 | 实体类型(见下文各类型定义) |
|
||||
| `description` | String | 否 | 实体描述 |
|
||||
| `graph_id` | String (UUID) | 是 | 所属图谱 ID,用于多租户隔离 |
|
||||
| `source_id` | String | 否 | 来源记录 ID(MySQL 主键或外部系统 ID) |
|
||||
| `source_type` | String | 否 | 来源类型:`SYNC`(MySQL 同步)、`EXTRACTION`(LLM 抽取)、`MANUAL`(人工创建) |
|
||||
| `confidence` | Double | 否 | 置信度 0.0-1.0(同步数据默认 1.0,抽取数据由模型评分) |
|
||||
| `properties_json` | String (JSON) | 否 | 类型特有 properties 的 JSON 序列化,各类型的 properties 定义见下文 |
|
||||
| `created_at` | LocalDateTime | 是 | 创建时间 |
|
||||
|
||||
---
|
||||
|
||||
## 1. Dataset(数据集)
|
||||
|
||||
数据集是 DataMate 的核心资产,代表一组结构化或非结构化数据的集合。
|
||||
|
||||
**对应代码模型**:`data-management-service` 的 `Dataset.java`
|
||||
|
||||
### properties(properties_json 字段)
|
||||
|
||||
| property | 类型 | 必填 | 约束 | 说明 |
|
||||
|----------|------|------|------|------|
|
||||
| `dataset_type` | String | 是 | `IMAGE` / `TEXT` / `QA` / `MULTIMODAL` / `OTHER` | 数据集类型 |
|
||||
| `status` | String | 是 | `DRAFT` / `ACTIVE` / `ARCHIVED` | 数据集状态 |
|
||||
| `category` | String | 否 | 最长 50 字符 | 业务分类 |
|
||||
| `format` | String | 否 | — | 数据格式(如 CSV、JSON、DICOM) |
|
||||
| `record_count` | Long | 否 | >= 0 | 记录/文件数量 |
|
||||
| `size_bytes` | Long | 否 | >= 0 | 数据集大小(字节) |
|
||||
| `version` | Integer | 否 | >= 1 | 版本号 |
|
||||
| `tags` | List\<String\> | 否 | — | 标签列表 |
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
// 创建 Dataset 实体(类型 properties 序列化到 properties_json)
|
||||
CREATE (d:Entity {
|
||||
id: 'a1b2c3d4-...',
|
||||
name: '用户行为日志-v2',
|
||||
type: 'Dataset',
|
||||
description: '2025年Q4用户行为埋点数据',
|
||||
graph_id: $graphId,
|
||||
source_id: '12345',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"dataset_type":"TEXT","status":"ACTIVE","category":"用户行为","format":"JSON","record_count":1500000,"size_bytes":2147483648,"version":2,"tags":["behavior","production"]}',
|
||||
created_at: datetime()
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Field(字段)
|
||||
|
||||
字段代表数据集中的列或属性元数据,是数据血缘分析和影响评估的基础单元。
|
||||
|
||||
**对应代码模型**:从 `DatasetFile` 的 schema 元数据中提取
|
||||
|
||||
### properties(properties_json 字段)
|
||||
|
||||
| property | 类型 | 必填 | 约束 | 说明 |
|
||||
|----------|------|------|------|------|
|
||||
| `data_type` | String | 是 | — | 数据类型(如 STRING、INT、FLOAT、DATETIME、JSON) |
|
||||
| `nullable` | Boolean | 否 | — | 是否允许空值 |
|
||||
| `is_primary_key` | Boolean | 否 | — | 是否为主键 |
|
||||
| `default_value` | String | 否 | — | 默认值 |
|
||||
| `sample_values` | List\<String\> | 否 | 最多 5 个 | 示例值 |
|
||||
| `statistics` | String | 否 | JSON 格式 | 字段统计信息(null 率、唯一值数等) |
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
CREATE (f:Entity {
|
||||
id: 'f1e2d3c4-...',
|
||||
name: 'user_id',
|
||||
type: 'Field',
|
||||
description: '用户唯一标识符',
|
||||
graph_id: $graphId,
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"data_type":"STRING","nullable":false,"is_primary_key":true,"sample_values":["U001","U002","U003"]}',
|
||||
created_at: datetime()
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. LabelTask(标注任务)
|
||||
|
||||
标注任务代表一次数据标注活动,包括人工标注和自动标注。
|
||||
|
||||
**对应代码模型**:`data-annotation-service` 的 `LabelingProject`、`AutoAnnotationTask`;`task-coordination-service` 的 `TaskMeta`
|
||||
|
||||
### properties(properties_json 字段)
|
||||
|
||||
| property | 类型 | 必填 | 约束 | 说明 |
|
||||
|----------|------|------|------|------|
|
||||
| `task_mode` | String | 是 | `MANUAL` / `AUTO` / `HYBRID` | 标注模式 |
|
||||
| `data_type` | String | 否 | `image` / `text` / `audio` / `video` / `pdf` 等 | 标注数据类型 |
|
||||
| `labeling_type` | String | 否 | — | 标注类型(如 NER、目标检测、情感分析) |
|
||||
| `status` | String | 是 | `PENDING` / `IN_PROGRESS` / `COMPLETED` / `FAILED` / `STOPPED` | 任务状态 |
|
||||
| `progress` | Double | 否 | 0.0-100.0 | 完成进度百分比 |
|
||||
| `template_name` | String | 否 | — | 使用的标注模板名称 |
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
CREATE (t:Entity {
|
||||
id: 'e5f6a7b8-...',
|
||||
name: '医学图像病灶标注-批次3',
|
||||
type: 'LabelTask',
|
||||
description: 'CT影像中肺结节目标检测标注',
|
||||
graph_id: $graphId,
|
||||
source_id: '67890',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"task_mode":"HYBRID","data_type":"image","labeling_type":"object_detection","status":"IN_PROGRESS","progress":45.5,"template_name":"医学目标检测"}',
|
||||
created_at: datetime()
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Workflow(工作流)
|
||||
|
||||
工作流代表一组数据处理步骤的编排定义,涵盖数据清洗、数据合成、数据评估等处理管道。
|
||||
|
||||
**对应代码模型**:`data-cleaning-service` 的 `CleaningTemplate`;`data-collection-service` 的 `CollectionTemplate`;算子编排 `Operator`
|
||||
|
||||
### properties(properties_json 字段)
|
||||
|
||||
| property | 类型 | 必填 | 约束 | 说明 |
|
||||
|----------|------|------|------|------|
|
||||
| `workflow_type` | String | 是 | `CLEANING` / `SYNTHESIS` / `EVALUATION` / `COLLECTION` / `CUSTOM` | 工作流类型 |
|
||||
| `status` | String | 否 | `DRAFT` / `ACTIVE` / `DEPRECATED` | 工作流状态 |
|
||||
| `version` | String | 否 | — | 版本号 |
|
||||
| `operator_count` | Integer | 否 | >= 0 | 包含的算子数量 |
|
||||
| `schedule` | String | 否 | Cron 表达式 | 调度表达式(用于定时工作流) |
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
CREATE (w:Entity {
|
||||
id: 'c9d0e1f2-...',
|
||||
name: '文本去重清洗管道',
|
||||
type: 'Workflow',
|
||||
description: '基于SimHash的文本去重 + 格式标准化 + 质量过滤',
|
||||
graph_id: $graphId,
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"workflow_type":"CLEANING","status":"ACTIVE","version":"2.1","operator_count":3}',
|
||||
created_at: datetime()
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Job(作业)
|
||||
|
||||
作业代表一次具体的任务执行实例,是工作流的运行时实体,记录输入输出和执行状态。
|
||||
|
||||
**对应代码模型**:`CleaningTask`、`DataSynthInstance`、`EvaluationTask`、`CollectionTask`、`TaskExecution`
|
||||
|
||||
### properties(properties_json 字段)
|
||||
|
||||
| property | 类型 | 必填 | 约束 | 说明 |
|
||||
|----------|------|------|------|------|
|
||||
| `job_type` | String | 是 | `CLEANING` / `SYNTHESIS` / `EVALUATION` / `COLLECTION` / `ANNOTATION` | 作业类型 |
|
||||
| `status` | String | 是 | `PENDING` / `RUNNING` / `COMPLETED` / `FAILED` / `STOPPED` / `CANCELLED` | 执行状态 |
|
||||
| `started_at` | String | 否 | ISO 8601 | 开始时间 |
|
||||
| `completed_at` | String | 否 | ISO 8601 | 完成时间 |
|
||||
| `duration_seconds` | Long | 否 | >= 0 | 执行耗时(秒) |
|
||||
| `input_count` | Long | 否 | >= 0 | 输入记录/文件数 |
|
||||
| `output_count` | Long | 否 | >= 0 | 输出记录/文件数 |
|
||||
| `error_message` | String | 否 | — | 错误信息(失败时) |
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
CREATE (j:Entity {
|
||||
id: 'd3e4f5a6-...',
|
||||
name: '清洗作业-20260215-001',
|
||||
type: 'Job',
|
||||
description: '用户行为日志去重清洗',
|
||||
graph_id: $graphId,
|
||||
source_id: '54321',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"job_type":"CLEANING","status":"COMPLETED","started_at":"2026-02-15T10:00:00","completed_at":"2026-02-15T10:35:00","duration_seconds":2100,"input_count":1500000,"output_count":1380000}',
|
||||
created_at: datetime()
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. User(用户)
|
||||
|
||||
用户代表 DataMate 平台的操作人员,用于追踪数据资产的责任人和任务的执行者。
|
||||
|
||||
**对应代码模型**:`User.java`(`user` 表)
|
||||
|
||||
### properties(properties_json 字段)
|
||||
|
||||
| property | 类型 | 必填 | 约束 | 说明 |
|
||||
|----------|------|------|------|------|
|
||||
| `username` | String | 是 | 唯一 | 登录用户名 |
|
||||
| `email` | String | 否 | — | 邮箱地址 |
|
||||
| `role` | String | 否 | `ADMIN` / `USER` | 角色 |
|
||||
| `enabled` | Boolean | 否 | — | 是否启用 |
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
CREATE (u:Entity {
|
||||
id: 'b7c8d9e0-...',
|
||||
name: '张三',
|
||||
type: 'User',
|
||||
graph_id: $graphId,
|
||||
source_id: '1001',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"username":"zhangsan","email":"zhangsan@example.com","role":"USER","enabled":true}',
|
||||
created_at: datetime()
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Org(组织)
|
||||
|
||||
组织代表企业内部的团队或部门,用于数据资产的归属管理和权限隔离。
|
||||
|
||||
**对应代码模型**:从 `User.organization` 字段聚合派生
|
||||
|
||||
### properties(properties_json 字段)
|
||||
|
||||
| property | 类型 | 必填 | 约束 | 说明 |
|
||||
|----------|------|------|------|------|
|
||||
| `org_code` | String | 否 | 唯一 | 组织编码 |
|
||||
| `parent_org_id` | String | 否 | UUID | 上级组织 ID |
|
||||
| `level` | Integer | 否 | >= 1 | 组织层级 |
|
||||
| `member_count` | Integer | 否 | >= 0 | 成员数量 |
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
CREATE (o:Entity {
|
||||
id: 'a0b1c2d3-...',
|
||||
name: '数据工程部',
|
||||
type: 'Org',
|
||||
description: '负责数据采集、清洗和标注',
|
||||
graph_id: $graphId,
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"org_code":"DE","level":2,"member_count":15}',
|
||||
created_at: datetime()
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. KnowledgeSet(知识集)
|
||||
|
||||
知识集代表经过整理和验证的知识资产集合,是 RAG 检索和知识问答的基础。
|
||||
|
||||
**对应代码模型**:`KnowledgeSet.java`(`knowledge_set` 表)
|
||||
|
||||
### properties(properties_json 字段)
|
||||
|
||||
| property | 类型 | 必填 | 约束 | 说明 |
|
||||
|----------|------|------|------|------|
|
||||
| `status` | String | 是 | `DRAFT` / `PUBLISHED` / `ARCHIVED` / `DEPRECATED` | 知识集状态 |
|
||||
| `domain` | String | 否 | — | 知识领域 |
|
||||
| `business_line` | String | 否 | — | 业务线 |
|
||||
| `sensitivity` | String | 否 | `PUBLIC` / `INTERNAL` / `CONFIDENTIAL` / `SECRET` | 敏感级别 |
|
||||
| `item_count` | Integer | 否 | >= 0 | 包含的知识条目数 |
|
||||
| `valid_from` | String | 否 | ISO 8601 | 有效期开始 |
|
||||
| `valid_to` | String | 否 | ISO 8601 | 有效期结束 |
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
CREATE (k:Entity {
|
||||
id: 'f4e5d6c7-...',
|
||||
name: '医学影像标注规范知识库',
|
||||
type: 'KnowledgeSet',
|
||||
description: 'CT/MRI影像标注标准和常见病灶特征知识',
|
||||
graph_id: $graphId,
|
||||
source_id: '777',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"status":"PUBLISHED","domain":"医学影像","business_line":"AI辅助诊断","sensitivity":"INTERNAL","item_count":320,"valid_from":"2026-01-01T00:00:00","valid_to":"2027-01-01T00:00:00"}',
|
||||
created_at: datetime()
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 实体类型汇总
|
||||
|
||||
| 实体类型 | Neo4j type 值 | 核心用途 | 来源 |
|
||||
|---------|--------------|---------|------|
|
||||
| Dataset | `Dataset` | 数据资产管理、血缘追踪 | MySQL 同步 |
|
||||
| Field | `Field` | 字段级血缘、影响分析 | MySQL 同步 / Schema 解析 |
|
||||
| LabelTask | `LabelTask` | 标注任务追踪、人员管理 | MySQL 同步 |
|
||||
| Workflow | `Workflow` | 流程编排、复用管理 | MySQL 同步 |
|
||||
| Job | `Job` | 执行追踪、输入输出血缘 | MySQL 同步 |
|
||||
| User | `User` | 责任人追踪、权限管理 | MySQL 同步 |
|
||||
| Org | `Org` | 组织归属、资产隔离 | MySQL 同步 / 派生 |
|
||||
| KnowledgeSet | `KnowledgeSet` | 知识资产管理、RAG 检索 | MySQL 同步 |
|
||||
|
||||
## 扩展说明
|
||||
|
||||
- **自定义实体类型**:除上述 8 类核心实体外,用户可通过 LLM 抽取或手动创建自定义实体类型。自定义实体使用相同的 `Entity` 标签和公共属性结构,`type` 字段可为任意字符串。
|
||||
- **属性存储**:类型特有 properties 存储在 `properties_json` 字段中(JSON 序列化),不直接作为 Neo4j 节点属性。这保证了 schema 的灵活性,同时通过 `type` 字段实现类型区分。
|
||||
- **索引策略**:`id`、`graph_id`、`type`、`name` 字段建立 Neo4j 索引,`properties_json` 中的 properties 不建立索引。如果某个 property 需要高频查询,应提升为节点顶层属性并建立索引。
|
||||
298
docs/knowledge-graph/schema/er-diagram.md
Normal file
298
docs/knowledge-graph/schema/er-diagram.md
Normal file
@@ -0,0 +1,298 @@
|
||||
# DataMate 知识图谱 - 实体关系图
|
||||
|
||||
> Schema 版本:1.0.0
|
||||
> 更新日期:2026-02-17
|
||||
|
||||
## 核心实体关系总览
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
%% 实体定义
|
||||
Dataset["<b>Dataset</b><br/>数据集"]
|
||||
Field["<b>Field</b><br/>字段"]
|
||||
LabelTask["<b>LabelTask</b><br/>标注任务"]
|
||||
Workflow["<b>Workflow</b><br/>工作流"]
|
||||
Job["<b>Job</b><br/>作业"]
|
||||
User["<b>User</b><br/>用户"]
|
||||
Org["<b>Org</b><br/>组织"]
|
||||
KnowledgeSet["<b>KnowledgeSet</b><br/>知识集"]
|
||||
|
||||
%% 关系连接
|
||||
Dataset -->|HAS_FIELD| Field
|
||||
Dataset -->|DERIVED_FROM| Dataset
|
||||
Dataset -->|BELONGS_TO| Org
|
||||
|
||||
Job -->|USES_DATASET| Dataset
|
||||
Job -->|PRODUCES| Dataset
|
||||
Job -->|DEPENDS_ON| Job
|
||||
|
||||
Workflow -->|TRIGGERS| Job
|
||||
Workflow -->|USES_DATASET| Dataset
|
||||
|
||||
LabelTask -->|USES_DATASET| Dataset
|
||||
LabelTask -->|ASSIGNED_TO| User
|
||||
|
||||
User -->|BELONGS_TO| Org
|
||||
|
||||
Field -->|IMPACTS| Field
|
||||
|
||||
KnowledgeSet -->|SOURCED_FROM| Dataset
|
||||
|
||||
%% 样式
|
||||
classDef dataAsset fill:#4A90D9,stroke:#2C5F8A,color:#fff,stroke-width:2px
|
||||
classDef task fill:#7B68EE,stroke:#5A4CB5,color:#fff,stroke-width:2px
|
||||
classDef actor fill:#50C878,stroke:#3A9B5B,color:#fff,stroke-width:2px
|
||||
classDef knowledge fill:#FFB347,stroke:#CC8F39,color:#fff,stroke-width:2px
|
||||
|
||||
class Dataset,Field dataAsset
|
||||
class LabelTask,Workflow,Job task
|
||||
class User,Org actor
|
||||
class KnowledgeSet knowledge
|
||||
```
|
||||
|
||||
## 分领域视图
|
||||
|
||||
### 数据血缘视图
|
||||
|
||||
展示数据集之间的派生关系和字段级血缘。
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph 源数据层
|
||||
DS_RAW["Dataset<br/>原始数据集"]
|
||||
F1["Field: user_id"]
|
||||
F2["Field: event_type"]
|
||||
F3["Field: timestamp"]
|
||||
end
|
||||
|
||||
subgraph 处理层
|
||||
JOB_CLEAN["Job<br/>清洗作业"]
|
||||
JOB_SYNTH["Job<br/>合成作业"]
|
||||
end
|
||||
|
||||
subgraph 产出数据层
|
||||
DS_CLEAN["Dataset<br/>清洗后数据集"]
|
||||
DS_SYNTH["Dataset<br/>合成数据集"]
|
||||
F1_CLEAN["Field: user_id"]
|
||||
F4["Field: user_segment"]
|
||||
end
|
||||
|
||||
DS_RAW -->|HAS_FIELD| F1
|
||||
DS_RAW -->|HAS_FIELD| F2
|
||||
DS_RAW -->|HAS_FIELD| F3
|
||||
|
||||
JOB_CLEAN -->|USES_DATASET| DS_RAW
|
||||
JOB_CLEAN -->|PRODUCES| DS_CLEAN
|
||||
JOB_SYNTH -->|USES_DATASET| DS_CLEAN
|
||||
JOB_SYNTH -->|PRODUCES| DS_SYNTH
|
||||
|
||||
DS_CLEAN -->|DERIVED_FROM| DS_RAW
|
||||
DS_SYNTH -->|DERIVED_FROM| DS_CLEAN
|
||||
|
||||
DS_CLEAN -->|HAS_FIELD| F1_CLEAN
|
||||
DS_SYNTH -->|HAS_FIELD| F4
|
||||
|
||||
F1 -->|IMPACTS| F1_CLEAN
|
||||
F1_CLEAN -->|IMPACTS| F4
|
||||
|
||||
classDef source fill:#E8F4FD,stroke:#4A90D9,color:#333
|
||||
classDef process fill:#F3E8FF,stroke:#7B68EE,color:#333
|
||||
classDef output fill:#E8FFF0,stroke:#50C878,color:#333
|
||||
|
||||
class DS_RAW,F1,F2,F3 source
|
||||
class JOB_CLEAN,JOB_SYNTH process
|
||||
class DS_CLEAN,DS_SYNTH,F1_CLEAN,F4 output
|
||||
```
|
||||
|
||||
### 任务编排视图
|
||||
|
||||
展示工作流、作业和任务之间的编排关系。
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph 工作流定义
|
||||
WF_CLEAN["Workflow<br/>清洗管道"]
|
||||
WF_EVAL["Workflow<br/>评估管道"]
|
||||
end
|
||||
|
||||
subgraph 作业执行
|
||||
JOB1["Job<br/>清洗作业 #1"]
|
||||
JOB2["Job<br/>清洗作业 #2"]
|
||||
JOB3["Job<br/>评估作业"]
|
||||
end
|
||||
|
||||
subgraph 标注任务
|
||||
LT1["LabelTask<br/>人工标注"]
|
||||
LT2["LabelTask<br/>自动标注"]
|
||||
end
|
||||
|
||||
subgraph 人员
|
||||
U1["User<br/>张三"]
|
||||
U2["User<br/>李四"]
|
||||
end
|
||||
|
||||
WF_CLEAN -->|TRIGGERS| JOB1
|
||||
WF_CLEAN -->|TRIGGERS| JOB2
|
||||
WF_EVAL -->|TRIGGERS| JOB3
|
||||
|
||||
JOB2 -->|DEPENDS_ON| JOB1
|
||||
JOB3 -->|DEPENDS_ON| JOB2
|
||||
|
||||
LT1 -->|ASSIGNED_TO| U1
|
||||
LT2 -->|ASSIGNED_TO| U2
|
||||
|
||||
classDef wf fill:#7B68EE,stroke:#5A4CB5,color:#fff
|
||||
classDef job fill:#9B8FFF,stroke:#7B68EE,color:#fff
|
||||
classDef task fill:#B8A9FF,stroke:#9B8FFF,color:#fff
|
||||
classDef user fill:#50C878,stroke:#3A9B5B,color:#fff
|
||||
|
||||
class WF_CLEAN,WF_EVAL wf
|
||||
class JOB1,JOB2,JOB3 job
|
||||
class LT1,LT2 task
|
||||
class U1,U2 user
|
||||
```
|
||||
|
||||
### 组织归属视图
|
||||
|
||||
展示用户、数据集与组织的归属关系。
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph 组织
|
||||
ORG1["Org<br/>数据工程部"]
|
||||
ORG2["Org<br/>AI研发部"]
|
||||
end
|
||||
|
||||
subgraph 人员
|
||||
U1["User: 张三"]
|
||||
U2["User: 李四"]
|
||||
U3["User: 王五"]
|
||||
end
|
||||
|
||||
subgraph 数据资产
|
||||
DS1["Dataset: 用户行为日志"]
|
||||
DS2["Dataset: 医学影像集"]
|
||||
DS3["Dataset: 训练数据集"]
|
||||
end
|
||||
|
||||
U1 -->|BELONGS_TO| ORG1
|
||||
U2 -->|BELONGS_TO| ORG1
|
||||
U3 -->|BELONGS_TO| ORG2
|
||||
|
||||
DS1 -->|BELONGS_TO| ORG1
|
||||
DS2 -->|BELONGS_TO| ORG2
|
||||
DS3 -->|BELONGS_TO| ORG2
|
||||
|
||||
classDef org fill:#FFB347,stroke:#CC8F39,color:#fff
|
||||
classDef user fill:#50C878,stroke:#3A9B5B,color:#fff
|
||||
classDef data fill:#4A90D9,stroke:#2C5F8A,color:#fff
|
||||
|
||||
class ORG1,ORG2 org
|
||||
class U1,U2,U3 user
|
||||
class DS1,DS2,DS3 data
|
||||
```
|
||||
|
||||
### 知识溯源视图
|
||||
|
||||
展示知识集与数据集的溯源关系。
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
subgraph 数据源
|
||||
DS1["Dataset<br/>用户行为日志"]
|
||||
DS2["Dataset<br/>产品文档"]
|
||||
end
|
||||
|
||||
subgraph 知识资产
|
||||
KS1["KnowledgeSet<br/>用户行为知识库"]
|
||||
end
|
||||
|
||||
subgraph 标注
|
||||
LT["LabelTask<br/>知识标注"]
|
||||
end
|
||||
|
||||
KS1 -->|SOURCED_FROM| DS1
|
||||
KS1 -->|SOURCED_FROM| DS2
|
||||
LT -->|USES_DATASET| DS1
|
||||
|
||||
classDef data fill:#4A90D9,stroke:#2C5F8A,color:#fff
|
||||
classDef knowledge fill:#FFB347,stroke:#CC8F39,color:#fff
|
||||
classDef task fill:#7B68EE,stroke:#5A4CB5,color:#fff
|
||||
|
||||
class DS1,DS2 data
|
||||
class KS1 knowledge
|
||||
class LT task
|
||||
```
|
||||
|
||||
## 综合示例:完整数据流
|
||||
|
||||
展示从原始数据到知识资产的完整处理链路。
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
%% 组织和人员
|
||||
ORG["Org: 数据工程部"]
|
||||
USER["User: 张三"]
|
||||
|
||||
%% 数据资产
|
||||
DS_RAW["Dataset: 原始日志"]
|
||||
DS_CLEAN["Dataset: 清洗数据"]
|
||||
F_UID_RAW["Field: user_id (原始)"]
|
||||
F_UID_CLEAN["Field: user_id (清洗)"]
|
||||
|
||||
%% 处理流程
|
||||
WF["Workflow: 清洗管道"]
|
||||
JOB["Job: 清洗作业"]
|
||||
LT["LabelTask: 情感标注"]
|
||||
|
||||
%% 知识
|
||||
KS["KnowledgeSet: 行为知识库"]
|
||||
|
||||
%% 组织归属
|
||||
USER -->|BELONGS_TO| ORG
|
||||
DS_RAW -->|BELONGS_TO| ORG
|
||||
|
||||
%% 数据结构
|
||||
DS_RAW -->|HAS_FIELD| F_UID_RAW
|
||||
DS_CLEAN -->|HAS_FIELD| F_UID_CLEAN
|
||||
|
||||
%% 处理链路
|
||||
WF -->|TRIGGERS| JOB
|
||||
JOB -->|USES_DATASET| DS_RAW
|
||||
JOB -->|PRODUCES| DS_CLEAN
|
||||
DS_CLEAN -->|DERIVED_FROM| DS_RAW
|
||||
|
||||
%% 字段血缘
|
||||
F_UID_RAW -->|IMPACTS| F_UID_CLEAN
|
||||
|
||||
%% 任务分配
|
||||
LT -->|USES_DATASET| DS_CLEAN
|
||||
LT -->|ASSIGNED_TO| USER
|
||||
|
||||
%% 知识溯源
|
||||
KS -->|SOURCED_FROM| DS_CLEAN
|
||||
|
||||
%% 样式
|
||||
classDef org fill:#FFB347,stroke:#CC8F39,color:#fff,stroke-width:2px
|
||||
classDef user fill:#50C878,stroke:#3A9B5B,color:#fff,stroke-width:2px
|
||||
classDef data fill:#4A90D9,stroke:#2C5F8A,color:#fff,stroke-width:2px
|
||||
classDef field fill:#87CEEB,stroke:#4A90D9,color:#333,stroke-width:1px
|
||||
classDef process fill:#7B68EE,stroke:#5A4CB5,color:#fff,stroke-width:2px
|
||||
classDef knowledge fill:#FF6B6B,stroke:#CC5555,color:#fff,stroke-width:2px
|
||||
|
||||
class ORG org
|
||||
class USER user
|
||||
class DS_RAW,DS_CLEAN data
|
||||
class F_UID_RAW,F_UID_CLEAN field
|
||||
class WF,JOB,LT process
|
||||
class KS knowledge
|
||||
```
|
||||
|
||||
## 图例
|
||||
|
||||
| 颜色 | 分类 | 包含实体 |
|
||||
|------|------|---------|
|
||||
| 蓝色 | 数据资产 | Dataset, Field |
|
||||
| 紫色 | 任务/流程 | Workflow, Job, LabelTask |
|
||||
| 绿色 | 人员 | User, Org |
|
||||
| 橙色/红色 | 知识 | KnowledgeSet |
|
||||
585
docs/knowledge-graph/schema/relationships.md
Normal file
585
docs/knowledge-graph/schema/relationships.md
Normal file
@@ -0,0 +1,585 @@
|
||||
# DataMate 知识图谱 - 核心关系定义
|
||||
|
||||
> Schema 版本:1.0.0
|
||||
> 更新日期:2026-02-17
|
||||
|
||||
## 概述
|
||||
|
||||
DataMate 知识图谱定义了 **10 类核心关系**,覆盖数据血缘、任务编排、组织归属和知识溯源四大场景。
|
||||
|
||||
所有关系在 Neo4j 中统一使用 `RELATED_TO` 关系类型,通过 `relation_type` 属性区分语义类型。每个关系都包含以下公共属性:
|
||||
|
||||
| 公共属性 | 类型 | 必填 | 说明 |
|
||||
|---------|------|------|------|
|
||||
| `id` | String (UUID) | 是 | 关系唯一标识符 |
|
||||
| `relation_type` | String | 是 | 语义关系类型(见下文各类型定义) |
|
||||
| `graph_id` | String (UUID) | 是 | 所属图谱 ID |
|
||||
| `weight` | Double | 否 | 关系权重 0.0-1.0(默认 1.0) |
|
||||
| `confidence` | Double | 否 | 置信度 0.0-1.0(同步数据默认 1.0,抽取数据由模型评分) |
|
||||
| `source_id` | String | 否 | 来源记录 ID |
|
||||
| `properties_json` | String | 否 | 扩展属性 JSON |
|
||||
| `created_at` | LocalDateTime | 是 | 创建时间 |
|
||||
|
||||
### 关系方向约定
|
||||
|
||||
所有关系均为有向关系。方向表示语义上的"主动方 → 被动方"关系:
|
||||
- `(A)-[:RELATED_TO {relation_type: 'HAS_FIELD'}]->(B)` 表示 A 拥有 B
|
||||
- 查询时应注意方向,反向查询需要使用 `<-[]-` 语法
|
||||
|
||||
---
|
||||
|
||||
## 1. HAS_FIELD(包含字段)
|
||||
|
||||
**方向**:`Dataset → Field`
|
||||
|
||||
表示数据集包含某个字段/列。这是数据血缘分析的基础关系,支撑字段级影响评估。
|
||||
|
||||
### 关系属性
|
||||
|
||||
| 属性 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `ordinal` | Integer | 否 | 字段在数据集中的位置(从 0 开始) |
|
||||
| `required` | Boolean | 否 | 是否为必填字段 |
|
||||
|
||||
### 约束
|
||||
|
||||
- 源实体类型必须为 `Dataset`
|
||||
- 目标实体类型必须为 `Field`
|
||||
- 同一 Dataset → Field 对不应重复
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
// 创建 HAS_FIELD 关系
|
||||
MATCH (d:Entity {id: $datasetId, graph_id: $graphId})
|
||||
MATCH (f:Entity {id: $fieldId, graph_id: $graphId})
|
||||
CREATE (d)-[r:RELATED_TO {
|
||||
id: randomUUID(),
|
||||
relation_type: 'HAS_FIELD',
|
||||
graph_id: $graphId,
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
source_id: '',
|
||||
properties_json: '{"ordinal": 0, "required": true}',
|
||||
created_at: datetime()
|
||||
}]->(f)
|
||||
|
||||
// 查询数据集的所有字段
|
||||
MATCH (d:Entity {id: $datasetId, graph_id: $graphId})
|
||||
-[r:RELATED_TO {relation_type: 'HAS_FIELD', graph_id: $graphId}]->
|
||||
(f:Entity {graph_id: $graphId})
|
||||
RETURN f ORDER BY r.properties_json
|
||||
```
|
||||
|
||||
### 业务场景
|
||||
|
||||
- 查看数据集包含哪些字段
|
||||
- 字段搜索:找到包含 `user_id` 字段的所有数据集
|
||||
- Schema 对比:比较两个数据集的字段差异
|
||||
|
||||
---
|
||||
|
||||
## 2. DERIVED_FROM(派生自)
|
||||
|
||||
**方向**:`Dataset → Dataset`
|
||||
|
||||
表示数据集之间的血缘关系:目标数据集是源数据集经过某种处理后派生出来的。涵盖数据清洗、数据合成、版本迭代等场景。
|
||||
|
||||
### 关系属性
|
||||
|
||||
| 属性 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `derivation_type` | String | 是 | 派生类型:`CLEANING`(清洗)/ `SYNTHESIS`(合成)/ `SPLIT`(拆分)/ `MERGE`(合并)/ `VERSION`(版本迭代) |
|
||||
| `job_id` | String | 否 | 产生该派生关系的作业 ID |
|
||||
| `transformation` | String | 否 | 转换描述(如"去重 + 格式标准化") |
|
||||
|
||||
### 约束
|
||||
|
||||
- 源实体和目标实体类型均为 `Dataset`
|
||||
- 不允许自引用(源 ≠ 目标)
|
||||
- 建议检查避免循环依赖
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
// 创建清洗派生关系
|
||||
MATCH (output:Entity {id: $outputDatasetId, graph_id: $graphId})
|
||||
MATCH (input:Entity {id: $inputDatasetId, graph_id: $graphId})
|
||||
CREATE (output)-[r:RELATED_TO {
|
||||
id: randomUUID(),
|
||||
relation_type: 'DERIVED_FROM',
|
||||
graph_id: $graphId,
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
properties_json: '{"derivation_type":"CLEANING","job_id":"d3e4f5a6-...","transformation":"SimHash去重 + 空值过滤"}',
|
||||
created_at: datetime()
|
||||
}]->(input)
|
||||
|
||||
// 追踪数据血缘(最多 5 跳)
|
||||
MATCH path = (d:Entity {id: $datasetId, graph_id: $graphId})
|
||||
-[:RELATED_TO *1..5 {relation_type: 'DERIVED_FROM'}]->
|
||||
(ancestor:Entity {graph_id: $graphId})
|
||||
RETURN path
|
||||
```
|
||||
|
||||
### 业务场景
|
||||
|
||||
- **数据血缘追踪**:追溯数据集的来源链路
|
||||
- **影响分析**:当源数据集变更时,哪些下游数据集受影响
|
||||
- **版本管理**:查看数据集的版本演进历史
|
||||
|
||||
---
|
||||
|
||||
## 3. USES_DATASET(使用数据集)
|
||||
|
||||
**方向**:`Job | LabelTask | Workflow → Dataset`
|
||||
|
||||
表示作业、标注任务或工作流使用某个数据集作为输入。
|
||||
|
||||
### 关系属性
|
||||
|
||||
| 属性 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `usage_role` | String | 否 | 使用角色:`INPUT`(输入)/ `REFERENCE`(参考)/ `VALIDATION`(验证) |
|
||||
|
||||
### 约束
|
||||
|
||||
- 源实体类型为 `Job`、`LabelTask` 或 `Workflow`
|
||||
- 目标实体类型为 `Dataset`
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
// 创建使用关系
|
||||
MATCH (j:Entity {id: $jobId, graph_id: $graphId})
|
||||
MATCH (d:Entity {id: $datasetId, graph_id: $graphId})
|
||||
CREATE (j)-[r:RELATED_TO {
|
||||
id: randomUUID(),
|
||||
relation_type: 'USES_DATASET',
|
||||
graph_id: $graphId,
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
properties_json: '{"usage_role":"INPUT"}',
|
||||
created_at: datetime()
|
||||
}]->(d)
|
||||
|
||||
// 查询数据集被哪些作业使用
|
||||
MATCH (j:Entity {graph_id: $graphId})
|
||||
-[r:RELATED_TO {relation_type: 'USES_DATASET', graph_id: $graphId}]->
|
||||
(d:Entity {id: $datasetId, graph_id: $graphId})
|
||||
RETURN j
|
||||
```
|
||||
|
||||
### 业务场景
|
||||
|
||||
- 查看数据集的消费者:谁在使用这个数据集
|
||||
- 评估数据集的重要程度:被多少任务依赖
|
||||
- 任务输入追溯:任务使用了哪些数据集
|
||||
|
||||
---
|
||||
|
||||
## 4. PRODUCES(产出)
|
||||
|
||||
**方向**:`Job → Dataset`
|
||||
|
||||
表示作业执行后产出了一个新的数据集。与 `USES_DATASET` 相对,构成完整的输入输出链路。
|
||||
|
||||
### 关系属性
|
||||
|
||||
| 属性 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `output_type` | String | 否 | 产出类型:`PRIMARY`(主输出)/ `SECONDARY`(副产物,如日志、统计报告) |
|
||||
|
||||
### 约束
|
||||
|
||||
- 源实体类型为 `Job`
|
||||
- 目标实体类型为 `Dataset`
|
||||
- 一个 Job 可以产出多个 Dataset(如主输出 + 统计报告)
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
// 创建产出关系
|
||||
MATCH (j:Entity {id: $jobId, graph_id: $graphId})
|
||||
MATCH (d:Entity {id: $outputDatasetId, graph_id: $graphId})
|
||||
CREATE (j)-[r:RELATED_TO {
|
||||
id: randomUUID(),
|
||||
relation_type: 'PRODUCES',
|
||||
graph_id: $graphId,
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
properties_json: '{"output_type":"PRIMARY"}',
|
||||
created_at: datetime()
|
||||
}]->(d)
|
||||
|
||||
// 查看作业的完整输入输出
|
||||
MATCH (input:Entity {graph_id: $graphId})
|
||||
<-[:RELATED_TO {relation_type: 'USES_DATASET'}]-
|
||||
(j:Entity {id: $jobId, graph_id: $graphId})
|
||||
-[:RELATED_TO {relation_type: 'PRODUCES'}]->
|
||||
(output:Entity {graph_id: $graphId})
|
||||
RETURN input, j, output
|
||||
```
|
||||
|
||||
### 业务场景
|
||||
|
||||
- **端到端血缘**:结合 `USES_DATASET` 查看 Input → Job → Output 完整链路
|
||||
- **产出追踪**:查看作业产出了哪些数据集
|
||||
- **成本归因**:将产出数据集的成本归因到执行作业
|
||||
|
||||
---
|
||||
|
||||
## 5. ASSIGNED_TO(分配给)
|
||||
|
||||
**方向**:`LabelTask | Job → User`
|
||||
|
||||
表示任务被分配给某个用户执行。
|
||||
|
||||
### 关系属性
|
||||
|
||||
| 属性 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `assigned_at` | String | 否 | 分配时间(ISO 8601) |
|
||||
| `role` | String | 否 | 分配角色:`EXECUTOR`(执行者)/ `REVIEWER`(审核者)/ `OWNER`(负责人) |
|
||||
|
||||
### 约束
|
||||
|
||||
- 源实体类型为 `LabelTask` 或 `Job`
|
||||
- 目标实体类型为 `User`
|
||||
- 同一任务可分配给多个用户(不同角色)
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
// 创建分配关系
|
||||
MATCH (t:Entity {id: $taskId, graph_id: $graphId})
|
||||
MATCH (u:Entity {id: $userId, graph_id: $graphId})
|
||||
CREATE (t)-[r:RELATED_TO {
|
||||
id: randomUUID(),
|
||||
relation_type: 'ASSIGNED_TO',
|
||||
graph_id: $graphId,
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
properties_json: '{"assigned_at":"2026-02-15T10:00:00","role":"EXECUTOR"}',
|
||||
created_at: datetime()
|
||||
}]->(u)
|
||||
|
||||
// 查询用户的所有待办任务
|
||||
MATCH (t:Entity {graph_id: $graphId})
|
||||
-[r:RELATED_TO {relation_type: 'ASSIGNED_TO', graph_id: $graphId}]->
|
||||
(u:Entity {id: $userId, graph_id: $graphId})
|
||||
RETURN t
|
||||
```
|
||||
|
||||
### 业务场景
|
||||
|
||||
- **工作量分析**:查看用户被分配了多少任务
|
||||
- **任务追踪**:查看任务的执行者和审核者
|
||||
- **人员负载均衡**:分析团队内任务分配情况
|
||||
|
||||
---
|
||||
|
||||
## 6. BELONGS_TO(归属于)
|
||||
|
||||
**方向**:`User → Org` 或 `Dataset → Org`
|
||||
|
||||
表示用户属于某个组织,或数据集归属于某个组织。
|
||||
|
||||
### 关系属性
|
||||
|
||||
| 属性 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `membership_type` | String | 否 | 归属类型:`PRIMARY`(主归属)/ `SECONDARY`(兼任/共享) |
|
||||
| `since` | String | 否 | 归属起始时间(ISO 8601) |
|
||||
|
||||
### 约束
|
||||
|
||||
- 源实体类型为 `User` 或 `Dataset`
|
||||
- 目标实体类型为 `Org`
|
||||
- User → Org 通常为 1:1(主归属),但允许兼任
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
// 用户归属组织
|
||||
MATCH (u:Entity {id: $userId, graph_id: $graphId})
|
||||
MATCH (o:Entity {id: $orgId, graph_id: $graphId})
|
||||
CREATE (u)-[r:RELATED_TO {
|
||||
id: randomUUID(),
|
||||
relation_type: 'BELONGS_TO',
|
||||
graph_id: $graphId,
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
properties_json: '{"membership_type":"PRIMARY","since":"2025-03-01T00:00:00"}',
|
||||
created_at: datetime()
|
||||
}]->(o)
|
||||
|
||||
// 查询组织下的所有数据资产
|
||||
MATCH (d:Entity {type: 'Dataset', graph_id: $graphId})
|
||||
-[:RELATED_TO {relation_type: 'BELONGS_TO', graph_id: $graphId}]->
|
||||
(o:Entity {id: $orgId, graph_id: $graphId})
|
||||
RETURN d
|
||||
```
|
||||
|
||||
### 业务场景
|
||||
|
||||
- **组织资产看板**:查看组织拥有的所有数据集
|
||||
- **权限继承**:基于组织关系推导数据访问权限
|
||||
- **跨组织协作**:发现共享数据集的组织关系
|
||||
|
||||
---
|
||||
|
||||
## 7. TRIGGERS(触发)
|
||||
|
||||
**方向**:`Workflow → Job`
|
||||
|
||||
表示工作流触发了一次作业执行。
|
||||
|
||||
### 关系属性
|
||||
|
||||
| 属性 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `trigger_type` | String | 否 | 触发方式:`MANUAL`(手动)/ `SCHEDULED`(定时)/ `EVENT`(事件驱动) |
|
||||
| `triggered_at` | String | 否 | 触发时间(ISO 8601) |
|
||||
|
||||
### 约束
|
||||
|
||||
- 源实体类型为 `Workflow`
|
||||
- 目标实体类型为 `Job`
|
||||
- 一个 Workflow 可触发多个 Job(每次执行产生一个)
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
// 创建触发关系
|
||||
MATCH (w:Entity {id: $workflowId, graph_id: $graphId})
|
||||
MATCH (j:Entity {id: $jobId, graph_id: $graphId})
|
||||
CREATE (w)-[r:RELATED_TO {
|
||||
id: randomUUID(),
|
||||
relation_type: 'TRIGGERS',
|
||||
graph_id: $graphId,
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
properties_json: '{"trigger_type":"SCHEDULED","triggered_at":"2026-02-15T10:00:00"}',
|
||||
created_at: datetime()
|
||||
}]->(j)
|
||||
|
||||
// 查询工作流的执行历史
|
||||
MATCH (w:Entity {id: $workflowId, graph_id: $graphId})
|
||||
-[r:RELATED_TO {relation_type: 'TRIGGERS', graph_id: $graphId}]->
|
||||
(j:Entity {graph_id: $graphId})
|
||||
RETURN j ORDER BY r.created_at DESC
|
||||
```
|
||||
|
||||
### 业务场景
|
||||
|
||||
- **执行历史**:查看工作流的所有执行记录
|
||||
- **故障排查**:定位工作流最近一次失败的作业
|
||||
- **运行统计**:统计工作流的执行频率和成功率
|
||||
|
||||
---
|
||||
|
||||
## 8. DEPENDS_ON(依赖)
|
||||
|
||||
**方向**:`Job → Job`
|
||||
|
||||
表示作业之间的执行依赖关系:源作业的执行依赖于目标作业的完成。
|
||||
|
||||
### 关系属性
|
||||
|
||||
| 属性 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `dependency_type` | String | 否 | 依赖类型:`STRICT`(强依赖,必须成功)/ `SOFT`(弱依赖,失败可继续) |
|
||||
|
||||
### 约束
|
||||
|
||||
- 源实体和目标实体类型均为 `Job`
|
||||
- 不允许自引用
|
||||
- 不允许循环依赖(应用层校验)
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
// 创建依赖关系
|
||||
MATCH (j1:Entity {id: $jobId, graph_id: $graphId})
|
||||
MATCH (j2:Entity {id: $dependsOnJobId, graph_id: $graphId})
|
||||
CREATE (j1)-[r:RELATED_TO {
|
||||
id: randomUUID(),
|
||||
relation_type: 'DEPENDS_ON',
|
||||
graph_id: $graphId,
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
properties_json: '{"dependency_type":"STRICT"}',
|
||||
created_at: datetime()
|
||||
}]->(j2)
|
||||
|
||||
// 查询作业的完整依赖链
|
||||
MATCH path = (j:Entity {id: $jobId, graph_id: $graphId})
|
||||
-[:RELATED_TO *1..10 {relation_type: 'DEPENDS_ON'}]->
|
||||
(dep:Entity {graph_id: $graphId})
|
||||
RETURN path
|
||||
```
|
||||
|
||||
### 业务场景
|
||||
|
||||
- **DAG 执行调度**:确定作业执行顺序
|
||||
- **失败传播分析**:当某个作业失败,哪些下游作业受影响
|
||||
- **关键路径分析**:找到最长依赖链,识别瓶颈
|
||||
|
||||
---
|
||||
|
||||
## 9. IMPACTS(影响)
|
||||
|
||||
**方向**:`Field → Field`
|
||||
|
||||
表示字段之间的影响关系:源字段的变更会影响目标字段。这是跨数据集的字段级血缘关系。
|
||||
|
||||
### 关系属性
|
||||
|
||||
| 属性 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `impact_type` | String | 否 | 影响类型:`DIRECT`(直接映射)/ `TRANSFORM`(转换派生)/ `AGGREGATE`(聚合计算) |
|
||||
| `transformation_rule` | String | 否 | 转换规则描述(如"UPPER(source.name)") |
|
||||
| `job_id` | String | 否 | 建立该影响关系的作业 ID |
|
||||
|
||||
### 约束
|
||||
|
||||
- 源实体和目标实体类型均为 `Field`
|
||||
- 通常跨越不同 Dataset(但同 Dataset 内的字段派生也允许)
|
||||
- 不允许自引用
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
// 创建字段影响关系
|
||||
MATCH (f1:Entity {id: $sourceFieldId, graph_id: $graphId})
|
||||
MATCH (f2:Entity {id: $targetFieldId, graph_id: $graphId})
|
||||
CREATE (f1)-[r:RELATED_TO {
|
||||
id: randomUUID(),
|
||||
relation_type: 'IMPACTS',
|
||||
graph_id: $graphId,
|
||||
weight: 0.8,
|
||||
confidence: 0.9,
|
||||
properties_json: '{"impact_type":"TRANSFORM","transformation_rule":"TRIM(LOWER(source))","job_id":"d3e4f5a6-..."}',
|
||||
created_at: datetime()
|
||||
}]->(f2)
|
||||
|
||||
// 查询字段的影响范围(下游)
|
||||
MATCH (f:Entity {id: $fieldId, graph_id: $graphId})
|
||||
-[:RELATED_TO *1..5 {relation_type: 'IMPACTS'}]->
|
||||
(downstream:Entity {graph_id: $graphId})
|
||||
RETURN downstream
|
||||
```
|
||||
|
||||
### 业务场景
|
||||
|
||||
- **字段级血缘**:追踪字段从源到目标的完整链路
|
||||
- **影响评估**:修改某个字段前,评估下游影响范围
|
||||
- **数据质量追溯**:发现下游字段质量问题时,回溯源头
|
||||
|
||||
---
|
||||
|
||||
## 10. SOURCED_FROM(来源于)
|
||||
|
||||
**方向**:`KnowledgeSet → Dataset`
|
||||
|
||||
表示知识集的知识内容来源于某个数据集,是知识溯源的基础关系。
|
||||
|
||||
### 关系属性
|
||||
|
||||
| 属性 | 类型 | 必填 | 说明 |
|
||||
|------|------|------|------|
|
||||
| `extraction_method` | String | 否 | 抽取方式:`LLM`(LLM 抽取)/ `RULE`(规则抽取)/ `MANUAL`(人工整理) |
|
||||
| `extracted_at` | String | 否 | 抽取时间(ISO 8601) |
|
||||
| `item_count` | Integer | 否 | 从该数据集抽取的知识条目数 |
|
||||
|
||||
### 约束
|
||||
|
||||
- 源实体类型为 `KnowledgeSet`
|
||||
- 目标实体类型为 `Dataset`
|
||||
- 一个 KnowledgeSet 可来源于多个 Dataset
|
||||
|
||||
### Cypher 示例
|
||||
|
||||
```cypher
|
||||
// 创建来源关系
|
||||
MATCH (k:Entity {id: $knowledgeSetId, graph_id: $graphId})
|
||||
MATCH (d:Entity {id: $datasetId, graph_id: $graphId})
|
||||
CREATE (k)-[r:RELATED_TO {
|
||||
id: randomUUID(),
|
||||
relation_type: 'SOURCED_FROM',
|
||||
graph_id: $graphId,
|
||||
weight: 1.0,
|
||||
confidence: 0.85,
|
||||
properties_json: '{"extraction_method":"LLM","extracted_at":"2026-02-10T14:30:00","item_count":120}',
|
||||
created_at: datetime()
|
||||
}]->(d)
|
||||
|
||||
// 查询知识集的所有数据来源
|
||||
MATCH (k:Entity {id: $knowledgeSetId, graph_id: $graphId})
|
||||
-[r:RELATED_TO {relation_type: 'SOURCED_FROM', graph_id: $graphId}]->
|
||||
(d:Entity {graph_id: $graphId})
|
||||
RETURN d, r.properties_json AS extraction_info
|
||||
```
|
||||
|
||||
### 业务场景
|
||||
|
||||
- **知识溯源**:查看知识集基于哪些数据构建
|
||||
- **数据变更通知**:当源数据集更新时,提醒知识集需要刷新
|
||||
- **知识覆盖分析**:查看哪些数据集尚未被纳入知识管理
|
||||
|
||||
---
|
||||
|
||||
## 关系类型汇总
|
||||
|
||||
| 关系类型 | 方向 | relation_type 值 | 核心用途 |
|
||||
|---------|------|-----------------|---------|
|
||||
| HAS_FIELD | Dataset → Field | `HAS_FIELD` | 数据集字段结构 |
|
||||
| DERIVED_FROM | Dataset → Dataset | `DERIVED_FROM` | 数据集级血缘 |
|
||||
| USES_DATASET | Job/LabelTask/Workflow → Dataset | `USES_DATASET` | 输入依赖 |
|
||||
| PRODUCES | Job → Dataset | `PRODUCES` | 输出产出 |
|
||||
| ASSIGNED_TO | LabelTask/Job → User | `ASSIGNED_TO` | 任务分配 |
|
||||
| BELONGS_TO | User/Dataset → Org | `BELONGS_TO` | 组织归属 |
|
||||
| TRIGGERS | Workflow → Job | `TRIGGERS` | 流程触发 |
|
||||
| DEPENDS_ON | Job → Job | `DEPENDS_ON` | 作业依赖 |
|
||||
| IMPACTS | Field → Field | `IMPACTS` | 字段级血缘 |
|
||||
| SOURCED_FROM | KnowledgeSet → Dataset | `SOURCED_FROM` | 知识溯源 |
|
||||
|
||||
## 典型查询模式
|
||||
|
||||
### 1. 端到端数据血缘
|
||||
|
||||
```cypher
|
||||
// 从最终数据集追溯到原始数据集,经过的所有处理步骤
|
||||
MATCH path = (final:Entity {id: $datasetId, graph_id: $graphId})
|
||||
-[:RELATED_TO *1..10]->
|
||||
(origin:Entity {graph_id: $graphId})
|
||||
WHERE ALL(r IN relationships(path) WHERE r.relation_type IN ['DERIVED_FROM', 'USES_DATASET', 'PRODUCES'])
|
||||
RETURN path
|
||||
```
|
||||
|
||||
### 2. 数据集影响分析
|
||||
|
||||
```cypher
|
||||
// 查找修改某数据集后,所有受影响的下游实体
|
||||
MATCH (d:Entity {id: $datasetId, graph_id: $graphId})
|
||||
<-[:RELATED_TO {relation_type: 'USES_DATASET'}]-
|
||||
(consumer:Entity {graph_id: $graphId})
|
||||
RETURN consumer.type AS entity_type, consumer.name AS entity_name, consumer.id AS entity_id
|
||||
```
|
||||
|
||||
### 3. 用户工作看板
|
||||
|
||||
```cypher
|
||||
// 查询用户相关的所有实体和关系
|
||||
MATCH (u:Entity {id: $userId, type: 'User', graph_id: $graphId})
|
||||
OPTIONAL MATCH (task:Entity)-[:RELATED_TO {relation_type: 'ASSIGNED_TO'}]->(u)
|
||||
OPTIONAL MATCH (u)-[:RELATED_TO {relation_type: 'BELONGS_TO'}]->(org:Entity)
|
||||
RETURN u, collect(DISTINCT task) AS tasks, collect(DISTINCT org) AS orgs
|
||||
```
|
||||
|
||||
## 扩展说明
|
||||
|
||||
- **自定义关系类型**:除上述 10 类核心关系外,用户可通过 LLM 抽取或手动创建自定义关系类型。自定义关系使用相同的 `RELATED_TO` Neo4j 关系类型和公共属性结构,`relation_type` 字段可为任意字符串。
|
||||
- **双向关系**:所有关系均为单向。如果需要表达双向关系(如"A 和 B 互相影响"),应创建两条方向相反的关系。
|
||||
- **关系去重**:应用层应在创建关系前检查是否已存在相同的(source, target, relation_type)组合,避免重复。
|
||||
469
docs/knowledge-graph/schema/schema.cypher
Normal file
469
docs/knowledge-graph/schema/schema.cypher
Normal file
@@ -0,0 +1,469 @@
|
||||
// =============================================================================
|
||||
// DataMate 知识图谱 - Neo4j Schema 初始化脚本
|
||||
// Schema 版本:1.0.0
|
||||
// 更新日期:2026-02-17
|
||||
//
|
||||
// 使用方式:
|
||||
// 1. 通过 Cypher Shell 执行:
|
||||
// cat schema.cypher | cypher-shell -u neo4j -p <password>
|
||||
// 2. 或在 Neo4j Browser 中逐段执行
|
||||
//
|
||||
// 注意:
|
||||
// - 所有索引和约束使用 IF NOT EXISTS,可重复执行
|
||||
// - 约束自动创建对应索引,无需重复创建
|
||||
// - 关系属性索引需要 Neo4j Enterprise Edition,社区版使用属性内联匹配
|
||||
// =============================================================================
|
||||
|
||||
// =============================================================================
|
||||
// 第 1 部分:节点约束
|
||||
// =============================================================================
|
||||
|
||||
// Entity 节点 ID 唯一性约束(自动创建索引)
|
||||
CREATE CONSTRAINT entity_id_unique IF NOT EXISTS
|
||||
FOR (n:Entity) REQUIRE n.id IS UNIQUE;
|
||||
|
||||
// =============================================================================
|
||||
// 第 2 部分:节点索引
|
||||
// =============================================================================
|
||||
|
||||
// graph_id 索引 —— 多租户隔离的核心索引,所有查询都会带上 graph_id
|
||||
CREATE INDEX entity_graph_id IF NOT EXISTS
|
||||
FOR (n:Entity) ON (n.graph_id);
|
||||
|
||||
// type 索引 —— 按实体类型过滤
|
||||
CREATE INDEX entity_type IF NOT EXISTS
|
||||
FOR (n:Entity) ON (n.type);
|
||||
|
||||
// name 索引 —— 按名称搜索
|
||||
CREATE INDEX entity_name IF NOT EXISTS
|
||||
FOR (n:Entity) ON (n.name);
|
||||
|
||||
// source_id 索引 —— MySQL → Neo4j 同步时按源 ID 查找
|
||||
CREATE INDEX entity_source_id IF NOT EXISTS
|
||||
FOR (n:Entity) ON (n.source_id);
|
||||
|
||||
// 复合索引:(graph_id, type) —— 查询某图谱内指定类型的实体
|
||||
CREATE INDEX entity_graph_id_type IF NOT EXISTS
|
||||
FOR (n:Entity) ON (n.graph_id, n.type);
|
||||
|
||||
// 复合索引:(graph_id, id) —— 精确查找实体(最常用查询路径)
|
||||
CREATE INDEX entity_graph_id_id IF NOT EXISTS
|
||||
FOR (n:Entity) ON (n.graph_id, n.id);
|
||||
|
||||
// 复合索引:(graph_id, source_id) —— 同步时按源 ID 查找
|
||||
CREATE INDEX entity_graph_id_source_id IF NOT EXISTS
|
||||
FOR (n:Entity) ON (n.graph_id, n.source_id);
|
||||
|
||||
// created_at 索引 —— 按创建时间排序
|
||||
CREATE INDEX entity_created_at IF NOT EXISTS
|
||||
FOR (n:Entity) ON (n.created_at);
|
||||
|
||||
// =============================================================================
|
||||
// 第 3 部分:全文索引(用于模糊搜索)
|
||||
// =============================================================================
|
||||
|
||||
// Entity name + description 全文索引
|
||||
CREATE FULLTEXT INDEX entity_fulltext IF NOT EXISTS
|
||||
FOR (n:Entity) ON EACH [n.name, n.description];
|
||||
|
||||
// =============================================================================
|
||||
// 第 3.1 部分:SyncHistory 约束和索引(同步元数据节点)
|
||||
// =============================================================================
|
||||
|
||||
// (graph_id, sync_id) 唯一约束 —— 防止 syncId 碰撞产生重复记录
|
||||
CREATE CONSTRAINT sync_history_graph_sync_unique IF NOT EXISTS
|
||||
FOR (h:SyncHistory) REQUIRE (h.graph_id, h.sync_id) IS UNIQUE;
|
||||
|
||||
// (graph_id, started_at) 索引 —— 加速按时间范围查询同步历史
|
||||
CREATE INDEX sync_history_graph_started IF NOT EXISTS
|
||||
FOR (h:SyncHistory) ON (h.graph_id, h.started_at);
|
||||
|
||||
// (graph_id, status, started_at) 索引 —— 加速按状态+时间的过滤查询
|
||||
CREATE INDEX sync_history_graph_status_started IF NOT EXISTS
|
||||
FOR (h:SyncHistory) ON (h.graph_id, h.status, h.started_at);
|
||||
|
||||
// =============================================================================
|
||||
// 第 4 部分:关系属性说明
|
||||
// =============================================================================
|
||||
|
||||
// Neo4j 社区版不支持关系属性索引。
|
||||
// 所有关系查询通过节点索引定位后,在关系上使用属性内联匹配:
|
||||
// -[r:RELATED_TO {graph_id: $graphId, relation_type: $type}]->
|
||||
//
|
||||
// 如果使用 Neo4j Enterprise Edition,可取消以下注释创建关系索引:
|
||||
//
|
||||
// CREATE INDEX rel_graph_id IF NOT EXISTS
|
||||
// FOR ()-[r:RELATED_TO]-() ON (r.graph_id);
|
||||
//
|
||||
// CREATE INDEX rel_relation_type IF NOT EXISTS
|
||||
// FOR ()-[r:RELATED_TO]-() ON (r.relation_type);
|
||||
//
|
||||
// CREATE INDEX rel_id IF NOT EXISTS
|
||||
// FOR ()-[r:RELATED_TO]-() ON (r.id);
|
||||
|
||||
// =============================================================================
|
||||
// 第 5 部分:示例数据(可选,用于验证 Schema)
|
||||
// =============================================================================
|
||||
|
||||
// 以下示例数据使用固定的 graph_id,用于开发和测试环境。
|
||||
// 生产环境中 graph_id 由应用层生成和管理。
|
||||
|
||||
// --- 创建示例组织 ---
|
||||
CREATE (org:Entity {
|
||||
id: '00000000-0000-0000-0000-000000000001',
|
||||
name: '数据工程部',
|
||||
type: 'Org',
|
||||
description: '负责数据采集、清洗和标注',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
source_type: 'MANUAL',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"org_code":"DE","level":1,"member_count":15}',
|
||||
created_at: datetime()
|
||||
});
|
||||
|
||||
// --- 创建示例用户 ---
|
||||
CREATE (user:Entity {
|
||||
id: '00000000-0000-0000-0000-000000000002',
|
||||
name: '张三',
|
||||
type: 'User',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"username":"zhangsan","email":"zhangsan@example.com","role":"USER","enabled":true}',
|
||||
created_at: datetime()
|
||||
});
|
||||
|
||||
// --- 创建示例数据集(源) ---
|
||||
CREATE (ds1:Entity {
|
||||
id: '00000000-0000-0000-0000-000000000010',
|
||||
name: '用户行为日志-原始',
|
||||
type: 'Dataset',
|
||||
description: '原始用户行为埋点数据',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
source_id: '100',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"dataset_type":"TEXT","status":"ACTIVE","category":"用户行为","format":"JSON","record_count":2000000,"size_bytes":3221225472}',
|
||||
created_at: datetime()
|
||||
});
|
||||
|
||||
// --- 创建示例数据集(清洗后) ---
|
||||
CREATE (ds2:Entity {
|
||||
id: '00000000-0000-0000-0000-000000000011',
|
||||
name: '用户行为日志-清洗后',
|
||||
type: 'Dataset',
|
||||
description: '经过去重和格式标准化的用户行为数据',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
source_id: '101',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"dataset_type":"TEXT","status":"ACTIVE","category":"用户行为","format":"JSON","record_count":1500000,"size_bytes":2147483648,"version":1}',
|
||||
created_at: datetime()
|
||||
});
|
||||
|
||||
// --- 创建示例字段 ---
|
||||
CREATE (f1:Entity {
|
||||
id: '00000000-0000-0000-0000-000000000020',
|
||||
name: 'user_id',
|
||||
type: 'Field',
|
||||
description: '用户唯一标识符',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"data_type":"STRING","nullable":false,"is_primary_key":true}',
|
||||
created_at: datetime()
|
||||
});
|
||||
|
||||
CREATE (f2:Entity {
|
||||
id: '00000000-0000-0000-0000-000000000021',
|
||||
name: 'event_type',
|
||||
type: 'Field',
|
||||
description: '事件类型',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"data_type":"STRING","nullable":false,"sample_values":["click","view","purchase"]}',
|
||||
created_at: datetime()
|
||||
});
|
||||
|
||||
CREATE (f3:Entity {
|
||||
id: '00000000-0000-0000-0000-000000000022',
|
||||
name: 'user_id',
|
||||
type: 'Field',
|
||||
description: '用户唯一标识符(清洗后)',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"data_type":"STRING","nullable":false,"is_primary_key":true}',
|
||||
created_at: datetime()
|
||||
});
|
||||
|
||||
// --- 创建示例工作流 ---
|
||||
CREATE (wf:Entity {
|
||||
id: '00000000-0000-0000-0000-000000000030',
|
||||
name: '文本去重清洗管道',
|
||||
type: 'Workflow',
|
||||
description: 'SimHash去重 + 格式标准化 + 空值过滤',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"workflow_type":"CLEANING","status":"ACTIVE","version":"1.0","operator_count":3}',
|
||||
created_at: datetime()
|
||||
});
|
||||
|
||||
// --- 创建示例作业 ---
|
||||
CREATE (job:Entity {
|
||||
id: '00000000-0000-0000-0000-000000000040',
|
||||
name: '清洗作业-20260215-001',
|
||||
type: 'Job',
|
||||
description: '用户行为日志去重清洗',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
source_id: '500',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"job_type":"CLEANING","status":"COMPLETED","started_at":"2026-02-15T10:00:00","completed_at":"2026-02-15T10:35:00","duration_seconds":2100,"input_count":2000000,"output_count":1500000}',
|
||||
created_at: datetime()
|
||||
});
|
||||
|
||||
// --- 创建示例标注任务 ---
|
||||
CREATE (lt:Entity {
|
||||
id: '00000000-0000-0000-0000-000000000050',
|
||||
name: '情感分析标注-批次1',
|
||||
type: 'LabelTask',
|
||||
description: '用户评论情感标注(正面/负面/中性)',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
source_id: '600',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"task_mode":"MANUAL","data_type":"text","labeling_type":"sentiment_analysis","status":"IN_PROGRESS","progress":30.0,"template_name":"情感分析"}',
|
||||
created_at: datetime()
|
||||
});
|
||||
|
||||
// --- 创建示例知识集 ---
|
||||
CREATE (ks:Entity {
|
||||
id: '00000000-0000-0000-0000-000000000060',
|
||||
name: '用户行为分析知识库',
|
||||
type: 'KnowledgeSet',
|
||||
description: '从用户行为数据中提取的业务规则和洞察',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
source_type: 'SYNC',
|
||||
confidence: 1.0,
|
||||
properties_json: '{"status":"PUBLISHED","domain":"用户行为","business_line":"数据分析","sensitivity":"INTERNAL","item_count":85}',
|
||||
created_at: datetime()
|
||||
});
|
||||
|
||||
// =============================================================================
|
||||
// 第 6 部分:示例关系
|
||||
// =============================================================================
|
||||
|
||||
// HAS_FIELD:源数据集 → 字段
|
||||
MATCH (ds1:Entity {id: '00000000-0000-0000-0000-000000000010'})
|
||||
MATCH (f1:Entity {id: '00000000-0000-0000-0000-000000000020'})
|
||||
CREATE (ds1)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000001',
|
||||
relation_type: 'HAS_FIELD',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
source_id: '',
|
||||
properties_json: '{"ordinal":0,"required":true}',
|
||||
created_at: datetime()
|
||||
}]->(f1);
|
||||
|
||||
MATCH (ds1:Entity {id: '00000000-0000-0000-0000-000000000010'})
|
||||
MATCH (f2:Entity {id: '00000000-0000-0000-0000-000000000021'})
|
||||
CREATE (ds1)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000002',
|
||||
relation_type: 'HAS_FIELD',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
source_id: '',
|
||||
properties_json: '{"ordinal":1,"required":true}',
|
||||
created_at: datetime()
|
||||
}]->(f2);
|
||||
|
||||
// HAS_FIELD:清洗后数据集 → 字段
|
||||
MATCH (ds2:Entity {id: '00000000-0000-0000-0000-000000000011'})
|
||||
MATCH (f3:Entity {id: '00000000-0000-0000-0000-000000000022'})
|
||||
CREATE (ds2)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000003',
|
||||
relation_type: 'HAS_FIELD',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
source_id: '',
|
||||
properties_json: '{"ordinal":0,"required":true}',
|
||||
created_at: datetime()
|
||||
}]->(f3);
|
||||
|
||||
// DERIVED_FROM:清洗后数据集 → 源数据集
|
||||
MATCH (ds2:Entity {id: '00000000-0000-0000-0000-000000000011'})
|
||||
MATCH (ds1:Entity {id: '00000000-0000-0000-0000-000000000010'})
|
||||
CREATE (ds2)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000004',
|
||||
relation_type: 'DERIVED_FROM',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
source_id: '',
|
||||
properties_json: '{"derivation_type":"CLEANING","job_id":"00000000-0000-0000-0000-000000000040","transformation":"SimHash去重 + 空值过滤"}',
|
||||
created_at: datetime()
|
||||
}]->(ds1);
|
||||
|
||||
// TRIGGERS:工作流 → 作业
|
||||
MATCH (wf:Entity {id: '00000000-0000-0000-0000-000000000030'})
|
||||
MATCH (job:Entity {id: '00000000-0000-0000-0000-000000000040'})
|
||||
CREATE (wf)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000005',
|
||||
relation_type: 'TRIGGERS',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
source_id: '',
|
||||
properties_json: '{"trigger_type":"MANUAL","triggered_at":"2026-02-15T10:00:00"}',
|
||||
created_at: datetime()
|
||||
}]->(job);
|
||||
|
||||
// USES_DATASET:作业 → 源数据集
|
||||
MATCH (job:Entity {id: '00000000-0000-0000-0000-000000000040'})
|
||||
MATCH (ds1:Entity {id: '00000000-0000-0000-0000-000000000010'})
|
||||
CREATE (job)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000006',
|
||||
relation_type: 'USES_DATASET',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
source_id: '',
|
||||
properties_json: '{"usage_role":"INPUT"}',
|
||||
created_at: datetime()
|
||||
}]->(ds1);
|
||||
|
||||
// PRODUCES:作业 → 清洗后数据集
|
||||
MATCH (job:Entity {id: '00000000-0000-0000-0000-000000000040'})
|
||||
MATCH (ds2:Entity {id: '00000000-0000-0000-0000-000000000011'})
|
||||
CREATE (job)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000007',
|
||||
relation_type: 'PRODUCES',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
source_id: '',
|
||||
properties_json: '{"output_type":"PRIMARY"}',
|
||||
created_at: datetime()
|
||||
}]->(ds2);
|
||||
|
||||
// ASSIGNED_TO:标注任务 → 用户
|
||||
MATCH (lt:Entity {id: '00000000-0000-0000-0000-000000000050'})
|
||||
MATCH (user:Entity {id: '00000000-0000-0000-0000-000000000002'})
|
||||
CREATE (lt)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000008',
|
||||
relation_type: 'ASSIGNED_TO',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
source_id: '',
|
||||
properties_json: '{"assigned_at":"2026-02-14T09:00:00","role":"EXECUTOR"}',
|
||||
created_at: datetime()
|
||||
}]->(user);
|
||||
|
||||
// USES_DATASET:标注任务 → 清洗后数据集
|
||||
MATCH (lt:Entity {id: '00000000-0000-0000-0000-000000000050'})
|
||||
MATCH (ds2:Entity {id: '00000000-0000-0000-0000-000000000011'})
|
||||
CREATE (lt)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000009',
|
||||
relation_type: 'USES_DATASET',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
source_id: '',
|
||||
properties_json: '{"usage_role":"INPUT"}',
|
||||
created_at: datetime()
|
||||
}]->(ds2);
|
||||
|
||||
// BELONGS_TO:用户 → 组织
|
||||
MATCH (user:Entity {id: '00000000-0000-0000-0000-000000000002'})
|
||||
MATCH (org:Entity {id: '00000000-0000-0000-0000-000000000001'})
|
||||
CREATE (user)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000010',
|
||||
relation_type: 'BELONGS_TO',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
source_id: '',
|
||||
properties_json: '{"membership_type":"PRIMARY","since":"2025-03-01T00:00:00"}',
|
||||
created_at: datetime()
|
||||
}]->(org);
|
||||
|
||||
// BELONGS_TO:源数据集 → 组织
|
||||
MATCH (ds1:Entity {id: '00000000-0000-0000-0000-000000000010'})
|
||||
MATCH (org:Entity {id: '00000000-0000-0000-0000-000000000001'})
|
||||
CREATE (ds1)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000011',
|
||||
relation_type: 'BELONGS_TO',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 1.0,
|
||||
source_id: '',
|
||||
properties_json: '{"membership_type":"PRIMARY"}',
|
||||
created_at: datetime()
|
||||
}]->(org);
|
||||
|
||||
// IMPACTS:源字段 → 清洗后字段
|
||||
MATCH (f1:Entity {id: '00000000-0000-0000-0000-000000000020'})
|
||||
MATCH (f3:Entity {id: '00000000-0000-0000-0000-000000000022'})
|
||||
CREATE (f1)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000012',
|
||||
relation_type: 'IMPACTS',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 0.95,
|
||||
source_id: '',
|
||||
properties_json: '{"impact_type":"DIRECT","job_id":"00000000-0000-0000-0000-000000000040"}',
|
||||
created_at: datetime()
|
||||
}]->(f3);
|
||||
|
||||
// SOURCED_FROM:知识集 → 清洗后数据集
|
||||
MATCH (ks:Entity {id: '00000000-0000-0000-0000-000000000060'})
|
||||
MATCH (ds2:Entity {id: '00000000-0000-0000-0000-000000000011'})
|
||||
CREATE (ks)-[:RELATED_TO {
|
||||
id: '00000000-0000-0000-0000-100000000013',
|
||||
relation_type: 'SOURCED_FROM',
|
||||
graph_id: '11111111-1111-1111-1111-111111111111',
|
||||
weight: 1.0,
|
||||
confidence: 0.85,
|
||||
source_id: '',
|
||||
properties_json: '{"extraction_method":"LLM","extracted_at":"2026-02-16T14:30:00","item_count":85}',
|
||||
created_at: datetime()
|
||||
}]->(ds2);
|
||||
|
||||
// =============================================================================
|
||||
// 第 7 部分:验证查询
|
||||
// =============================================================================
|
||||
|
||||
// 验证节点数量
|
||||
// MATCH (n:Entity {graph_id: '11111111-1111-1111-1111-111111111111'})
|
||||
// RETURN n.type AS type, count(*) AS count
|
||||
// ORDER BY count DESC;
|
||||
|
||||
// 验证关系数量
|
||||
// MATCH (:Entity {graph_id: '11111111-1111-1111-1111-111111111111'})
|
||||
// -[r:RELATED_TO {graph_id: '11111111-1111-1111-1111-111111111111'}]->
|
||||
// (:Entity {graph_id: '11111111-1111-1111-1111-111111111111'})
|
||||
// RETURN r.relation_type AS type, count(*) AS count
|
||||
// ORDER BY count DESC;
|
||||
|
||||
// 验证端到端血缘
|
||||
// MATCH path = (ds2:Entity {name: '用户行为日志-清洗后'})
|
||||
// -[:RELATED_TO *1..5]->
|
||||
// (origin:Entity)
|
||||
// WHERE ALL(r IN relationships(path) WHERE r.graph_id = '11111111-1111-1111-1111-111111111111')
|
||||
// RETURN path;
|
||||
|
||||
// =============================================================================
|
||||
// 第 8 部分:清理示例数据(可选)
|
||||
// =============================================================================
|
||||
|
||||
// 如需清理示例数据,执行以下语句:
|
||||
// MATCH (n:Entity {graph_id: '11111111-1111-1111-1111-111111111111'})
|
||||
// DETACH DELETE n;
|
||||
Reference in New Issue
Block a user