You've already forked DataMate
update data synthesis page ui (#60)
* feat: Update site name to DataMate and refine text for AI data processing * feat: Refactor settings page and implement model access functionality - Created a new ModelAccess component for managing model configurations. - Removed the old Settings component and replaced it with a new SettingsPage component that integrates ModelAccess, SystemConfig, and WebhookConfig. - Added SystemConfig component for managing system settings. - Implemented WebhookConfig component for managing webhook configurations. - Updated API functions for model management in settings.apis.ts. - Adjusted routing to point to the new SettingsPage component. * feat: Implement Data Collection Page with Task Management and Execution Log - Created DataCollectionPage component to manage data collection tasks. - Added TaskManagement and ExecutionLog components for task handling and logging. - Integrated task operations including start, stop, edit, and delete functionalities. - Implemented filtering and searching capabilities in task management. - Introduced SimpleCronScheduler for scheduling tasks with cron expressions. - Updated CreateTask component to utilize new scheduling and template features. - Enhanced BasicInformation component to conditionally render fields based on visibility settings. - Refactored ImportConfiguration component to remove NAS import section. * feat: Update task creation API endpoint and enhance task creation form with new fields and validation * Refactor file upload and operator management components - Removed unnecessary console logs from file download and export functions. - Added size property to TaskItem interface for better task management. - Simplified TaskUpload component by utilizing useFileSliceUpload hook for file upload logic. - Enhanced OperatorPluginCreate component to handle file uploads and parsing more efficiently. - Updated ConfigureStep component to use Ant Design Form for better data handling and validation. - Improved PreviewStep component to navigate back to the operator market. - Added support for additional file types in UploadStep component. - Implemented delete operator functionality in OperatorMarketPage with confirmation prompts. - Cleaned up unused API functions in operator.api.ts to streamline the codebase. - Fixed number formatting utility to handle zero values correctly. * Refactor Knowledge Generation to Knowledge Base - Created new API service for Knowledge Base operations including querying, creating, updating, and deleting knowledge bases and files. - Added constants for Knowledge Base status and type mappings. - Defined models for Knowledge Base and related files. - Removed obsolete Knowledge Base creation and home components, replacing them with new implementations under the Knowledge Base structure. - Updated routing to reflect the new Knowledge Base paths. - Adjusted menu items to align with the new Knowledge Base terminology. - Modified ModelAccess interface to include modelName and type properties. * feat: Implement Knowledge Base Page with CRUD operations and data management - Added KnowledgeBasePage component for displaying and managing knowledge bases. - Integrated search and filter functionalities with SearchControls component. - Implemented CreateKnowledgeBase component for creating and editing knowledge bases. - Enhanced AddDataDialog for file uploads and dataset selections. - Introduced TableTransfer component for managing data transfers between tables. - Updated API functions for knowledge base operations, including file management. - Refactored knowledge base model to include file status and metadata. - Adjusted routing to point to the new KnowledgeBasePage. * feat: enhance OperatorPluginCreate and ConfigureStep for better upload handling and UI updates * refactor: remove unused components and clean up API logging in KnowledgeBase * feat: update icons in various components and improve styling for better UI consistency * fix: adjust upload step handling and improve error display in configuration step * feat: Add RatioTransfer component for dataset selection and configuration - Implemented RatioTransfer component to manage dataset selection and ratio configuration. - Integrated dataset fetching with search and filter capabilities. - Added RatioConfig component for displaying and updating selected datasets' configurations. - Enhanced SelectDataset component with improved UI and functionality for dataset selection. - Updated RatioTasksPage to utilize new ratio task status mapping and improved error handling for task deletion. - Refactored ratio model and constants for better type safety and clarity. - Changed Vite configuration to use local backend service for development.
This commit is contained in:
chenghh-9609
GitHub
@@ -1,5 +1,5 @@
|
||||
import React from "react";
|
||||
import { Badge, Card, Input, Progress } from "antd";
|
||||
import React, { useMemo, useState } from "react";
|
||||
import { Badge, Card, Input, Progress, Button, Divider } from "antd";
|
||||
import { BarChart3 } from "lucide-react";
|
||||
import type { Dataset } from "@/pages/DataManagement/dataset.model.ts";
|
||||
|
||||
@@ -16,32 +16,150 @@ interface RatioConfigProps {
|
||||
ratioType: "dataset" | "label";
|
||||
selectedDatasets: string[];
|
||||
datasets: Dataset[];
|
||||
ratioConfigs: RatioConfigItem[];
|
||||
totalTargetCount: number;
|
||||
distributions: Record<string, Record<string, number>>;
|
||||
onUpdateDatasetQuantity: (datasetId: string, quantity: number) => void;
|
||||
onUpdateLabelQuantity: (datasetId: string, label: string, quantity: number) => void;
|
||||
onChange?: (configs: RatioConfigItem[]) => void;
|
||||
}
|
||||
|
||||
const RatioConfig: React.FC<RatioConfigProps> = ({
|
||||
ratioType,
|
||||
selectedDatasets,
|
||||
datasets,
|
||||
ratioConfigs,
|
||||
totalTargetCount,
|
||||
distributions,
|
||||
onUpdateDatasetQuantity,
|
||||
onUpdateLabelQuantity,
|
||||
onChange,
|
||||
}) => {
|
||||
const totalConfigured = ratioConfigs.reduce((sum, c) => sum + (c.quantity || 0), 0);
|
||||
const [ratioConfigs, setRatioConfigs] = useState<RatioConfigItem[]>([]);
|
||||
|
||||
// 配比项总数
|
||||
const totalConfigured = useMemo(
|
||||
() => ratioConfigs.reduce((sum, c) => sum + (c.quantity || 0), 0),
|
||||
[ratioConfigs]
|
||||
);
|
||||
|
||||
// 更新数据集配比项
|
||||
const updateDatasetQuantity = (datasetId: string, quantity: number) => {
|
||||
setRatioConfigs((prev) => {
|
||||
const existingIndex = prev.findIndex(
|
||||
(config) => config.source === datasetId
|
||||
);
|
||||
const totalOtherQuantity = prev
|
||||
.filter((config) => config.source !== datasetId)
|
||||
.reduce((sum, config) => sum + config.quantity, 0);
|
||||
|
||||
const dataset = datasets.find((d) => String(d.id) === datasetId);
|
||||
const newConfig: RatioConfigItem = {
|
||||
id: datasetId,
|
||||
name: dataset?.name || datasetId,
|
||||
type: ratioType,
|
||||
quantity: Math.min(quantity, totalTargetCount - totalOtherQuantity),
|
||||
percentage: Math.round((quantity / totalTargetCount) * 100),
|
||||
source: datasetId,
|
||||
};
|
||||
|
||||
let newConfigs;
|
||||
if (existingIndex >= 0) {
|
||||
newConfigs = [...prev];
|
||||
newConfigs[existingIndex] = newConfig;
|
||||
} else {
|
||||
newConfigs = [...prev, newConfig];
|
||||
}
|
||||
onChange?.(newConfigs);
|
||||
return newConfigs;
|
||||
});
|
||||
};
|
||||
|
||||
// 自动平均分配
|
||||
const generateAutoRatio = () => {
|
||||
const selectedCount = selectedDatasets.length;
|
||||
if (selectedCount === 0) return;
|
||||
const baseQuantity = Math.floor(totalTargetCount / selectedCount);
|
||||
const remainder = totalTargetCount % selectedCount;
|
||||
const newConfigs = selectedDatasets.map((datasetId, index) => {
|
||||
const dataset = datasets.find((d) => String(d.id) === datasetId);
|
||||
const quantity = baseQuantity + (index < remainder ? 1 : 0);
|
||||
return {
|
||||
id: datasetId,
|
||||
name: dataset?.name || datasetId,
|
||||
type: ratioType,
|
||||
quantity,
|
||||
percentage: Math.round((quantity / totalTargetCount) * 100),
|
||||
source: datasetId,
|
||||
};
|
||||
});
|
||||
setRatioConfigs(newConfigs);
|
||||
onChange?.(newConfigs);
|
||||
};
|
||||
|
||||
// 标签模式下，更新某数据集的某个标签的数量
|
||||
const updateLabelQuantity = (
|
||||
datasetId: string,
|
||||
label: string,
|
||||
quantity: number
|
||||
) => {
|
||||
const sourceKey = `${datasetId}_${label}`;
|
||||
setRatioConfigs((prev) => {
|
||||
const existingIndex = prev.findIndex((c) => c.source === sourceKey);
|
||||
const totalOtherQuantity = prev
|
||||
.filter((c) => c.source !== sourceKey)
|
||||
.reduce((sum, c) => sum + c.quantity, 0);
|
||||
const dist = distributions[datasetId] || {};
|
||||
const labelMax = dist[label] ?? Infinity;
|
||||
const cappedQuantity = Math.max(
|
||||
0,
|
||||
Math.min(quantity, totalTargetCount - totalOtherQuantity, labelMax)
|
||||
);
|
||||
const newConfig: RatioConfigItem = {
|
||||
id: sourceKey,
|
||||
name: label,
|
||||
type: "label",
|
||||
quantity: cappedQuantity,
|
||||
percentage: Math.round((cappedQuantity / totalTargetCount) * 100),
|
||||
source: sourceKey,
|
||||
};
|
||||
let newConfigs;
|
||||
if (existingIndex >= 0) {
|
||||
newConfigs = [...prev];
|
||||
newConfigs[existingIndex] = newConfig;
|
||||
} else {
|
||||
newConfigs = [...prev, newConfig];
|
||||
}
|
||||
onChange?.(newConfigs);
|
||||
return newConfigs;
|
||||
});
|
||||
};
|
||||
|
||||
// 选中数据集变化时，移除未选中的配比项
|
||||
React.useEffect(() => {
|
||||
setRatioConfigs((prev) => {
|
||||
const next = prev.filter((c) => {
|
||||
const id = String(c.source);
|
||||
const dsId = id.includes("_") ? id.split("_")[0] : id;
|
||||
return selectedDatasets.includes(dsId);
|
||||
});
|
||||
if (next !== prev) onChange?.(next);
|
||||
return next;
|
||||
});
|
||||
// eslint-disable-next-line
|
||||
}, [selectedDatasets]);
|
||||
|
||||
return (
|
||||
<div className="mb-4">
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-sm font-medium">配比设置</span>
|
||||
<span className="text-xs text-gray-500">
|
||||
已配置: {totalConfigured} / {totalTargetCount}
|
||||
<div className="border-card flex-1 flex flex-col min-w-[320px]">
|
||||
<div className="flex items-center justify-between p-4 border-bottom">
|
||||
<span className="text-sm font-bold">
|
||||
配比配置
|
||||
<span className="text-xs text-gray-500">
|
||||
(已配置:{totalConfigured}/{totalTargetCount}条)
|
||||
</span>
|
||||
</span>
|
||||
<Button
|
||||
type="link"
|
||||
size="small"
|
||||
onClick={generateAutoRatio}
|
||||
disabled={selectedDatasets.length === 0}
|
||||
>
|
||||
平均分配
|
||||
</Button>
|
||||
</div>
|
||||
{selectedDatasets.length === 0 ? (
|
||||
<div className="text-center py-8 text-gray-500">
|
||||
@@ -49,80 +167,150 @@ const RatioConfig: React.FC<RatioConfigProps> = ({
|
||||
<p className="text-sm">请先选择数据集</p>
|
||||
</div>
|
||||
) : (
|
||||
<div style={{ maxHeight: 500, overflowY: "auto" }}>
|
||||
{selectedDatasets.map((datasetId) => {
|
||||
const dataset = datasets.find((d) => String(d.id) === datasetId);
|
||||
const config = ratioConfigs.find((c) => c.source === datasetId);
|
||||
const currentQuantity = config?.quantity || 0;
|
||||
if (!dataset) return null;
|
||||
return (
|
||||
<Card key={datasetId} size="small" className="mb-2">
|
||||
<div className="flex items-center justify-between mb-3">
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="font-medium text-sm">{dataset.name}</span>
|
||||
<Badge color="gray">{dataset.fileCount}条</Badge>
|
||||
<div className="flex-overflow-auto gap-4 p-4">
|
||||
{/* 配比预览 */}
|
||||
{ratioConfigs.length > 0 && (
|
||||
<div>
|
||||
<div className="p-3 bg-gray-50 rounded-lg">
|
||||
<div className="grid grid-cols-2 gap-4 text-sm">
|
||||
<div>
|
||||
<span className="text-gray-500">总配比数量:</span>
|
||||
<span className="ml-2 font-medium">
|
||||
{ratioConfigs
|
||||
.reduce((sum, config) => sum + config.quantity, 0)
|
||||
.toLocaleString()}
|
||||
</span>
|
||||
</div>
|
||||
<div>
|
||||
<span className="text-gray-500">目标数量:</span>
|
||||
<span className="ml-2 font-medium">
|
||||
{totalTargetCount.toLocaleString()}
|
||||
</span>
|
||||
</div>
|
||||
<div>
|
||||
<span className="text-gray-500">配比项目:</span>
|
||||
<span className="ml-2 font-medium">
|
||||
{ratioConfigs.length}个
|
||||
</span>
|
||||
</div>
|
||||
<div className="text-xs text-gray-500">{config?.percentage || 0}%</div>
|
||||
</div>
|
||||
{ratioType === "dataset" ? (
|
||||
<div>
|
||||
<div className="flex items-center gap-2 mb-2">
|
||||
<span className="text-xs">数量:</span>
|
||||
<Input
|
||||
type="number"
|
||||
value={currentQuantity}
|
||||
onChange={(e) => onUpdateDatasetQuantity(datasetId, Number(e.target.value))}
|
||||
style={{ width: 80 }}
|
||||
min={0}
|
||||
max={Math.min(dataset.fileCount || 0, totalTargetCount)}
|
||||
/>
|
||||
<span className="text-xs text-gray-500">条</span>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
<div className="flex-1 overflow-auto">
|
||||
{selectedDatasets.map((datasetId) => {
|
||||
const dataset = datasets.find((d) => String(d.id) === datasetId);
|
||||
const config = ratioConfigs.find((c) => c.source === datasetId);
|
||||
const currentQuantity = config?.quantity || 0;
|
||||
if (!dataset) return null;
|
||||
return (
|
||||
<Card key={datasetId} size="small" className="mb-2">
|
||||
<div className="flex items-center justify-between mb-3">
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="font-medium text-sm">
|
||||
{dataset.name}
|
||||
</span>
|
||||
<Badge color="gray">{dataset.fileCount}条</Badge>
|
||||
</div>
|
||||
<div className="text-xs text-gray-500">
|
||||
{config?.percentage || 0}%
|
||||
</div>
|
||||
<Progress
|
||||
percent={Math.round((currentQuantity / totalTargetCount) * 100)}
|
||||
size="small"
|
||||
/>
|
||||
</div>
|
||||
) : (
|
||||
<div>
|
||||
{!distributions[String(dataset.id)] ? (
|
||||
<div className="text-xs text-gray-400">加载标签分布...</div>
|
||||
) : Object.entries(distributions[String(dataset.id)]).length === 0 ? (
|
||||
<div className="text-xs text-gray-400">该数据集暂无标签</div>
|
||||
) : (
|
||||
<div className="flex flex-col gap-2">
|
||||
{Object.entries(distributions[String(dataset.id)]).map(([label, count]) => {
|
||||
const sourceKey = `${datasetId}_${label}`;
|
||||
const labelConfig = ratioConfigs.find((c) => c.source === sourceKey);
|
||||
const labelQuantity = labelConfig?.quantity || 0;
|
||||
return (
|
||||
<div key={label} className="flex items-center justify-between gap-2">
|
||||
<div className="flex items-center gap-2">
|
||||
<Badge color="gray">{label}</Badge>
|
||||
<span className="text-xs text-gray-500">{count}条</span>
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="text-xs">数量:</span>
|
||||
<Input
|
||||
type="number"
|
||||
value={labelQuantity}
|
||||
onChange={(e) => onUpdateLabelQuantity(datasetId, label, Number(e.target.value))}
|
||||
style={{ width: 80 }}
|
||||
min={0}
|
||||
max={Math.min(Number(count) || 0, totalTargetCount)}
|
||||
/>
|
||||
<span className="text-xs text-gray-500">条</span>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
{ratioType === "dataset" ? (
|
||||
<div>
|
||||
<div className="flex items-center gap-2 mb-2">
|
||||
<span className="text-xs">数量:</span>
|
||||
<Input
|
||||
type="number"
|
||||
value={currentQuantity}
|
||||
onChange={(e) =>
|
||||
updateDatasetQuantity(
|
||||
datasetId,
|
||||
Number(e.target.value)
|
||||
)
|
||||
}
|
||||
style={{ width: 80 }}
|
||||
min={0}
|
||||
max={Math.min(
|
||||
dataset.fileCount || 0,
|
||||
totalTargetCount
|
||||
)}
|
||||
/>
|
||||
<span className="text-xs text-gray-500">条</span>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</Card>
|
||||
);
|
||||
})}
|
||||
<Progress
|
||||
percent={Math.round(
|
||||
(currentQuantity / totalTargetCount) * 100
|
||||
)}
|
||||
size="small"
|
||||
/>
|
||||
</div>
|
||||
) : (
|
||||
<div>
|
||||
{!distributions[String(dataset.id)] ? (
|
||||
<div className="text-xs text-gray-400">
|
||||
加载标签分布...
|
||||
</div>
|
||||
) : Object.entries(distributions[String(dataset.id)])
|
||||
.length === 0 ? (
|
||||
<div className="text-xs text-gray-400">
|
||||
该数据集暂无标签
|
||||
</div>
|
||||
) : (
|
||||
<div className="flex flex-col gap-2">
|
||||
{Object.entries(
|
||||
distributions[String(dataset.id)]
|
||||
).map(([label, count]) => {
|
||||
const sourceKey = `${datasetId}_${label}`;
|
||||
const labelConfig = ratioConfigs.find(
|
||||
(c) => c.source === sourceKey
|
||||
);
|
||||
const labelQuantity = labelConfig?.quantity || 0;
|
||||
return (
|
||||
<div
|
||||
key={label}
|
||||
className="flex items-center justify-between gap-2"
|
||||
>
|
||||
<div className="flex items-center gap-2">
|
||||
<Badge color="gray">{label}</Badge>
|
||||
<span className="text-xs text-gray-500">
|
||||
{count}条
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="text-xs">数量:</span>
|
||||
<Input
|
||||
type="number"
|
||||
value={labelQuantity}
|
||||
onChange={(e) =>
|
||||
updateLabelQuantity(
|
||||
datasetId,
|
||||
label,
|
||||
Number(e.target.value)
|
||||
)
|
||||
}
|
||||
style={{ width: 80 }}
|
||||
min={0}
|
||||
max={Math.min(
|
||||
Number(count) || 0,
|
||||
totalTargetCount
|
||||
)}
|
||||
/>
|
||||
<span className="text-xs text-gray-500">
|
||||
条
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</Card>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
Reference in New Issue
Block a user