feat: File and Annotation 2-way sync implementation (#63)

* feat: Refactor configuration and sync logic for improved dataset handling and logging

* feat: Enhance annotation synchronization and dataset file management

- Added new fields `tags_updated_at` to `DatasetFiles` model for tracking the last update time of tags.
- Implemented new asynchronous methods in the Label Studio client for fetching, creating, updating, and deleting task annotations.
- Introduced bidirectional synchronization for annotations between DataMate and Label Studio, allowing for flexible data management.
- Updated sync service to handle annotation conflicts based on timestamps, ensuring data integrity during synchronization.
- Enhanced dataset file response model to include tags and their update timestamps.
- Modified database initialization script to create a new column for `tags_updated_at` in the dataset files table.
- Updated requirements to ensure compatibility with the latest dependencies.
This commit is contained in:
Jason Wang
2025-11-07 15:03:07 +08:00
committed by GitHub
parent d136bad38c
commit 78f50ea520
16 changed files with 1336 additions and 290 deletions

View File

@@ -45,7 +45,7 @@ async def lifespan(app: FastAPI):
yield
# @shutdown
logger.info("DataMate Python Backend shutting down ...")
logger.info("DataMate Python Backend shutting down ...\n\n")
# 创建FastAPI应用
app = FastAPI(
@@ -69,11 +69,7 @@ app = FastAPI(
app.include_router(router)
# 输出注册的路由(每行一个)
logger.debug("Registered routes:")
for route in app.routes:
route_path = getattr(route, "path", None)
if route_path:
logger.debug(f" {route_path}")
logger.debug(f"Registered routes refer to http://localhost:{settings.port}/redoc")
# 注册全局异常处理器
app.add_exception_handler(StarletteHTTPException, starlette_http_exception_handler) # type: ignore
@@ -102,7 +98,7 @@ async def root():
data={
"message": f"{settings.app_name} is running",
"version": settings.app_version,
"docs_url": "/docs",
"docs_url": "/redoc",
"label_studio_url": settings.label_studio_base_url
}
)