jp/rakuten-api
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(gateway): 定时下派通道——周期性盘点账号订单并回写网关编目

Browse Source 
新增 §12 定时下派通道：网关自己定期派 order_list / order_detail 查询，把账号
真实订单沉淀进新表 account_orders（回写网关），上游可直接查 GET /api/account/orders
拿到账号里实际有哪些订单，不必自己记 order_number。

- collector.py: OrderDiscoveryCollector 常驻后台任务（与 sweep 并列）。每隔
  account_discovery_interval_seconds 派 order_list，收割结果后把「编目里没有的
  订单」upsert 进编目，再逐笔派 order_detail 沉淀 delivery_status/order_state。
  状态无痕：不新增编排跟踪表，仅内存 _pending + _detail_dispatched_this_day；
  detail query_id 按天分段（discover-detail-<order>-<日期>），同天幂等防重复派。
  边界：collector 只消费 worker 结果的规范化字段，绝不解析 raw/raw_pages。
- db.py: account_orders 表 + AccountOrderRow + upsert/single/list/stale 访问；
  upsert 以 order_number 为主键，重复采集只刷新，列表重扫不抹详情节点。
- 路由: GET /api/account/orders（列编目）、GET /api/account/orders/{n}(6005)、
  POST /api/account/discovery/trigger（手动立即派一轮 list）。
- config: RAKUTEN_ACCOUNT_DISCOVERY_ENABLED / _INTERVAL_SECONDS / _MAX_PAGES /
  RAKUTEN_ACCOUNT_DETAIL_REFRESH_SECONDS。
- 既有网关测试夹具统一关 discovery（会启动即派扫描污染查询队列语义），
  新表测试在 tests/test_gateway_discovery.py（13 用例，真实 QueryQueue+DB 装配）。

472 tests passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
q792602257
2026-08-16 23:35:51 +08:00
co-authored by Claude Opus 5
parent 6b77f3bcc3
commit f6c3976c0a
14 changed files with 1045 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app/gateway/db.py
+132
View File
@@ -71,6 +71,23 @@ CREATE TABLE IF NOT EXISTS account_queries (
completed_at TEXT
);
CREATE INDEX IF NOT EXISTS idx_queries_pending ON account_queries(status, created_at);
-- 定时下派通道（docs/order-gateway.md §12）：账号里真实订单的「已发现」目录。
-- collector 周期性派 order_list / order_detail 查询，把 worker 回结果里的
-- **规范化字段**（订单号/店铺/日期/配送状态）沉淀到这里——这是网关自己的编目，
-- 不是上游查询单的原样透传。只消费规范化契约，绝不碰结果里的站点原始 JSON。
CREATE TABLE IF NOT EXISTS account_orders (
order_number TEXT PRIMARY KEY,
shop_id TEXT,
shop_name TEXT,
order_date TEXT, -- 站点 ISO8601 字符串
delivery_status TEXT, -- 最近一次 detail 查询带回的站点码（如 CHECKING_ORDER）
order_state TEXT, -- 映射后的 OrderState（如 delivered）
discovered_at TEXT NOT NULL, -- 首次经 account 采集到该订单的时间
detail_fetched_at TEXT, -- 最近一次成功取到详情页的时间
last_seen_at TEXT NOT NULL -- 最近一次出现在订单列表里的时间
);
CREATE INDEX IF NOT EXISTS idx_account_orders_last_seen ON account_orders(last_seen_at);
"""
@@ -143,6 +160,21 @@ class QueryRow:
return json.loads(self.result_json) if self.result_json else None
@dataclass(slots=True)
class AccountOrderRow:
"""account_orders 表一行的强类型视图（定时下派通道的编目，见 §12）"""
order_number: str
shop_id: str | None
shop_name: str
order_date: str | None
delivery_status: str | None
order_state: str | None
discovered_at: str
detail_fetched_at: str | None
last_seen_at: str
def _row_to_task(row: aiosqlite.Row) -> TaskRow:
return TaskRow(
task_id=row["task_id"],
@@ -193,6 +225,20 @@ def _row_to_query(row: aiosqlite.Row) -> QueryRow:
)
def _row_to_account_order(row: aiosqlite.Row) -> AccountOrderRow:
return AccountOrderRow(
order_number=row["order_number"],
shop_id=row["shop_id"],
shop_name=row["shop_name"] or "",
order_date=row["order_date"],
delivery_status=row["delivery_status"],
order_state=row["order_state"],
discovered_at=row["discovered_at"],
detail_fetched_at=row["detail_fetched_at"],
last_seen_at=row["last_seen_at"],
)
class GatewayDB:
"""网关 SQLite 访问对象
@@ -577,3 +623,89 @@ class GatewayDB:
)
await self.conn.commit()
return cursor.rowcount or 0
# ---- account_orders（定时下派通道的编目，见 docs/order-gateway.md §12）----
async def upsert_account_order(self, row: AccountOrderRow) -> bool:
"""把一笔账号订单写进编目（回写）。存在则刷新，返回 True=新写入
这正是「采集到网关中不存在的订单就写回」的落点：主键是 order_number，
重复采集同一笔订单只是刷新 last_seen 等字段，不会产生重复行。
"""
async with self.conn.execute(
"SELECT 1 FROM account_orders WHERE order_number = ?", (row.order_number,)
) as cur:
existed = await cur.fetchone() is not None
await self.conn.execute(
"INSERT INTO account_orders (order_number, shop_id, shop_name, order_date, "
"delivery_status, order_state, discovered_at, detail_fetched_at, last_seen_at) "
"VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) "
"ON CONFLICT(order_number) DO UPDATE SET "
"shop_id = excluded.shop_id, shop_name = excluded.shop_name, "
"order_date = excluded.order_date, delivery_status = excluded.delivery_status, "
"order_state = excluded.order_state, detail_fetched_at = excluded.detail_fetched_at, "
"last_seen_at = excluded.last_seen_at",
(
row.order_number,
row.shop_id,
row.shop_name,
row.order_date,
row.delivery_status,
row.order_state,
row.discovered_at,
row.detail_fetched_at,
row.last_seen_at,
),
)
await self.conn.commit()
return not existed
async def get_account_order(self, order_number: str) -> AccountOrderRow | None:
async with self.conn.execute(
"SELECT * FROM account_orders WHERE order_number = ?", (order_number,)
) as cur:
row = await cur.fetchone()
return _row_to_account_order(row) if row else None
async def list_account_orders(
self,
*,
state: str | None = None,
limit: int = 50,
offset: int = 0,
) -> tuple[list[AccountOrderRow], int]:
"""分页列出编目订单，按 last_seen（最近出现）倒序"""
where = []
params: list[Any] = []
if state:
where.append("order_state = ?")
params.append(state)
clause = f"WHERE {' AND '.join(where)}" if where else ""
async with self.conn.execute(
f"SELECT COUNT(*) FROM account_orders {clause}", params
) as cur:
total = (await cur.fetchone())[0]
sql = (
f"SELECT * FROM account_orders {clause} "
"ORDER BY last_seen_at DESC LIMIT ? OFFSET ?"
)
async with self.conn.execute(sql, [*params, limit, offset]) as cur:
rows = await cur.fetchall()
return [_row_to_account_order(r) for r in rows], total
async def list_account_orders_needing_detail(self, cutoff_iso: str) -> list[AccountOrderRow]:
"""取「还没成功取过详情」或「详情已过刷新期（detail_fetched_at < cutoff）」的编目订单
collector 据此派新一批 order_detail 查询。校验用 detail_fetched_at
IS NULL OR < cutoff，保证不会对同一笔订单反复派（详情查询结果回来后才
落 detail_fetched_at）。
"""
async with self.conn.execute(
"SELECT * FROM account_orders WHERE detail_fetched_at IS NULL "
"OR detail_fetched_at < ? ORDER BY last_seen_at ASC",
(cutoff_iso,),
) as cur:
rows = await cur.fetchall()
return [_row_to_account_order(r) for r in rows]