Compare commits

...
11 Commits
Author SHA1 Message Date
q792602257 f5d717dae8 fix(gateway): document multi-item order intent schema 2026-08-31 12:30:09 +08:00
q792602257 f76c0a1518 feat(trading): support multi-item purchase intents 2026-08-31 11:41:39 +08:00
q792602257andClaude Opus 5 865bbe3724 feat(observability): 抓取会话逐次尝试与升级路径记成 event
重试链路的关键信息是「升级路径」,而属性表达不了过程:同名属性后写覆盖先写,
三次尝试跑完只剩最后一次的状态码,前两次为什么失败、升到哪一级全被盖掉。

- site_session / rakuma_session:每次尝试各记一条 scrape.attempt event,带
  attempt / outcome / status_code / 截断后的错误串;outcome 区分 http_error、
  bad_status、challenge、validate_failed
- site_session 的每次升级各记一条 scrape.escalate:rewarm_on_home、浏览器兜底
  recovered / failed。兜底失败也要记——不然链路里只剩「最终失败」,看不出浏览器
  这一级试过没有,而「没装 playwright」和「装了也被挡」是两个查法(reason 取
  BrowserFallback.unavailable_reason,为 None 时由 add_event 跳过)
- 三处 span.record_exception 换成 record_error,失败的错误码与可重试位跟着落下来
- 失败页面快照带上 url 与 profile,省得回头对是哪条通道的哪个地址

测试:test_scrape_telemetry.py 补一条会话层用例,用 MockTransport 让所有请求都回
挑战页,断言三次尝试各留一条 event(而不是只剩最后一次)、升级路径依次是
rewarm_on_home → browser,以及浏览器那一级的 outcome=failed 与 reason。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 16:18:25 +08:00
q792602257andClaude Opus 5 fd7d89ab0a test(observability): 补异常处理器的 server span 埋点测试
8381896 的最后一块配套测试。挂真实的 FastAPIInstrumentor 中间件(而不是手工造
span),断言「一次真实请求打进来、失败返回之后,server span 上有什么」:

- AppError 默认 status_code=400,链路里光看状态码只知道「客户端错了」,分不出
  是反爬阻断(3002,可重试)还是别的;断言 error.code / error.retryable /
  api.* 都落到了 span 上,异常栈作为 event 保留
- 兜底分支最需要埋点:对外只回 "Internal server error",断言真正的
  RuntimeError 类型与消息在链路上看得到
- 校验失败记错误码与字段消息,但不记 pydantic 那条没有诊断价值的长栈
- 一条成功路径对照,证明 ERROR 状态与 api.* 属性不是恒真

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 16:10:16 +08:00
q792602257andClaude Opus 5 8403b9b586 test(observability): 补抓取客户端与网关客户端的埋点测试
上一提交(8381896)的配套测试,钉住两个失败可见性问题:

- 抓取侧回归:失败分支原先写的是无参 `span.record_exception()`,而 exception
  是必填位置参数——每次抓取失败都会在记异常时抛 TypeError,把真正的原因(反爬
  阻断 / 解析失败)整个顶替掉,失败页面快照也永远落不下来(抛错在 snapshot
  之前)。测试断言业务异常原样抛出、stage 能区分 fetch/parse/delegate、
  parse 阶段确实落了快照且带得上 URL
- 网关侧:success=false code=6002 在 HTTP 层是 200,断言 span 记下了信封结论;
  以及空转长轮询在 suppressed() 里确实不产 span,抑制之外照常产(防止空断言
  是因为埋点根本没生效)
- 两个文件都各带一条成功路径对照,证明 ERROR 状态不是恒真

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 16:08:35 +08:00
q792602257andClaude Opus 5 b577d3ac8d fix(trading): 必填选项自动填值跳过「選択してください」占位项,并把选项开放给接口
trading 自动填 choice 时取 values[0],而必填 select 的 values[0] 恒为 id=0 的
「選択してください」——等于把「请选择」当答案提交。4 份真实样本一致(真值从
id=200 起)。同时 /api/item_detail 完全不返回 options,调用方即使想显式指定
choice 也无从知道合法取值。

- purchase_contract.py:新增 ItemOption / ItemOptionValue 与 parse_options /
  auto_choice_for / format_choice。占位判定以结构为主(value_id == 0),日文
  文案仅作兜底。放 shared 是因为「接口声明的合法取值」与「下单实际提交的值」
  必须同源,否则两边各判一次迟早再次分叉
- item.py / scrape.py:ItemDetailData 增 options、has_required_options、
  unfillable_required_options;只解析一次,两个派生结果都取自同一份结果
- site_interact.py:auto_choice_for 取第一个非占位候选;必填项填不出值时
  报错点名是哪些选项,让调用方知道该在 intent.choice 里补什么
- auto_choice_for 只自动填必填项:非必填项要不要选是业务决定,不是我们该替
  调用方做的选择
- README / docs:补 options[] → intent.choice、variants[] → intent.variant_id
  的对照,修掉 order-gateway 示例里已不存在的 "options": {} 字段

真账号验证(scripts/probe_option_choice.py,仅加购不结算不支付):两个商品
提交 確認した / 了解致しました。均被站点接受,购物车 count=2,跑完清空恢复
原状。探针刻意走生产的 add_to_cart_payload 并从其日志截获实际 payload——
probe_purchase_block_v2.py 自己抄了一遍字段构造,与生产代码同错,正是这个
bug 当初藏住的原因。

未覆盖:这两家店铺本身不校验该选项(旧的占位值当年也被收下),所以只证明新值
走得通、语义上才是真答案,证明不了旧值会被拒;必填自由文本项(
unfillable_required_options)无真实样本,仅离线测试覆盖。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 16:07:33 +08:00
q792602257andClaude Opus 5 8381896eeb feat(observability): 失败响应与网关信封进链路,手工埋点尊重 suppressed
失败此前在 trace 里近乎不可见:异常处理器把异常吃掉换成信封响应,自动
instrumentation 只看到一个 HTTP 状态码,而 AppError 默认 400、信封里
success=false,跟正常返回分不出来。

- api.py:四个异常处理器(对外失败的唯一出口)各记一次 span;兜底处理器额外
  record_error——对外只回一句无信息量的错误文案,异常类型与栈只在本地日志里
- telemetry.py:新增 record_envelope / record_parse_failure /
  span_unless_suppressed;record_error 补 error.message / retryable /
  status_code;snapshot 支持 extra 带上「这份 HTML 是哪来的」
- worker/client.py:_request 自建 span,活到解信封之后。httpx 那个 CLIENT span
  在 request() 返回时就结束,此时信封还没解——success=false code=6002(租约
  无效)在它看来是完成的 200 请求
- span_unless_suppressed:suppress_instrumentation 只被 instrumentation 库尊重,
  手工 span 不看它,lease 空转长轮询会从这个口子把孤立 trace 放回来
- 两个 scraping client 的解析失败分支收拢到 record_parse_failure;shop_items
  显式标 stage=delegate 且不落快照(它自己不抓页面,按 html 推断只会得出
  「fetch 失败」的错误结论)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 16:07:00 +08:00
q792602257andClaude Opus 5 4cc30bc058 feat(observability): /health 不再产生 server span
容器 HEALTHCHECK 每 30 秒探一次、上游也在轮询,这些请求各自是一条孤立
trace,量大且没有信息量——和 worker 空转长轮询同一个问题,把观测后台刷满
的正是它们。

instrument_app 传 excluded_urls,由新增的 RAKUTEN_OTEL_EXCLUDED_URLS 控制
(默认 /health$)。按 search 匹配完整 URL,锚点保证不误伤 /api/health-*;
留空时传 None,回落到 OTel 自己的 OTEL_PYTHON_FASTAPI_EXCLUDED_URLS。

三个服务共用 instrument_app,因此抓取/交易/网关一并生效。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 15:24:52 +08:00
q792602257andClaude Opus 5 3c7618a1d6 feat(observability): 抓取去掉首页预热,交易补齐链路埋点
两个问题一起处理,都与「出站请求与可观测性」有关。

## 抓取:正常路径不再多打一次首页

site_session 原先每条通道每 30 分钟打一次 www.rakuten.co.jp/ 做预热,而且预热
返回非 2xx 时 warmed_at 不置位——那种情况下每个请求前都会再打一次首页。

Akamai 的 cookie 随任意页面响应下发,目标页自己就会带回来,专门先打一次首页除了
多一个出站请求(以及多一次被风控计数的机会)之外没有额外收益:首个请求无论打哪个
URL 都是冷的 ~11s,之后都复用 cookie。

改为 cookie 由目标页响应建立(_note_cookies)、超 TTL 主动清空
(_drop_expired_cookies)。首页只保留在失败修复路径上(_rewarm_on_home):目标页
已经吃了挑战页时,拿首页换一套干净 cookie 比继续撞同一个 URL 更安全。happy path
的出站请求数 2 → 1。

_note_cookies 刻意不在每次响应时刷新时刻:TTL 要从「这套 cookie 第一次出现」算起,
每次都刷新会让一套 cookie 被无限续命,反而绕过了 session_ttl_seconds 的本意。

profile_status() 的 warmed 字段名保留(上游健康检查看板在用),语义改为「当前有
可复用的 Akamai cookie」,不再代表「已专门预热过首页」。

## 交易:此前没有任何有意义的链路数据

根因是 trading 的实际工作两类自动埋点都覆盖不到:站点交互走 Playwright(不经
httpx),worker 主循环是后台 asyncio 任务(没有 HTTP 入口,因此没有根 span)。
于是发给网关的每次 httpx 调用各自成为孤立 trace——观测后台上只剩一堆请求记录。

新增手工埋点:

- order.task:一笔下单的根 span,一个 task_id 一条 trace,带 order.route
  (execute / recovery / already_finished)与终态 order.terminal_status
- order.step.*:清车 → 加购 → 校验 → 确认 → 提交 → 付款,每步一个子 span,
  带 order.evidence_ref,可从 span 直接定位落盘证据
- site.*:12 个 Playwright 交互方法(用 traced 装饰器而非 with 块——这些方法的
  函数体本就很长,再加一层缩进不利于阅读)
- account_query:只读查询单的根 span,带 query.outcome

空转的长轮询(30 秒一次、绝大多数返回空)用 suppressed() 屏蔽:量大且没有信息量,
把观测后台刷满的正是它们。领到任务后的网关调用都在任务根 span 底下,不受影响。

闸门 / 风控拦截会被 _execute_with_renewal 吞掉转 needs_human,异常冒不到根 span,
被拦下的单在 trace 里跟成功下单一模一样。加 _execute_recording_errors 一层统一
记录,比每个 except 分支各写一遍省事,也不会漏掉后续新增的分支。

_report_safe 写 span 属性前判断 is_recording():付款后监控是 create_task 起的,
asyncio 在创建时就把 context 复制了进去,等它真正跑起来根 span 早已结束——
get_current_span() 拿到的仍是那个已结束的 span(不是 INVALID_SPAN),写属性会打
"Setting attribute on ended span"。当前监控路径不传 terminal_status 走不到那里,
这道判断是防以后。

## 顺带修掉:instrument_app 从未生效

instrument_app 用 _provider is None 做前置判断,但三个服务都在模块导入时执行
app = create_app(),而 setup_telemetry 要等 lifespan 才跑——那时 _provider 还是
None,照着判断直接 return。**FastAPI 从来没被打桩过,三个服务一条 server span
都没有。**

实测确认两件事:导入期打桩能出 span,lifespan 内打桩出不来(instrument_app 是加
中间件,应用开始服务后加进去不生效);provider 后设也不影响 ProxyTracer 委托到
真实 provider。所以只能在导入期装,判断条件改为 otel_enabled。

app/gateway/main.py 此前完全没接 telemetry,worker 出站请求带过来的 traceparent
没人接上,一条下单链路在网关这里断掉,只看得到 worker 侧那半截。补上
setup_telemetry(service_name="rakuten-gateway") 与 instrument_app / shutdown。

## 验证

新增 8 个用例:首页零请求、cookie 复用与过期清空、失败后用首页换 cookie、一任务
一 trace 的父子结构、闸门失败标 ERROR、空转不埋点,以及 instrument_app 调用顺序
的回归测试。全量 526 passed。

Playwright 那些 site.* 埋点只做了静态验证(测试用桩替换站点方法),没有跑真实
浏览器下单确认 span 真的落地。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 14:56:49 +08:00
q792602257 bba45d7f7c fix(tests): isolate trading API from Playwright 2026-08-20 16:51:24 +08:00
q792602257andClaude Opus 5 3284e086ad feat(trading): 浏览器掉线兜底——任务边界自愈重建,中途掉线转 needs_human
SiteInteractor 的 Chromium 是进程级单例,此前启动后即假定永远活着:全仓唯一的
is_connected() 探活在 scraping 侧,交易侧既不探活也不重启。容器里 Chromium 崩溃
是有真实前提的(/dev/shm 不足、OOM kill、seccomp 挡 sandbox,docker-compose.yml
里已有相关注释),一旦发生,进程还活着但之后每一单都会失败,且 /health 恒返回
ok,restart: unless-stopped 永远不会被触发。

更隐蔽的一条:clear_cart / enter_checkout / pay 等处的 new_page()、context.request
写在 try 之外,掉线抛的 TargetClosedError 不是 AppError,会穿过 runner 的
except AppError 落到主循环那个只记日志的兜底里——任务一次都不上报,网关侧要干等
整个 lease_ttl(默认 300s)才被 sweep 置 stale。clear_cart 是 execute() 的 step 0,
浏览器死时最先撞上的正是它。

分三层处理:

- 任务边界自愈。_launch() 从 start() 抽出,_context_options() 统一 context 参数
  (重建必须与启动完全一致,指纹漂移就是一次风控事件);_refresh_context_if_stale
  改名 _ensure_context_ready(),先探活重建再做原有的 storage_state mtime 检查
  (顺序不可换,mtime 重建要用 self._browser)。重建前丢弃 _checkout_pages 里的
  残留确认页并记 warning——那些 Page 已随浏览器一起没了。

- 中途掉线不重建,抛 BrowserDeadError(新增,5006)。新增 _new_page() 与
  _request() 两个壳收口裸异常;_request() 只在确认浏览器真死了时才改写异常,
  站点 5xx 这类正常业务失败原样抛出。submit_order / pay 入口用
  _require_live_browser() 直接拒绝:这两步复用 enter_checkout 留存的 Page,
  重建救不回服务端订单草稿,而 pay 跑的时候订单已经真的提交了。顺带修掉一个
  误诊——浏览器死时 submit_order 原先报「未找到确认按钮」,把「浏览器崩了」
  说成「站点改版了」,两者的处置方式完全不同。

- runner 把 BrowserDeadError 转 needs_human 而非 failed,except 分支排在
  except AppError 之前(子类,顺序反了就报 failed)。掉线发生在动作中途,
  站点侧生效与否无从判断,不能给上游「明确失败」的结论。

/health 暴露 browser 状态,掉线时 degraded + HTTP 503 + code 5006,让
Dockerfile.trading 的 HEALTHCHECK 探到并重启容器。重启不会导致重复下单:网关侧
任务绝不自动重投,租约过期只置 stale 等人工 reclaim(docs/order-gateway.md §5),
重启只是恢复领新任务的能力。启动窗口期 started=False 不算掉线。

README 错误码表补 5005(此前遗漏)与 5006。

新增 15 个用例覆盖探活三态、is_connected() 自身抛错、边界重建/不重建/丢弃残留页/
重建失败、两个包装壳的分支、submit/pay 拒绝、runner 转 needs_human、/health 503。
真实 Chromium 崩溃无法在离线测试里制造,用例模拟的是 is_connected() 返回 False
这个唯一可观测信号,覆盖的是代码对该信号的反应而非崩溃本身;容器 HEALTHCHECK
真的触发重启这条链路尚未实跑验证。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 16:01:47 +08:00
36 changed files with 3694 additions and 336 deletions
+6 -1
View File
@@ -41,7 +41,8 @@ RAKUTEN_REQUEST_TIMEOUT_SECONDS=30
RAKUTEN_MAX_SITE_CONCURRENCY=8
# 单次抓取的最大尝试次数(含首次):1 次直发,2 次换 cookie 重试,3 次起动用浏览器兜底
RAKUTEN_HTTP_MAX_ATTEMPTS=3
# Akamai cookie 最长复用时长(秒),超时后重新预热
# Akamai cookie 最长复用时长(秒),超时后清空、由下一次请求的响应重新建立。
# cookie 是随目标页响应下发的,正常路径不会额外访问站点首页。
RAKUTEN_SESSION_TTL_SECONDS=1800
# 浏览器兜底:纯 HTTP 被反爬拦截时用 Playwright 取回 cookie 再回灌重试。
@@ -76,6 +77,10 @@ RAKUTEN_OTEL_SERVICE_NAME=
RAKUTEN_OTEL_HEADERS=
# 失败 HTML 快照上限字节(默认 2MB,超出截断并标注 truncated=true)
RAKUTEN_OTEL_SNAPSHOT_MAX_BYTES=2000000
# 不上报 server span 的路径(逗号分隔正则,按 search 匹配完整 URL)。
# 默认排掉 /health:容器 HEALTHCHECK 每 30 秒探一次、上游也在轮询,
# 这些请求各自是一条孤立 trace,量大且没有信息量。留空表示不排除。
RAKUTEN_OTEL_EXCLUDED_URLS=/health$
# ---- 以下仅交易服务使用 ----
# 人工登录后落盘的 cookie 目录(相对项目根目录)。
+5
View File
@@ -168,6 +168,11 @@ USER rakuten
RUN /app/.venv/bin/playwright install chromium
# /health 只读缓存登录态,不触发站点请求,适合高频探活。
# 交易服务的 /health 在 Playwright 浏览器掉线时返回 503(curl -f 视为失败),
# 于是这里的 retries 用尽后容器被 restart 掉——进程还活着但浏览器已死时,所有站点
# 操作都做不了,重启是最后一道兜底。重启不会导致重复下单:网关侧任务绝不自动重投
# (租约过期只置 stale 等人工 reclaim,见 docs/order-gateway.md §5)。
# start-period 覆盖启动期,那时 browser 还没 start(),不算掉线。
# 端口取 RAKUTEN_HEALTH_PORT,未设时用交易端口;跑网关的容器把它设成 31109。
HEALTHCHECK --interval=30s --timeout=5s --start-period=45s --retries=3 \
CMD curl -fsS http://127.0.0.1:${RAKUTEN_HEALTH_PORT:-${RAKUTEN_TRADING_PORT}}/health || exit 1
+61 -6
View File
@@ -93,8 +93,8 @@ PC UA 在搜索页、详情页、店铺页上都能拿到完整模板。因此
- FastAPI 提供 HTTP API
- Bearer Token 接口鉴权(`Authorization: Bearer <token>`
- Akamai cookie 自动预热与复用,过期自动重新预热(仅乐天链路)
- 失败逐级升级:重新预热 → Playwright 兜底取 cookie → 结构化报错
- Akamai cookie 随目标页响应自动建立与复用,过期自动清空重建(仅乐天链路;正常路径不额外访问首页
- 失败逐级升级:首页换一套 cookie → Playwright 兜底取 cookie → 结构化报错
- 搜索结果自动剔除混入的 CPC 广告位(乐天)
- 识别站点深翻页「静默回绕到第 1 页」的行为,避免上游重复入库(乐天)
- 商品详情按落地域名分派解析,覆盖楽天ブックス / Rakuten Fashion / ビックカメラ 等官方旗舰子站
@@ -125,7 +125,7 @@ PC UA 在搜索页、详情页、店铺页上都能拿到完整模板。因此
| 接口 | 说明 |
| --- | --- |
| `GET /health` | 健康检查,含乐天账号登录态(只读缓存,不打站点) |
| `GET /health` | 健康检查,含乐天账号登录态(只读缓存,不打站点)与浏览器连接状态;浏览器掉线时返回 **503** |
| `POST /api/auth/status` | 查询登录态,默认真实探测一次 |
| `POST /api/auth/login` | 按 `account.yaml` 自动登录(已登录则跳过;撞验证码要人工接管) |
| `POST /api/auth/reload` | 人工重新登录后免重启换上新 cookie |
@@ -139,7 +139,7 @@ PC UA 在搜索页、详情页、店铺页上都能拿到完整模板。因此
| 接口 | 说明 |
| --- | --- |
| `GET /health` | 健康检查,含 worker 心跳、长时间无人领任务告警 |
| `POST /api/orders` | 上游提交下单意图(幂等) |
| `POST /api/orders` | 上游提交下单意图(幂等)`intent.items` 支持一次购买多个商品,旧版 `intent.item_url` 仍兼容 |
| `GET /api/orders/lease` | 本地 worker 长轮询领取(全局并发度 1) |
| `POST /api/orders/{id}/renew` | 续租(worker 在长任务里每 60s 调一次) |
| `POST /api/orders/{id}/report` | 本地回报订单状态(同 state 重复上报幂等) |
@@ -373,6 +373,20 @@ docker compose --profile gateway up -d
搜索结果中的 `shop.shop_code` + `item_code` 可直接用作本接口入参。
**下单前要决定的参数全在响应里**(规格与选项是两套东西,下单时走不同字段):
| 字段 | 下单时对应 | 说明 |
| --- | --- | --- |
| `sku.variants[].variant_id` | `intent.variant_id` | 规格组合(颜色 × 尺码等) |
| `options[]` | `intent.choice` | 店铺自定义选项,格式「选项名:取值名」 |
| `has_required_options` | — | `true` 时不给 `choice` 会被站点拒绝加购 |
| `unfillable_required_options` | — | 必填但无法自动选值的选项名,**必须**由调用方给值 |
`options[].values[].is_placeholder` 标出「選択してください」这类占位项——它们不是
合法取值,拼 `choice` 时要跳过(`selectable_value_count` 已是剔除占位项后的数量)。
`type="text"` 的选项是自由文本(如「【お名前】4文字まで」),站点不给候选值,
必填时只能由调用方给值。
部分官方店的商品页会跳转到独立子站,响应里的 `source` / `source_url` 会标明数据来源,
字段覆盖差异见下一节。
@@ -432,6 +446,7 @@ docker compose --profile gateway up -d
| `review` | ✅ | ✅ | ❌ 异步加载 | ❌ 异步加载 |
| `sku.variants`(规格组合) | ✅ | — 图书无规格 | ✅ 颜色 × 尺码 | — 单一规格 |
| `sku.attributes`(规格表) | ✅ | ✅ 出版社/ISBN 等 | 挂在各 variant 上 | ❌ |
| `options`(店铺自定义选项) | ✅ | ❌ 未解析 | ❌ 未解析 | ❌ 未解析 |
| `shipping` 运费明细 | ✅ | ❌ 仅库存措辞 | ❌ | 仅「是否含运费」 |
| `shop.shop_id` | ✅ | ❌ | ✅ | ✅ |
| `breadcrumbs[].url` | ✅ | ✅ | ❌ 站内分类编码,拼不出链接 | ✅ |
@@ -602,6 +617,8 @@ docker compose --profile gateway up -d
| `purchase_unit` | 起订单位 |
| `sku.inventory_type` | `multiple` 表示多规格,要看 `sku.variants[]` |
| `sku.variants[].variant_id` | 多规格商品的规格 ID(trading 加购多规格时按此选择) |
| `options[]` | 店铺自定义选项(必填项不给值站点拒绝加购),加购时走 `choice` 字段 |
| `unfillable_required_options` | 必填但无法自动选值的选项名,必须由调用方显式给值 |
交易服务对外接口(:31108,全部需要 Bearer token):
@@ -611,8 +628,13 @@ docker compose --profile gateway up -d
- `POST /api/cart/clear` — 清空购物车(UI 点击 `button[aria-label="削除"]`
- `POST /api/cart/remove` — 删除指定 `item_id`
底层共用 `app/shared/purchase_contract.py` 的常量字段构造(与 scraping 模型解耦)。
trading 加购时的字段选择策略:多规格挑第一个非售罄的 variant;必填选项拼「名:值」。
底层共用 `app/shared/purchase_contract.py` 的常量字段构造与**选项解析**(与 scraping
模型解耦)——`/api/item_detail` 对外暴露的 `options[]` 和 trading 自动填 `choice` 用的
是同一份解析与占位项判定,避免「接口说能选的值」与「下单实际填的值」不一致。
trading 加购时的字段选择策略:多规格挑第一个非售罄的 variant;必填选项拼「名:值」,
取第一个**非占位**候选值(`values[0]` 往往是「選択してください」,填它等于没选)。
必填项自动填不出来(自由文本项、候选值只剩占位项)且调用方没给 `choice` 时当场报错
并点名是哪几项,不拿占位值凑数去撞站点。
站点端点 `basketDomain` 逐商品不同(实测有 `sp.basket…``ts.sp.basket…`),不能写死。
几点子站差异(仅信息,trading 不覆盖子站加购):
@@ -639,6 +661,8 @@ trading 加购时的字段选择策略:多规格挑第一个非售罄的 varia
| 5002 | 加购失败(交易服务) | 400 |
| 5003 | 下单失败(交易服务) | 400 |
| 5004 | 下单安全闸门未通过:未显式确认或金额超上限(交易服务) | 400 |
| 5005 | 结算被站点风控拦截:session upgrade / 3DS 等需人工验证(交易服务) | 400 |
| 5006 | Playwright 浏览器已掉线(交易服务;也是 `/health` degraded 时的 code) | 400 / 503 |
| 6001 | 任务不存在(网关) | 404 |
| 6002 | 租约无效:不是持有者、已过期或任务已终结(网关) | 409 |
| 6003 | 任务状态不允许该操作(如对已终结任务 reclaim)(网关) | 409 |
@@ -649,11 +673,41 @@ trading 加购时的字段选择策略:多规格挑第一个非售罄的 varia
错误码在两站、三个服务之间通用。ラクマ 链路不会出现 `3002`(无反爬拦截行为)
`4002`(无子站跳转);`5xxx` 只会来自交易服务——抓取服务全程匿名,不会有登录态问题。
`5001``5004` 都标记为不可重试:前者要人工重新登录,后者要调用方改入参。
`5005``5006` 同样不可重试,且都转 `needs_human`:前者是站点主动要求人工验证,
后者是浏览器在动作中途没了、站点侧生效与否无从判断——下单不可逆,这种时候必须
停下来等人核对,不能赌。浏览器在**任务边界**掉线不会产生 `5006`,交易服务会就地
重建一套继续跑;重建失败或掉线发生在一次调用途中,才抛这个码。
`6xxx` 只会来自网关:`6001``6004`(下单任务通道)全部标记为不可重试——任务编排侧
重试无意义,部分场景(如租约过期)重试可能变成重复下单;`6005``6006`
(账号只读查询通道)**可以重试**——查询只读,重发没有副作用
(见 [docs/order-gateway.md §11](docs/order-gateway.md#11-账号只读查询通道))。
## 链路追踪
可选,默认关闭。`RAKUTEN_OTEL_ENABLED=true` + `RAKUTEN_OTEL_ENDPOINT` 指向 OTLP/HTTP
端点后,三个服务分别以 `rakuten-scraping` / `rakuten-trading` / `rakuten-gateway` 上报。
FastAPI 与 httpx 走自动埋点,但那只覆盖「收到 HTTP 请求」与「发出 httpx 请求」两类
边界。交易侧的实际工作两者都不是——站点交互走 Playwright,worker 主循环是后台任务,
所以这一侧手工埋点:
- `order.task`:一笔下单任务的**根 span**(一个 `task_id` 一条 trace),带
`order.route``execute` / `recovery` / `already_finished`)与终态
`order.terminal_status`;被闸门或风控拦下时置 ERROR 并带 `error.code`
- `order.step.*`:清车 → 加购 → 校验 → 确认页 → 提交 → 付款,每步一个子 span,
`order.evidence_ref`(可直接定位落盘证据)
- `site.*`:Playwright 站点交互(`site.add_to_cart``site.enter_checkout`
`site.submit_order``site.pay` 等)
- `account_query`:一张只读查询单的根 span,带 `query.outcome`
worker 空转的长轮询(每 30 秒问一次网关有没有活干)刻意不埋点——它们没有信息量,
量却极大,会把观测后台刷满。`/health` 同理:容器 HEALTHCHECK 每 30 秒探一次、上游
也在轮询,默认由 `RAKUTEN_OTEL_EXCLUDED_URLS`(默认 `/health$`)挡在 server span
之外。该项是逗号分隔的正则、按 search 匹配完整 URL,留空则不排除任何路径。
> 解析失败时页面 HTML 会作为 span event 上报(`RAKUTEN_OTEL_SNAPSHOT_MAX_BYTES`
> 控制上限,默认 2MB),用于事后复现「抓到的内容为什么解析不出预期字段」。
## 常用环境变量
完整列表见 [.env.example](.env.example)。配置项前缀统一为 `RAKUTEN_`,三个服务共用同一份
@@ -669,6 +723,7 @@ trading 加购时的字段选择策略:多规格挑第一个非售罄的 varia
- 抓取:`RAKUTEN_MAX_SITE_CONCURRENCY`(默认 `8`,两站各自独立计数)、`RAKUTEN_HTTP_MAX_ATTEMPTS`(默认 `3`)、`RAKUTEN_SESSION_TTL_SECONDS`(默认 `1800`,仅乐天)
- 浏览器兜底(仅乐天):`RAKUTEN_BROWSER_FALLBACK_ENABLED``RAKUTEN_BROWSER_HEADLESS``RAKUTEN_BROWSER_CHANNEL`
- 代理(需日本 IP 时):`RAKUTEN_PROXY_SERVER``RAKUTEN_PROXY_USERNAME``RAKUTEN_PROXY_PASSWORD``RAKUTEN_PROXY_BYPASS`
- 链路追踪(可选,默认关闭;三个服务共用):`RAKUTEN_OTEL_ENABLED``RAKUTEN_OTEL_ENDPOINT``RAKUTEN_OTEL_HEADERS``RAKUTEN_OTEL_SNAPSHOT_MAX_BYTES``RAKUTEN_OTEL_EXCLUDED_URLS`(默认 `/health$`
> 从中国大陆直连实测可用、无需代理;`RAKUTEN_PROXY_SERVER` 留空即可。设置后,所有面向
> 外部站点的 HTTPX 与 Playwright 流量都会经代理;本机和 Docker 服务间地址由
+8
View File
@@ -27,6 +27,7 @@ from app.gateway.task_queue import TaskQueue
from app.shared.api import register_exception_handlers
from app.shared.config import get_settings
from app.shared.logging_setup import configure_logging
from app.shared.telemetry import instrument_app, setup_telemetry, shutdown_telemetry
logger = logging.getLogger(__name__)
@@ -96,6 +97,11 @@ async def lifespan(app: FastAPI):
app.state.container = container
configure_logging(container.settings)
# 网关这一侧原先完全没接 telemetry,导致 worker 出站请求带过来的 traceparent
# 没人接上:一条下单链路在网关这里断掉,观测后台上只看得到 worker 侧那半截。
# CallbackNotifier 的 httpx 客户端是每次投递时才建的(不是 __init__ 里),
# 所以放在 build_container() 之后仍然赶在客户端创建之前。
setup_telemetry(container.settings, service_name="rakuten-gateway")
logger.info(
"网关启动:%s:%s", container.settings.gateway_host, container.settings.gateway_port
)
@@ -141,6 +147,7 @@ async def lifespan(app: FastAPI):
# 等在途回调发完(各次发送有超时兜底),避免关停时静默丢通知
await container.notifier.aclose()
await container.db.close()
shutdown_telemetry()
def create_app() -> FastAPI:
@@ -151,6 +158,7 @@ def create_app() -> FastAPI:
app.include_router(queries_router)
app.include_router(account_router)
register_exception_handlers(app)
instrument_app(app)
return app
+87 -7
View File
@@ -6,10 +6,10 @@ intent 字段刻意保留成 `dict[str, Any]`——网关不解释下单意图
"""
from __future__ import annotations
from typing import Any
from typing import Annotated, Any
from urllib.parse import urlsplit
from pydantic import BaseModel, Field, field_validator
from pydantic import BaseModel, Field, WithJsonSchema, field_validator
from app.shared.task_state import AccountQueryKind, OrderState, QueryStatus, TaskStatus
@@ -17,6 +17,79 @@ from app.shared.task_state import AccountQueryKind, OrderState, QueryStatus, Tas
# ---- POST /api/orders ----
# The gateway intentionally keeps intent open-ended at runtime. This schema documents
# the stable trading fields without preventing future fields from being passed through.
OrderIntent = Annotated[
dict[str, Any],
WithJsonSchema(
{
"type": "object",
"additionalProperties": True,
"description": (
"下单意图。推荐使用 items;旧版 item_url 等同级字段继续兼容。"
),
"properties": {
"items": {
"type": "array",
"minItems": 1,
"description": "本次购买的商品列表,按顺序加入同一购物车",
"items": {
"oneOf": [
{
"type": "object",
"additionalProperties": True,
"properties": {
"item_url": {
"type": "string",
"description": "商品页 URL",
},
"quantity": {
"type": "integer",
"minimum": 1,
"default": 1,
},
"variant_id": {"type": "string"},
"choice": {
"oneOf": [
{"type": "string"},
{
"type": "array",
"items": {"type": "string"},
},
]
},
},
"required": ["item_url"],
},
{
"type": "string",
"description": "商品页 URL(简写)",
},
]
},
},
"item_url": {
"type": "string",
"description": "旧版单商品商品页 URL",
},
"quantity": {"type": "integer", "minimum": 1, "default": 1},
"variant_id": {"type": "string"},
"choice": {
"oneOf": [
{"type": "string"},
{"type": "array", "items": {"type": "string"}},
]
},
"max_total_yen": {
"type": "integer",
"description": "本次订单允许的最高应付金额(日元)",
},
},
}
),
]
class SubmitOrderRequest(BaseModel):
"""上游提交下单意图
@@ -34,19 +107,26 @@ class SubmitOrderRequest(BaseModel):
description="站点标识。交易服务只覆盖乐天市场,固定 rakuten",
examples=["rakuten"],
)
intent: dict[str, Any] = Field(
intent: OrderIntent = Field(
description=(
"下单意图原文。网关不解释内容,原样存库并透传给本地 worker,结构由 trading 侧定义:"
"item_url(必填,商品页 URL);quantity(可选,默认 1);"
"推荐使用 items(非空数组,每项含 item_url,及可选 quantity/variant_id/choice);"
"为兼容已发布客户端,也支持单商品 item_url(及同级 quantity/variant_id/choice);"
"quantity(可选,默认 1);"
"variant_id(多规格商品必填,取自 /api/item_detail 的 variants,不传时 worker 自动选第一个非售罄规格);"
"choice(可选,商品选项,如 \"颜色:赤\",可传字符串或字符串列表);"
"max_total_yen(可选,本次金额上限:确认页实际应付超过即中止并报 needs_human,"
"缺省用服务端 RAKUTEN_ORDER_MAX_TOTAL_YEN)"
),
examples=[{
"item_url": "https://item.rakuten.co.jp/shop/code/",
"quantity": 1,
"variant_id": "1001",
"items": [{
"item_url": "https://item.rakuten.co.jp/shop/code/",
"quantity": 1,
"variant_id": "1001",
}, {
"item_url": "https://item.rakuten.co.jp/shop/another-code/",
"quantity": 2,
}],
"max_total_yen": 30000,
}],
)
+54 -1
View File
@@ -338,6 +338,43 @@ class SkuInfo(BaseModel):
variant_count: int = Field(default=0, description="SKU 组合总数;不受 include_sku_variants 影响,始终为真实组合数")
class ItemOptionValue(BaseModel):
"""店铺自定义选项的一个候选取值"""
value_id: int | None = Field(default=None, description="站点 values[].id 原值;给不出数字时为 null")
name: str = Field(default="", description="取值展示名,下单时 choice 里要用这个原文")
is_placeholder: bool = Field(
default=False,
description="是否为「選択してください」这类占位项。占位项不是合法取值,"
"提交它等于没选,构造 choice 时必须跳过",
)
class ItemOption(BaseModel):
"""一个店铺自定义选项(下单必填项的来源)
与 SKU 规格不是一回事:规格走 `sku.variants[].variant_id`,选项走下单接口的
`choice` 字段。店铺用它承载「名入れ文字」「配送方式确认」「レビュー依頼」等,
`is_required=true` 的选项不给值时站点会直接拒绝加购。
"""
option_id: int | None = Field(default=None, description="站点 options[].id 原值")
name: str = Field(default="", description="选项名,下单时 choice 的「名」部分要用这个原文")
type: str = Field(
default="",
description="站点原值:select(下拉,看 values)/ text(自由文本,values 为空)",
)
is_required: bool = Field(default=False, description="是否必填;必填项不给值时站点拒绝加购")
values: list[ItemOptionValue] = Field(
default_factory=list, description="候选取值;type=text 时为空"
)
selectable_value_count: int = Field(
default=0,
description="剔除占位项后真正可提交的候选数。为 0 且 is_required=true 时"
"无法自动选值,必须由调用方在 choice 里显式给出",
)
class ShippingInfo(BaseModel):
"""配送与运费信息"""
@@ -365,7 +402,8 @@ class ItemDetailData(BaseModel):
加购(构造 cart 请求)不在本服务范围内——加购需要已登录的乐天账号会话,
归 trading 服务(app.trading)。本响应只描述「商品状态」:能不能买
(purchase_condition / is_sold_out)、规格(sku.variants)、起订单位等。
(purchase_condition / is_sold_out)、规格(sku.variants)、起订单位
店铺自定义选项(options)等,即「下单前需要先决定哪些参数」。
"""
source: str = Field(default="ichiba", description="数据来源站点:ichiba / books / brandavenue / biccamera")
@@ -390,6 +428,21 @@ class ItemDetailData(BaseModel):
breadcrumbs: list[Breadcrumb] = Field(default_factory=list, description="分类面包屑")
shipping: ShippingInfo = Field(default_factory=ShippingInfo, description="该商品的配送与运费信息")
sku: SkuInfo = Field(default_factory=SkuInfo, description="SKU 信息")
options: list[ItemOption] = Field(
default_factory=list,
description="店铺自定义选项(站点 purchase.information.options)。与 SKU 规格不同:"
"规格选 sku.variants[].variant_id,选项走下单接口的 choice 字段。"
"空列表表示该商品页没有选项;子站(books / brandavenue / biccamera)暂不解析",
)
has_required_options: bool = Field(
default=False,
description="是否存在必填选项。为 true 时下单必须给 choice,否则站点拒绝加购",
)
unfillable_required_options: list[str] = Field(
default_factory=list,
description="必填但无法自动选值的选项名(自由文本项,或候选值只有占位项)。"
"非空时**必须**由调用方在下单 intent.choice 里显式给出这些项的取值",
)
class HealthData(BaseModel):
+42 -1
View File
@@ -11,9 +11,13 @@ from __future__ import annotations
from typing import Any
from app.shared.errors import ScrapeParseError
from app.shared.purchase_contract import ItemOption as SharedItemOption
from app.shared.purchase_contract import auto_choice_for, parse_options
from app.scraping.models.scrape import (
Breadcrumb,
ItemDetailData,
ItemOption,
ItemOptionValue,
ReviewSummary,
ShippingInfo,
ShopSummary,
@@ -31,6 +35,33 @@ _PURCHASABLE_CONDITION = "enabled"
# 库存类型 → 加购表单里的 inventory_flag(常量与基础字段构造在 app.shared.purchase_contract)
def _to_item_options(options: list[SharedItemOption]) -> list[ItemOption]:
"""把共用契约的 ItemOption dataclass 搬成对外的 pydantic 模型
解析与占位项判定都在 `app.shared.purchase_contract.parse_options`trading
下单时用同一份逻辑挑 choice 取值两侧对哪个取值是合法的必须完全一致
否则本接口告诉上游能选的值下单时却填了别的本函数只做搬运不加判断
"""
return [
ItemOption(
option_id=option.option_id,
name=option.name,
type=option.type,
is_required=option.is_required,
values=[
ItemOptionValue(
value_id=value.value_id,
name=value.name,
is_placeholder=value.is_placeholder,
)
for value in option.values
],
selectable_value_count=len(option.selectable_values),
)
for option in options
]
def _parse_attributes(raw: Any) -> list[SkuAttribute]:
return [
SkuAttribute(title=as_str(attr.get("title")), value=as_str(attr.get("value")))
@@ -115,6 +146,13 @@ def parse_item_detail(
sell_type = _pick_sell_type(as_dict(purchase.get("sellType")))
purchase_condition = as_str(sell_type.get("purchaseCondition"))
purchase_information = as_dict(purchase.get("information"))
# 只解析一次,两个派生结果都从这份结果来
shared_options = parse_options(purchase_information)
# 无法自动选值的必填项:与 trading 自动填 choice 时的判定同源,上游据此知道
# 「哪些项必须自己给值」,而不是等下单时才被站点拒绝
_, unfillable_required = auto_choice_for(shared_options)
raw_sku = as_dict(purchase.get("sku"))
variants = _parse_variants(raw_sku.get("variants"))
sku = SkuInfo(
@@ -188,7 +226,7 @@ def parse_item_detail(
purchase_condition=purchase_condition,
# purchaseCondition 是站点判定能否下单的直接依据;缺失时不臆断为售罄
is_sold_out=bool(purchase_condition) and purchase_condition != _PURCHASABLE_CONDITION,
purchase_unit=as_int(as_dict(purchase.get("information")).get("unit")),
purchase_unit=as_int(purchase_information.get("unit")),
images=images,
shop=shop,
review=review,
@@ -196,4 +234,7 @@ def parse_item_detail(
breadcrumbs=breadcrumbs,
shipping=shipping,
sku=sku,
options=_to_item_options(shared_options),
has_required_options=any(option.is_required for option in shared_options),
unfillable_required_options=unfillable_required,
)
+26 -21
View File
@@ -37,7 +37,7 @@ from app.scraping.utils.rakuma_urls import (
split_item_url,
split_shop_url,
)
from app.shared.telemetry import snapshot
from app.shared.telemetry import record_parse_failure
logger = logging.getLogger(__name__)
tracer = trace.get_tracer(__name__)
@@ -77,10 +77,11 @@ class RakumaClient:
url, len(result.items), result.total_count,
)
return result
except Exception:
span.record_exception()
span.set_attribute("parse.fail_reason", "parse_error")
snapshot(span, "parse.failed_html", html, self._settings.otel_snapshot_max_bytes)
except Exception as exc:
record_parse_failure(
span, exc, html=html,
max_bytes=self._settings.otel_snapshot_max_bytes, url=url,
)
raise
async def categories(self, payload: RakumaCategoryRequest) -> RakumaCategoryData:
@@ -110,10 +111,11 @@ class RakumaClient:
data.category_id, data.name, len(data.children), data.total_count,
)
return data
except Exception:
span.record_exception()
span.set_attribute("parse.fail_reason", "parse_error")
snapshot(span, "parse.failed_html", html, self._settings.otel_snapshot_max_bytes)
except Exception as exc:
record_parse_failure(
span, exc, html=html,
max_bytes=self._settings.otel_snapshot_max_bytes, url=url,
)
raise
async def item_detail(self, payload: RakumaItemDetailRequest) -> RakumaItemDetailData:
@@ -139,10 +141,11 @@ class RakumaClient:
url, detail.item_name[:40], detail.price, detail.is_sold_out,
)
return detail
except Exception:
span.record_exception()
span.set_attribute("parse.fail_reason", "parse_error")
snapshot(span, "parse.failed_html", html, self._settings.otel_snapshot_max_bytes)
except Exception as exc:
record_parse_failure(
span, exc, html=html,
max_bytes=self._settings.otel_snapshot_max_bytes, url=url,
)
raise
async def shop_detail(self, payload: RakumaShopDetailRequest) -> RakumaShopDetailData:
@@ -177,10 +180,11 @@ class RakumaClient:
shop_id, detail.shop_name, detail.item_count, detail.review_count,
)
return detail
except Exception:
span.record_exception()
span.set_attribute("parse.fail_reason", "parse_error")
snapshot(span, "parse.failed_html", html, self._settings.otel_snapshot_max_bytes)
except Exception as exc:
record_parse_failure(
span, exc, html=html,
max_bytes=self._settings.otel_snapshot_max_bytes, url=url,
)
raise
async def shop_items(self, payload: RakumaShopItemsRequest) -> RakumaShopItemsData:
@@ -205,10 +209,11 @@ class RakumaClient:
shop_id, len(result.items), result.total_count,
)
return result
except Exception:
span.record_exception()
span.set_attribute("parse.fail_reason", "parse_error")
snapshot(span, "parse.failed_html", html, self._settings.otel_snapshot_max_bytes)
except Exception as exc:
record_parse_failure(
span, exc, html=html,
max_bytes=self._settings.otel_snapshot_max_bytes, url=url,
)
raise
@staticmethod
+26 -5
View File
@@ -29,10 +29,14 @@ from app.shared.errors import (
UpstreamBlockedError,
UpstreamRequestError,
)
from app.shared.telemetry import snapshot
from app.shared.telemetry import add_event, record_error, snapshot
logger = logging.getLogger(__name__)
# 逐次尝试 event 里错误串的截断长度:httpx 的异常消息可能很长(带完整 URL 与
# 底层 socket 错误),而这里只是用来区分「这次是怎么失败的」,前半句就够
_ERROR_MAX_CHARS = 200
class RakumaSession:
"""ラクマ 站点抓取会话,管理并发限流与失败重试"""
@@ -108,7 +112,7 @@ class RakumaSession:
)
except TimeoutError as exc:
err = ResourceBusyError()
span.record_exception(err)
record_error(span, err)
span.set_attribute("scrape.fail_reason", "ResourceBusyError")
raise err from exc
@@ -119,6 +123,13 @@ class RakumaSession:
response = await client.get(url)
except httpx.HTTPError as exc:
last_error = f"{type(exc).__name__}: {exc}"
# 每次尝试各记一条 event:属性同名后写覆盖先写,三次尝试
# 跑完只剩最后一次的状态,中间那两次为什么失败全被盖掉
add_event(span, "scrape.attempt", {
"attempt": attempt,
"outcome": "http_error",
"error": last_error[:_ERROR_MAX_CHARS],
})
logger.warning(
"ラクマ 抓取请求异常:url=%s attempt=%s/%s err=%s",
url, attempt, max_attempts, last_error,
@@ -127,7 +138,7 @@ class RakumaSession:
if response.status_code == 404:
err = ItemNotFoundError(f"Page not found: {url}")
span.record_exception(err)
record_error(span, err)
span.set_attribute("scrape.fail_reason", "ItemNotFoundError")
raise err
@@ -145,6 +156,12 @@ class RakumaSession:
last_text = response.text
span.set_attribute("scrape.last_status_code", response.status_code)
span.set_attribute("scrape.html_bytes", len(response.text))
add_event(span, "scrape.attempt", {
"attempt": attempt,
"outcome": "bad_status",
"status_code": response.status_code,
"error": last_error[:_ERROR_MAX_CHARS],
})
logger.warning(
"ラクマ 抓取结果异常:url=%s attempt=%s/%s %s",
url, attempt, max_attempts, last_error,
@@ -154,9 +171,13 @@ class RakumaSession:
err = UpstreamRequestError(f"Upstream request failed: {last_error}")
else:
err = UpstreamBlockedError(f"Failed to fetch {url}: {last_error}")
span.record_exception(err)
record_error(span, err)
span.set_attribute("scrape.fail_reason", type(err).__name__)
snapshot(span, "scrape.failed_html", last_text, self._settings.otel_snapshot_max_bytes)
snapshot(
span, "scrape.failed_html", last_text,
self._settings.otel_snapshot_max_bytes,
extra={"scrape.url": url, "scrape.profile": "rakuma"},
)
raise err
finally:
self._semaphore.release()
+26 -20
View File
@@ -45,7 +45,7 @@ from app.scraping.utils.urls import (
split_item_url,
split_shop_url,
)
from app.shared.telemetry import snapshot
from app.shared.telemetry import record_parse_failure
logger = logging.getLogger(__name__)
tracer = trace.get_tracer(__name__)
@@ -87,10 +87,11 @@ class RakutenClient:
url, len(result.items), result.ad_count, result.total_count,
)
return result
except Exception:
span.record_exception()
span.set_attribute("parse.fail_reason", "parse_error")
snapshot(span, "parse.failed_html", html, self._settings.otel_snapshot_max_bytes)
except Exception as exc:
record_parse_failure(
span, exc, html=html,
max_bytes=self._settings.otel_snapshot_max_bytes, url=url,
)
raise
async def genres(self, payload: GenreRequest) -> GenreData:
@@ -113,10 +114,11 @@ class RakutenClient:
result.genre_id or "root", result.name, len(result.children),
)
return result
except Exception:
span.record_exception()
span.set_attribute("parse.fail_reason", "parse_error")
snapshot(span, "parse.failed_html", html, self._settings.otel_snapshot_max_bytes)
except Exception as exc:
record_parse_failure(
span, exc, html=html,
max_bytes=self._settings.otel_snapshot_max_bytes, url=url,
)
raise
async def shop_detail(self, payload: ShopDetailRequest) -> ShopDetailData:
@@ -142,10 +144,11 @@ class RakutenClient:
result.shop_code, result.shop_id, result.shop_name, result.review_count,
)
return result
except Exception:
span.record_exception()
span.set_attribute("parse.fail_reason", "parse_error")
snapshot(span, "parse.failed_html", html, self._settings.otel_snapshot_max_bytes)
except Exception as exc:
record_parse_failure(
span, exc, html=html,
max_bytes=self._settings.otel_snapshot_max_bytes, url=url,
)
raise
async def shop_items(self, payload: ShopItemsRequest) -> SearchResultData:
@@ -170,9 +173,11 @@ class RakutenClient:
span.set_attribute("parse.items", len(result.items))
span.set_attribute("parse.total", result.total_count)
return result
except Exception:
span.record_exception()
span.set_attribute("parse.fail_reason", "parse_error")
except Exception as exc:
# 本方法自己不抓页面,失败一定发生在它转调的 shop_detail / search
# 里(那两个 span 已各自记了自己的失败页面),所以显式标 delegate、
# 不落快照:按 html 推断只会得出「fetch 失败」的错误结论。
record_parse_failure(span, exc, stage="delegate")
raise
async def item_detail(self, payload: ItemDetailRequest) -> ItemDetailData:
@@ -226,8 +231,9 @@ class RakutenClient:
url, detail.source, detail.item_name[:40], detail.price, detail.sku.variant_count,
)
return detail
except Exception:
span.record_exception()
span.set_attribute("parse.fail_reason", "parse_error")
snapshot(span, "parse.failed_html", html, self._settings.otel_snapshot_max_bytes)
except Exception as exc:
record_parse_failure(
span, exc, html=html,
max_bytes=self._settings.otel_snapshot_max_bytes, url=url,
)
raise
+125 -51
View File
@@ -8,7 +8,13 @@
画像详情页只有手机 UA 才返回带 __INITIAL_STATE__ 的统一模板两条通道
各自持有独立 cookie 避免把 PC 指纹拿到的 cookie 混用到手机请求上
抓取失败时的升级路径重新预热 浏览器兜底取 cookie 放弃
**不预热首页**Akamai cookie 是随任意一个页面响应下发的目标页自己就会带回
专门先打一次 `www.rakuten.co.jp/` 除了多一个出站请求以及多一次被风控计数
的机会之外没有额外收益首个请求无论打哪个 URL 都是冷的 ~11s之后都复用
cookie首页只在**失败修复**路径上使用目标页已经被挑战时拿首页换一套干净
cookie 比继续拿目标页去撞更安全 `_rewarm_on_home`
抓取失败时的升级路径 cookie首页重新预热 浏览器兜底取 cookie 放弃
"""
from __future__ import annotations
@@ -32,10 +38,14 @@ from app.shared.errors import (
)
from app.scraping.parsers.state import PageValidator, require_state_marker
from app.scraping.services.browser_fallback import BrowserFallback
from app.shared.telemetry import snapshot
from app.shared.telemetry import add_event, record_error, snapshot
logger = logging.getLogger(__name__)
# 逐次尝试 event 里错误串的截断长度:httpx 异常消息可能很长(带完整 URL 与底层
# socket 错误),这里只用来区分「这次是怎么失败的」,前半句够了
_ERROR_MAX_CHARS = 200
@dataclass(slots=True)
class FetchedPage:
@@ -62,12 +72,18 @@ class _Profile:
mobile: bool
client: httpx.AsyncClient
lock: asyncio.Lock = field(default_factory=asyncio.Lock)
warmed_at: float = 0.0
# Akamai cookie 罐的建立时刻(monotonic)。0 表示当前没有可复用的 cookie。
# 超过 session_ttl_seconds 就主动清空:拿着过期 cookie 去撞反而更容易被挑战。
cookies_at: float = 0.0
@property
def cookie_names(self) -> set[str]:
return {cookie.name for cookie in self.client.cookies.jar}
@property
def akamai_cookies(self) -> set[str]:
return self.cookie_names & set(site.AKAMAI_COOKIE_NAMES)
class SiteSession:
"""乐天站点抓取会话,管理 cookie 预热、并发限流与失败升级"""
@@ -115,13 +131,17 @@ class SiteSession:
# ---- 状态 ----
def profile_status(self) -> dict[str, dict[str, Any]]:
"""各通道的预热状态,供健康检查展示"""
"""各通道的 cookie 状态,供健康检查展示
`warmed` 保留原字段名上游健康检查看板在用语义是当前有可复用的
Akamai cookie不再代表已专门预热过首页因为正常路径不打首页了
"""
now = time.monotonic()
return {
name: {
"warmed": profile.warmed_at > 0,
"age_seconds": round(now - profile.warmed_at, 1) if profile.warmed_at else None,
"cookies": sorted(profile.cookie_names & set(site.AKAMAI_COOKIE_NAMES)),
"warmed": profile.cookies_at > 0,
"age_seconds": round(now - profile.cookies_at, 1) if profile.cookies_at else None,
"cookies": sorted(profile.akamai_cookies),
}
for name, profile in self._profiles.items()
}
@@ -142,7 +162,8 @@ class SiteSession:
) -> FetchedPage:
"""抓取页面,返回 HTML 与最终落地地址
失败时按 重新预热 浏览器兜底 的顺序逐级升级重试
直接打目标页不先访问首页失败时按 cookie 浏览器兜底 的顺序
逐级升级重试
Args:
validator: 页面校验器默认要求页面含 __INITIAL_STATE__跨站抓取时
@@ -174,27 +195,40 @@ class SiteSession:
)
except TimeoutError as exc:
err = ResourceBusyError()
span.record_exception(err)
record_error(span, err)
span.set_attribute("scrape.fail_reason", "ResourceBusyError")
raise err from exc
try:
for attempt in range(1, max_attempts + 1):
span.set_attribute("scrape.attempts", attempt)
await self._ensure_warm(profile)
# 过期 cookie 主动丢掉:带着它去撞比裸请求更容易吃挑战页
await self._drop_expired_cookies(profile)
try:
response = await profile.client.get(url)
except httpx.HTTPError as exc:
last_error = f"{type(exc).__name__}: {exc}"
# 每次尝试各记一条 event。属性同名后写覆盖先写,三次尝试跑完
# 只剩最后一次的状态码,前两次为什么失败、升级到哪一级全被
# 盖掉——而这条链路的关键信息恰恰是「升级路径」。
add_event(span, "scrape.attempt", {
"attempt": attempt,
"outcome": "http_error",
"error": last_error[:_ERROR_MAX_CHARS],
})
logger.warning(
"抓取请求异常:url=%s profile=%s attempt=%s/%s err=%s",
url, profile.name, attempt, max_attempts, last_error,
)
continue
# 任何响应都可能带 set-cookie(Akamai 不保证每次下发),拿到就
# 记下时刻,后续请求复用到 TTL 为止。
self._note_cookies(profile)
if response.status_code == 404:
err = ItemNotFoundError(f"Page not found: {url}")
span.record_exception(err)
record_error(span, err)
span.set_attribute("scrape.fail_reason", "ItemNotFoundError")
raise err
@@ -210,11 +244,21 @@ class SiteSession:
if reason is None:
return FetchedPage(html=text, url=final_url)
last_error = self._refine_failure(reason, text)
if last_error.startswith("challenge page detected"):
challenged = last_error.startswith("challenge page detected")
if challenged:
span.set_attribute("scrape.challenge_detected", True)
outcome = "challenge" if challenged else "validate_failed"
else:
last_error = self._describe_error_status(response.status_code)
outcome = "bad_status"
last_text = text
add_event(span, "scrape.attempt", {
"attempt": attempt,
"outcome": outcome,
"status_code": response.status_code,
"html_bytes": len(text),
"error": last_error[:_ERROR_MAX_CHARS],
})
logger.warning(
"抓取结果异常:url=%s profile=%s attempt=%s/%s %s",
url, profile.name, attempt, max_attempts, last_error,
@@ -223,23 +267,44 @@ class SiteSession:
if attempt >= max_attempts:
break
# 第一次失败先便宜地换一套 cookie;仍失败才动用浏览器
# 第一次失败先便宜地换一套 cookie(丢掉旧的,用首页换新的);
# 仍失败才动用浏览器
if attempt == 1:
await self._invalidate(profile)
await self._rewarm_on_home(profile)
span.set_attribute("scrape.rewarmed_on_home", True)
add_event(span, "scrape.escalate", {
"attempt": attempt, "to": "rewarm_on_home",
})
else:
page = await self._escalate_to_browser(profile, url, validate)
if page is not None:
span.set_attribute("scrape.fell_back_to_browser", True)
span.set_attribute("scrape.final_url", page.url)
add_event(span, "scrape.escalate", {
"attempt": attempt, "to": "browser", "outcome": "recovered",
})
return page
# 兜底没救回来(浏览器不可用,或取回的页面仍不合格)。不记
# 这条的话链路里只看得到「最终失败」,看不出浏览器这一级到底
# 试过没有——而「没装 playwright」和「装了也被挡」要分开查。
add_event(span, "scrape.escalate", {
"attempt": attempt,
"to": "browser",
"outcome": "failed",
"reason": self._browser.unavailable_reason,
})
if self._is_server_error(last_error):
err = UpstreamRequestError(f"Upstream request failed: {last_error}")
else:
err = UpstreamBlockedError(f"Blocked while fetching {url}: {last_error}")
span.record_exception(err)
record_error(span, err)
span.set_attribute("scrape.fail_reason", type(err).__name__)
snapshot(span, "scrape.failed_html", last_text, self._settings.otel_snapshot_max_bytes)
snapshot(
span, "scrape.failed_html", last_text,
self._settings.otel_snapshot_max_bytes,
extra={"scrape.url": url, "scrape.profile": profile.name},
)
raise err
finally:
self._semaphore.release()
@@ -265,44 +330,53 @@ class SiteSession:
def _is_server_error(reason: str) -> bool:
return reason.startswith("upstream status") or reason.startswith("httpx") or "Error:" in reason
async def _ensure_warm(self, profile: _Profile) -> None:
"""确保通道有一次新鲜的首页预热;过期时重新访问首页"""
if self._is_warm(profile):
def _note_cookies(self, profile: _Profile) -> None:
"""目标页响应带回 Akamai cookie 时记下时刻,作为 TTL 起点
已经在计时的不重置TTL 要从这套 cookie 第一次出现算起每次响应都
刷新会让一套 cookie 被无限续命反而绕过了 session_ttl_seconds 的本意
"""
if profile.cookies_at:
return
if profile.akamai_cookies:
profile.cookies_at = time.monotonic()
async with profile.lock:
if self._is_warm(profile):
return
try:
response = await profile.client.get(self._settings.home_url)
# Akamai 不保证每次都下发 cookie;首页探测成功本身就是可复用的
# 预热结果,cookie 只用于观测和失败升级时的回灌。
if response.status_code < 400:
profile.warmed_at = time.monotonic()
logger.info(
"会话预热完成:profile=%s status=%s cookies=%s",
profile.name,
response.status_code,
sorted(profile.cookie_names & set(site.AKAMAI_COOKIE_NAMES)),
)
except httpx.HTTPError as exc:
# 预热失败不阻断本次抓取:直连目标页仍可能成功,只是慢
logger.warning("会话预热失败:profile=%s err=%s", profile.name, exc)
profile.warmed_at = 0.0
def _is_warm(self, profile: _Profile) -> bool:
if not profile.warmed_at:
return False
if time.monotonic() - profile.warmed_at > self._settings.session_ttl_seconds:
return False
return True
async def _invalidate(self, profile: _Profile) -> None:
"""清空通道 cookie 并强制下次重新预热"""
async def _drop_expired_cookies(self, profile: _Profile) -> None:
"""cookie 罐超过 TTL 时清空,让下一次请求裸奔换一套新的"""
if not profile.cookies_at:
return
if time.monotonic() - profile.cookies_at <= self._settings.session_ttl_seconds:
return
async with profile.lock:
profile.client.cookies.clear()
profile.warmed_at = 0.0
logger.info("已清空会话 cookie,将重新预热:profile=%s", profile.name)
profile.cookies_at = 0.0
logger.info("会话 cookie 已过期,已清空:profile=%s", profile.name)
async def _rewarm_on_home(self, profile: _Profile) -> None:
"""首次失败后的修复:丢掉旧 cookie,用首页换一套新的
这是首页 URL 唯一的用途目标页已经吃了挑战页继续拿同一个 URL 去撞
只会把挑战坐实首页是站点最"无害"的入口 cookie 的成功率更高
失败不阻断本次抓取下一次 attempt 会裸请求目标页只是慢所以这里
只记日志
"""
async with profile.lock:
profile.client.cookies.clear()
profile.cookies_at = 0.0
try:
response = await profile.client.get(self._settings.home_url)
except httpx.HTTPError as exc:
logger.warning("首页换 cookie 失败:profile=%s err=%s", profile.name, exc)
return
if profile.akamai_cookies:
profile.cookies_at = time.monotonic()
logger.info(
"已用首页换一套新 cookie:profile=%s status=%s cookies=%s",
profile.name,
response.status_code,
sorted(profile.akamai_cookies),
)
async def _escalate_to_browser(
self, profile: _Profile, url: str, validate: PageValidator
@@ -332,7 +406,7 @@ class SiteSession:
domain=cookie.get("domain") or "",
path=cookie.get("path") or "/",
)
profile.warmed_at = time.monotonic()
profile.cookies_at = time.monotonic()
# 浏览器不回报最终 URL,这里以请求地址为准;跨站跳转场景下 HTTP 通道已先行
# 报错,走不到这一步。
+44 -3
View File
@@ -16,10 +16,12 @@ from typing import Any, Generic, TypeVar
from fastapi import Depends, FastAPI, Request
from fastapi.exceptions import RequestValidationError
from fastapi.responses import JSONResponse
from opentelemetry import trace
from pydantic import BaseModel, Field, ValidationError
from starlette.exceptions import HTTPException as StarletteHTTPException
from app.shared.errors import AppError, AuthenticationError
from app.shared.telemetry import record_envelope, record_error
logger = logging.getLogger(__name__)
@@ -98,16 +100,43 @@ def jsonable_errors(errors: list[dict]) -> list[dict]:
return [{key: value for key, value in error.items() if key != "ctx"} for error in errors]
def _trace_failure(
exc: BaseException | None, *, err_code: int, msg: str, status_code: int
) -> None:
"""把一次失败响应记到当前 server span 上(FastAPI 自动 instrumentation 建的那个)
这些处理器是所有对外失败的**唯一出口**也是链路上唯一还知道异常长什么样
的地方它们把异常吃掉换成 200/4xx 的信封响应异常不再向上冒自动
instrumentation 只看得到一个 HTTP 状态码尤其 AppError 默认 status_code=400
信封里 `success=false` trace 里跟正常返回几乎分不出来不在这里记一次
上游报调用失败了时链路里根本找不到对应的错误
span 没在录otel 关闭 /health 这类被 excluded_urls 排除的路径
`set_attributes` / `record_error` 都作用在 NonRecordingSpan noop
不必额外判断
"""
span = trace.get_current_span()
if exc is not None:
record_error(span, exc)
record_envelope(span, success=False, err_code=err_code, msg=msg, status_code=status_code)
def register_exception_handlers(app: FastAPI) -> None:
"""给应用挂上全套异常处理器
两个入口都调用它保证抓取失败与下单失败返回的错误结构完全一致
上游只需要按 code 分支不必区分是哪个服务回的
每个处理器除了构造响应还把这次失败记到当前 server span
`_trace_failure`处理器是失败的唯一出口不记就等于链路里没有这次失败
"""
@app.exception_handler(AppError)
async def app_error_handler(_: Request, exc: AppError) -> JSONResponse:
"""业务异常处理器:返回结构化的错误响应"""
_trace_failure(
exc, err_code=exc.err_code, msg=exc.message, status_code=exc.status_code
)
return JSONResponse(
status_code=exc.status_code,
content=ApiResponse[None](
@@ -123,11 +152,15 @@ def register_exception_handlers(app: FastAPI) -> None:
async def validation_error_handler(_: Request, exc: RequestValidationError) -> JSONResponse:
"""请求参数校验异常处理器"""
errors = exc.errors()
msg = _format_validation_msg(errors)
# 校验失败不记异常本身(pydantic 的 ValidationError 栈很长且没有诊断价值),
# 只留错误码与整理后的字段消息——排查要看的是「哪个字段不合法」
_trace_failure(None, err_code=1002, msg=msg, status_code=422)
return JSONResponse(
status_code=422,
content=ApiResponse[object](
success=False,
msg=_format_validation_msg(errors),
msg=msg,
data=jsonable_errors(errors),
code=1002,
).model_dump(),
@@ -137,11 +170,13 @@ def register_exception_handlers(app: FastAPI) -> None:
async def pydantic_validation_error_handler(_: Request, exc: ValidationError) -> JSONResponse:
"""Pydantic 模型校验异常处理器"""
errors = exc.errors()
msg = _format_validation_msg(errors)
_trace_failure(None, err_code=1002, msg=msg, status_code=422)
return JSONResponse(
status_code=422,
content=ApiResponse[object](
success=False,
msg=_format_validation_msg(errors),
msg=msg,
data=jsonable_errors(errors),
code=1002,
).model_dump(),
@@ -152,11 +187,13 @@ def register_exception_handlers(app: FastAPI) -> None:
"""HTTP 异常处理器(404、500 等)"""
status_code = int(getattr(exc, "status_code", 500) or 500)
err_code = 1404 if status_code == 404 else 1500
detail = str(getattr(exc, "detail", "HTTP error"))
_trace_failure(None, err_code=err_code, msg=detail, status_code=status_code)
return JSONResponse(
status_code=status_code,
content=ApiResponse[None](
success=False,
msg=str(getattr(exc, "detail", "HTTP error")),
msg=detail,
data=None,
code=err_code,
).model_dump(),
@@ -167,6 +204,10 @@ def register_exception_handlers(app: FastAPI) -> None:
async def unhandled_exception_handler(_: Request, exc: Exception) -> JSONResponse:
"""兜底异常处理器:捕获所有未处理的异常"""
logger.exception("未处理异常:%s", exc)
# 这里最需要 record_error:对外只回一句无信息量的 "Internal server error",
# 真正的异常类型与栈只在本进程日志里。记到 span 上,链路里就能直接看到
# 是什么炸了,不必再去捞日志按时间对。
_trace_failure(exc, err_code=1500, msg="Internal server error", status_code=500)
return JSONResponse(
status_code=500,
content=ApiResponse[None](
+8 -1
View File
@@ -73,7 +73,9 @@ class Settings(BaseSettings):
request_timeout_seconds: float = 30.0
max_site_concurrency: int = 8 # 对站点的最大并发请求数
http_max_attempts: int = 3 # 单次抓取的最大尝试次数(含首次)
session_ttl_seconds: float = 1800.0 # Akamai cookie 会话最长复用时长超时后重新预热
# Akamai cookie 最长复用时长超时后清空 cookie 罐,由下一次响应重新建立。
# cookie 随目标页响应下发,正常路径不额外访问站点首页(见 site_session.py)。
session_ttl_seconds: float = 1800.0
# ---- 浏览器兜底配置(仅抓取服务)----
# 纯 HTTP 被 Akamai 拦截时,用 Playwright 打开页面取回 cookie 再回灌给
@@ -101,6 +103,11 @@ class Settings(BaseSettings):
otel_service_name: str = "rakuten" # 仅作兜底;实际值由两侧 main.py 显式覆盖
otel_headers: str | None = None # OTLP 鉴权头,形如 "k=v,k=v";当前 endpoint 裸跑,留空
otel_export_interval_ms: int = 5000
# 不产生 server span 的路径(逗号分隔正则,按 search 匹配完整 URL)。
# 默认排掉 /health:容器 HEALTHCHECK 每 30 秒探一次、上游也在轮询,这些请求
# 各自成为一条孤立 trace,量大且没有信息量——和 worker 空转长轮询同一个问题
# (见 telemetry.py::suppressed)。留空表示不排除任何路径。
otel_excluded_urls: str = "/health$"
# 解析失败时把页面 HTML 作为 span event 上报的上限字节;超出截断并标注。
# 单个搜索页 HTML 可达 200KB-2MB,调高时同步关注 OTLP 单次请求大小限制。
otel_snapshot_max_bytes: int = 2_000_000
+24
View File
@@ -172,6 +172,30 @@ class OrderGuardError(AppError):
super().__init__(message=message, code="ORDER_GUARD", err_code=5004, retryable=False)
class BrowserDeadError(AppError):
"""Playwright 浏览器已掉线(Chromium 崩溃 / 被 OOM kill / 驱动连接断开)
单独成一类而不是复用 OrderOperationError有两个理由
1. **必须是 AppError**掉线时 Playwright 抛的是 `TargetClosedError` 这类
AppError 异常会直接穿过 `runner._execute_with_renewal`
`except AppError`落到主循环那个只记日志的兜底里任务一次都不上报
网关侧要干等整个 lease_ttl默认 300s才被 sweep stale包成 AppError
是为了让 worker **立刻**回报
2. **结论必须是 needs_human 而不是 failed**浏览器是在动作中途没的站点侧
到底生效没有无从判断尤其 submit_order / pay 之后明确失败上报
会误导上游`runner` 为此单独接这一类 _execute_with_renewal
掉线本身能自愈的部分在 `SiteInteractor._ensure_browser_alive()`**任务边界**
上探到浏览器没了会就地重建那条路径不抛本异常抛到这里的都是重建也救不回来
的场景重建失败掉线发生在一次调用中途submit/pay 复用的确认页已随浏览器
一起消失
"""
def __init__(self, message: str = "浏览器已掉线"):
super().__init__(message=message, code="BROWSER_DEAD", err_code=5006, retryable=False)
class CheckoutBlockedError(AppError):
"""结算流程被站点风控拦截(session upgrade 二次验证 / 3DS / 短信验证等)
+174 -2
View File
@@ -9,11 +9,13 @@
- `inventory_flag_for(inventory_type)` 多规格判定
- `basket_domain_of(sell_type)` basketDomain + 反转义 `\\u002F`
- `base_form_fields(shop_id, item_id, inventory_flag)` 加购表单四件套
- `parse_options(information)` 店铺自定义选项`purchase.information.options[]`
的结构化解析含占位值识别scraping 用它对外暴露选项trading 用它自动填 choice
- `auto_choice_for(options)` / `format_choice(...)` choice 表单值的构造
****放这里
- `PurchaseInfo` pydantic 模型 scraping 的对外契约4 个子站共用留在 scraping/models
- variant_id / choice 自动选择策略 trading 独有scraping 把决策权留给上游
- options 结构化解析 scraping 独有trading 用原始 dict
- variant_id 自动选择策略 trading 独有scraping 把决策权留给上游
- 子站books/biccamera/brandavenue的加购契约 各自独立不走这里
依赖约束 `typing.Any`标准库 import scraping / trading / gateway
@@ -21,6 +23,7 @@
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any
# 站点前端构造加购表单时固定带的事件标识(普通购买 normalPurchase)
@@ -76,3 +79,172 @@ def base_form_fields(
"inventory_flag": inventory_flag,
"__event": NORMAL_PURCHASE_EVENT,
}
# ---- 店铺自定义选项(purchase.information.options[])----
#
# 「規格(SKU)」与「選項(option)」是两套完全不同的东西,加购表单里也走不同字段:
# - 規格:sku.variants[].variantId → 表单 variant_id
# - 選項:店铺自己配的下拉/文本框(名入れ文字、配送方式确认、レビュー依頼 等)
# → 表单 choice,格式「名:值」,多项用「,」连接
#
# 2026-08-28 用 .probe/ 下 3 份真实商品页样本 + tests/fixtures/item_with_options_state.json
# 核对过结构,得到两条**结构性**判据(比按日文文案猜稳):
#
# 1. `values[].id == 0` 是「請選擇」占位项,不是可提交的取值。三份真实样本里所有
# 必填 select 的 values[0] 都是 id=0 的「選択してください」,真实可选值从 id=200
# 起编号。此前 trading 侧自动填 choice 时取的就是 values[0],等于把「選択して
# ください」当答案提交上去(.probe/checkout/probe-v2.txt 里两条 "成功" 记录的
# choice 值就是它)——站点当时收下了,但那是店铺没做校验,不代表填对了。
# 2. `type == "text"` 的选项没有 values[](自由文本,如「【お名前】4文字まで」),
# **无法**自动填。必填且为 text 时只能交由调用方给值,不猜。
#
# 占位项文案本身(「選択してください」)作为辅助判据一起保留:id 编号规则是从 4 份
# 样本归纳的,万一某店铺不按 200 起编号,文案还能兜一层。两条判据命中任一即占位。
_PLACEHOLDER_VALUE_ID: int = 0
_PLACEHOLDER_VALUE_NAMES: frozenset[str] = frozenset({"選択してください", "選択して下さい"})
# 选项类型:下拉(有候选值)与自由文本(无候选值)
OPTION_TYPE_SELECT: str = "select"
OPTION_TYPE_TEXT: str = "text"
# choice 表单值的分隔符:项间用「,」,名与值之间用「:」
_CHOICE_PAIR_SEPARATOR: str = ","
_CHOICE_NAME_VALUE_SEPARATOR: str = ":"
@dataclass(slots=True)
class ItemOptionValue:
"""选项的一个候选取值
`is_placeholder=True` 表示这是選択してください这类占位项提交它等于没选
"""
value_id: int | None
name: str
is_placeholder: bool = False
@dataclass(slots=True)
class ItemOption:
"""一个店铺自定义选项
`is_free_text` True values 必然为空站点 type="text"取值只能由
调用方给`selectable_values` 是剔掉占位项后真正可提交的候选
"""
option_id: int | None
name: str
type: str
is_required: bool
values: list[ItemOptionValue] = field(default_factory=list)
@property
def is_free_text(self) -> bool:
return self.type == OPTION_TYPE_TEXT
@property
def selectable_values(self) -> list[ItemOptionValue]:
return [value for value in self.values if not value.is_placeholder]
@property
def can_auto_fill(self) -> bool:
"""能否在不问调用方的情况下自动给出一个合法取值"""
return bool(self.selectable_values)
def _coerce_option_id(raw: Any) -> int | None:
"""选项/取值的 id 收敛为 int;给不出数字时返回 None(不编造 0——0 有含义)"""
if isinstance(raw, bool) or not isinstance(raw, (int, float, str)):
return None
try:
return int(raw)
except (TypeError, ValueError):
return None
def _parse_option_value(raw: Any) -> ItemOptionValue | None:
"""解析单个候选取值;不是 dict 或没有名字的直接丢掉"""
if not isinstance(raw, dict):
return None
name = raw.get("name")
if not isinstance(name, str) or not name:
return None
value_id = _coerce_option_id(raw.get("id"))
is_placeholder = value_id == _PLACEHOLDER_VALUE_ID or name.strip() in _PLACEHOLDER_VALUE_NAMES
return ItemOptionValue(value_id=value_id, name=name, is_placeholder=is_placeholder)
def parse_options(information: Any) -> list[ItemOption]:
"""从 `purchase.information` 解析店铺自定义选项列表
`information` 传站点原始 dict不是 options 数组本身结构不符合预期时返回
空列表**没有选项与解析不出选项在站点数据上长得一样** options
这一层区分不出来也不该假装能区分
"""
if not isinstance(information, dict):
return []
raw_options = information.get("options")
if not isinstance(raw_options, list):
return []
options: list[ItemOption] = []
for raw in raw_options:
if not isinstance(raw, dict):
continue
name = raw.get("name")
if not isinstance(name, str) or not name:
continue
raw_type = raw.get("type")
option_type = raw_type if isinstance(raw_type, str) else ""
values = [
value
for value in (_parse_option_value(item) for item in raw.get("values") or [])
if value is not None
]
options.append(
ItemOption(
option_id=_coerce_option_id(raw.get("id")),
name=name,
type=option_type,
is_required=bool(raw.get("isRequired")),
values=values,
)
)
return options
def format_choice(pairs: Any) -> str:
"""把「名:值」序列拼成站点 choice 表单值
接受 str原样透传调用方自己拼好的或可迭代的字符串序列
"""
if isinstance(pairs, str):
return pairs
return _CHOICE_PAIR_SEPARATOR.join(str(pair) for pair in pairs)
def auto_choice_for(options: list[ItemOption]) -> tuple[str, list[str]]:
"""为必填选项自动挑取值,返回 (choice 表单值, 无法自动填的必填项名)
策略每个必填选项取**第一个非占位**候选值拼成:自由文本必填项
type="text"没有候选值与候选值全是占位项的选项都自动填不了其名字进
第二个返回值调用方据此决定报错还是要求上游显式给 choice**不拿占位值
凑数**那正是本次修复的问题
只处理必填项非必填项站点不强制替上游擅自选比如置き配を希望する
等于替人做了业务决定
"""
pairs: list[str] = []
unfillable: list[str] = []
for option in options:
if not option.is_required:
continue
selectable = option.selectable_values
if not selectable:
unfillable.append(option.name)
continue
pairs.append(
f"{option.name}{_CHOICE_NAME_VALUE_SEPARATOR}{selectable[0].name}"
)
return format_choice(pairs), unfillable
+251 -7
View File
@@ -11,30 +11,54 @@ instrumentation(FastAPI、httpx)。失败时(如 endpoint 不可达)不
OTel SDK 默认的 ProxyTracerProvider setup 之前就能用noop span所以
其它代码里直接 `trace.get_tracer(__name__)` + `start_as_current_span` 即可
不必关心 telemetry 是否启用禁用时 span 不会真正产生与上报
自动 instrumentationFastAPI + httpx只覆盖进程收到 HTTP 请求进程发出
httpx 请求两类边界交易侧的实际工作两者都不是站点交互走 Playwright不经
httpxworker 主循环是后台 asyncio 任务没有 HTTP 入口所以那一侧必须手工
埋点否则 trace 里只剩 worker 与网关之间的往返记录看不到任何业务链路本模块
为此提供三件东西
- `traced` async 方法套一层 span异常自动记录站点交互各步骤在用
- `set_attributes` / `record_error`批量写属性统一记异常
- `suppressed`屏蔽空转长轮询产生的孤立 trace
"""
from __future__ import annotations
import functools
import logging
from typing import TYPE_CHECKING
from collections.abc import Awaitable, Callable, Iterator, Mapping
from contextlib import contextmanager
from typing import TYPE_CHECKING, ParamSpec, TypeVar
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
from opentelemetry.instrumentation.utils import (
is_instrumentation_enabled,
suppress_instrumentation,
)
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.sdk.trace.sampling import ALWAYS_ON
from opentelemetry.trace import Span
from opentelemetry.trace import Span, SpanKind, Status, StatusCode
from opentelemetry.util.types import AttributeValue
from app.shared.config import Settings
from app.shared.config import Settings, get_settings
if TYPE_CHECKING:
from fastapi import FastAPI
P = ParamSpec("P")
R = TypeVar("R")
logger = logging.getLogger(__name__)
# 错误信息类属性的截断长度。站点错误页抽出来的 msg 可能很长(_extract_error_message
# 拼两条提示),而属性值过长会把 OTLP 请求撑大;排查看的是前半句,够了。
_MSG_MAX_CHARS = 512
# 全局 provider 引用,用于 instrument_app / shutdown 时判断当前是否已初始化。
# 显式持有比依赖 trace.get_tracer_provider() 的类型判断更稳——后者在测试场景
# 下可能被其它用例改动全局状态。
@@ -88,10 +112,33 @@ def setup_telemetry(settings: Settings, *, service_name: str) -> None:
def instrument_app(app: "FastAPI") -> None:
"""FastAPI 应用打桩;未初始化时 noop,调用顺序无要求。"""
if _provider is None:
"""FastAPI 应用打桩。必须在应用开始服务之前调用,与 setup_telemetry 的先后无关。
**不能用 `_provider is None` 做前置判断**三个服务都在模块导入时执行
`app = create_app()` `setup_telemetry` 要等 lifespan 启动才跑那时
`_provider` 还是 None照着判断就会直接 returnFastAPI 永远没被打桩
观测后台里一条 server span 都不会有
反过来 lifespan 里再打桩也不行instrument_app 是往应用上加中间件
应用一旦开始服务加进去的中间件不生效实测 lifespan 内调用后 server span
为空所以只能在这里在导入期就装上
provider 尚未设置时拿到的是 ProxyTracer它在 `set_tracer_provider` 之后会
自动委托到真实 provider实测导入期打桩 + lifespan 内设 provider请求
照样产生 span所以顺序不构成问题
otel 关闭时跳过省掉一层用不上的中间件
`excluded_urls` 把健康检查挡在 server span 之外默认 `/health$`容器
HEALTHCHECK 30 秒探一次上游也在轮询这些请求各自是一条孤立 trace
量大且没有信息量配置留空时传 None OTel 回落到它自己的
`OTEL_PYTHON_FASTAPI_EXCLUDED_URLS` 环境变量
"""
settings = get_settings()
if not settings.otel_enabled:
return
FastAPIInstrumentor.instrument_app(app)
excluded = settings.otel_excluded_urls.strip() or None
FastAPIInstrumentor.instrument_app(app, excluded_urls=excluded)
def shutdown_telemetry() -> None:
@@ -107,11 +154,21 @@ def shutdown_telemetry() -> None:
_provider = None
def snapshot(span: Span, name: str, html: str | None, max_bytes: int) -> None:
def snapshot(
span: Span,
name: str,
html: str | None,
max_bytes: int,
*,
extra: Mapping[str, AttributeValue | None] | None = None,
) -> None:
"""把 HTML 作为 span event 上报,超 max_bytes 截断并标注。
用于解析失败时复现页面span 自身只放结构化指标items source
完整 HTML 体量大含商品/价格内容仅在失败分支通过 event 携带
`extra` 用来带上这份 HTML 是哪来的落地 URL页面标题证据文件路径
之类光有一坨 HTML 还得自己回头对是哪一步的产物附在同一条 event 上省事
"""
if html is None or not html:
return
@@ -124,9 +181,196 @@ def snapshot(span: Span, name: str, html: str | None, max_bytes: int) -> None:
}
if truncated:
attributes["snapshot.truncated"] = True
for key, value in (extra or {}).items():
if value is not None:
attributes[key] = value
span.add_event(name, attributes=attributes)
@contextmanager
def suppressed() -> Iterator[None]:
"""在这个上下文里不产生任何自动 instrumentation span。
空转的长轮询worker 30 秒问一次网关有没有活干绝大多数时候
返回空这些请求各自成为一条孤立 trace量大且没有信息量把观测后台刷满
的正是它们领到任务后的每一次网关调用都在任务根 span 底下不受影响
**只挡自动 instrumentation**OTel 那个上下文标记是给 instrumentation 库看的
手工 `start_as_current_span` 不看它照样会建 span所以在这个上下文里手工埋点
要走 `span_unless_suppressed`否则空转长轮询会从另一个口子把孤立 trace 放回来
"""
with suppress_instrumentation():
yield
@contextmanager
def span_unless_suppressed(
tracer: trace.Tracer, name: str, *, kind: SpanKind = SpanKind.INTERNAL
) -> Iterator[Span]:
"""同 `start_as_current_span`,但在 `suppressed()` 里退化成 noop span。
手工埋点与 `suppressed()` 的配套件`suppress_instrumentation` 只被
instrumentation 库尊重手工建的 span 不受它影响worker `lease` /
`lease_query` 正是在 `suppressed()` 里调 `GatewayClient._request` 那里若
无条件建 span空转的长轮询就会每 30 秒产出一条孤立 trace等于绕开了
`suppressed()` 本来要解决的问题
退化时给的是 `INVALID_SPAN`NonRecordingSpan`set_attributes` /
`record_error` / `record_envelope` 作用在它上面全是 noop调用方不必分支
"""
if not is_instrumentation_enabled():
yield trace.INVALID_SPAN
return
with tracer.start_as_current_span(name, kind=kind) as span:
yield span
def record_error(span: Span, exc: BaseException) -> None:
"""把异常记到 span 上并置 ERROR 状态。
单独抽出来是因为 AppError 带的那几个字段对外错误码 `err_code`是否可重试
`retryable`HTTP 状态码正是排查时真正要看的东西按码筛比按异常类名筛更
贴近上游看到的结果 `retryable` 直接决定这次失败该不该重来异常消息也单独
落一个属性`record_exception` 记的 event 在多数观测后台里要展开才看得到
列表页按 `error.message` 筛不出来
"""
span.record_exception(exc)
retryable = getattr(exc, "retryable", None)
set_attributes(
span,
{
"error.type": type(exc).__name__,
"error.message": str(exc)[:_MSG_MAX_CHARS],
"error.code": err_code if isinstance(err_code := getattr(exc, "err_code", None), int) else None,
"error.retryable": retryable if isinstance(retryable, bool) else None,
"error.status_code": sc if isinstance(sc := getattr(exc, "status_code", None), int) else None,
},
)
span.set_status(Status(StatusCode.ERROR, f"{type(exc).__name__}: {exc}"))
def record_envelope(
span: Span,
*,
success: bool,
err_code: int | None = None,
msg: str | None = None,
status_code: int | None = None,
) -> None:
"""把 `ApiResponse` 信封的结果记到 span 上。
自动 instrumentation 只看 HTTP 而本项目的失败**在信封里**一次
`success=false, code=6002` 的回报HTTP 层跟成功的调用长得一模一样很多还
200于是 trace 里只剩调用发生过这次到底成没成错在哪个码
全在 body 不显式记就永远看不到这正是只知道调用了不知道异常怎么
来的的直接原因
出入两侧都用它服务端异常处理器往 server span 上记 `shared.api`
worker 解信封时往 client span 上记 `trading.worker.client`同一套
`api.*` 属性名一条 trace 里两侧的结论可以直接对上
"""
set_attributes(
span,
{
"api.success": success,
"api.code": err_code,
"api.status_code": status_code,
"api.msg": msg[:_MSG_MAX_CHARS] if msg else None,
},
)
if not success:
span.set_status(Status(StatusCode.ERROR, msg or f"api.code={err_code}"))
def add_event(
span: Span, name: str, attributes: Mapping[str, AttributeValue | None]
) -> None:
"""记一条 span event,跳过 None 值属性。
过程不能用属性表达同名属性后写覆盖先写三次抓取尝试写完只剩最后一次
的状态码中间那两次为什么失败升级到哪一级全被盖掉了每次尝试各记一条
event链路里才看得出升级路径
"""
span.add_event(
name,
attributes={key: value for key, value in attributes.items() if value is not None},
)
def record_parse_failure(
span: Span,
exc: BaseException,
*,
html: str | None = None,
max_bytes: int = 0,
url: str | None = None,
stage: str | None = None,
) -> None:
"""页面类操作失败的统一记法:错误详情 + 失败阶段 + 失败页面快照。
抓取侧与 ラクマ 侧一共十个接口的失败分支原本各写一遍同样三步且都漏了最关键
的一件事**失败在哪一步**`html` 是否已拿到恰好就是判据还是空说明页面根本
没取回来通道 / 反爬 / 上游 5xx非空说明取回了但解析不出多半站点改版
两者的排查方向完全相反所以作为属性直接落下来不让人对着一坨 HTML
`stage` 可显式覆盖 `shop_items` 那种转调另外两个接口的编排方法失败
既不在自己的 fetch 也不在自己的 parse html 推断只会给出错的结论
"""
record_error(span, exc)
set_attributes(
span,
{
"parse.stage": stage or ("fetch" if not html else "parse"),
# 保留原有属性名(观测后台的既有筛选条件),但值改成真实异常类名——
# 原先无论什么失败都写死 "parse_error",把反爬阻断也说成解析失败。
"parse.fail_reason": type(exc).__name__,
},
)
snapshot(span, "parse.failed_html", html, max_bytes, extra={"parse.url": url})
def traced(
name: str,
*,
kind: SpanKind = SpanKind.INTERNAL,
) -> Callable[[Callable[P, Awaitable[R]]], Callable[P, Awaitable[R]]]:
"""给 async 方法套一层 span,异常自动记录后原样抛出。
交易侧的实际工作是 Playwright 页面操作httpx 自动 instrumentation 完全看不到
浏览器请求不走 httpx所以这些步骤必须手工埋点否则 trace 里只剩 worker
与网关之间的 HTTP 往返用装饰器而不是在每个方法里写 with 是因为这些方法
的函数体都已经很长再加一层缩进不利于阅读
未启用 telemetry tracer noop装饰器只多一次函数调用可以无条件套
"""
def decorate(fn: Callable[P, Awaitable[R]]) -> Callable[P, Awaitable[R]]:
@functools.wraps(fn)
async def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
tracer = trace.get_tracer(fn.__module__)
with tracer.start_as_current_span(name, kind=kind) as span:
try:
return await fn(*args, **kwargs)
except Exception as exc:
record_error(span, exc)
raise
return wrapper
return decorate
def set_attributes(span: Span, attributes: Mapping[str, AttributeValue | None]) -> None:
"""批量设置属性,跳过 None 值。
站点交互里大量字段是可选的site_order_id 要到提交后才有payable_yen 只在
确认页解析后才有逐个 if 判断会把埋点代码写得比业务逻辑还长
"""
for key, value in attributes.items():
if value is not None:
span.set_attribute(key, value)
def _parse_headers(raw: str | None) -> list[tuple[str, str]] | None:
"""解析 "k1=v1,k2=v2" 形式的 header 配置;空输入返回 None。"""
if not raw or not raw.strip():
+30 -5
View File
@@ -1,5 +1,5 @@
"""交易服务健康检查路由"""
from fastapi import APIRouter, Depends
from fastapi import APIRouter, Depends, Response
from app.shared.api import ApiResponse, get_container
from app.trading.container import TradingContainer
@@ -10,6 +10,7 @@ router = APIRouter(tags=["health"])
@router.get("/health", response_model=ApiResponse[TradingHealthData])
async def health(
response: Response,
container: TradingContainer = Depends(get_container),
) -> ApiResponse[TradingHealthData]:
"""交易服务健康状态
@@ -18,10 +19,34 @@ async def health(
上游高频轮询时反复打站点要实时结果请用 POST /api/auth/status
注意 `logged_in=null` 表示服务启动后还没探测过不等于未登录
browser 反映 Playwright 浏览器的连接状态**浏览器已掉线时本接口返回 503**
这个进程的所有站点操作都要靠那一个浏览器它没了以后进程虽然还活着端口还
却已经什么都干不了每单都会失败返回 503 是为了让容器 HEALTHCHECK
Dockerfile.trading能探到并触发重启这是掉线自愈的最后一道兜底前面还有
SiteInteractor 在任务边界的就地重建
重启是安全的网关侧的下单任务绝不自动重投租约过期只置 stale 等人工
reclaim docs/order-gateway.md §5所以重启只是让 worker 恢复领****
任务的能力不会让任何一笔已经在跑的订单被重复下单
尚未 start() 的启动窗口期不算掉线HEALTHCHECK start-period 兜着
只有起过浏览器且现在连不上才转 degraded
"""
# site 恒非空(lifespan 必建),留 None 分支只是不为一个健康检查赌这一点
browser = container.site.browser_status() if container.site is not None else {}
degraded = bool(browser.get("started")) and not browser.get("alive")
if degraded:
response.status_code = 503
return ApiResponse[TradingHealthData](
success=True,
msg="success",
data=TradingHealthData(status="ok", auth=container.auth_session.status_all()),
code=0,
success=not degraded,
msg="浏览器已掉线" if degraded else "success",
data=TradingHealthData(
status="degraded" if degraded else "ok",
auth=container.auth_session.status_all(),
browser=browser,
),
# 与 BrowserDeadError 用同一个错误码,上游按同一张错误码表分支
code=5006 if degraded else 0,
)
+11 -1
View File
@@ -85,10 +85,20 @@ class TradingHealthData(BaseModel):
只读缓存的登录态不触发网络探测健康检查会被高频轮询实时结果请用
POST /api/auth/status
browser 是唯一一个实时字段但它只读 Playwright 的本地连接状态
`browser.is_connected()`不打站点也不进事件循环高频探活没有代价
"""
status: str = Field(description="服务状态,健康为 ok")
status: str = Field(description="服务状态:ok 正常,degraded 表示浏览器已掉线(HTTP 503)")
auth: dict[str, Any] = Field(default_factory=dict, description="缓存的各站点登录态(不触发网络探测)")
browser: dict[str, Any] = Field(
default_factory=dict,
description=(
"Playwright 浏览器状态:started 是否已跑过 start(),alive 连接是否正常,"
"pending_checkout_tasks 仍留有下单确认页的 task_id,detail 文字说明"
),
)
# ---- 购物车接口(/api/cart/*)----
+60 -19
View File
@@ -15,15 +15,25 @@ import logging
from typing import Any
import httpx
from opentelemetry import trace
from opentelemetry.trace import SpanKind
from app.shared.config import Settings
from app.shared.errors import AppError
from app.shared.proxy import httpx_client_options
from app.shared.task_state import OrderState, TaskStatus
from app.shared.telemetry import (
record_envelope,
record_error,
set_attributes,
span_unless_suppressed,
)
from app.trading.worker.models import LeaseTask, QueryTask
logger = logging.getLogger(__name__)
tracer = trace.get_tracer(__name__)
class GatewayClient:
"""网关 HTTP 客户端
@@ -49,27 +59,58 @@ class GatewayClient:
# ---- 基础封装 ----
async def _request(self, method: str, path: str, **kwargs: Any) -> dict[str, Any]:
"""发起请求并解信封。失败(success=False)抛 AppError"""
response = await self._client.request(method, path, **kwargs)
try:
body = response.json()
except ValueError as exc:
raise AppError(
message=f"网关响应不是合法 JSON:HTTP {response.status_code}",
code="GATEWAY_BAD_BODY",
err_code=3001,
retryable=True,
) from exc
"""发起请求并解信封。失败(success=False)抛 AppError
if not body.get("success"):
raise AppError(
message=body.get("msg", "网关返回失败"),
code="GATEWAY_ERROR",
err_code=int(body.get("code", 1500)),
retryable=False,
status_code=response.status_code,
整段套一个自己的 span而不是依赖 httpx 自动 instrumentation 那个 CLIENT
span**网关的失败在信封里不在 HTTP 状态码上**httpx 那个 span
`request()` 返回时就结束了此时信封还没解一次 `success=false, code=6002`
租约无效的调用在它看来是完成的 200 请求链路里跟成功毫无区别
span 活到解信封之后所以能把这次调用的结论记下来
"""
with span_unless_suppressed(
tracer,
f"gateway.{path.strip('/').replace('/', '.')}",
kind=SpanKind.CLIENT,
) as span:
set_attributes(span, {"gateway.method": method, "gateway.path": path})
response = await self._client.request(method, path, **kwargs)
try:
body = response.json()
except ValueError as exc:
err = AppError(
message=f"网关响应不是合法 JSON:HTTP {response.status_code}",
code="GATEWAY_BAD_BODY",
err_code=3001,
retryable=True,
)
record_error(span, err)
span.set_attribute("gateway.status_code", response.status_code)
raise err from exc
if not body.get("success"):
err_code = int(body.get("code", 1500))
msg = body.get("msg", "网关返回失败")
# 记成信封结果而不是只抛异常:上报被网关拒时,链路里能直接按
# api.code 筛出是哪一类拒绝(6002 租约无效 / 6003 状态不允许…),
# 不必回头翻 worker 日志。
record_envelope(
span,
success=False,
err_code=err_code,
msg=msg,
status_code=response.status_code,
)
raise AppError(
message=msg,
code="GATEWAY_ERROR",
err_code=err_code,
retryable=False,
status_code=response.status_code,
)
record_envelope(
span, success=True, status_code=response.status_code
)
return body
return body
# ---- 接口 ----
+39 -3
View File
@@ -21,8 +21,12 @@ import logging
from datetime import datetime, timezone
from typing import TYPE_CHECKING, Any
from opentelemetry import trace
from opentelemetry.trace import SpanKind
from app.shared.errors import AppError, InvalidRequestError
from app.shared.task_state import AccountQueryKind
from app.shared.telemetry import record_error, set_attributes, suppressed
from app.trading.worker.client import GatewayClient
from app.trading.worker.models import QueryTask
from app.trading.worker.site_interact import SiteInteractor
@@ -32,6 +36,8 @@ if TYPE_CHECKING:
logger = logging.getLogger(__name__)
tracer = trace.get_tracer(__name__)
# 没给 since 时的窗口下界。用 epoch 而不是 None,是为了让翻页终止条件
# (`order_date < since`)与「不设下界」共用同一条代码路径,不多一个分支。
_EPOCH = datetime(1970, 1, 1, tzinfo=timezone.utc)
@@ -100,7 +106,9 @@ class QueryRunner:
logger.info("账号查询 worker 启动:worker_id=%s", self.worker_id)
while self._running:
try:
query = await self._gateway.lease_query(self.worker_id, wait=30)
# 与下单主循环同样:空转的长轮询不埋点,见 runner.py::run
with suppressed():
query = await self._gateway.lease_query(self.worker_id, wait=30)
except AppError as exc:
logger.warning("查询 lease 失败:%s (err=%s)", exc.message, exc.err_code)
await asyncio.sleep(5)
@@ -122,7 +130,27 @@ class QueryRunner:
# ---- 单张查询单 ----
async def handle(self, query: QueryTask) -> None:
"""执行一张查询单并回报结果。任何失败都转成一次「失败回报」,不抛出去"""
"""执行一张查询单并回报结果。任何失败都转成一次「失败回报」,不抛出去
这里开的 span 是一张查询单的根底下挂着站点读取`site.*`与回报网关的
HTTP 调用一个 query_id 对应一条 trace查询失败不抛出去都转成失败
回报所以失败信息由各分支显式记到 span 否则 trace 里会显示成功
"""
with tracer.start_as_current_span("account_query", kind=SpanKind.CONSUMER) as span:
set_attributes(
span,
{
"query.query_id": query.query_id,
"query.kind": query.kind,
"query.site": query.site,
"query.attempt": query.attempt,
"query.worker_id": self.worker_id,
},
)
await self._handle_traced(query, span)
async def _handle_traced(self, query: QueryTask, span: "trace.Span") -> None:
"""handle() 的实际执行体,拆出来只为让根 span 的 with 块保持一层缩进"""
logger.info(
"领到查询单:query_id=%s kind=%s attempt=%s",
query.query_id, query.kind, query.attempt,
@@ -132,11 +160,13 @@ class QueryRunner:
self.execute(query),
timeout=self._settings.account_query_timeout_seconds,
)
except asyncio.TimeoutError:
except asyncio.TimeoutError as exc:
logger.warning(
"查询超时(%s 秒,多半是下单任务正占着账号锁):query_id=%s",
self._settings.account_query_timeout_seconds, query.query_id,
)
span.set_attribute("query.outcome", "timeout")
record_error(span, exc)
await self._report_safe(
query,
success=False,
@@ -152,12 +182,16 @@ class QueryRunner:
"查询失败:query_id=%s code=%s msg=%s",
query.query_id, exc.err_code, exc.message,
)
span.set_attribute("query.outcome", "failed")
record_error(span, exc)
await self._report_safe(
query, success=False, error_code=exc.err_code, error_message=exc.message
)
return
except Exception as exc: # noqa: BLE001
logger.exception("查询未预期异常:query_id=%s", query.query_id)
span.set_attribute("query.outcome", "unexpected_error")
record_error(span, exc)
await self._report_safe(
query,
success=False,
@@ -168,10 +202,12 @@ class QueryRunner:
payload, oversized = self._enforce_result_size(result)
if oversized is not None:
span.set_attribute("query.outcome", "oversized")
await self._report_safe(
query, success=False, error_code=1003, error_message=oversized
)
return
span.set_attribute("query.outcome", "succeeded")
await self._report_safe(query, success=True, result=payload)
async def execute(self, query: QueryTask) -> dict[str, Any]:
+157 -36
View File
@@ -28,8 +28,17 @@ import contextlib
import logging
from typing import TYPE_CHECKING
from app.shared.errors import AppError, CheckoutBlockedError, OrderGuardError
from opentelemetry import trace
from opentelemetry.trace import SpanKind, Status, StatusCode
from app.shared.errors import (
AppError,
BrowserDeadError,
CheckoutBlockedError,
OrderGuardError,
)
from app.shared.task_state import OrderState, TaskStatus
from app.shared.telemetry import record_error, set_attributes, suppressed
from app.trading.worker import verify
from app.trading.worker.client import GatewayClient
from app.trading.worker.evidence import EvidenceStore
@@ -42,6 +51,8 @@ if TYPE_CHECKING:
logger = logging.getLogger(__name__)
tracer = trace.get_tracer(__name__)
def _coerce_state(value: str | None) -> OrderState:
"""把网关返回的 state 字符串安全地包成 OrderState;None 或非法值回落到 CREATED"""
@@ -110,7 +121,11 @@ class WorkerRunner:
logger.info("worker 启动:worker_id=%s", self.worker_id)
while self._running:
try:
task = await self._gateway.lease(self.worker_id, wait=30)
# 空转的长轮询不埋点:30 秒一次、绝大多数返回空,每次都会变成一条
# 孤立 trace 把观测后台刷满。领到任务后的调用都在 handle() 的根
# span 底下,不受这里影响。
with suppressed():
task = await self._gateway.lease(self.worker_id, wait=30)
except AppError as exc:
logger.warning("lease 失败:%s (err=%s)", exc.message, exc.err_code)
await asyncio.sleep(5)
@@ -132,13 +147,42 @@ class WorkerRunner:
# ---- 单任务调度 ----
async def handle(self, task: LeaseTask) -> None:
"""单任务调度入口:本地幂等闸门 → 恢复核对 → 执行"""
"""单任务调度入口:本地幂等闸门 → 恢复核对 → 执行
这里开的 span **整条下单链路的根**往下的每一步站点交互每一次回报
网关都挂在它底下一个 task_id 对应观测后台里的一条 traceworker 是后台
asyncio 任务没有 HTTP 入口不开这个根 span 的话下游 httpx 调用会各自
散成孤立 trace这正是只有请求记录没有链路的原因
"""
with tracer.start_as_current_span("order.task", kind=SpanKind.CONSUMER) as span:
intent = task.intent or {}
set_attributes(
span,
{
"order.task_id": task.task_id,
"order.site": task.site,
"order.worker_id": self.worker_id,
"order.lease_count": task.lease_count,
"order.known_state": task.known_state,
"order.item_url": intent.get("item_url"),
"order.quantity": intent.get("quantity"),
},
)
try:
await self._dispatch(task, span)
except Exception as exc:
record_error(span, exc)
raise
async def _dispatch(self, task: LeaseTask, span: "trace.Span") -> None:
"""handle() 的实际分支逻辑,拆出来只为让根 span 的 with 块保持一层缩进"""
# 本地幂等闸门:之前已完成过的任务不再执行
if await self._db.has_finished(task.task_id):
final_state = await self._db.final_state(task.task_id)
logger.info(
"本地已完成,补报终态:task_id=%s state=%s", task.task_id, final_state
)
span.set_attribute("order.route", "already_finished")
await self._report_safe(
task,
state=_coerce_state(final_state),
@@ -149,10 +193,12 @@ class WorkerRunner:
# 恢复领取:lease_count > 1 表示 stale → reclaim,必须先核对站点订单
if task.lease_count > 1:
span.set_attribute("order.route", "recovery")
await self._handle_recovery(task)
return
# 常规执行
span.set_attribute("order.route", "execute")
await self._execute_with_renewal(task)
async def _handle_recovery(self, task: LeaseTask) -> None:
@@ -192,11 +238,24 @@ class WorkerRunner:
# ---- 常规执行 ----
async def _execute_recording_errors(self, task: LeaseTask) -> None:
"""execute() 外面加一层:异常先记到任务根 span 上,再原样抛出
下面那一串 except 分支会把异常**吞掉**转成 needs_human / failed 回报
异常冒不到 handle() 的根 span在这里统一记一次比每个分支各写一遍省事
也不会漏掉新增的分支
"""
try:
await self.execute(task)
except Exception as exc:
record_error(trace.get_current_span(), exc)
raise
async def _execute_with_renewal(self, task: LeaseTask) -> None:
"""在租约自动续期的上下文里执行任务"""
async with self._renew_lease_every(task, interval=60):
try:
await self.execute(task)
await self._execute_recording_errors(task)
except NotImplementedError as exc:
# 站点交互未实现(规格 §10):上报 needs_human,不视为 worker 失败
logger.warning(
@@ -240,6 +299,23 @@ class WorkerRunner:
terminal=True,
terminal_status=TaskStatus.NEEDS_HUMAN,
)
except BrowserDeadError as exc:
# 浏览器在执行途中没了:站点侧到底生效没有无从判断(尤其掉线发生在
# submit_order / pay 前后),按 needs_human 交人工核对订单列表。
# 必须排在 except AppError 前面——BrowserDeadError 是 AppError 的
# 子类,顺序反了就会被当成普通失败报 failed。
logger.error(
"浏览器掉线导致任务中断,转 needs_human:task_id=%s msg=%s",
task.task_id,
exc.message,
)
await self._report_safe(
task,
state=_coerce_state(task.known_state),
detail=exc.message,
terminal=True,
terminal_status=TaskStatus.NEEDS_HUMAN,
)
except AppError as exc:
logger.warning(
"执行失败:task_id=%s code=%s msg=%s",
@@ -499,38 +575,65 @@ class WorkerRunner:
证据载体 html / screenshot 即本步骤要落盘的页面与整页截图
显式传入的 `html` / `png` 优先级更高 step3/step4 等已经单独拿到
页面的调用点使用
每步一个 span挂在 handle() 的任务根 span 底下一条 trace 就是一单的
完整流水清车 加购 校验 确认 提交 付款卡在哪一步每步
耗时多少证据落在哪个 evidence_ref 都能直接读出来
"""
result = await action()
if isinstance(result, PageSnapshot):
if html is None:
html = result.html or None
if png is None:
png = result.screenshot or None
elif html is None and isinstance(result, str):
# 旧契约兼容:站点方法返回字符串时视为页面 HTML
html = result
meta = {
"step": step_name,
"state": state.value,
**(evidence_meta or {}),
}
evidence_ref = self._evidence.write_step(
task.task_id, step_no, step_name, html=html, png=png, meta=meta
)
await self._db.index_evidence(task.task_id, step_no, step_name, evidence_ref)
await self._db.record_event(
task.task_id, state.value, detail=detail, evidence_ref=evidence_ref
)
await self._gateway.report(
task.task_id,
self.worker_id,
state=state,
payable_yen=payable_yen,
pay_deadline=pay_deadline,
site_order_id=site_order_id,
evidence_ref=evidence_ref,
detail=detail,
)
with tracer.start_as_current_span(f"order.step.{step_name}") as span:
set_attributes(
span,
{
"order.task_id": task.task_id,
"order.step_no": step_no,
"order.step_name": step_name,
"order.state": state.value,
},
)
try:
result = await action()
except Exception as exc:
record_error(span, exc)
raise
if isinstance(result, PageSnapshot):
if html is None:
html = result.html or None
if png is None:
png = result.screenshot or None
elif html is None and isinstance(result, str):
# 旧契约兼容:站点方法返回字符串时视为页面 HTML
html = result
meta = {
"step": step_name,
"state": state.value,
**(evidence_meta or {}),
}
evidence_ref = self._evidence.write_step(
task.task_id, step_no, step_name, html=html, png=png, meta=meta
)
set_attributes(
span,
{
"order.evidence_ref": evidence_ref,
"order.site_order_id": site_order_id,
"order.payable_yen": payable_yen,
},
)
await self._db.index_evidence(task.task_id, step_no, step_name, evidence_ref)
await self._db.record_event(
task.task_id, state.value, detail=detail, evidence_ref=evidence_ref
)
await self._gateway.report(
task.task_id,
self.worker_id,
state=state,
payable_yen=payable_yen,
pay_deadline=pay_deadline,
site_order_id=site_order_id,
evidence_ref=evidence_ref,
detail=detail,
)
async def _report_safe(
self,
@@ -544,7 +647,25 @@ class WorkerRunner:
payable_yen: int | None = None,
pay_deadline: str | None = None,
) -> None:
"""回报 gateway,失败只记日志不抛——主循环不能因为回报失败退出"""
"""回报 gateway,失败只记日志不抛——主循环不能因为回报失败退出
顺带把终态标到当前 span 这一步是必要的`_execute_with_renewal` 会把
闸门拦截风控拦截浏览器掉线等异常**吞掉**转成 needs_human 回报异常
不会冒到 handle() 的根 span于是一笔被拦下的单在 trace 里看起来跟成功
下单一模一样所有这些分支都汇到这个方法标在这里最省事也最不容易漏
`is_recording()` 那道判断不是多余的付款后监控_monitor_order
`create_task` 起的后台任务 asyncio 在创建时就把当时的 context 复制了
进去等它真正跑起来任务根 span 早已结束 `get_current_span()` 拿到
的仍是那个**已结束** span不是 INVALID_SPAN往上写属性会打
"Setting attribute on ended span" 警告当前监控路径不传 terminal_status
走不到这里但这道判断保证以后传了也不会污染已完成的任务 span
"""
span = trace.get_current_span()
if terminal_status is not None and span.is_recording():
span.set_attribute("order.terminal_status", terminal_status.value)
if terminal_status in (TaskStatus.NEEDS_HUMAN, TaskStatus.FAILED):
span.set_status(Status(StatusCode.ERROR, detail))
try:
await self._gateway.report(
task.task_id,
+389 -97
View File
@@ -94,9 +94,12 @@ import logging
import re
from dataclasses import dataclass, field
from datetime import datetime
from typing import TYPE_CHECKING, TypeVar
from typing import TYPE_CHECKING, Any, TypeVar
from opentelemetry.trace import SpanKind
from app.shared.errors import (
BrowserDeadError,
CartOperationError,
CheckoutBlockedError,
InvalidRequestError,
@@ -106,17 +109,23 @@ from app.shared.errors import (
from app.shared.purchase_contract import (
INVENTORY_FLAG_DEFAULT,
INVENTORY_FLAG_MULTIPLE,
auto_choice_for,
base_form_fields,
basket_domain_of,
format_choice,
inventory_flag_for,
parse_options,
)
from app.shared.proxy import playwright_launch_proxy
from app.shared.task_state import OrderState
from app.shared.telemetry import traced
from app.trading.core import auth_site
from app.trading.worker.models import LeaseTask
if TYPE_CHECKING:
from collections.abc import Awaitable, Callable
from pathlib import Path
from playwright.async_api import Page
@@ -620,15 +629,29 @@ class SiteInteractor:
浏览器 context 复用同一份登录态所有任务串行self._lock + worker 主循环本就串行
不需要为每个任务开新 context开销大且 cookie 状态会乱
**掉线兜底**Chromium 可能崩溃容器里 /dev/shm 不足 OOM kill
seccomp 挡住 sandbox docker-compose.yml 注释而这个浏览器是进程级
单例一旦没了后续每一单都会失败处理分两层
- 任务边界各公开方法入口的 `_ensure_context_ready()`探到掉线就**就地重建**
一套完全一样的 playwright/browser/context调用方无感知
- 一次调用**中途**掉线不重建站点侧生效与否无从判断统一抛 `BrowserDeadError`
AppError 的一种 runner needs_human 交人工核对submit_order / pay
因为复用 enter_checkout 留存的 Page入口处直接用 `_require_live_browser()`
拒绝不走重建
`browser_status()` 把这份状态暴露给交易服务 /health容器 HEALTHCHECK 据此
在进程还活着但浏览器已死时触发重启
调用方
- worker runner传入 LeaseTask add_to_cart(task) / verify_cart(task)
- HTTP 路由 /api/cart/* add_to_cart_payload(...) / cart_status() /
clear_cart() / remove_item(item_id)
"""
# 每任务保留的临时状态:task_id → {"item_id": str, "shop_bid": str}
# 用于 add_to_cart 把抓出来的 item_id / shop_bid 喂给 verify_cart
_per_task_state: dict[str, dict[str, str]]
# 每任务保留的临时状态:task_id → 首个商品字段及全部 item_ids。
# 旧调用方仍读取 item_id,新调用方由 item_ids 校验整组商品。
_per_task_state: dict[str, dict[str, Any]]
# 每任务保留的下单确认页 Page:enter_checkout 落地后不关闭页面,存在这里,
# submit_order / pay 复用同一个页面继续操作——下单确认页是服务端会话态,
@@ -654,28 +677,56 @@ class SiteInteractor:
# ---- 生命周期 ----
def _state_path(self) -> "Path":
"""rakuten 的 storage_state 文件路径"""
return self._settings.auth_state_path / auth_site.profile("rakuten").state_filename
@staticmethod
def _context_options(storage_state: str | None) -> dict:
"""new_context 的参数。启动与掉线重建必须用同一套,指纹漂移会触发风控"""
return {
"storage_state": storage_state,
"user_agent": auth_site.RAKUTEN_USER_AGENT,
"locale": "ja-JP",
"timezone_id": "Asia/Tokyo",
"viewport": {"width": 390, "height": 844},
"is_mobile": True,
"has_touch": True,
}
async def start(self) -> None:
"""启动 Playwright 与带 cookie 的浏览器 context
登录态文件不存在时同样启动context cookie后续 add_to_cart
require_logged_in 里报错这样保持启动路径一致
"""
await self._launch()
logger.info("SiteInteractor 已就绪:storage_state=%s", self._state_path())
async def _launch(self) -> None:
"""真正拉起 playwright + browser + context。start() 与掉线重建共用
抽出来的唯一目的就是让 `_ensure_browser_alive()` 能原样重放一遍启动流程
重建出来的浏览器必须和启动时**完全一致**同样的 UA / viewport / proxy /
launch args否则 Akamai 那边指纹一变就是一次风控事件
"""
from playwright.async_api import async_playwright
state_path = self._settings.auth_state_path / auth_site.profile("rakuten").state_filename
state_path = self._state_path()
storage_state = str(state_path) if state_path.exists() else None
if storage_state is None:
logger.warning(
"登录态文件不存在:site_interactor 以无 cookie 状态启动,"
"加购请求会被站点拒认"
)
self._state_mtime = None
else:
self._state_mtime = state_path.stat().st_mtime
self._playwright = await async_playwright().start()
self._browser = await self._playwright.chromium.launch(
# 2026-08-14 实测确认:headless=True 会让站点的购物车/结算 SPA 表现
# 异常(入手続き点了不跳转、shopUrlList 渲染不出来),换 headless=False
# 异常(入手続き点了不跳转、shopUrlList 渲染不出来),换 headless=False
# 后行为与真实下单一致(能正常触发 session upgrade)——所有会改动站点
# 状态的操作(加购/结算/支付)都必须走非无头浏览器
headless=False,
@@ -683,26 +734,150 @@ class SiteInteractor:
proxy=playwright_launch_proxy(self._settings),
args=["--no-first-run", "--disable-blink-features=AutomationControlled"],
)
self._context = await self._browser.new_context(
storage_state=storage_state,
user_agent=auth_site.RAKUTEN_USER_AGENT,
locale="ja-JP",
timezone_id="Asia/Tokyo",
viewport={"width": 390, "height": 844},
is_mobile=True,
has_touch=True,
)
logger.info("SiteInteractor 已就绪:storage_state=%s", storage_state or "(none)")
self._context = await self._browser.new_context(**self._context_options(storage_state))
async def _refresh_context_if_stale(self) -> None:
"""检查 storage_state 文件 mtime,变化则重建 context
# ---- 掉线探活与重建 ----
AuthSession.try_relogin 成功后会重写 storage_state 文件 context 启动时
用快照式 storage_state 创建cookie 不会自动同步必须关掉旧 context
用新文件重建 add_to_cart / verify_cart 开头各调一次开销可接受
只在 mtime 变了才重建
@property
def browser_alive(self) -> bool:
"""浏览器进程当前是否还连着
/health `_ensure_browser_alive()` 判断`is_connected()` 是同步调用
不打站点不进事件循环高频探活没有代价
"""
state_path = self._settings.auth_state_path / auth_site.profile("rakuten").state_filename
browser = self._browser
if browser is None:
return False
try:
return bool(browser.is_connected())
except Exception:
return False
def browser_status(self) -> dict:
"""浏览器状态快照,供交易服务 /health 暴露
`started` `alive` 要分开看`started=False` 还没跑 start()正常的
启动窗口期`started=True, alive=False` 才是浏览器死了只有后者该
让健康检查转 degraded
"""
started = self._browser is not None
alive = self.browser_alive
if not started:
detail = "尚未启动"
elif alive:
detail = "连接正常"
else:
detail = "浏览器已掉线(Chromium 崩溃 / 被 kill),需要重建或重启进程"
return {
"started": started,
"alive": alive,
"pending_checkout_tasks": sorted(self._checkout_pages),
"detail": detail,
}
async def _ensure_browser_alive(self) -> None:
"""任务边界上的探活:浏览器没了就地重建一套
**只在任务边界重建**各公开方法入口动作还没发出去的时候中途掉线不
走这里那时站点侧生不生效已经无从判断只能抛 BrowserDeadError 交人工
`_new_page()` `_require_live_browser()`
重建前会把 `_checkout_pages` 里残留的确认页丢掉那些 Page 已经随浏览器
一起没了留着只会让后续 submit_order / pay 拿到一个必然报错的死对象
丢弃前记一条 warning留了确认页说明上一单卡在已进确认页可能已提交
的中间态这条日志是人工核对的线索
Raises:
BrowserDeadError: 重建失败Chromium 起不来下一次调用会再试一次
"""
if self.browser_alive:
return
orphaned = sorted(self._checkout_pages)
if orphaned:
logger.warning(
"浏览器掉线时仍留有未收尾的下单确认页,随浏览器一起丢弃,"
"这些任务的站点侧状态需人工核对:task_ids=%s",
orphaned,
)
self._checkout_pages.clear()
logger.warning("检测到浏览器已掉线,就地重建 Playwright / 浏览器 / context")
await self._teardown()
try:
await self._launch()
except Exception as exc:
# 重建失败时把半成品收干净,让下一次调用从 _browser=None 重新试
await self._teardown()
raise BrowserDeadError(
f"浏览器掉线后重建失败:{type(exc).__name__}: {exc}"
) from exc
logger.info("浏览器已重建完成")
def _require_live_browser(self, where: str) -> None:
"""确认浏览器还活着,否则抛 BrowserDeadError(**不重建**)
submit_order / pay 这种复用 enter_checkout 留存的 Page的步骤用
重建一套新浏览器救不回那份服务端订单草稿而且这两步正好卡在提交前后
站点侧生效与否不能猜直接抛出去让 runner needs_human
"""
if self.browser_alive:
return
raise BrowserDeadError(
f"{where}:浏览器已掉线,enter_checkout 留存的下单确认页已随之失效,"
"本单站点侧是否已提交无法判断,需人工核对订单列表(不要按失败重试)"
)
async def _new_page(self) -> "Page":
"""开一个新页面,掉线时抛 BrowserDeadError 而不是裸的 TargetClosedError
各方法里 `new_page()` 都写在 try 之外页面还没拿到没有要 close 的东西
裸异常会穿过 runner `except AppError` 一路飘到主循环任务一次都不上报
统一走这个壳把它包成 AppError
"""
try:
return await self._context.new_page()
except Exception as exc:
raise BrowserDeadError(
f"打开新页面失败,浏览器可能已掉线:{type(exc).__name__}: {exc}"
) from exc
async def _request(self, method: str, url: str, **kwargs):
"""走 context.request 发请求(共享 cookie),掉线时转成 BrowserDeadError
`_new_page()` 同样的理由裸的 TargetClosedError 会绕过 runner
`except AppError`区别是这里的请求本来就可能因为站点/网络原因失败那些
是正常的业务失败所以**只在确认浏览器真的没了时**才改写异常其余原样
抛出交给各调用点既有的判据处理
"""
try:
return await getattr(self._context.request, method)(url, **kwargs)
except Exception as exc:
if self.browser_alive:
raise
raise BrowserDeadError(
f"{method.upper()} {url} 失败且浏览器已掉线:{type(exc).__name__}: {exc}"
) from exc
async def _ensure_context_ready(self) -> None:
"""每个站点动作前的统一前置:浏览器还活着 + context 拿的是最新 cookie
两件事按顺序做顺序不能换mtime 重建要用 `self._browser`浏览器已经
死了的话得先重建出来
1. `_ensure_browser_alive()`掉线就地重建任务边界
2. storage_state mtime 变了就重建 contextAuthSession.try_relogin 成功后会
重写该文件而本 context 启动时拿的是快照式 storage_statecookie 不会
自动同步必须关掉旧 context 用新文件重建
只在 mtime 真的变了才重建每个动作前调一次的开销可接受
Raises:
BrowserDeadError: 浏览器掉线且重建失败
"""
await self._ensure_browser_alive()
state_path = self._state_path()
if not state_path.exists():
return
mtime = state_path.stat().st_mtime
@@ -718,15 +893,14 @@ class SiteInteractor:
await self._context.close()
except Exception:
logger.debug("关闭旧 context 失败", exc_info=True)
self._context = await self._browser.new_context(
storage_state=str(state_path),
user_agent=auth_site.RAKUTEN_USER_AGENT,
locale="ja-JP",
timezone_id="Asia/Tokyo",
viewport={"width": 390, "height": 844},
is_mobile=True,
has_touch=True,
)
try:
self._context = await self._browser.new_context(
**self._context_options(str(state_path))
)
except Exception as exc:
raise BrowserDeadError(
f"重建 context 失败,浏览器可能已掉线:{type(exc).__name__}: {exc}"
) from exc
self._state_mtime = mtime
async def _read_with_relogin_retry(
@@ -751,7 +925,7 @@ class SiteInteractor:
await self._auth_session.require_logged_in("rakuten")
# 重登会重写 storage_state,本 context 的 cookie 是启动时的快照,
# 必须在每次 attempt 前都刷一遍,否则重登后仍拿旧 cookie 去读。
await self._refresh_context_if_stale()
await self._ensure_context_ready()
try:
return await read()
except _LoggedOutMidRead as exc:
@@ -778,6 +952,16 @@ class SiteInteractor:
"""关闭 context、browser、playwright,吞掉单个 close 异常"""
for task_id in list(self._checkout_pages):
await self._discard_checkout_page(task_id)
await self._teardown()
async def _teardown(self) -> None:
"""释放 context / browser / playwright 三件套并置空
close() 分开是因为掉线重建`_ensure_browser_alive`也要走一遍那条
路径不能碰 `_checkout_pages`那些 Page 已经死了`page.close()` 只会再抛
一次异常它自己负责丢弃每个 close 独立 try已经死掉的浏览器上
close 本来就会抛不能因此漏掉后面两个资源的释放
"""
for resource, name in (
(self._context, "context"),
(self._browser, "browser"),
@@ -796,48 +980,46 @@ class SiteInteractor:
# ---- 已实现:add_to_cart / verify_cart / cart_status ----
@traced("site.add_to_cart", kind=SpanKind.CLIENT)
async def add_to_cart(self, task: LeaseTask) -> PageSnapshot:
"""加购(worker 入口):从 task.intent 取字段,调 _add_to_cart_with_fields
调用方 task.intent 提供
- item_url: 商品详情页 URL必填
- quantity: 数量默认 1
- variant_id: 多规格商品的 variant_id不传则从 sku.variants[] 自动选第一个非售罄
- choice: 必填选项的取值列表不传则每个必填选项用第一个候选值站点不严格校验
调用方 task.intent 提供 `items` 商品数组数组元素字段与旧版单商品
字段相同item_url / quantity / variant_id / choice为兼容已发布的
调用方也接受顶层 item_url 等旧字段并自动包装成单元素数组
Returns:
PageSnapshot加购响应的落地页 HTML + 商品页整页截图 runner 落证据
Raises:
InvalidRequestError: intent.item_url 缺失
InvalidRequestError: intent.item_url 缺失 intent.items 格式非法
NotLoggedInError: 登录态失效
CartOperationError: 商品页打不开state 解析失败商品不可购买加购返回错误页
"""
intent = task.intent or {}
item_url = intent.get("item_url")
if not item_url:
raise InvalidRequestError("intent.item_url 必填")
quantity = int(intent.get("quantity") or 1)
if quantity <= 0:
raise InvalidRequestError(f"intent.quantity 必须为正整数,收到 {quantity}")
items = _normalize_intent_items(task.intent or {})
async with self._lock:
result = await self._add_to_cart_with_fields(
item_url=item_url,
quantity=quantity,
variant_id=intent.get("variant_id"),
choice=intent.get("choice"),
)
self._per_task_state[task.task_id] = {
"item_id": result["item_id"],
"shop_bid": result["shop_bid"],
"basket_domain": result["basket_domain"],
results = []
for item in items:
results.append(await self._add_to_cart_with_fields(**item))
first = results[0]
state: dict[str, Any] = {
# 保留旧字段,避免已有 verify/监控代码及外部桩失效。
"item_id": first["item_id"],
"shop_bid": first["shop_bid"],
"basket_domain": first["basket_domain"],
}
if len(results) > 1:
state["item_ids"] = [result["item_id"] for result in results]
state["items"] = results
self._per_task_state[task.task_id] = state
result = results[-1]
return PageSnapshot(
html=result.get("response_html") or "",
screenshot=result.get("screenshot") or b"",
)
@traced("site.add_to_cart_payload", kind=SpanKind.CLIENT)
async def add_to_cart_payload(
self,
*,
@@ -880,7 +1062,7 @@ class SiteInteractor:
response_html / screenshot worker 入口包成 PageSnapshot 落证据
"""
await self._auth_session.require_logged_in("rakuten")
await self._refresh_context_if_stale()
await self._ensure_context_ready()
intent_for_extract: dict = {}
if variant_id is not None:
@@ -888,7 +1070,7 @@ class SiteInteractor:
if choice is not None:
intent_for_extract["choice"] = choice
page = await self._context.new_page()
page = await self._new_page()
try:
try:
await page.goto(item_url, wait_until="domcontentloaded", timeout=30_000)
@@ -919,10 +1101,18 @@ class SiteInteractor:
raise CartOperationError(
"多规格商品未选 variant,且 sku.variants 全部售罄或为空"
)
# 必填选项要求填了 choice
if fields["has_required_options"] and not fields["form_fields"].get(fields["options_field"]):
# 必填选项要求填了 choice。调用方没给 choice 时,只要存在「自动填不了」
# 的必填项(自由文本项,或候选值只剩占位项)就当场失败并点名是哪几项——
# 这些项非人工给值不可能成功,继续 POST 只会拿站点的
# 「未選択の項目からどれか1つ選んでください。」错误页,排查成本更高
if fields["has_required_options"] and not fields["form_fields"].get(
fields["options_field"]
):
unfillable = fields["unfillable_required_options"]
detail = f"{unfillable}" if unfillable else ""
raise CartOperationError(
"商品有必填选项但未提供 choice,且选项无候选值"
"商品有必填选项但未提供 choice,且这些必填项无法自动选值"
f"(需在 intent.choice 里按「选项名:取值名」显式给出){detail}"
)
payload = dict(fields["form_fields"])
@@ -932,7 +1122,8 @@ class SiteInteractor:
"加购请求:basket=%s payload=%s", fields["basket_domain"], payload,
)
resp = await self._context.request.post(
resp = await self._request(
"post",
fields["basket_domain"],
form=payload,
max_redirects=5,
@@ -985,6 +1176,7 @@ class SiteInteractor:
"screenshot": screenshot,
}
@traced("site.verify_cart", kind=SpanKind.CLIENT)
async def verify_cart(self, task: LeaseTask) -> PageSnapshot:
"""校验购物车里有没有刚加的商品
@@ -1000,11 +1192,18 @@ class SiteInteractor:
"""
async with self._lock:
await self._auth_session.require_logged_in("rakuten")
await self._refresh_context_if_stale()
await self._ensure_context_ready()
per_task = self._per_task_state.get(task.task_id, {})
item_id = (task.intent or {}).get("item_id") or per_task.get("item_id")
if not item_id:
intent_item_id = (task.intent or {}).get("item_id")
item_ids = (
[intent_item_id]
if intent_item_id
else (per_task.get("item_ids") or [])
)
if not item_ids and per_task.get("item_id"):
item_ids = [per_task["item_id"]]
if not item_ids:
raise CartOperationError(
"无法确定 item_id:intent 未提供且 add_to_cart 未记录"
)
@@ -1015,9 +1214,17 @@ class SiteInteractor:
raise CartOperationError("购物车为空,加购可能未生效")
logger.info("cart count=%s task_id=%s", count, task.task_id)
# 2. 渲染 cart 页确认 item_id 在里面
return await self._verify_item_in_cart_html(item_id, label=f"task_id={task.task_id}")
# 2. 渲染一次 cart 页确认本任务的全部商品都在里面
# 单商品继续走旧 helper,保留原有测试桩与内部调用契约。
if len(item_ids) == 1:
return await self._verify_item_in_cart_html(
str(item_ids[0]), label=f"task_id={task.task_id}"
)
return await self._verify_items_in_cart_html(
[str(item_id) for item_id in item_ids], label=f"task_id={task.task_id}"
)
@traced("site.cart_status", kind=SpanKind.CLIENT)
async def cart_status(self) -> dict:
"""轻量查询购物车状态:调 cart count JSONP API,不渲染整页
@@ -1028,7 +1235,7 @@ class SiteInteractor:
"""
async with self._lock:
await self._auth_session.require_logged_in("rakuten")
await self._refresh_context_if_stale()
await self._ensure_context_ready()
raw_status, count = await self._query_cart_count()
return {
"logged_in": True,
@@ -1038,6 +1245,7 @@ class SiteInteractor:
# ---- 已实现:clear_cart / remove_item(Playwright UI 点击)----
@traced("site.clear_cart", kind=SpanKind.CLIENT)
async def clear_cart(self) -> dict:
"""清空购物车:渲染 cart SPA → 反复点第一个「削除」按钮 → count API 校验
@@ -1057,9 +1265,9 @@ class SiteInteractor:
"""
async with self._lock:
await self._auth_session.require_logged_in("rakuten")
await self._refresh_context_if_stale()
await self._ensure_context_ready()
page = await self._context.new_page()
page = await self._new_page()
removed = 0
html = ""
screenshot = b""
@@ -1126,6 +1334,7 @@ class SiteInteractor:
"screenshot": screenshot,
}
@traced("site.remove_item", kind=SpanKind.CLIENT)
async def remove_item(self, item_id: str) -> dict:
"""删除购物车里指定 item_id 的商品
@@ -1148,9 +1357,9 @@ class SiteInteractor:
raise InvalidRequestError("item_id 必填")
async with self._lock:
await self._auth_session.require_logged_in("rakuten")
await self._refresh_context_if_stale()
await self._ensure_context_ready()
page = await self._context.new_page()
page = await self._new_page()
try:
await page.goto(_CART_PAGE, wait_until="domcontentloaded", timeout=30_000)
await self._wait_cart_rendered(page, label=f"remove_item {item_id}")
@@ -1212,7 +1421,8 @@ class SiteInteractor:
才是获取失败 CartOperationError常见原因是 Referer 错或
cookie 失效
"""
resp = await self._context.request.get(
resp = await self._request(
"get",
_CART_COUNT_API + "?sid=1010",
headers={"Referer": _CART_PAGE},
)
@@ -1236,7 +1446,13 @@ class SiteInteractor:
返回渲染后的 cart HTML + 整页截图校验通过时供调用方落证据
"""
page = await self._context.new_page()
return await self._verify_items_in_cart_html([item_id], label=label)
async def _verify_items_in_cart_html(
self, item_ids: list[str], *, label: str
) -> PageSnapshot:
"""渲染一次 cart SPA,确认多个 item_id 都存在,避免多商品任务重复开页。"""
page = await self._new_page()
try:
await page.goto(_CART_PAGE, wait_until="domcontentloaded", timeout=30_000)
await self._wait_cart_rendered(page, label=label)
@@ -1248,11 +1464,12 @@ class SiteInteractor:
site="rakuten",
detail="购物车页出现旧版未登录 marker",
)
if str(item_id) not in html:
missing = [item_id for item_id in item_ids if str(item_id) not in html]
if missing:
raise CartOperationError(
f"购物车页未找到 item_id={item_id}(加购可能被服务端静默丢弃)"
f"购物车页未找到 item_id={missing}(加购可能被服务端静默丢弃)"
)
logger.info("cart 校验通过:%s item_id=%s in cart HTML", label, item_id)
logger.info("cart 校验通过:%s item_ids=%s in cart HTML", label, item_ids)
# 校验通过的 cart 页整页截图随结果带出;失败不掩盖校验结果
screenshot = b""
try:
@@ -1327,6 +1544,7 @@ class SiteInteractor:
# ---- 已实现:enter_checkout(到下单确认页,中间步骤未经真实 HTML 验证)----
@traced("site.enter_checkout", kind=SpanKind.CLIENT)
async def enter_checkout(self, task: LeaseTask) -> PageSnapshot:
"""进入下单确认页:购物车 → 点「購入手続き」→ 依次处理中间步骤 → 落地确认页
@@ -1367,9 +1585,9 @@ class SiteInteractor:
"""
async with self._lock:
await self._auth_session.require_logged_in("rakuten")
await self._refresh_context_if_stale()
await self._ensure_context_ready()
page = await self._context.new_page()
page = await self._new_page()
success = False
try:
await page.goto(_CART_PAGE, wait_until="domcontentloaded", timeout=30_000)
@@ -1838,6 +2056,7 @@ class SiteInteractor:
"""
return _parse_checkout_summary(html)
@traced("site.submit_order", kind=SpanKind.CLIENT)
async def submit_order(self, task: LeaseTask) -> SubmitOutcome:
"""点击下单确认页的最终确认按钮,提交订单
@@ -1856,6 +2075,10 @@ class SiteInteractor:
供人工核对详见 _discard_checkout_page 的调用取舍
"""
async with self._lock:
# 掉线检查必须在取 page 之前:浏览器没了的话 _checkout_pages 里那个
# Page 是个死对象(不是 None),下面的 selector 轮询会全部超时,最后
# 报「未找到确认按钮」——把「浏览器崩了」误诊成「站点改版了」
self._require_live_browser("submit_order")
page = self._checkout_pages.get(task.task_id)
if page is None:
raise OrderOperationError(
@@ -1935,6 +2158,7 @@ class SiteInteractor:
evidence=PageSnapshot(html=html, screenshot=screenshot),
)
@traced("site.pay", kind=SpanKind.CLIENT)
async def pay(self, task: LeaseTask, site_order_id: str) -> PageSnapshot:
"""检查提交下单后是否已完成付款 / 是否触发了需要人工介入的验证环节
@@ -1952,6 +2176,10 @@ class SiteInteractor:
整页截图 runner step 5 落证据
"""
async with self._lock:
# 这一步最危险:submit_order 已经真的提交过订单了。浏览器在这时候没了,
# 「付款到底完成没有」完全无从判断,必须当场抛出去转人工,不能让裸的
# TargetClosedError 飘到主循环变成「一次都不上报」。
self._require_live_browser("pay")
page = self._checkout_pages.pop(task.task_id, None)
if page is None:
raise OrderOperationError(
@@ -1987,8 +2215,14 @@ class SiteInteractor:
logger.warning("pay 页面截图失败(不影响付款判定)", exc_info=True)
return PageSnapshot(html=html, screenshot=screenshot)
finally:
await page.close()
# 关页面失败不能掩盖 try 里的原始异常——浏览器要是在本方法执行途中
# 没的,close() 自己也会抛,那条异常没有任何诊断价值
try:
await page.close()
except Exception:
logger.debug("pay 关闭确认页失败", exc_info=True)
@traced("site.check_order_status", kind=SpanKind.CLIENT)
async def check_order_status(self, site_order_id: str) -> OrderStatusSnapshot:
"""付款后监控的单次探测:查一次订单详情页的配送阶段,不循环
@@ -2006,6 +2240,7 @@ class SiteInteractor:
"""
return (await self.fetch_order_detail(site_order_id)).status
@traced("site.fetch_order_detail", kind=SpanKind.CLIENT)
async def fetch_order_detail(self, site_order_id: str) -> OrderDetailSnapshot:
"""读一次订单详情页:配送阶段 + 页面原始 __INITIAL_STATE__
@@ -2034,7 +2269,7 @@ class SiteInteractor:
url = _ORDER_DETAIL_URL_TEMPLATE.format(order_number=site_order_id, shop_id=shop_id)
async def read() -> OrderDetailSnapshot:
page = await self._context.new_page()
page = await self._new_page()
try:
try:
await page.goto(url, wait_until="domcontentloaded", timeout=30_000)
@@ -2077,6 +2312,7 @@ class SiteInteractor:
f"check_order_status site_order_id={site_order_id}", read
)
@traced("site.list_recent_orders", kind=SpanKind.CLIENT)
async def list_recent_orders(
self, *, since: datetime, max_pages: int | None = None
) -> OrderListWindow:
@@ -2112,7 +2348,7 @@ class SiteInteractor:
acc = _OrderListAccumulator()
stop = False
page = await self._context.new_page()
page = await self._new_page()
try:
for page_num in range(1, page_limit + 1):
url = _ORDER_LIST_URL if page_num == 1 else f"{_ORDER_LIST_URL}?page={page_num}"
@@ -2151,6 +2387,63 @@ class SiteInteractor:
# ---- 模块级辅助函数(纯函数,便于单测)----
def _normalize_intent_items(intent: dict[str, Any]) -> list[dict[str, Any]]:
"""把新旧下单意图统一成加购参数列表。
新格式是 ``{"items": [{"item_url": ..., "quantity": ...}, ...]}``
旧格式的 ``item_url/quantity/variant_id/choice`` 仍直接支持字符串元素
也接受方便只传多个 URL 的调用方
"""
raw_items = intent.get("items")
legacy_single = raw_items is None
if raw_items is None:
if not intent.get("item_url"):
# 保持已发布的单商品错误契约不变。
raise InvalidRequestError("intent.item_url 必填")
raw_items = [intent]
if not isinstance(raw_items, list) or not raw_items:
raise InvalidRequestError("intent.items 必须是非空数组")
normalized: list[dict[str, Any]] = []
for index, raw in enumerate(raw_items):
if isinstance(raw, str):
raw = {"item_url": raw}
if not isinstance(raw, dict):
raise InvalidRequestError(f"intent.items[{index}] 必须是对象")
item_url = raw.get("item_url")
if not item_url:
raise InvalidRequestError(f"intent.items[{index}].item_url 必填")
try:
quantity = int(raw.get("quantity") or 1)
except (TypeError, ValueError) as exc:
field_name = (
"intent.quantity"
if legacy_single
else f"intent.items[{index}].quantity"
)
raise InvalidRequestError(
f"{field_name} 必须为正整数"
) from exc
if quantity <= 0:
field_name = (
"intent.quantity"
if legacy_single
else f"intent.items[{index}].quantity"
)
raise InvalidRequestError(
f"{field_name} 必须为正整数,收到 {quantity}"
)
normalized.append(
{
"item_url": str(item_url),
"quantity": quantity,
"variant_id": raw.get("variant_id"),
"choice": raw.get("choice"),
}
)
return normalized
def _parse_initial_state(html: str) -> dict | None:
"""从商品页 HTML 抽 window.__INITIAL_STATE__ 并解析为 dict"""
m = re.search(
@@ -2213,22 +2506,18 @@ def _extract_purchase_fields(state: dict, *, intent_override: dict | None) -> di
elif inventory_flag == INVENTORY_FLAG_DEFAULT and item.get("variantId"):
form_fields["variant_id"] = str(item.get("variantId"))
# 必填选项:调用方覆盖 > 自动填第一个候选值
options = information.get("options") or []
required_options = [o for o in options if o.get("isRequired")]
has_required = bool(required_options)
# 必填选项:调用方覆盖 > 自动填第一个**非占位**候选值
# 解析与占位项判定走 app.shared.purchase_contract(与 scraping 的
# /api/item_detail 同源),旧实现直接取 values[0],而必填 select 的 values[0]
# 恰恰是「選択してください」占位项,等于把「请选择」当答案提交上去。
options = parse_options(information)
has_required = any(option.is_required for option in options)
auto_choice, unfillable_required = auto_choice_for(options)
if intent_override.get("choice"):
# 调用方给的可能是 list 或 str
c = intent_override["choice"]
form_fields["choice"] = ",".join(c) if isinstance(c, list) else str(c)
elif has_required:
pairs: list[str] = []
for opt in required_options:
values = opt.get("values") or []
if values:
pairs.append(f"{opt.get('name')}:{values[0].get('name')}")
if pairs:
form_fields["choice"] = ",".join(pairs)
# 调用方给的可能是 list 或 str,两种都交给共用的格式化
form_fields["choice"] = format_choice(intent_override["choice"])
elif auto_choice:
form_fields["choice"] = auto_choice
return {
"basket_domain": basket_domain,
@@ -2238,6 +2527,9 @@ def _extract_purchase_fields(state: dict, *, intent_override: dict | None) -> di
"options_field": "choice" if options else "",
"options": options,
"has_required_options": has_required,
# 必填但自动填不了的选项名(自由文本项,或候选值只有占位项)。调用方没给
# choice 时这就是「非人工介入不可能成功」的直接依据,见 _add_to_cart_with_fields
"unfillable_required_options": unfillable_required,
"inventory_flag": inventory_flag,
"purchase_condition": sell_type.get("purchaseCondition"),
"min_price": sell_type.get("minPrice"),
+38 -13
View File
@@ -15,7 +15,7 @@ from __future__ import annotations
from dataclasses import dataclass
from enum import StrEnum
from typing import TYPE_CHECKING
from typing import TYPE_CHECKING, Any
from urllib.parse import urlsplit
from app.shared.errors import AppError
@@ -55,24 +55,44 @@ def _normalize_item_url(url: str | None) -> str | None:
return f"{parts.scheme}://{parts.netloc}{parts.path.rstrip('/')}"
def _intent_item_urls(intent: dict[str, Any]) -> list[str]:
"""读取新旧意图中的商品 URL,供恢复核对使用。"""
raw_items = intent.get("items")
if raw_items is None:
raw_items = [intent]
if not isinstance(raw_items, list):
return []
urls: list[str] = []
for item in raw_items:
if isinstance(item, str):
url = item
elif isinstance(item, dict):
url = item.get("item_url")
else:
url = None
normalized = _normalize_item_url(url)
if normalized:
urls.append(normalized)
return urls
async def verify_on_site(
task: LeaseTask, *, gateway: "GatewayClient", site: "SiteInteractor"
) -> VerifyResult:
"""核对一笔任务是否已在站点上下过单
核对链路intent.item_url 查任务创建时间GET /api/orders/{task_id}
LeaseTask 本身不带 created_at 创建时间之后的订单列表 按商品 URL
比对任何一环拿不到足够信息都返回 UNKNOWN不猜尤其是 NOT_ORDERED
核对链路intent.items或兼容的 intent.item_url 查任务创建时间
GET /api/orders/{task_id}LeaseTask 本身不带 created_at 创建时间之后
的订单列表 按商品 URL 比对任何一环拿不到足够信息都返回 UNKNOWN不猜尤其是 NOT_ORDERED
只有在确认翻完了窗口内的全部订单后才允许返回否则没找到可能只是没翻
到那一页
"""
intent = task.intent or {}
item_url = intent.get("item_url")
if not item_url:
targets = set(_intent_item_urls(intent))
if not targets:
return VerifyResult(
VerifyVerdict.UNKNOWN, detail="intent 缺 item_url,无法比对商品"
VerifyVerdict.UNKNOWN, detail="intent 缺商品 URL(item_url/items),无法比对商品"
)
target = _normalize_item_url(item_url)
try:
task_detail = await gateway.get_task(task.task_id)
@@ -96,11 +116,16 @@ async def verify_on_site(
detail=f"订单列表查询失败:{type(exc).__name__}: {exc}",
)
matches = [
entry
for entry in window.entries
if any(_normalize_item_url(it.item_url) == target for it in entry.items)
]
matches = []
for entry in window.entries:
entry_urls = {
normalized
for normalized in (_normalize_item_url(it.item_url) for it in entry.items)
if normalized
}
# 多商品任务必须在同一笔订单中全部命中,避免部分匹配误判为已下单。
if targets.issubset(entry_urls):
matches.append(entry)
if len(matches) == 1:
return VerifyResult(
VerifyVerdict.ALREADY_ORDERED,
+10 -5
View File
@@ -112,16 +112,22 @@ CREATE TABLE workers (
"task_id": "po-20260727-0001", // 可选,上游自带的幂等键;不传则服务端生成
"site": "rakuten",
"intent": { // gateway 原样透传,结构由 trading 侧定义
"item_url": "https://item.rakuten.co.jp/shop/code/",
"quantity": 1,
"variant_id": "...", // 多规格商品必填
"options": {},
"items": [{ // 新格式:一次购买多个商品
"item_url": "https://item.rakuten.co.jp/shop/code/",
"quantity": 1,
"variant_id": "...", // 多规格商品必填,取自 /api/item_detail 的 sku.variants[]
"choice": ["名入れ:希望する"] // 店铺自定义必填选项,格式「选项名:取值名」
}],
"max_total_yen": 30000 // 可选,覆盖本次的金额上限
},
"callback_url": "https://upstream.example.com/hooks/rakuten-order" // 可选,终结类事件通知,见 §4.8
}
```
`items` 必须是非空数组,worker 会按顺序将每项加入同一购物车后再进入结算。
为兼容已发布客户端,也可继续使用旧格式:`intent.item_url` 加同级的
`quantity` / `variant_id` / `choice`,其语义等同于只有一个元素的 `items`
响应 `data``{"task_id": "...", "status": "queued", "created": true}`
**幂等**:同一个 `task_id` 重复提交不新建任务,返回既有任务且 `created=false`
@@ -614,4 +620,3 @@ collector 本身不回写「已经存在于下单任务表 `tasks` 的订单」
- [x] 手动 trigger 立即派一轮 list,返回的单在查询队列里可见
- [x] `GET /api/account/orders` 列编目;单笔不存在返回 6005
- [x] collector 只读规范化字段,不解析 raw/raw_pages 站点原始 JSON
+245
View File
@@ -0,0 +1,245 @@
"""必填选项 choice 取值的真账号验证:占位项修复是否真的被站点接受
背景2026-08-28 发现 trading 自动填 choice 时取 `values[0]`而必填 select
values[0] 恒为 id=0 選択してください占位项等于把请选择当答案提交
修复后取第一个非占位候选 app/shared/purchase_contract.py::auto_choice_for
离线测试只能证明我们填的值变了证明不了站点接受这个值后者必须真账号
实测本探针就为这一件事
**刻意走生产代码路径**SiteInteractor.add_to_cart_payload不重写字段构造逻辑
上一版探针 scripts/probe_purchase_block_v2.py 自己抄了一遍 form 构造结果那份
抄写与生产代码一起用了 values[0]两边同错就测不出问题实际提交的 payload
生产代码自己的 INFO 日志里抓加购请求basket=... payload=...确保记录的
就是真正发出去的东西
安全边界
- 只做清空购物车 加购 校验 清空购物车**绝不进入结算/支付**
- 加购完立即清空账号购物车恢复原状加购本身可逆不产生订单不扣款
- 不触碰 enter_checkout / submit_order / pay
用法
.venv/Scripts/python.exe scripts/probe_option_choice.py
"""
from __future__ import annotations
import asyncio
import json
import logging
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from app.shared.config import get_settings # noqa: E402
from app.shared.errors import AppError # noqa: E402
from app.shared.purchase_contract import auto_choice_for, parse_options # noqa: E402
from app.trading.services.auth_session import AuthSession # noqa: E402
from app.trading.worker.site_interact import ( # noqa: E402
SiteInteractor,
_parse_initial_state,
)
PROBE_DIR = Path(__file__).resolve().parent.parent / ".probe" / "options_choice"
PROBE_DIR.mkdir(parents=True, exist_ok=True)
# 目标商品:从 .probe/checkout/ 的历史样本里挑「有必填选项且占位项在 values[0]」的两个。
# 这两个正是旧实现填「選択してください」也被站点收下的商品——它们不能证明旧值是对的
# (店铺没做校验),但可以验证新值同样被接受,且是站点自己给的唯一非占位候选。
TARGETS = (
"https://item.rakuten.co.jp/waabbit/088-4p",
"https://item.rakuten.co.jp/aoimorinomise/121",
)
class _PayloadCapture(logging.Handler):
"""截获生产代码 `加购请求:basket=%s payload=%s` 这条 INFO 日志
目的是记录**真正发出去的 payload**而不是探针自己再算一遍
"""
def __init__(self) -> None:
super().__init__(level=logging.INFO)
self.payloads: list[dict] = []
def emit(self, record: logging.LogRecord) -> None:
if record.msg == "加购请求:basket=%s payload=%s":
basket, payload = record.args
self.payloads.append({"basket_domain": basket, "payload": dict(payload)})
def save(name: str, content: str) -> Path:
path = PROBE_DIR / name
path.write_text(content, encoding="utf-8")
print(f" saved -> {path.name} ({len(content)} bytes)")
return path
async def inspect_options(site: SiteInteractor, item_url: str) -> list[dict]:
"""只读打开商品页,把解析出的选项结构落盘,供人工核对占位项判定是否符合真实页面"""
page = await site._context.new_page()
try:
await page.goto(item_url, wait_until="domcontentloaded", timeout=30_000)
await page.wait_for_function(
"() => window.__INITIAL_STATE__ && window.__INITIAL_STATE__.purchase",
timeout=10_000,
)
html = await page.content()
finally:
await page.close()
state = _parse_initial_state(html) or {}
information = (state.get("purchase") or {}).get("information") or {}
options = parse_options(information)
auto_choice, unfillable = auto_choice_for(options)
slug = item_url.rstrip("/").replace("https://item.rakuten.co.jp/", "").replace("/", "_")
save(
f"options-{slug}.json",
json.dumps(
{
"item_url": item_url,
"raw_options": information.get("options"),
"parsed": [
{
"name": option.name,
"type": option.type,
"is_required": option.is_required,
"values": [
{
"value_id": value.value_id,
"name": value.name,
"is_placeholder": value.is_placeholder,
}
for value in option.values
],
}
for option in options
],
"auto_choice": auto_choice,
"unfillable_required_options": unfillable,
},
ensure_ascii=False,
indent=1,
),
)
for option in options:
if not option.is_required:
continue
names = [
f"{value.name}{'(占位)' if value.is_placeholder else ''}" for value in option.values
]
print(f" 必填项「{option.name[:30]}」候选={names}")
print(f" auto_choice = {auto_choice!r}")
if unfillable:
print(f" !! 自动填不了的必填项:{unfillable}")
return [{"name": o.name, "is_required": o.is_required} for o in options]
async def run() -> int:
settings = get_settings()
capture = _PayloadCapture()
logging.basicConfig(level=logging.WARNING)
# 必须显式把这个 logger 抬到 INFO:basicConfig 把 root 设成 WARNING,而目标
# logger 自身是 NOTSET,有效级别继承 root——INFO 记录会在到达 handler 之前
# 就被丢掉,capture 全程收不到东西(第一次跑就是这样,choice 打印成 None)
target_logger = logging.getLogger("app.trading.worker.site_interact")
target_logger.setLevel(logging.INFO)
target_logger.addHandler(capture)
auth = AuthSession(settings)
await auth.start()
print("=== 0. 登录态探测(storage_state 落盘于 2026-08-11,可能已过期)===")
status = await auth.check("rakuten")
print(f" logged_in={status.logged_in} detail={status.detail}")
if not status.logged_in:
print(" 登录态已失效——需要先跑 scripts/login.py 重新登录(可能要人工过验证码)")
await auth.close()
return 2
site = SiteInteractor(auth_session=auth, settings=settings)
await site.start()
results: list[dict] = []
try:
print("\n=== 1. 清空购物车(排除历史残留污染校验)===")
cleared = await site.clear_cart()
print(f" removed={cleared['removed_count']} cart_count={cleared['cart_count']}")
for index, item_url in enumerate(TARGETS, start=1):
print(f"\n=== 2.{index} {item_url} ===")
print(" -- 只读核对选项结构 --")
try:
await inspect_options(site, item_url)
except Exception as exc: # noqa: BLE001
print(f" 选项结构读取失败(不影响加购验证):{type(exc).__name__}: {exc}")
print(" -- 走生产路径加购(add_to_cart_payload)--")
before = len(capture.payloads)
entry: dict = {"item_url": item_url}
try:
added = await site.add_to_cart_payload(item_url=item_url, quantity=1)
entry["ok"] = True
entry["item_id"] = added["item_id"]
entry["cart_count"] = added["cart_count"]
print(
f" 加购成功 item_id={added['item_id']} cart_count={added['cart_count']}"
)
save(f"add-response-{index}.html", added.get("response_html") or "")
except AppError as exc:
entry["ok"] = False
entry["error"] = exc.message
print(f" 加购失败:{exc.message}")
except Exception as exc: # noqa: BLE001
entry["ok"] = False
entry["error"] = f"{type(exc).__name__}: {exc}"
print(f" 加购异常:{type(exc).__name__}: {exc}")
# 真正发出去的 payload(从生产代码日志截获)
sent = capture.payloads[before:]
if not sent:
# 抓不到就等于这次验证什么也没证明——必须显眼报出来,不能让
# 「加购成功」把它盖过去(第一次跑就是 logger 级别没抬导致静默为 None)
entry["capture_failed"] = True
print(" !! 未截获到 payload 日志,本次无法确认实际提交的 choice")
else:
entry["sent_payload"] = sent[-1]["payload"]
choice = sent[-1]["payload"].get("choice")
print(f" 实际提交 choice = {choice!r}")
if choice is None:
print(" !! payload 里没有 choice 字段")
elif "選択してください" in choice:
print(" !! 提交值里仍含占位项——修复未生效")
results.append(entry)
print("\n=== 3. 购物车状态(确认商品真的进车了)===")
try:
status_after = await site.cart_status()
print(f" {status_after}")
except AppError as exc:
print(f" 查询失败:{exc.message}")
print("\n=== 4. 清空购物车(恢复账号原状,不进入结算)===")
final = await site.clear_cart()
print(f" removed={final['removed_count']} cart_count={final['cart_count']}")
finally:
await site.close()
await auth.close()
save("summary.json", json.dumps(results, ensure_ascii=False, indent=1))
print("\n========== 汇总 ==========")
for entry in results:
mark = "OK " if entry.get("ok") else "FAIL"
choice = (entry.get("sent_payload") or {}).get("choice")
print(f" {mark} {entry['item_url']}")
print(f" choice={choice!r}")
if not entry.get("ok"):
print(f" error={entry.get('error')}")
print(f"\n探针输出目录:{PROBE_DIR}")
return 0
if __name__ == "__main__":
raise SystemExit(asyncio.run(run()))
+154
View File
@@ -0,0 +1,154 @@
"""对外失败响应的埋点测试:异常处理器是失败的唯一出口
`register_exception_handlers` 里的处理器把异常吃掉换成 `ApiResponse` 信封返回
异常不再向上冒**自动 instrumentation 之后只看得到一个 HTTP 状态码**
`AppError` 默认 `status_code=400`信封里 `success=false`在链路上跟正常返回几乎
分不出来兜底处理器更是只回一句无信息量的 "Internal server error"真正的异常
类型与栈只在本进程日志里所以处理器必须把这次失败记到当前 server span
这里挂真实的 FastAPIInstrumentor 中间件而不是手工造 span断言的就是一次
真实请求打进来失败返回之后server span 上有什么
"""
from __future__ import annotations
import pytest
from fastapi import APIRouter, FastAPI
from fastapi.testclient import TestClient
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
from opentelemetry.trace import StatusCode
from pydantic import BaseModel
from app.shared.api import ApiResponse, register_exception_handlers
from app.shared.errors import ItemNotFoundError, UpstreamBlockedError
class _Body(BaseModel):
keyword: str
@pytest.fixture
def app_and_spans():
"""挂了异常处理器 + 真实 OTel ASGI 中间件的最小应用
provider 显式传给 instrument_app避免碰全局 provider只允许设置一次
测试间共享会互相污染
"""
exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))
router = APIRouter()
@router.get("/api/blocked")
async def blocked() -> ApiResponse[None]:
raise UpstreamBlockedError("Blocked while fetching: challenge page detected")
@router.get("/api/missing")
async def missing() -> ApiResponse[None]:
raise ItemNotFoundError("Page not found: https://item.rakuten.co.jp/x/y/")
@router.get("/api/boom")
async def boom() -> ApiResponse[None]:
raise RuntimeError("unexpected explosion")
@router.post("/api/validated")
async def validated(_body: _Body) -> ApiResponse[None]:
return ApiResponse[None](success=True, msg="success", data=None, code=0)
app = FastAPI()
app.include_router(router)
register_exception_handlers(app)
FastAPIInstrumentor.instrument_app(app, tracer_provider=provider)
try:
yield app, exporter
finally:
FastAPIInstrumentor.uninstrument_app(app)
def _server_span(exporter: InMemorySpanExporter):
spans = exporter.get_finished_spans()
assert spans, "没有产生 server span"
return spans[-1]
def test_app_error_is_recorded_on_server_span(app_and_spans):
"""业务异常:错误码、可重试、异常类型都要落在 span 上
AppError 默认 status_code=400链路里光看状态码只知道客户端错了
不知道是被反爬阻断3002可重试还是别的什么
"""
app, exporter = app_and_spans
with TestClient(app) as http:
response = http.get("/api/blocked")
assert response.status_code == 400
assert response.json()["code"] == 3002
span = _server_span(exporter)
assert span.status.status_code is StatusCode.ERROR
assert span.attributes["error.type"] == "UpstreamBlockedError"
assert span.attributes["error.code"] == 3002
# 上游据此判断该不该重试,是失败分类里最要紧的一位
assert span.attributes["error.retryable"] is True
assert span.attributes["api.success"] is False
assert span.attributes["api.code"] == 3002
assert "challenge page" in span.attributes["api.msg"]
# 异常栈作为 event 保留,需要细看时能展开
assert any(event.name == "exception" for event in span.events)
def test_not_found_records_its_own_code(app_and_spans):
"""404 类业务异常与阻断类要能按码区分(4004 不可重试)"""
app, exporter = app_and_spans
with TestClient(app) as http:
assert http.get("/api/missing").status_code == 404
span = _server_span(exporter)
assert span.attributes["error.code"] == 4004
assert span.attributes["error.retryable"] is False
def test_unhandled_exception_records_real_cause(app_and_spans):
"""兜底分支最需要埋点:对外只回 "Internal server error",真因只在日志里"""
app, exporter = app_and_spans
with TestClient(app, raise_server_exceptions=False) as http:
response = http.get("/api/boom")
assert response.status_code == 500
assert response.json()["msg"] == "Internal server error"
span = _server_span(exporter)
assert span.status.status_code is StatusCode.ERROR
# 响应体里查不到的真因,链路上能直接看到
assert span.attributes["error.type"] == "RuntimeError"
assert "unexpected explosion" in span.attributes["error.message"]
assert span.attributes["api.code"] == 1500
def test_validation_error_records_field_message(app_and_spans):
"""参数校验失败记错误码与字段消息,但不记 pydantic 那条没有诊断价值的长栈"""
app, exporter = app_and_spans
with TestClient(app) as http:
response = http.post("/api/validated", json={})
assert response.status_code == 422
assert response.json()["code"] == 1002
span = _server_span(exporter)
assert span.status.status_code is StatusCode.ERROR
assert span.attributes["api.code"] == 1002
# 排查要看的是「哪个字段不合法」
assert "keyword" in span.attributes["api.msg"]
assert "error.type" not in span.attributes
def test_success_leaves_server_span_ok(app_and_spans):
"""成功请求不被误标 ERROR——作为对照说明失败断言不是恒真"""
app, exporter = app_and_spans
with TestClient(app) as http:
assert http.post("/api/validated", json={"keyword": "switch"}).status_code == 200
span = _server_span(exporter)
assert span.status.status_code is not StatusCode.ERROR
assert "api.success" not in span.attributes
assert "error.type" not in span.attributes
+13
View File
@@ -103,3 +103,16 @@ def test_all_refs_resolve(spec):
names = set(spec["components"]["schemas"])
refs = set(re.findall(r"#/components/schemas/([^\"]+)", json.dumps(spec)))
assert not refs - names
def test_submit_order_intent_schema_documents_multi_item_and_legacy_fields(spec):
"""intent 保持透传对象,同时在 OpenAPI 中明确展示新旧两种商品格式。"""
intent = spec["components"]["schemas"]["SubmitOrderRequest"]["properties"]["intent"]
properties = intent["properties"]
assert intent["additionalProperties"] is True
assert properties["items"]["type"] == "array"
assert properties["items"]["minItems"] == 1
item_object = properties["items"]["items"]["oneOf"][0]
assert item_object["required"] == ["item_url"]
assert "item_url" in properties
assert "quantity" in properties
+73
View File
@@ -142,6 +142,79 @@ def test_item_detail_falls_back_to_request_shop_code_when_state_lacks_it(item_st
assert detail.shop.shop_code == "fallback"
# ---- 店铺自定义选项(下单必填项)----
def test_item_detail_exposes_shop_options_for_ordering(item_with_options_state):
"""下单要用的选项必须能从接口拿到:名字、类型、是否必填、候选取值"""
detail = parse_item_detail(
item_with_options_state, item_url="https://x", shop_code="kizamu",
include_sku_variants=True,
)
assert detail.has_required_options is True
# 真实样本:1 个必填 select(名入れ)+ 1 个非必填 select + 2 个自由文本
assert [option.name for option in detail.options] == [
"名入れ",
" 【お名前】※漢字・かな文字:4文字まで",
"【定型】",
"【定型】※〇寿・〇婚式・〇長を選択の場合のみ(1文字まで)",
]
required = detail.options[0]
assert required.is_required is True
assert required.type == "select"
assert required.option_id == 100
def test_item_detail_marks_placeholder_option_values_as_unselectable(item_with_options_state):
"""「選択してください」是占位项,不能作为下单取值——这是 choice 填错的根因"""
detail = parse_item_detail(
item_with_options_state, item_url="https://x", shop_code="kizamu",
include_sku_variants=True,
)
required = detail.options[0]
placeholder, first_real = required.values[0], required.values[1]
assert placeholder.name == "選択してください"
assert placeholder.is_placeholder is True
assert first_real.name == "希望する【次の項目で入力】"
assert first_real.is_placeholder is False
# 3 个候选里只有 2 个真正可提交
assert len(required.values) == 3
assert required.selectable_value_count == 2
def test_item_detail_free_text_option_has_no_values(item_with_options_state):
"""type=text 是自由文本,站点不给候选值,无法自动选"""
detail = parse_item_detail(
item_with_options_state, item_url="https://x", shop_code="kizamu",
include_sku_variants=True,
)
free_text = detail.options[1]
assert free_text.type == "text"
assert free_text.values == []
assert free_text.selectable_value_count == 0
def test_item_detail_reports_required_options_that_cannot_be_auto_filled(item_with_options_state):
"""必填自由文本项无法自动选值,必须点名交给调用方,不能静默蒙一个值"""
options = item_with_options_state["purchase"]["information"]["options"]
# 把那个自由文本项改成必填:店铺要求填「お名前」时就是这种形态
options[1]["isRequired"] = True
detail = parse_item_detail(
item_with_options_state, item_url="https://x", shop_code="kizamu",
include_sku_variants=True,
)
assert detail.unfillable_required_options == [" 【お名前】※漢字・かな文字:4文字まで"]
def test_item_detail_without_options_reports_empty(item_state):
"""没有选项的商品(真实样本 purchase.information 只有 unit)不应凭空造出选项"""
detail = parse_item_detail(
item_state, item_url="https://x", shop_code="s", include_sku_variants=False,
)
assert detail.options == []
assert detail.has_required_options is False
assert detail.unfillable_required_options == []
def test_item_detail_rejects_state_without_item_node():
with pytest.raises(ScrapeParseError):
parse_item_detail({}, item_url="https://x", shop_code="s", include_sku_variants=True)
+296
View File
@@ -0,0 +1,296 @@
"""抓取客户端失败分支的埋点测试:失败原因必须能从 span 上读出来
这里钉住的核心是一个真实存在过的 bug失败分支里写的是无参
`span.record_exception()`而该方法的 `exception` 是必填位置参数**每一次抓取
失败都会在记录异常时抛 TypeError**把真正的失败原因反爬阻断 / 解析失败 /
404整个替换掉链路上只剩调用发生过异常怎么来的完全看不到失败页面
快照也永远落不下来抛错发生在 snapshot 之前
用独立的 InMemory provider 断言 span不碰全局 providerOTel 的全局 provider
只允许设置一次测试间共享会互相污染 test_telemetry.py 同一套思路
"""
from __future__ import annotations
import json
import httpx
import pytest
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
from opentelemetry.trace import StatusCode
from app.scraping.models.scrape import (
ItemDetailRequest,
RakumaSearchRequest,
SearchRequest,
ShopItemsRequest,
)
from app.scraping.services import rakuma_client as rakuma_module
from app.scraping.services import rakuten_client as rakuten_module
from app.scraping.services import site_session as session_module
from app.scraping.services.rakuma_client import RakumaClient
from app.scraping.services.rakuten_client import RakutenClient
from app.scraping.services.site_session import SiteSession
from app.shared.config import Settings
from app.shared.errors import AppError, ScrapeParseError, UpstreamBlockedError
@pytest.fixture
def spans(monkeypatch) -> InMemorySpanExporter:
"""把两个抓取客户端模块级 tracer 换成写内存的,用于断言 span"""
exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))
tracer = provider.get_tracer("test")
monkeypatch.setattr(rakuten_module, "tracer", tracer)
monkeypatch.setattr(rakuma_module, "tracer", tracer)
return exporter
def _settings() -> Settings:
return Settings(_env_file=None, otel_snapshot_max_bytes=2_000_000)
class _StubSession:
"""按需返回 HTML 或抛异常的会话桩;两站的 fetch_html 签名差异在这里吸收"""
def __init__(self, *, html: str = "", error: Exception | None = None):
self._html = html
self._error = error
async def fetch_html(self, url: str, *, mobile: bool = False) -> str:
if self._error is not None:
raise self._error
return self._html
async def fetch(self, url: str, *, mobile: bool = False, validator=None):
if self._error is not None:
raise self._error
raise AssertionError("本测试不该走到 fetch 的成功分支")
def _span(exporter: InMemorySpanExporter, name: str):
return next(s for s in exporter.get_finished_spans() if s.name == name)
async def test_parse_failure_preserves_original_exception(spans):
"""回归:解析失败要原样抛出业务异常,不能被埋点自身的 TypeError 顶替
这是原 bug 最直接的后果上游拿到的不再是 4001解析失败而是一个
TypeError 兜底成的 500错误码表整个失效
"""
client = RakutenClient(_settings(), _StubSession(html="<html>no state</html>"))
with pytest.raises(ScrapeParseError) as excinfo:
await client.search(SearchRequest(keyword="switch"))
# 是业务异常而不是埋点炸出来的 TypeError
assert isinstance(excinfo.value, AppError)
assert excinfo.value.err_code == 4001
async def test_parse_failure_records_reason_and_page_snapshot(spans):
"""页面取回来了但解析不出:stage=parse,且失败页面 HTML 落成 event
站点改版了只能靠当时那份 HTML 判断所以快照必须真的落下来 bug
记异常那步先抛了snapshot 这行永远执行不到
"""
html = "<html>changed layout</html>"
client = RakutenClient(_settings(), _StubSession(html=html))
with pytest.raises(ScrapeParseError):
await client.search(SearchRequest(keyword="switch"))
span = _span(spans, "parse.rakuten.search")
assert span.status.status_code is StatusCode.ERROR
assert span.attributes["parse.stage"] == "parse"
# 真实异常类名,不是一律写死的 "parse_error"
assert span.attributes["parse.fail_reason"] == "ScrapeParseError"
assert span.attributes["error.code"] == 4001
assert span.attributes["error.retryable"] is False
snapshot = next(e for e in span.events if e.name == "parse.failed_html")
assert snapshot.attributes["snapshot.html"] == html
# 快照要能对上是哪个地址的页面
assert "search.rakuten.co.jp" in snapshot.attributes["parse.url"]
async def test_fetch_failure_marks_fetch_stage_without_snapshot(spans):
"""页面根本没取回来:stage=fetch,没有快照可落
parse 阶段的排查方向相反通道/反爬/上游 5xx而不是站点改版
所以要能直接按 parse.stage 分流而不是对着有没有 HTML
"""
blocked = UpstreamBlockedError("Blocked while fetching: challenge page detected")
client = RakutenClient(_settings(), _StubSession(error=blocked))
with pytest.raises(UpstreamBlockedError):
await client.search(SearchRequest(keyword="switch"))
span = _span(spans, "parse.rakuten.search")
assert span.attributes["parse.stage"] == "fetch"
assert span.attributes["parse.fail_reason"] == "UpstreamBlockedError"
# 反爬阻断是可重试的,这一位直接决定上游要不要重来
assert span.attributes["error.retryable"] is True
assert not [e for e in span.events if e.name == "parse.failed_html"]
async def test_item_detail_failure_records_reason(spans):
"""详情接口走的是 fetch()(带校验器)而非 fetch_html,失败分支同样要记全"""
blocked = UpstreamBlockedError("challenge page detected")
client = RakutenClient(_settings(), _StubSession(error=blocked))
with pytest.raises(UpstreamBlockedError):
await client.item_detail(
ItemDetailRequest(shop_code="someshop", item_code="10000001")
)
span = _span(spans, "parse.rakuten.item_detail")
assert span.status.status_code is StatusCode.ERROR
assert span.attributes["parse.fail_reason"] == "UpstreamBlockedError"
async def test_shop_items_marks_delegate_stage(spans):
"""shop_items 自己不抓页面:失败在转调的 shop_detail / search 里
html 推断会得出fetch 失败的错误结论它手里从来没有 html
所以这里显式标 delegate且不落快照真正的现场在被转调那个 span
"""
blocked = UpstreamBlockedError("challenge page detected")
client = RakutenClient(_settings(), _StubSession(error=blocked))
with pytest.raises(UpstreamBlockedError):
await client.shop_items(ShopItemsRequest(shop_code="someshop"))
span = _span(spans, "parse.rakuten.shop_items")
assert span.attributes["parse.stage"] == "delegate"
assert not [e for e in span.events if e.name == "parse.failed_html"]
# 被转调的那一步才是现场所在,它自己落了快照
inner = _span(spans, "parse.rakuten.shop_detail")
assert inner.attributes["parse.fail_reason"] == "UpstreamBlockedError"
async def test_rakuma_parse_failure_records_reason_and_snapshot(spans):
"""ラクマ 侧五个接口是同一套失败分支,同样要能读出原因与现场"""
html = "<html>rakuma changed</html>"
client = RakumaClient(_settings(), _StubSession(html=html))
with pytest.raises(AppError):
await client.search(RakumaSearchRequest(keyword="switch"))
span = _span(spans, "parse.rakuma.search")
assert span.status.status_code is StatusCode.ERROR
assert span.attributes["parse.stage"] == "parse"
assert "error.code" in span.attributes
snapshot = next(e for e in span.events if e.name == "parse.failed_html")
assert snapshot.attributes["snapshot.html"] == html
# ---- 会话层:逐次尝试与升级路径 ----
#
# 这一层的关键信息是「升级路径」:换 cookie → 浏览器兜底 → 放弃。属性表达不了过程
# (同名后写覆盖先写,三次尝试跑完只剩最后一次),所以每次尝试与每次升级各记一条
# event。site_session / rakuma_session 都在 fetch 内部 `trace.get_tracer(__name__)`,
# 所以这里替 `trace.get_tracer` 本身(与 test_telemetry.py 的 spans 夹具同一手法)。
@pytest.fixture
def session_spans(monkeypatch) -> InMemorySpanExporter:
exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))
monkeypatch.setattr(session_module.trace, "get_tracer", provider.get_tracer)
return exporter
class _FakeBrowser:
"""浏览器兜底替身:visit 恒不可用,让升级链走完整条路"""
def __init__(self) -> None:
self.unavailable_reason = "playwright is not installed"
async def visit(self, url: str, *, mobile: bool):
return None
async def close(self) -> None:
pass
async def _blocked_session() -> SiteSession:
"""所有请求都回 Akamai 挑战页的会话,用 MockTransport 拦截"""
settings = Settings(
_env_file=None, http_max_attempts=3, request_timeout_seconds=5.0,
max_site_concurrency=4,
)
session = SiteSession(settings, _FakeBrowser())
await session.start()
for profile in session._profiles.values():
await profile.client.aclose()
profile.client = httpx.AsyncClient(
transport=httpx.MockTransport(
lambda _r: httpx.Response(
200, text="<html>Access Denied. Reference #18.abc</html>"
)
),
follow_redirects=True,
)
return session
async def test_retry_attempts_and_escalation_are_recorded_as_events(session_spans):
"""每次尝试与每次升级各留一条 event,链路上能读出完整升级路径
这些信息无法用属性表达`scrape.attempts` 只剩最后一次的值前两次为什么失败
换过 cookie 没有浏览器兜底试过没有全被覆盖掉
"""
session = await _blocked_session()
try:
with pytest.raises(UpstreamBlockedError):
await session.fetch_html(
"https://search.rakuten.co.jp/search/mall/x/", mobile=False
)
finally:
await session.close()
span = _span(session_spans, "scrape.fetch")
assert span.status.status_code is StatusCode.ERROR
assert span.attributes["scrape.fail_reason"] == "UpstreamBlockedError"
assert span.attributes["scrape.challenge_detected"] is True
# 三次尝试都留下了自己的记录,而不是只剩最后一次
attempts = [e for e in span.events if e.name == "scrape.attempt"]
assert [e.attributes["attempt"] for e in attempts] == [1, 2, 3]
assert {e.attributes["outcome"] for e in attempts} == {"challenge"}
# 升级路径:第 1 次失败后换 cookie,第 2 次失败后动用浏览器(本例不可用)
escalations = [e for e in span.events if e.name == "scrape.escalate"]
assert [e.attributes["to"] for e in escalations] == ["rewarm_on_home", "browser"]
browser_step = escalations[-1]
assert browser_step.attributes["outcome"] == "failed"
# 「没装 playwright」与「装了也被挡」要能分开查
assert browser_step.attributes["reason"] == "playwright is not installed"
# 最终失败页面的快照带上来源,便于确认是哪条通道的哪个地址
failed = next(e for e in span.events if e.name == "scrape.failed_html")
assert failed.attributes["scrape.profile"] == "pc"
assert "search.rakuten.co.jp" in failed.attributes["scrape.url"]
async def test_successful_scrape_leaves_span_ok(spans, search_state):
"""成功路径不被误标 ERROR,不落失败快照,且照常记结果指标
作为上面那些失败断言的对照证明 ERROR 状态与 parse.fail_reason 是真的由失败
触发的不是每条 span 都长这样
"""
html = f"<script>window.__INITIAL_STATE__ = {json.dumps(search_state)};</script>"
client = RakutenClient(_settings(), _StubSession(html=html))
result = await client.search(SearchRequest(keyword="switch"))
span = _span(spans, "parse.rakuten.search")
assert span.status.status_code is not StatusCode.ERROR
assert "parse.fail_reason" not in span.attributes
assert "error.type" not in span.attributes
assert not [e for e in span.events if e.name == "parse.failed_html"]
# 成功时记的是结果指标,与失败侧属性互不重叠
assert span.attributes["parse.items"] == len(result.items)
+397 -2
View File
@@ -11,6 +11,7 @@ clear_cart 与 _dump_debug_snapshot 用不依赖 Playwright 的 fake page 覆盖
"""
from __future__ import annotations
import asyncio
import json
from pathlib import Path
from typing import Any
@@ -20,6 +21,7 @@ import pytest
from datetime import datetime, timezone
from app.shared.errors import (
BrowserDeadError,
CartOperationError,
InvalidRequestError,
NotLoggedInError,
@@ -38,6 +40,7 @@ from app.trading.worker.site_interact import (
_DELETE_BUTTON_SELECTOR,
_extract_error_message,
_extract_purchase_fields,
_normalize_intent_items,
_OrderListAccumulator,
_parse_checkout_summary,
_parse_initial_state,
@@ -69,6 +72,82 @@ def _wrap_state(state: dict[str, Any]) -> str:
)
# ---- 下单意图兼容 ----
def test_normalize_intent_items_accepts_legacy_single_item():
assert _normalize_intent_items({
"item_url": "https://item.rakuten.co.jp/shop/x/",
"quantity": 2,
}) == [{
"item_url": "https://item.rakuten.co.jp/shop/x/",
"quantity": 2,
"variant_id": None,
"choice": None,
}]
def test_normalize_intent_items_accepts_multiple_items_and_string_urls():
assert _normalize_intent_items({
"items": [
{"item_url": "https://item.rakuten.co.jp/shop/x/", "quantity": 2},
"https://item.rakuten.co.jp/shop/y/",
]
}) == [
{
"item_url": "https://item.rakuten.co.jp/shop/x/",
"quantity": 2,
"variant_id": None,
"choice": None,
},
{
"item_url": "https://item.rakuten.co.jp/shop/y/",
"quantity": 1,
"variant_id": None,
"choice": None,
},
]
def test_normalize_intent_items_rejects_empty_items():
with pytest.raises(InvalidRequestError):
_normalize_intent_items({"items": []})
async def test_add_to_cart_processes_all_items_and_keeps_legacy_state():
site = SiteInteractor.__new__(SiteInteractor)
site._lock = asyncio.Lock()
site._per_task_state = {}
calls: list[dict[str, Any]] = []
async def fake_add(**kwargs):
calls.append(kwargs)
index = len(calls)
return {
"item_id": str(index),
"shop_bid": "shop",
"basket_domain": "https://basket",
"response_html": f"html-{index}",
"screenshot": b"",
}
site._add_to_cart_with_fields = fake_add
snapshot = await site.add_to_cart(_make_task(intent={
"items": [
{"item_url": "https://item.rakuten.co.jp/shop/x/"},
{"item_url": "https://item.rakuten.co.jp/shop/y/", "quantity": 3},
]
}))
assert [call["item_url"] for call in calls] == [
"https://item.rakuten.co.jp/shop/x/",
"https://item.rakuten.co.jp/shop/y/",
]
assert calls[1]["quantity"] == 3
assert site._per_task_state["t1"]["item_id"] == "1"
assert site._per_task_state["t1"]["item_ids"] == ["1", "2"]
assert snapshot.html == "html-2"
# ---- _parse_initial_state ----
@@ -179,6 +258,95 @@ def test_extract_fields_required_options_auto_picks_first_value():
assert fields["form_fields"]["choice"] == "サイズ:S"
def test_extract_fields_required_options_skips_placeholder_value():
"""回归:必填 select 的 values[0] 是「選択してください」占位项,不能当答案提交
真实商品页.probe/checkout/03-add-fail-2.html 3 份样本里必填 select
第一个候选恒为 id=0 的占位项真实取值从 id=200 旧实现直接取 values[0]
等于把請選擇填进 choice 提交上去
"""
state = {
"item": {"itemId": 1, "variantId": "v"},
"purchase": {
"sku": {"inventoryType": "single"},
"sellType": {"normalPurchase": {"basketDomain": "https://x/add", "purchaseCondition": "enabled"}},
"information": {
"options": [
{
"id": 100,
"name": "この商品は「トライタン製/プラスチック」です",
"type": "select",
"isRequired": True,
"values": [
{"id": 0, "name": "選択してください"},
{"id": 200, "name": "確認した"},
],
}
]
},
},
"shop": {"information": {"shopId": 1}},
}
fields = _extract_purchase_fields(state, intent_override={})
assert fields["has_required_options"] is True
assert fields["form_fields"]["choice"] == "この商品は「トライタン製/プラスチック」です:確認した"
assert fields["unfillable_required_options"] == []
def test_extract_fields_required_free_text_option_cannot_be_auto_filled():
"""必填自由文本项(type=text,无候选值)自动填不了,要点名报出来"""
state = {
"item": {"itemId": 1, "variantId": "v"},
"purchase": {
"sku": {"inventoryType": "single"},
"sellType": {"normalPurchase": {"basketDomain": "https://x/add", "purchaseCondition": "enabled"}},
"information": {
"options": [
{"id": 101, "name": "【お名前】", "type": "text", "isRequired": True},
]
},
},
"shop": {"information": {"shopId": 1}},
}
fields = _extract_purchase_fields(state, intent_override={})
assert fields["has_required_options"] is True
# 填不出来就不填,也不拿占位/空值凑数
assert "choice" not in fields["form_fields"]
assert fields["unfillable_required_options"] == ["【お名前】"]
def test_extract_fields_skips_optional_options_when_auto_filling():
"""非必填项不替上游做业务决定(如「置き配を希望する」),只自动填必填项"""
state = {
"item": {"itemId": 1, "variantId": "v"},
"purchase": {
"sku": {"inventoryType": "single"},
"sellType": {"normalPurchase": {"basketDomain": "https://x/add", "purchaseCondition": "enabled"}},
"information": {
"options": [
{
"id": 100,
"name": "必須確認",
"type": "select",
"isRequired": True,
"values": [{"id": 0, "name": "選択してください"}, {"id": 200, "name": "了解"}],
},
{
"id": 101,
"name": "「置き配」希望について",
"type": "select",
"isRequired": False,
"values": [{"id": 200, "name": "置き配を希望しない"}],
},
]
},
},
"shop": {"information": {"shopId": 1}},
}
fields = _extract_purchase_fields(state, intent_override={})
assert fields["form_fields"]["choice"] == "必須確認:了解"
def test_extract_fields_intent_choice_override_accepts_list_and_str():
state = {
"item": {"itemId": 1, "variantId": "v"},
@@ -576,10 +744,24 @@ _ORDER_DETAIL_LANDED_URL = "https://order.my.rakuten.co.jp/purchase-history/?ord
_ORDER_LIST_LANDED_URL = "https://order.my.rakuten.co.jp/purchase-history/order-list"
class _FakeBrowser:
"""Browser 替身:只实现 _ensure_browser_alive 用到的 is_connected()
没有它`_ensure_context_ready` 会判定浏览器没起来并尝试真的 launch 一次
Chromiumconnected 可以翻成 False 来模拟掉线
"""
def __init__(self, connected: bool = True):
self.connected = connected
def is_connected(self) -> bool:
return self.connected
def _build_site(tmp_path: Path, context: _FakeContext, auth: _FakeAuthSession) -> SiteInteractor:
"""装配一个不依赖 Playwright 的 SiteInteractor
auth_state_dir 指向空目录_refresh_context_if_stale 见不到 storage_state
auth_state_dir 指向空目录_ensure_context_ready 见不到 storage_state
文件就直接返回不会试图重建 context重建需要真实 browser
"""
from app.shared.config import Settings
@@ -588,6 +770,7 @@ def _build_site(tmp_path: Path, context: _FakeContext, auth: _FakeAuthSession) -
auth_session=auth, # type: ignore[arg-type]
settings=Settings(auth_state_dir=str(tmp_path / "auth"), evidence_dir=str(tmp_path)),
)
site._browser = _FakeBrowser() # type: ignore[assignment]
site._context = context # type: ignore[assignment]
return site
@@ -980,13 +1163,16 @@ async def test_check_order_status_still_returns_only_status(tmp_path):
async def test_submit_order_without_checkout_page_raises():
"""浏览器活着但没有留存确认页 → OrderOperationError(不是掉线那类错误)"""
site = SiteInteractor(auth_session=None, settings=None) # type: ignore[arg-type]
site._browser = _FakeBrowser() # type: ignore[assignment]
with pytest.raises(OrderOperationError):
await site.submit_order(_make_task())
async def test_pay_without_checkout_page_raises():
site = SiteInteractor(auth_session=None, settings=None) # type: ignore[arg-type]
site._browser = _FakeBrowser() # type: ignore[assignment]
with pytest.raises(OrderOperationError):
await site.pay(_make_task(), "ord-1")
@@ -1314,15 +1500,17 @@ class _FakeLoggedInAuth:
def _build_clear_cart_site(*, delete_buttons: int, count_body: str, html: str) -> SiteInteractor:
"""构造 clear_cart 可离线跑起来的 SiteInteractor(fake page + fake count API)"""
site = SiteInteractor(auth_session=_FakeLoggedInAuth(), settings=None) # type: ignore[arg-type]
site._browser = _FakeBrowser() # type: ignore[assignment]
site._context = _FakeClearCartContext(
page=_FakeClearCartPage(delete_buttons=delete_buttons, html=html),
count_body=count_body,
) # type: ignore[assignment]
# settings=None,走不了真实的 mtime 检查;探活那半段由 _FakeBrowser 覆盖
async def _noop_refresh() -> None:
return None
site._refresh_context_if_stale = _noop_refresh # type: ignore[assignment]
site._ensure_context_ready = _noop_refresh # type: ignore[assignment]
return site
@@ -1390,3 +1578,210 @@ async def test_clear_cart_html_capture_failure_keeps_clear_result():
# ---- helper ----
# ---- 浏览器掉线:任务边界就地重建,中途掉线转 BrowserDeadError ----
#
# 真实的 Chromium 崩溃(容器 /dev/shm 不足、OOM kill、seccomp 挡 sandbox)没法在
# 离线测试里制造,这里用 _FakeBrowser.connected 翻成 False 模拟「is_connected()
# 返回 False」这个唯一的可观测信号,覆盖的是 SiteInteractor 对该信号的反应。
def _build_dead_browser_site(tmp_path: Path) -> SiteInteractor:
"""浏览器已掉线的 SiteInteractor:_browser 在位但 is_connected() 为 False"""
from app.shared.config import Settings
site = SiteInteractor(
auth_session=_FakeLoggedInAuth(), # type: ignore[arg-type]
settings=Settings(auth_state_dir=str(tmp_path / "auth"), evidence_dir=str(tmp_path)),
)
site._browser = _FakeBrowser(connected=False) # type: ignore[assignment]
return site
def test_browser_alive_reflects_is_connected(tmp_path):
"""browser_alive:没起过 / 已掉线 / 正常,三种情况分别为 False/False/True"""
site = SiteInteractor(auth_session=None, settings=None) # type: ignore[arg-type]
assert site.browser_alive is False # 还没 start()
site._browser = _FakeBrowser(connected=False) # type: ignore[assignment]
assert site.browser_alive is False
site._browser = _FakeBrowser(connected=True) # type: ignore[assignment]
assert site.browser_alive is True
def test_browser_alive_survives_is_connected_raising():
"""is_connected() 自己抛错(驱动已经没了)也要当掉线处理,不能把异常漏出去"""
class _ExplodingBrowser:
def is_connected(self) -> bool:
raise RuntimeError("driver connection closed")
site = SiteInteractor(auth_session=None, settings=None) # type: ignore[arg-type]
site._browser = _ExplodingBrowser() # type: ignore[assignment]
assert site.browser_alive is False
def test_browser_status_distinguishes_not_started_from_dead():
"""/health 要能分清「还没启动」和「起过但死了」——只有后者算 degraded"""
site = SiteInteractor(auth_session=None, settings=None) # type: ignore[arg-type]
not_started = site.browser_status()
assert not_started["started"] is False
assert not_started["alive"] is False
site._browser = _FakeBrowser(connected=False) # type: ignore[assignment]
dead = site.browser_status()
assert dead["started"] is True
assert dead["alive"] is False
site._browser = _FakeBrowser(connected=True) # type: ignore[assignment]
assert site.browser_status() == {
"started": True,
"alive": True,
"pending_checkout_tasks": [],
"detail": "连接正常",
}
async def test_ensure_browser_alive_relaunches_on_task_boundary(tmp_path):
"""任务边界探到掉线 → 重放一遍启动流程,调用方无感知"""
site = _build_dead_browser_site(tmp_path)
relaunched = []
async def _fake_launch() -> None:
relaunched.append(True)
site._browser = _FakeBrowser(connected=True) # type: ignore[assignment]
site._launch = _fake_launch # type: ignore[assignment]
await site._ensure_browser_alive()
assert relaunched == [True]
assert site.browser_alive is True
async def test_ensure_browser_alive_is_noop_when_connected(tmp_path):
"""浏览器好好的就不能重建——重建一次要几秒,还会丢掉当前 context 的 cookie"""
site = _build_dead_browser_site(tmp_path)
site._browser = _FakeBrowser(connected=True) # type: ignore[assignment]
async def _fail_launch() -> None:
raise AssertionError("浏览器连接正常时不应重建")
site._launch = _fail_launch # type: ignore[assignment]
await site._ensure_browser_alive()
async def test_ensure_browser_alive_drops_orphaned_checkout_pages(tmp_path):
"""重建前丢掉残留的确认页:那些 Page 已随浏览器一起没了,留着必然报错"""
site = _build_dead_browser_site(tmp_path)
site._checkout_pages["t-stuck"] = object() # type: ignore[assignment]
async def _fake_launch() -> None:
site._browser = _FakeBrowser(connected=True) # type: ignore[assignment]
site._launch = _fake_launch # type: ignore[assignment]
await site._ensure_browser_alive()
assert site._checkout_pages == {}
async def test_ensure_browser_alive_raises_browser_dead_when_relaunch_fails(tmp_path):
"""重建也起不来 → BrowserDeadError(AppError,runner 接得住),并把半成品收干净"""
site = _build_dead_browser_site(tmp_path)
async def _broken_launch() -> None:
raise RuntimeError("chromium 起不来")
site._launch = _broken_launch # type: ignore[assignment]
with pytest.raises(BrowserDeadError):
await site._ensure_browser_alive()
assert site._browser is None # 已 teardown,下次调用从头再试
async def test_new_page_wraps_target_closed_into_app_error():
"""中途掉线的裸 TargetClosedError 必须被包成 AppError
这是整条链最关键的一环不包的话异常会穿过 runner except AppError
落到主循环那个只记日志的兜底里任务一次都不上报网关要干等租约过期
"""
class _DeadContext:
async def new_page(self):
raise RuntimeError("Target page, context or browser has been closed")
site = SiteInteractor(auth_session=None, settings=None) # type: ignore[arg-type]
site._context = _DeadContext() # type: ignore[assignment]
with pytest.raises(BrowserDeadError) as excinfo:
await site._new_page()
from app.shared.errors import AppError
assert isinstance(excinfo.value, AppError)
assert excinfo.value.err_code == 5006
async def test_request_keeps_original_error_when_browser_is_alive():
"""浏览器活着时的请求失败是正常业务失败,不能被误标成掉线"""
class _FlakyRequest:
async def get(self, url: str, **kwargs):
raise RuntimeError("站点 502")
class _Context:
request = _FlakyRequest()
site = SiteInteractor(auth_session=None, settings=None) # type: ignore[arg-type]
site._browser = _FakeBrowser(connected=True) # type: ignore[assignment]
site._context = _Context() # type: ignore[assignment]
with pytest.raises(RuntimeError, match="站点 502"):
await site._request("get", "https://example.test/")
async def test_request_converts_to_browser_dead_when_browser_is_gone():
"""同样的请求失败,浏览器确实没了时才改写成 BrowserDeadError"""
class _FlakyRequest:
async def get(self, url: str, **kwargs):
raise RuntimeError("Target closed")
class _Context:
request = _FlakyRequest()
site = SiteInteractor(auth_session=None, settings=None) # type: ignore[arg-type]
site._browser = _FakeBrowser(connected=False) # type: ignore[assignment]
site._context = _Context() # type: ignore[assignment]
with pytest.raises(BrowserDeadError):
await site._request("get", "https://example.test/")
async def test_submit_order_reports_browser_dead_before_blaming_selectors(tmp_path):
"""浏览器死时 submit_order 要报掉线,而不是「找不到确认按钮」
留存的 Page 是个死对象不是 None不先探活的话 selector 轮询会全部超时
最后把浏览器崩了误诊成站点改版了这两个结论的处置方式完全不同
"""
site = _build_dead_browser_site(tmp_path)
site._checkout_pages["t1"] = object() # type: ignore[assignment]
with pytest.raises(BrowserDeadError):
await site.submit_order(_make_task(task_id="t1"))
async def test_pay_reports_browser_dead(tmp_path):
"""pay 之前 submit_order 已经真的下过单了,掉线必须当场抛出去交人工"""
site = _build_dead_browser_site(tmp_path)
site._checkout_pages["t1"] = object() # type: ignore[assignment]
with pytest.raises(BrowserDeadError):
await site.pay(_make_task(task_id="t1"), "ord-1")
+73 -19
View File
@@ -67,15 +67,17 @@ async def build_session(handler, *, browser=None, settings=None) -> SiteSession:
return session
async def test_warmup_happens_before_first_fetch_and_is_reused():
async def test_happy_path_never_touches_the_home_page():
"""目标页自己会带回 Akamai cookie,正常路径不该多打一次首页。"""
seen: list[str] = []
def handler(request: httpx.Request) -> httpx.Response:
seen.append(str(request.url))
headers = {}
if request.url.host == "www.rakuten.co.jp":
headers["set-cookie"] = "ak_bmsc=abc; Domain=.rakuten.co.jp; Path=/"
return httpx.Response(200, text=GOOD_PAGE, headers=headers)
return httpx.Response(
200,
text=GOOD_PAGE,
headers={"set-cookie": "ak_bmsc=abc; Domain=.rakuten.co.jp; Path=/"},
)
session = await build_session(handler)
try:
@@ -84,18 +86,20 @@ async def test_warmup_happens_before_first_fetch_and_is_reused():
finally:
await session.close()
# 首页预热只做一次,第二次抓取直接复用 cookie
assert seen.count("https://www.rakuten.co.jp/") == 1
assert seen.count(TARGET) == 2
assert seen == [TARGET, TARGET]
async def test_successful_warmup_is_reused_when_upstream_sets_no_cookie():
"""首页可能返回 200 但不下发 Akamai cookie,仍不得重复预热"""
seen: list[str] = []
async def test_cookies_from_target_page_are_reused_across_fetches():
"""第一次响应下发的 cookie 要带到后续请求上,不必再走首页"""
cookie_headers: list[str | None] = []
def handler(request: httpx.Request) -> httpx.Response:
seen.append(str(request.url))
return httpx.Response(200, text=GOOD_PAGE)
cookie_headers.append(request.headers.get("cookie"))
return httpx.Response(
200,
text=GOOD_PAGE,
headers={"set-cookie": "ak_bmsc=abc; Domain=.rakuten.co.jp; Path=/"},
)
session = await build_session(handler)
try:
@@ -104,8 +108,30 @@ async def test_successful_warmup_is_reused_when_upstream_sets_no_cookie():
finally:
await session.close()
assert seen.count("https://www.rakuten.co.jp/") == 1
assert seen.count(TARGET) == 2
assert cookie_headers[0] is None # 首个请求是冷的
assert "ak_bmsc=abc" in (cookie_headers[1] or "")
async def test_expired_cookies_are_dropped_before_next_fetch():
"""cookie 罐超过 session_ttl_seconds 后要清空,不能带着过期 cookie 去撞。"""
cookie_headers: list[str | None] = []
def handler(request: httpx.Request) -> httpx.Response:
cookie_headers.append(request.headers.get("cookie"))
return httpx.Response(
200,
text=GOOD_PAGE,
headers={"set-cookie": "ak_bmsc=abc; Domain=.rakuten.co.jp; Path=/"},
)
session = await build_session(handler, settings=make_settings(session_ttl_seconds=0.0))
try:
await session.fetch_html(TARGET, mobile=False)
await session.fetch_html(TARGET, mobile=False)
finally:
await session.close()
assert cookie_headers == [None, None] # 每次都从干净状态起步
async def test_missing_state_marker_is_treated_as_blocked():
@@ -139,6 +165,32 @@ async def test_retry_recovers_when_a_later_attempt_succeeds():
assert attempts["n"] == 2
async def test_first_failure_swaps_cookies_via_home_page():
"""首次失败时用首页换一套 cookie——这是首页 URL 唯一的用途。"""
seen: list[str] = []
def handler(request: httpx.Request) -> httpx.Response:
seen.append(str(request.url))
if request.url.host == "www.rakuten.co.jp":
return httpx.Response(
200,
text="home",
headers={"set-cookie": "ak_bmsc=fresh; Domain=.rakuten.co.jp; Path=/"},
)
if seen.count(TARGET) == 1:
return httpx.Response(200, text=BLOCK_PAGE)
return httpx.Response(200, text=GOOD_PAGE)
session = await build_session(handler)
try:
assert await session.fetch_html(TARGET, mobile=False) == GOOD_PAGE
finally:
await session.close()
# 首页只在失败之后出现一次,且排在两次目标页请求中间
assert seen == [TARGET, "https://www.rakuten.co.jp/", TARGET]
async def test_browser_fallback_supplies_cookies_and_page_on_persistent_block():
"""浏览器已经取到页面时应直接采用,不再多打一次 HTTP"""
browser = FakeBrowser(
@@ -264,9 +316,9 @@ async def test_search_and_detail_use_separate_cookie_jars():
def handler(request: httpx.Request) -> httpx.Response:
mobile = request.headers.get("sec-ch-ua-mobile") == "?1"
headers = {}
if request.url.host == "www.rakuten.co.jp":
headers["set-cookie"] = f"ak_bmsc={'sp' if mobile else 'pc'}; Domain=.rakuten.co.jp; Path=/"
headers = {
"set-cookie": f"ak_bmsc={'sp' if mobile else 'pc'}; Domain=.rakuten.co.jp; Path=/"
}
return httpx.Response(200, text=GOOD_PAGE, headers=headers)
session = await build_session(handler)
@@ -282,7 +334,9 @@ async def test_search_and_detail_use_separate_cookie_jars():
assert sp_cookie == "sp"
async def test_profile_status_reports_warmup_state():
async def test_profile_status_reports_cookie_state():
"""`warmed` 现在的语义是「当前有可复用的 Akamai cookie」,字段名为兼容保留。"""
def handler(request: httpx.Request) -> httpx.Response:
headers = {"set-cookie": "ak_bmsc=abc; Domain=.rakuten.co.jp; Path=/"}
return httpx.Response(200, text=GOOD_PAGE, headers=headers)
+317 -2
View File
@@ -1,22 +1,53 @@
"""telemetry 模块测试
验证两件事
验证
1. 默认配置otel_enabled=False setup noop不初始化任何 provider
2. enabled=true + endpoint setup 注册真实 TracerProvidershutdown 复位
3. instrument_app setup 之前调用也要真的装上中间件三个服务都是导入期打桩
4. traced / set_attributes / record_error 的行为
5. 采集返回结果这一侧record_envelope信封失败在 HTTP 层看不出来
record_parse_failure失败在 fetch 还是 parsesnapshot失败页面快照
以及 span_unless_suppressed suppressed() 的配套关系
不打真实网络OTLPSpanExporter 创建时不发请求BatchSpanProcessor 异步批量
上报在没有 span 产生时也不会触发
上报在没有 span 产生时也不会触发span 断言用独立的 InMemory provider不碰
全局 providerOTel 的全局 provider 只允许设置一次测试间共享会互相污染
"""
from __future__ import annotations
import pytest
from fastapi import FastAPI
from opentelemetry import trace
from opentelemetry.instrumentation import fastapi as otel_fastapi
from opentelemetry.instrumentation.asgi import OpenTelemetryMiddleware
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
from opentelemetry.trace import StatusCode
from opentelemetry.util.http import parse_excluded_urls
from app.shared import telemetry
from app.shared.config import Settings
from app.shared.errors import OrderGuardError, ScrapeParseError, UpstreamBlockedError
from app.shared.telemetry import is_initialized, setup_telemetry, shutdown_telemetry
@pytest.fixture
def spans(monkeypatch) -> InMemorySpanExporter:
"""把 telemetry 内部取到的 tracer 换成写内存的,用于断言 span
不用全局 provider`trace.set_tracer_provider` 只生效一次一旦别的用例先
设过这里再设就被忽略OTel 只打一条 warning断言会莫名其妙拿不到 span
"""
exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))
monkeypatch.setattr(telemetry.trace, "get_tracer", provider.get_tracer)
return exporter
def test_disabled_is_noop():
"""otel_enabled=False 时 setup/instrument/shutdown 都不初始化 provider"""
settings = Settings(_env_file=None) # 默认 otel_enabled=False
@@ -24,6 +55,290 @@ def test_disabled_is_noop():
assert not is_initialized()
def test_instrument_app_works_before_setup_telemetry(monkeypatch):
"""回归:instrument_app 必须在 setup_telemetry 之前也能真的装上中间件
三个服务都在模块导入时执行 `app = create_app()`内部调 instrument_app
setup_telemetry 要等 lifespan 才跑曾经这里用 `_provider is None` 做前置
判断于是导入期一律 returnFastAPI 从来没被打桩过一条 server span
没有这条用例把那个顺序钉住
"""
monkeypatch.setattr(
telemetry, "get_settings", lambda: Settings(_env_file=None, otel_enabled=True)
)
assert not is_initialized() # 尚未 setup,正是导入期的状态
app = FastAPI()
try:
telemetry.instrument_app(app)
assert app._is_instrumented_by_opentelemetry
finally:
FastAPIInstrumentor.uninstrument_app(app)
def _otel_middleware(app: FastAPI) -> OpenTelemetryMiddleware:
"""从构建好的中间件栈里挖出 ASGI 打桩中间件,用于断言它的排除规则"""
node = app.build_middleware_stack()
while node is not None:
if isinstance(node, OpenTelemetryMiddleware):
return node
node = getattr(node, "app", None)
raise AssertionError("中间件栈里没有 OpenTelemetryMiddleware")
def test_health_excluded_from_server_spans(monkeypatch):
"""/health 不产生 server span,其它路径照常
容器 HEALTHCHECK 30 秒探一次上游也在轮询这些请求各自是一条孤立 trace
量大且没有信息量排除规则按 search 匹配完整 URL所以要钉住两件事/health
真的被挡掉 `$` 锚点没有顺手把 /api/* 一起挡掉
"""
monkeypatch.setattr(
telemetry, "get_settings", lambda: Settings(_env_file=None, otel_enabled=True)
)
app = FastAPI()
try:
telemetry.instrument_app(app)
excluded = _otel_middleware(app).excluded_urls
assert excluded.url_disabled("http://127.0.0.1:31108/health")
assert not excluded.url_disabled("http://127.0.0.1:31108/api/cart/add")
# 前缀匹配会误伤的反例:锚点保证只有 /health 本身被排除
assert not excluded.url_disabled("http://127.0.0.1:31108/api/health-detail")
finally:
FastAPIInstrumentor.uninstrument_app(app)
def test_excluded_urls_empty_falls_back_to_env(monkeypatch):
"""配置留空时传 None,让 OTel 回落到它自己的环境变量而不是排除空字符串"""
monkeypatch.setattr(
telemetry,
"get_settings",
lambda: Settings(_env_file=None, otel_enabled=True, otel_excluded_urls=" "),
)
# OTel 那份环境变量在模块导入时就解析成常量了,setenv 已经晚了,只能直接替常量
monkeypatch.setattr(
otel_fastapi, "_excluded_urls_from_env", parse_excluded_urls("/metrics")
)
app = FastAPI()
try:
telemetry.instrument_app(app)
excluded = _otel_middleware(app).excluded_urls
assert excluded.url_disabled("http://127.0.0.1:31108/metrics")
assert not excluded.url_disabled("http://127.0.0.1:31108/health")
finally:
FastAPIInstrumentor.uninstrument_app(app)
def test_instrument_app_skipped_when_otel_disabled(monkeypatch):
"""otel 关闭时不装中间件,省掉一层用不上的开销"""
monkeypatch.setattr(
telemetry, "get_settings", lambda: Settings(_env_file=None, otel_enabled=False)
)
app = FastAPI()
telemetry.instrument_app(app)
assert not getattr(app, "_is_instrumented_by_opentelemetry", False)
async def test_traced_records_span_and_reraises(spans):
"""traced 成功时留一个 span;异常时记 ERROR 状态并原样抛出"""
@telemetry.traced("unit.ok")
async def ok() -> str:
return "done"
@telemetry.traced("unit.boom")
async def boom() -> None:
raise OrderGuardError("金额超限")
assert await ok() == "done"
with pytest.raises(OrderGuardError):
await boom()
finished = {s.name: s for s in spans.get_finished_spans()}
assert finished["unit.ok"].status.status_code is not StatusCode.ERROR
failed = finished["unit.boom"]
assert failed.status.status_code is StatusCode.ERROR
# AppError 的对外错误码要落在 span 上:排查时按码筛比按异常类名筛更贴近上游
assert failed.attributes["error.type"] == "OrderGuardError"
assert failed.attributes["error.code"] == OrderGuardError("x").err_code
async def test_traced_nests_under_caller_span(spans):
"""traced 出来的 span 要挂在调用方的 span 底下,而不是各自成为孤立 trace"""
@telemetry.traced("unit.child")
async def child() -> None:
return None
tracer = telemetry.trace.get_tracer(__name__)
with tracer.start_as_current_span("unit.root"):
await child()
by_name = {s.name: s for s in spans.get_finished_spans()}
assert by_name["unit.child"].parent.span_id == by_name["unit.root"].context.span_id
# 同一条 trace 才能在观测后台里连成一条链路
assert by_name["unit.child"].context.trace_id == by_name["unit.root"].context.trace_id
def test_set_attributes_skips_none(spans):
"""可选字段为 None 时不落属性,避免一堆 None 噪声"""
tracer = telemetry.trace.get_tracer(__name__)
with tracer.start_as_current_span("unit.attrs") as span:
telemetry.set_attributes(span, {"a": 1, "b": None, "c": "x"})
attributes = spans.get_finished_spans()[0].attributes
assert attributes["a"] == 1
assert attributes["c"] == "x"
assert "b" not in attributes
def test_record_error_keeps_app_error_fields(spans):
"""AppError 的排查字段(错误码/可重试/状态码/消息)都要落到属性上
只落 error.type 不够上游看到的是错误码该不该重试 retryable
record_exception 记的 event 在多数观测后台里要展开才看得到列表页筛不出来
"""
tracer = telemetry.trace.get_tracer(__name__)
with tracer.start_as_current_span("unit.err") as span:
telemetry.record_error(span, UpstreamBlockedError("Akamai 挑战页"))
attributes = spans.get_finished_spans()[0].attributes
assert attributes["error.type"] == "UpstreamBlockedError"
assert attributes["error.code"] == UpstreamBlockedError().err_code
assert attributes["error.retryable"] is True
assert attributes["error.status_code"] == 400
assert "Akamai 挑战页" in attributes["error.message"]
def test_record_error_on_plain_exception_omits_app_error_fields(spans):
"""非 AppError 不应凭空长出 error.code / error.retryable 属性"""
tracer = telemetry.trace.get_tracer(__name__)
with tracer.start_as_current_span("unit.plain") as span:
telemetry.record_error(span, RuntimeError("boom"))
attributes = spans.get_finished_spans()[0].attributes
assert attributes["error.type"] == "RuntimeError"
assert "error.code" not in attributes
assert "error.retryable" not in attributes
def test_record_envelope_failure_marks_span_error(spans):
"""信封失败要置 ERROR 并落错误码——HTTP 层看不出这次调用失败了"""
tracer = telemetry.trace.get_tracer(__name__)
with tracer.start_as_current_span("unit.envelope") as span:
telemetry.record_envelope(
span, success=False, err_code=6002, msg="租约无效", status_code=409
)
finished = spans.get_finished_spans()[0]
assert finished.status.status_code is StatusCode.ERROR
assert finished.attributes["api.success"] is False
assert finished.attributes["api.code"] == 6002
assert finished.attributes["api.status_code"] == 409
assert finished.attributes["api.msg"] == "租约无效"
def test_record_envelope_success_leaves_status_ok(spans):
tracer = telemetry.trace.get_tracer(__name__)
with tracer.start_as_current_span("unit.envelope_ok") as span:
telemetry.record_envelope(span, success=True, status_code=200)
finished = spans.get_finished_spans()[0]
assert finished.status.status_code is not StatusCode.ERROR
assert finished.attributes["api.success"] is True
def test_snapshot_truncates_and_carries_extra(spans):
"""超限 HTML 截断并标注,附带的来源信息(URL 等)也要落在同一条 event 上"""
tracer = telemetry.trace.get_tracer(__name__)
with tracer.start_as_current_span("unit.snapshot") as span:
telemetry.snapshot(
span, "parse.failed_html", "x" * 100, 10,
extra={"parse.url": "https://example.com/a", "parse.ignored": None},
)
event = spans.get_finished_spans()[0].events[0]
assert event.name == "parse.failed_html"
assert event.attributes["snapshot.html"] == "x" * 10
assert event.attributes["snapshot.original_bytes"] == 100
assert event.attributes["snapshot.truncated"] is True
assert event.attributes["parse.url"] == "https://example.com/a"
assert "parse.ignored" not in event.attributes
def test_snapshot_skips_empty_html(spans):
"""页面根本没取回来时不记空 event"""
tracer = telemetry.trace.get_tracer(__name__)
with tracer.start_as_current_span("unit.snapshot_empty") as span:
telemetry.snapshot(span, "parse.failed_html", None, 100)
telemetry.snapshot(span, "parse.failed_html", "", 100)
assert spans.get_finished_spans()[0].events == ()
def test_record_parse_failure_distinguishes_fetch_from_parse(spans):
"""失败阶段按「HTML 有没有拿到」区分:两者排查方向相反
html 为空页面没取回来通道/反爬/上游 5xx非空取回了但解析不出
多半站点改版fail_reason 要给真实异常类名不能一律写死 parse_error
"""
tracer = telemetry.trace.get_tracer(__name__)
with tracer.start_as_current_span("unit.fetch_fail") as span:
telemetry.record_parse_failure(
span, UpstreamBlockedError("挑战页"), html=None, max_bytes=100,
url="https://example.com/a",
)
with tracer.start_as_current_span("unit.parse_fail") as span:
telemetry.record_parse_failure(
span, ScrapeParseError("没有 state"), html="<html/>", max_bytes=100,
url="https://example.com/b",
)
with tracer.start_as_current_span("unit.delegate_fail") as span:
telemetry.record_parse_failure(
span, ScrapeParseError("转调失败"), stage="delegate",
)
by_name = {s.name: s for s in spans.get_finished_spans()}
fetch = by_name["unit.fetch_fail"]
assert fetch.attributes["parse.stage"] == "fetch"
assert fetch.attributes["parse.fail_reason"] == "UpstreamBlockedError"
# 页面没取回来,没有快照可落
assert [e.name for e in fetch.events] == ["exception"]
parsed = by_name["unit.parse_fail"]
assert parsed.attributes["parse.stage"] == "parse"
assert parsed.attributes["parse.fail_reason"] == "ScrapeParseError"
snapshot_event = next(e for e in parsed.events if e.name == "parse.failed_html")
assert snapshot_event.attributes["snapshot.html"] == "<html/>"
assert snapshot_event.attributes["parse.url"] == "https://example.com/b"
# 显式 stage 覆盖推断:编排方法的失败既不在自己的 fetch 也不在自己的 parse
assert by_name["unit.delegate_fail"].attributes["parse.stage"] == "delegate"
def test_span_unless_suppressed_is_noop_inside_suppressed(spans):
"""`suppressed()` 里手工埋点必须退化成 noop,否则空转长轮询绕开抑制刷满后台
`suppress_instrumentation` 只被 instrumentation 库尊重手工
`start_as_current_span` 不看它worker lease 正是在 suppressed() 里调
GatewayClient._request
"""
tracer = telemetry.trace.get_tracer(__name__)
with telemetry.suppressed():
with telemetry.span_unless_suppressed(tracer, "unit.suppressed") as span:
# 属性/异常写在 noop span 上不能报错,调用方不必分支
telemetry.set_attributes(span, {"a": 1})
telemetry.record_error(span, RuntimeError("boom"))
assert not span.is_recording()
assert spans.get_finished_spans() == ()
with telemetry.span_unless_suppressed(tracer, "unit.not_suppressed") as span:
assert span.is_recording()
assert [s.name for s in spans.get_finished_spans()] == ["unit.not_suppressed"]
def test_enabled_initializes_and_shutdown_releases():
"""enabled=true 时 setup 注册 TracerProvider,shutdown 后 _provider 复位"""
settings = Settings(
+53 -7
View File
@@ -11,7 +11,8 @@ from fastapi.testclient import TestClient
from app.shared.config import get_settings
from app.shared.errors import CartOperationError, NotLoggedInError
from app.trading.main import create_app
from app.trading import main as trading_main
from app.trading.container import TradingContainer
from app.trading.services.auth_session import AuthStatus
TOKEN = get_settings().bearer_token
@@ -68,6 +69,9 @@ class StubAuthSession:
if not self.logged_in:
raise NotLoggedInError(site=site, detail="stub")
async def start(self) -> None:
"""模拟 lifespan 启动;桩不持有真实客户端。"""
async def close(self) -> None:
"""lifespan 收尾会调用;桩没有真实客户端要关"""
@@ -85,6 +89,7 @@ class StubSiteInteractor:
self.clear_calls = 0
self.remove_calls: list[str] = []
self.fail_with: Exception | None = None
self.browser_alive = True
async def add_to_cart_payload(
self,
@@ -129,18 +134,39 @@ class StubSiteInteractor:
raise self.fail_with
return {"removed": True, "item_id": item_id}
def browser_status(self) -> dict:
"""/health 读的浏览器状态;browser_alive 可翻成 False 模拟掉线"""
return {
"started": True,
"alive": self.browser_alive,
"pending_checkout_tasks": [],
"detail": "连接正常" if self.browser_alive else "浏览器已掉线",
}
async def start(self) -> None:
"""模拟 lifespan 启动;桩不启动 Playwright。"""
async def close(self) -> None:
"""lifespan 收尾会调用;桩没有真实浏览器要关"""
@pytest.fixture
def client_and_stubs():
app = create_app()
def client_and_stubs(monkeypatch):
"""在 lifespan 之前注入桩,确保离线 API 测试不启动 Playwright。"""
stub = StubAuthSession()
stub_site = StubSiteInteractor()
settings = get_settings().model_copy(update={"auto_login_on_start": False})
def build_test_container() -> TradingContainer:
return TradingContainer(
settings=settings,
auth_session=stub, # type: ignore[arg-type]
site=stub_site,
)
monkeypatch.setattr(trading_main, "build_container", build_test_container)
app = trading_main.create_app()
with TestClient(app) as client:
stub = StubAuthSession()
stub_site = StubSiteInteractor()
app.state.container.auth_session = stub
app.state.container.site = stub_site
yield client, stub, stub_site
@@ -168,6 +194,26 @@ def test_health_needs_no_token(client):
body = response.json()
assert body["data"]["status"] == "ok"
assert set(body["data"]["auth"]) == {"rakuten"}
assert body["data"]["browser"]["alive"] is True
def test_health_returns_503_when_browser_is_dead(client, stub_site):
"""浏览器掉线 → 503,让容器 HEALTHCHECK 探到并触发重启
进程还活着端口还通但这个服务的所有站点操作都要靠那一个浏览器它没了以后
每一单都会失败返回 200 的话 HEALTHCHECK 永远绿灯缺口就一直挂在那儿
重启是安全的网关侧任务绝不自动重投docs/order-gateway.md §5
"""
stub_site.browser_alive = False
response = client.get("/health")
assert response.status_code == 503
body = response.json()
assert body["success"] is False
assert body["code"] == 5006 # 与 BrowserDeadError 同码
assert body["data"]["status"] == "degraded"
assert body["data"]["browser"]["alive"] is False
def test_health_does_not_probe_the_site(client, stub):
+48
View File
@@ -134,6 +134,54 @@ async def test_match_ignores_query_string_difference():
assert result.verdict == verify.VerifyVerdict.ALREADY_ORDERED
async def test_multiple_item_intent_requires_all_items_in_same_order():
gateway = FakeGateway()
site = FakeSite(
window=OrderListWindow(
entries=[
OrderListEntry(
order_number="o1",
order_date="2026-08-10T00:00:00Z",
items=[
OrderListItem(item_url="https://item.rakuten.co.jp/shop/x/"),
OrderListItem(item_url="https://item.rakuten.co.jp/shop/y/"),
],
)
],
window_fully_covered=True,
)
)
task = _make_task()
task.intent = {
"items": [
{"item_url": "https://item.rakuten.co.jp/shop/x/"},
{"item_url": "https://item.rakuten.co.jp/shop/y/"},
]
}
result = await verify.verify_on_site(task, gateway=gateway, site=site)
assert result.verdict == verify.VerifyVerdict.ALREADY_ORDERED
assert result.site_order_id == "o1"
async def test_multiple_item_intent_partial_order_match_is_not_ordered():
gateway = FakeGateway()
site = FakeSite(
window=OrderListWindow(
entries=[_entry("o1", "https://item.rakuten.co.jp/shop/x/")],
window_fully_covered=True,
)
)
task = _make_task()
task.intent = {
"items": [
{"item_url": "https://item.rakuten.co.jp/shop/x/"},
{"item_url": "https://item.rakuten.co.jp/shop/y/"},
]
}
result = await verify.verify_on_site(task, gateway=gateway, site=site)
assert result.verdict == verify.VerifyVerdict.NOT_ORDERED
# ---- 命中 0 笔且窗口确认覆盖完:未下单 ----
+158
View File
@@ -0,0 +1,158 @@
"""GatewayClient 埋点测试:信封里的失败必须在链路上看得见
要钉住的问题**网关的失败在响应信封里不在 HTTP 状态码上**一次
`success=false, code=6002`租约无效的回报httpx 自动 instrumentation 只看到
一个完成了的请求而且那个 CLIENT span `request()` 返回时就结束了此时信封
还没解所以它永远不可能带上这次调用的结论链路里于是只剩调用发生过
另一半是抑制worker 的空转长轮询在 `suppressed()` 里调 `_request`
`suppress_instrumentation` 只被 instrumentation 库尊重手工建的 span 不看它
手工埋点若不配合 `span_unless_suppressed`孤立 trace 会从这个口子重新灌回来
全部用 httpx.MockTransport 拦截不触达真实网关
"""
from __future__ import annotations
import httpx
import pytest
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
from opentelemetry.trace import SpanKind, StatusCode
from app.shared.config import Settings
from app.shared.errors import AppError
from app.shared.task_state import OrderState
from app.shared.telemetry import suppressed
from app.trading.worker import client as worker_client
from app.trading.worker.client import GatewayClient
@pytest.fixture
def spans(monkeypatch) -> InMemorySpanExporter:
"""把 client 模块级 tracer 换成写内存的(不碰全局 provider)"""
exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))
monkeypatch.setattr(worker_client, "tracer", provider.get_tracer("test"))
return exporter
def _client(handler) -> GatewayClient:
"""构建 GatewayClient 并把它的 httpx 客户端换成 MockTransport 版本"""
client = GatewayClient(
"https://gateway.example", "token", settings=Settings(_env_file=None)
)
client._client = httpx.AsyncClient(
base_url="https://gateway.example",
transport=httpx.MockTransport(handler),
)
return client
def _envelope(*, success: bool, code: int, msg: str, data=None, status: int = 200):
return httpx.Response(
status,
json={"success": success, "msg": msg, "data": data, "code": code},
)
async def test_envelope_failure_is_recorded_on_span(spans):
"""success=false 要把 span 置 ERROR 并落错误码——HTTP 层看这次是 200"""
gateway = _client(
lambda _r: _envelope(
success=False, code=6002, msg="租约无效:不是持有者", status=200
)
)
try:
with pytest.raises(AppError) as excinfo:
await gateway.report(
"task-1", "worker-1", state=OrderState.IN_CART, terminal=False
)
assert excinfo.value.err_code == 6002
finally:
await gateway.aclose()
span = spans.get_finished_spans()[0]
assert span.kind is SpanKind.CLIENT
assert span.status.status_code is StatusCode.ERROR
# 这次调用的结论——HTTP 状态码是 200,光看它分不出成败
assert span.attributes["api.success"] is False
assert span.attributes["api.code"] == 6002
assert span.attributes["api.status_code"] == 200
assert "租约无效" in span.attributes["api.msg"]
assert span.attributes["gateway.path"] == "/api/orders/task-1/report"
assert span.attributes["gateway.method"] == "POST"
async def test_successful_envelope_leaves_span_ok(spans):
"""success=true 不置 ERROR;作为对照说明上面的断言不是恒真"""
gateway = _client(
lambda _r: _envelope(success=True, code=0, msg="success", data={"recorded": True})
)
try:
await gateway.report("task-1", "worker-1", state=OrderState.IN_CART)
finally:
await gateway.aclose()
span = spans.get_finished_spans()[0]
assert span.status.status_code is not StatusCode.ERROR
assert span.attributes["api.success"] is True
assert "api.code" not in span.attributes
async def test_non_json_body_is_recorded_as_error(spans):
"""网关回了非 JSON(网关挂了/被反代拦了):记成可重试的 3001"""
gateway = _client(lambda _r: httpx.Response(502, text="<html>502 Bad Gateway</html>"))
try:
with pytest.raises(AppError) as excinfo:
await gateway.renew("task-1", "worker-1")
assert excinfo.value.err_code == 3001
finally:
await gateway.aclose()
span = spans.get_finished_spans()[0]
assert span.status.status_code is StatusCode.ERROR
assert span.attributes["error.code"] == 3001
# 这类失败是可重试的,与信封里的业务拒绝(retryable=False)要能区分开
assert span.attributes["error.retryable"] is True
assert span.attributes["gateway.status_code"] == 502
async def test_idle_long_poll_produces_no_span(spans):
"""回归:空转长轮询在 suppressed() 里不能产生 span
worker 30 lease 一次绝大多数返回空手工埋点若不看抑制标记这些
调用会各自成为一条孤立 trace 把观测后台刷满正是 suppressed() 要解决的问题
"""
gateway = _client(lambda _r: _envelope(success=True, code=0, msg="success", data=None))
try:
with suppressed():
assert await gateway.lease("worker-1", wait=0) is None
assert spans.get_finished_spans() == ()
# 抑制之外照常埋点,证明上面的空断言不是因为埋点根本没生效
assert await gateway.lease("worker-1", wait=0) is None
finally:
await gateway.aclose()
assert [s.attributes["gateway.path"] for s in spans.get_finished_spans()] == [
"/api/orders/lease"
]
async def test_leased_task_call_is_still_traced(spans):
"""领到任务后的调用不受抑制影响:它们挂在任务根 span 底下,是要看的那部分"""
gateway = _client(
lambda _r: _envelope(
success=True, code=0, msg="success",
data={"task_id": "task-1", "site": "rakuten", "lease_count": 1},
)
)
try:
task = await gateway.lease("worker-1", wait=0)
assert task is not None and task.task_id == "task-1"
finally:
await gateway.aclose()
assert len(spans.get_finished_spans()) == 1
+166 -1
View File
@@ -20,8 +20,14 @@ from typing import Any
import pytest
from app.shared.errors import CheckoutBlockedError, OrderGuardError
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor
from opentelemetry.sdk.trace.export.in_memory_span_exporter import InMemorySpanExporter
from opentelemetry.trace import StatusCode
from app.shared.errors import BrowserDeadError, CheckoutBlockedError, OrderGuardError
from app.shared.task_state import OrderState, TaskStatus
from app.trading.worker import runner as runner_module
from app.trading.worker import verify
from app.trading.worker.evidence import EvidenceStore
from app.trading.worker.local_db import LocalDB
@@ -352,6 +358,41 @@ async def test_checkout_blocked_becomes_needs_human(
assert "风控" in terminal["detail"]
# ---- 浏览器掉线 → needs_human(不是 failed)----
async def test_browser_dead_becomes_needs_human(runner: WorkerRunner):
"""执行途中浏览器掉线 → BrowserDeadError → 转 needs_human
两件事同时被这个用例守着
1. BrowserDeadError AppError 的子类能被 _execute_with_renewal 接住并**上报**
不是 AppError 的话它会一路飘到主循环的兜底日志里任务一次都不上报网关侧
要干等整个 lease_ttl 才被 sweep stale
2. 结论是 needs_human 而不是 failed`except BrowserDeadError` 必须排在
`except AppError` 前面顺序反了这个断言就会挂浏览器是在动作中途没的
站点侧生效与否无从判断不能给上游一个明确失败的结论
"""
async def _noop(task): # noqa: ANN001
return None
async def _browser_died(task): # noqa: ANN001
raise BrowserDeadError("pay:浏览器已掉线,本单是否已提交无法判断,需人工核对订单列表")
runner._site.add_to_cart = _noop # type: ignore[assignment]
runner._site.verify_cart = _noop # type: ignore[assignment]
runner._site.enter_checkout = _browser_died # type: ignore[assignment]
_set_site_clear(runner._site, {"removed_count": 0, "cart_count": 0})
await runner.handle(_make_task(task_id="t1"))
gateway: FakeGateway = runner._gateway_for_test # type: ignore[attr-defined]
terminal = gateway.last_terminal_report()
assert terminal["terminal_status"] == TaskStatus.NEEDS_HUMAN
assert "掉线" in terminal["detail"]
# ---- ラクマ 不在交易范围 → needs_human ----
@@ -865,6 +906,130 @@ async def test_monitor_order_stops_after_max_checks_without_finding_order(
assert gateway.reports == []
# ---- 链路追踪:一个任务一条 trace ----
@pytest.fixture
def spans(monkeypatch) -> InMemorySpanExporter:
"""把 runner 模块级 tracer 换成写内存的
不动全局 providerOTel 的全局 provider 只允许设置一次测试间共享会互相污染
"""
exporter = InMemorySpanExporter()
provider = TracerProvider()
provider.add_span_processor(SimpleSpanProcessor(exporter))
monkeypatch.setattr(runner_module, "tracer", provider.get_tracer("test"))
return exporter
def _stub_happy_path(runner: WorkerRunner) -> None:
"""把整条下单流程替换成桩,让 execute() 一路走到付款完成"""
async def _clear():
return {"removed_count": 0, "cart_count": 0, "html": "<html/>", "screenshot": b"p"}
async def _snapshot(*args, **kwargs):
return PageSnapshot(html="<html/>", screenshot=b"p")
async def _parse(html: str):
return CheckoutSummary(payable_yen=297)
async def _submit(task):
return SubmitOutcome(
site_order_id="ord-1", evidence=PageSnapshot(html="<html/>", screenshot=b"p")
)
async def _unchanged(order_id: str):
return OrderStatusSnapshot(found=True, order_state=None)
runner._site.clear_cart = _clear # type: ignore[assignment]
runner._site.add_to_cart = _snapshot # type: ignore[assignment]
runner._site.verify_cart = _snapshot # type: ignore[assignment]
runner._site.enter_checkout = _snapshot # type: ignore[assignment]
runner._site.parse_checkout = _parse # type: ignore[assignment]
runner._site.submit_order = _submit # type: ignore[assignment]
runner._site.pay = _snapshot # type: ignore[assignment]
runner._site.check_order_status = _unchanged # type: ignore[assignment]
async def test_order_task_emits_one_trace_with_a_span_per_step(
runner: WorkerRunner, spans: InMemorySpanExporter
):
"""一笔下单 = 一条 trace,每个步骤一个子 span,全部挂在任务根 span 底下
worker 是后台 asyncio 任务没有 HTTP 入口不开这个根 span 的话下游每次
出站调用都各自成为孤立 trace观测后台上就只剩一堆请求记录看不出
这是同一笔单也看不出卡在哪一步这条用例把链路结构钉住
"""
_stub_happy_path(runner)
await runner.handle(_make_task(task_id="t1", intent={"item_url": "https://x/y/", "quantity": 2}))
finished = spans.get_finished_spans()
root = next(s for s in finished if s.name == "order.task")
steps = [s for s in finished if s.name.startswith("order.step.")]
# 步骤 span 覆盖整条流水,且顺序与执行顺序一致
assert [s.name for s in steps] == [
"order.step.cart-add",
"order.step.cart-check",
"order.step.order-confirm",
"order.step.order-submit",
"order.step.payment",
]
# 每个步骤都挂在任务根底下,同一条 trace
for step in steps:
assert step.parent.span_id == root.context.span_id
assert step.context.trace_id == root.context.trace_id
assert root.attributes["order.task_id"] == "t1"
assert root.attributes["order.route"] == "execute"
assert root.attributes["order.item_url"] == "https://x/y/"
# 证据路径落在 span 上:看到失败步骤能直接找到现场
assert steps[0].attributes["order.evidence_ref"]
assert steps[3].attributes["order.site_order_id"] == "ord-1"
async def test_guard_failure_marks_task_span_as_error(
runner: WorkerRunner, spans: InMemorySpanExporter
):
"""金额守卫拦下时任务根 span 要是 ERROR,并带上对外错误码
闸门拦截会被 _execute_with_renewal 吞掉转 needs_human不抛出 handle
所以根 span 很容易显示成功这里确认它没有
"""
_stub_happy_path(runner)
async def _pricey(html: str):
return CheckoutSummary(payable_yen=999_999)
runner._site.parse_checkout = _pricey # type: ignore[assignment]
await runner.handle(_make_task(task_id="t1"))
root = next(s for s in spans.get_finished_spans() if s.name == "order.task")
assert root.status.status_code is StatusCode.ERROR
assert root.attributes["error.type"] == "OrderGuardError"
# 提交步骤没有发生:拦在确认页之后、提交之前
names = [s.name for s in spans.get_finished_spans()]
assert "order.step.order-submit" not in names
async def test_idle_lease_does_not_emit_spans(runner: WorkerRunner, spans: InMemorySpanExporter):
"""空转的长轮询不产生 span——否则每 30 秒一条孤立 trace 会把观测后台刷满"""
runner._running = True
async def _lease_then_stop(worker_id: str, *, wait: int = 30, site: str | None = None):
runner.stop() # 领到空就停,避免死循环
return None
runner._gateway.lease = _lease_then_stop # type: ignore[assignment]
await runner.run()
assert spans.get_finished_spans() == ()
async def test_monitor_order_swallows_check_errors_and_keeps_polling(
runner: WorkerRunner,
):