trading 新增 4 条购物车接口(POST /api/cart/{add,status,clear,remove}),
全部 Bearer 鉴权、走 SiteInteractor(Playwright + storage_state)。同步把
SiteInteractor 从 gateway URL 解耦——lifespan 总是构造与启停,container
字段 worker_site → site,加 asyncio.Lock 让 HTTP 与 worker 共用同一把锁
(同账号串行硬约束)。clear/remove 用 UI 点击 button[aria-label="削除"],
探针回报这是稳定 selector;真账号实测前先用此路径。
抽 ichiba 加购字段解析到 app/shared/purchase_contract.py(常量 +
inventory_flag_for + basket_domain_of + base_form_fields),原本 scraping
与 trading 重复实现同一段 __INITIAL_STATE__.purchase 解析。进一步发现
README 写的「purchase 块是两服务契约」实际未落地——trading 必须 Playwright
开页(httpx 被 TLS 指纹拦死),本地抽比再调 /api/item_detail 更快更新鲜。
删除 scraping 端 PurchaseInfo/PurchaseOption/PurchaseOptionValue 模型、
各站 _purchase_info 函数、tests/test_purchase.py。ItemDetailData 保留
purchase_condition / is_sold_out / purchase_unit / sku 等商品状态字段。
README「加购与下单」段重写。
328 测试全绿(含架构测试守住三方互不 import)。
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
785 lines
33 KiB
Python
785 lines
33 KiB
Python
"""站点交互:加购 / 校验 / 清空 / 删除 / 进入下单确认页 / 金额守卫 / 提交 / 付款 / 监控
|
|
|
|
实测进度(详见 project://jp-rakuten/checkout-flow-probe-findings):
|
|
- add_to_cart、verify_cart 已实测可用(Playwright + storage_state 走 SP 通道)
|
|
- clear_cart、remove_item 用 UI 点击 `button[aria-label="削除"]` 路径实现,
|
|
未做真账号实测;首次跑通后回填实测 selector 与 modal 行为到记忆节点
|
|
- enter_checkout 及之后**未实现**:Rakuten 对 checkout 这类敏感操作要求
|
|
session upgrade(重输密码),即使当前 SSO 已登录。这是自动 checkout 的硬墙,
|
|
比 3DS 还前置。本层留接口缝,未实测前调用直接抛「未实现」,不要写猜测的提交逻辑。
|
|
|
|
**httpx 不能用于带账号的写操作**:Rakuten 对账号操作有 TLS/HTTP2 指纹校验,
|
|
同一份 cookie Playwright 能用、httpx 不能。所以本模块全程使用 Playwright
|
|
的 BrowserContext + APIRequestContext(共享 cookie,绕过 CORS)。
|
|
|
|
**并发约束**:所有公开方法入口 acquire `self._lock`,保证 HTTP 路由与 worker
|
|
主循环不会同时操作同一个 Playwright context——同一账号必须串行
|
|
(per project://jp-rakuten/trading-split「全局并发度 1」)。
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import json
|
|
import logging
|
|
import re
|
|
from dataclasses import dataclass
|
|
from typing import TYPE_CHECKING
|
|
|
|
from app.shared.errors import (
|
|
CartOperationError,
|
|
InvalidRequestError,
|
|
NotLoggedInError,
|
|
)
|
|
from app.shared.purchase_contract import (
|
|
INVENTORY_FLAG_DEFAULT,
|
|
INVENTORY_FLAG_MULTIPLE,
|
|
base_form_fields,
|
|
basket_domain_of,
|
|
inventory_flag_for,
|
|
)
|
|
from app.trading.core import auth_site
|
|
from app.trading.worker.models import LeaseTask
|
|
|
|
if TYPE_CHECKING:
|
|
from app.shared.config import Settings
|
|
from app.trading.services.auth_session import AuthSession
|
|
|
|
logger = logging.getLogger(__name__)
|
|
|
|
# 普通购买事件标识、inventory_flag 映射、basketDomain 反转义、form_fields 基础构造
|
|
# 均在 app.shared.purchase_contract,与 scraping/parsers/item.py::_purchase_info 共用同一份契约
|
|
_BASKET_PATH = "/rms/mall/bss/cartadd/set"
|
|
|
|
# 购物车页(SP)与 cart 数量 JSONP API(探针实测)
|
|
_CART_PAGE = auth_site.RAKUTEN_CART_URL
|
|
_CART_COUNT_API = "https://cart-api.step.rakuten.co.jp/rms/mall/cart/count/all/jsonp/"
|
|
|
|
# 购物车页未登录标记(旧 marker;新 SPA 上不可靠,这里只作辅助判据)
|
|
_LEGACY_LOGGED_OUT_MARKER = auth_site.RAKUTEN_LOGGED_OUT_MARKER
|
|
|
|
# 删除按钮稳定 selector(探针回报:.probe/checkout/06-cart-with-items.html 4 处命中)
|
|
# aria-label 是 a11y 属性,比 CSS modules hash class 稳定;Rakuten 不用 data-testid
|
|
_DELETE_BUTTON_SELECTOR = 'button[aria-label="削除"]'
|
|
|
|
# clear_cart 安全上限:防止 SPA 异常时死循环
|
|
_CLEAR_CART_MAX_ITER = 50
|
|
|
|
# 确认 modal 候选 selector:点击「削除」后站点可能弹「本当に削除しますか?」
|
|
_CONFIRM_BUTTON_SELECTORS = ('button:has-text("はい")', 'button:has-text("OK")')
|
|
|
|
_NOT_IMPLEMENTED_MSG = (
|
|
"站点交互未实现:见 docs/order-gateway.md §10 与 "
|
|
"project://jp-rakuten/checkout-flow-probe-findings。"
|
|
"session upgrade 是 checkout 阶段的硬墙,未实测前不写猜测的提交逻辑。"
|
|
)
|
|
|
|
|
|
@dataclass(slots=True)
|
|
class CheckoutSummary:
|
|
"""下单确认页解析结果(待实测确认字段位置)
|
|
|
|
payable_yen 用于金额守卫;其他字段在实测确认后补全。
|
|
"""
|
|
|
|
payable_yen: int
|
|
site_order_id: str | None = None
|
|
pay_deadline: str | None = None
|
|
|
|
|
|
class SiteInteractor:
|
|
"""Rakuten 站点交互器:持有 Playwright 浏览器 context,复用账号 cookie
|
|
|
|
生命周期:
|
|
- start() 在 trading 服务 lifespan 启动时调用一次:启动 Playwright + 创建带 cookie 的 context
|
|
- add_to_cart / verify_cart / cart_status / clear_cart / remove_item 在任务或 HTTP 请求里调用
|
|
- close() 在服务关闭时调用
|
|
|
|
浏览器 context 复用同一份登录态,所有任务串行(self._lock + worker 主循环本就串行),
|
|
不需要为每个任务开新 context——开销大且 cookie 状态会乱。
|
|
|
|
调用方:
|
|
- worker runner:传入 LeaseTask,调 add_to_cart(task) / verify_cart(task)
|
|
- HTTP 路由 /api/cart/*:调 add_to_cart_payload(...) / cart_status() /
|
|
clear_cart() / remove_item(item_id)
|
|
"""
|
|
|
|
# 每任务保留的临时状态:task_id → {"item_id": str, "shop_bid": str}
|
|
# 用于 add_to_cart 把抓出来的 item_id / shop_bid 喂给 verify_cart
|
|
_per_task_state: dict[str, dict[str, str]]
|
|
|
|
# 所有公开方法共用一把锁:HTTP 接口与 worker 主循环都走它,串行化所有 Playwright 操作。
|
|
# 同一账号被并发操作 = 风控触发风险 + cart 状态错乱。
|
|
_lock: asyncio.Lock
|
|
|
|
def __init__(self, *, auth_session: "AuthSession", settings: "Settings"):
|
|
self._auth_session = auth_session
|
|
self._settings = settings
|
|
self._playwright = None
|
|
self._browser = None
|
|
self._context = None # playwright BrowserContext
|
|
self._per_task_state = {}
|
|
self._lock = asyncio.Lock()
|
|
# 上次 build context 时读到的 storage_state 文件 mtime。
|
|
# 用于在任务间检测 AuthSession 重登后产生的新 storage_state,触发 context 重建。
|
|
self._state_mtime: float | None = None
|
|
|
|
# ---- 生命周期 ----
|
|
|
|
async def start(self) -> None:
|
|
"""启动 Playwright 与带 cookie 的浏览器 context
|
|
|
|
登录态文件不存在时同样启动(context 没 cookie),后续 add_to_cart 会
|
|
在 require_logged_in 里报错。这样保持启动路径一致。
|
|
"""
|
|
from playwright.async_api import async_playwright
|
|
|
|
state_path = self._settings.auth_state_path / auth_site.profile("rakuten").state_filename
|
|
storage_state = str(state_path) if state_path.exists() else None
|
|
if storage_state is None:
|
|
logger.warning(
|
|
"登录态文件不存在:site_interactor 以无 cookie 状态启动,"
|
|
"加购请求会被站点拒认"
|
|
)
|
|
else:
|
|
self._state_mtime = state_path.stat().st_mtime
|
|
|
|
self._playwright = await async_playwright().start()
|
|
self._browser = await self._playwright.chromium.launch(
|
|
headless=True,
|
|
channel=self._settings.browser_channel or None,
|
|
args=["--no-first-run", "--disable-blink-features=AutomationControlled"],
|
|
)
|
|
self._context = await self._browser.new_context(
|
|
storage_state=storage_state,
|
|
user_agent=auth_site.RAKUTEN_USER_AGENT,
|
|
locale="ja-JP",
|
|
timezone_id="Asia/Tokyo",
|
|
viewport={"width": 390, "height": 844},
|
|
is_mobile=True,
|
|
has_touch=True,
|
|
)
|
|
logger.info("SiteInteractor 已就绪:storage_state=%s", storage_state or "(none)")
|
|
|
|
async def _refresh_context_if_stale(self) -> None:
|
|
"""检查 storage_state 文件 mtime,变化则重建 context
|
|
|
|
AuthSession.try_relogin 成功后会重写 storage_state 文件。本 context 启动时
|
|
用快照式 storage_state 创建,cookie 不会自动同步——必须关掉旧 context、
|
|
用新文件重建。在 add_to_cart / verify_cart 开头各调一次,开销可接受
|
|
(只在 mtime 变了才重建)。
|
|
"""
|
|
state_path = self._settings.auth_state_path / auth_site.profile("rakuten").state_filename
|
|
if not state_path.exists():
|
|
return
|
|
mtime = state_path.stat().st_mtime
|
|
if mtime == self._state_mtime:
|
|
return
|
|
|
|
logger.info(
|
|
"storage_state 文件变化(mtime %s → %s),重建 context",
|
|
self._state_mtime, mtime,
|
|
)
|
|
if self._context is not None:
|
|
try:
|
|
await self._context.close()
|
|
except Exception:
|
|
logger.debug("关闭旧 context 失败", exc_info=True)
|
|
self._context = await self._browser.new_context(
|
|
storage_state=str(state_path),
|
|
user_agent=auth_site.RAKUTEN_USER_AGENT,
|
|
locale="ja-JP",
|
|
timezone_id="Asia/Tokyo",
|
|
viewport={"width": 390, "height": 844},
|
|
is_mobile=True,
|
|
has_touch=True,
|
|
)
|
|
self._state_mtime = mtime
|
|
|
|
async def close(self) -> None:
|
|
"""关闭 context、browser、playwright,吞掉单个 close 异常"""
|
|
for resource, name in (
|
|
(self._context, "context"),
|
|
(self._browser, "browser"),
|
|
(self._playwright, "playwright"),
|
|
):
|
|
if resource is None:
|
|
continue
|
|
try:
|
|
closer = resource.close() if name != "playwright" else resource.stop()
|
|
await closer
|
|
except Exception:
|
|
logger.debug("关闭 %s 失败", name, exc_info=True)
|
|
self._context = None
|
|
self._browser = None
|
|
self._playwright = None
|
|
|
|
# ---- 已实现:add_to_cart / verify_cart / cart_status ----
|
|
|
|
async def add_to_cart(self, task: LeaseTask) -> None:
|
|
"""加购(worker 入口):从 task.intent 取字段,调 _add_to_cart_with_fields
|
|
|
|
调用方需在 task.intent 提供:
|
|
- item_url: 商品详情页 URL(必填)
|
|
- quantity: 数量,默认 1
|
|
- variant_id: 多规格商品的 variant_id;不传则从 sku.variants[] 自动选第一个非售罄
|
|
- choice: 必填选项的取值列表;不传则每个必填选项用第一个候选值(站点不严格校验)
|
|
|
|
Raises:
|
|
InvalidRequestError: intent.item_url 缺失
|
|
NotLoggedInError: 登录态失效
|
|
CartOperationError: 商品页打不开、state 解析失败、商品不可购买、加购返回错误页
|
|
"""
|
|
intent = task.intent or {}
|
|
item_url = intent.get("item_url")
|
|
if not item_url:
|
|
raise InvalidRequestError("intent.item_url 必填")
|
|
quantity = int(intent.get("quantity") or 1)
|
|
if quantity <= 0:
|
|
raise InvalidRequestError(f"intent.quantity 必须为正整数,收到 {quantity}")
|
|
|
|
async with self._lock:
|
|
result = await self._add_to_cart_with_fields(
|
|
item_url=item_url,
|
|
quantity=quantity,
|
|
variant_id=intent.get("variant_id"),
|
|
choice=intent.get("choice"),
|
|
)
|
|
self._per_task_state[task.task_id] = {
|
|
"item_id": result["item_id"],
|
|
"shop_bid": result["shop_bid"],
|
|
"basket_domain": result["basket_domain"],
|
|
}
|
|
|
|
async def add_to_cart_payload(
|
|
self,
|
|
*,
|
|
item_url: str,
|
|
quantity: int = 1,
|
|
variant_id: str | None = None,
|
|
choice: str | list[str] | None = None,
|
|
) -> dict:
|
|
"""加购(HTTP 入口):返回加购结果与最新 cart count,不写 _per_task_state
|
|
|
|
与 add_to_cart(task) 共享 _add_to_cart_with_fields,差异仅在:
|
|
- 入参形态(关键字 vs intent dict)
|
|
- 返回值(dict vs None,结果记在 _per_task_state)
|
|
- 不带 task_id(HTTP 调用方自己持有结果)
|
|
"""
|
|
if not item_url:
|
|
raise InvalidRequestError("item_url 必填")
|
|
if quantity <= 0:
|
|
raise InvalidRequestError(f"quantity 必须为正整数,收到 {quantity}")
|
|
async with self._lock:
|
|
return await self._add_to_cart_with_fields(
|
|
item_url=item_url,
|
|
quantity=quantity,
|
|
variant_id=variant_id,
|
|
choice=choice,
|
|
)
|
|
|
|
async def _add_to_cart_with_fields(
|
|
self,
|
|
*,
|
|
item_url: str,
|
|
quantity: int,
|
|
variant_id: str | None,
|
|
choice: str | list[str] | None,
|
|
) -> dict:
|
|
"""加购核心逻辑(不持锁,由调用方包裹 self._lock)
|
|
|
|
返回 dict:{item_id, shop_bid, basket_domain, cart_count}
|
|
cart_count 在加购成功后顺带查一次 cart count API,方便 HTTP 调用方一次性返回。
|
|
"""
|
|
await self._auth_session.require_logged_in("rakuten")
|
|
await self._refresh_context_if_stale()
|
|
|
|
intent_for_extract: dict = {}
|
|
if variant_id is not None:
|
|
intent_for_extract["variant_id"] = variant_id
|
|
if choice is not None:
|
|
intent_for_extract["choice"] = choice
|
|
|
|
page = await self._context.new_page()
|
|
try:
|
|
try:
|
|
await page.goto(item_url, wait_until="domcontentloaded", timeout=30_000)
|
|
await page.wait_for_function(
|
|
"() => window.__INITIAL_STATE__ && window.__INITIAL_STATE__.purchase",
|
|
timeout=10_000,
|
|
)
|
|
except Exception as exc:
|
|
raise CartOperationError(
|
|
f"打开商品页失败或反爬被触发:{type(exc).__name__}: {exc}"
|
|
) from exc
|
|
|
|
html = await page.content()
|
|
state = _parse_initial_state(html)
|
|
if not state:
|
|
raise CartOperationError("无法从商品页抽 __INITIAL_STATE__(可能 PC 模板或反爬)")
|
|
|
|
fields = _extract_purchase_fields(state, intent_override=intent_for_extract)
|
|
|
|
if fields["purchase_condition"] != "enabled":
|
|
raise CartOperationError(
|
|
f"商品不可购买:purchaseCondition={fields['purchase_condition']}"
|
|
)
|
|
if not fields["basket_domain"]:
|
|
raise CartOperationError("basketDomain 为空(商品可能下架)")
|
|
# 多规格商品要求选了 variant_id
|
|
if fields["inventory_flag"] == INVENTORY_FLAG_MULTIPLE and not fields["form_fields"].get("variant_id"):
|
|
raise CartOperationError(
|
|
"多规格商品未选 variant,且 sku.variants 全部售罄或为空"
|
|
)
|
|
# 必填选项要求填了 choice
|
|
if fields["has_required_options"] and not fields["form_fields"].get(fields["options_field"]):
|
|
raise CartOperationError(
|
|
"商品有必填选项但未提供 choice,且选项无候选值"
|
|
)
|
|
|
|
payload = dict(fields["form_fields"])
|
|
payload[fields["quantity_field"]] = str(quantity)
|
|
|
|
logger.info(
|
|
"加购请求:basket=%s payload=%s", fields["basket_domain"], payload,
|
|
)
|
|
|
|
resp = await self._context.request.post(
|
|
fields["basket_domain"],
|
|
form=payload,
|
|
max_redirects=5,
|
|
headers={
|
|
"Referer": item_url,
|
|
"Origin": "https://item.rakuten.co.jp",
|
|
},
|
|
)
|
|
final_url = str(resp.url)
|
|
if "/error" in final_url:
|
|
body = await resp.text()
|
|
msg = _extract_error_message(body) or f"错误页 {final_url}"
|
|
raise CartOperationError(f"加购失败:{msg}")
|
|
|
|
# 成功标志:URL 跳到 cart 且带 added_item 参数(探针实测的落地)
|
|
if "cart" not in final_url:
|
|
body = await resp.text()
|
|
raise CartOperationError(
|
|
f"加购响应异常:final_url={final_url} body_head={body[:200]!r}"
|
|
)
|
|
|
|
logger.info(
|
|
"加购成功:item_id=%s final=%s",
|
|
fields["form_fields"].get("item_id"), final_url,
|
|
)
|
|
finally:
|
|
await page.close()
|
|
|
|
# 加购成功后顺带查 cart count(best-effort:失败时返回 -1,不掩盖加购成功)
|
|
try:
|
|
_, cart_count = await self._query_cart_count()
|
|
except CartOperationError as exc:
|
|
logger.warning("加购后查 cart count 失败(不影响加购结果):%s", exc.message)
|
|
cart_count = -1
|
|
|
|
return {
|
|
"item_id": fields["form_fields"].get("item_id", ""),
|
|
"shop_bid": fields["form_fields"].get("shop_bid", ""),
|
|
"basket_domain": fields["basket_domain"],
|
|
"cart_count": cart_count,
|
|
}
|
|
|
|
async def verify_cart(self, task: LeaseTask) -> None:
|
|
"""校验购物车里有没有刚加的商品
|
|
|
|
策略(探针实测最稳的两步):
|
|
1. 打 cart count JSONP API:status=100 且 count>=1 才算「购物车非空」
|
|
2. 打开 cart 页等 SPA 渲染,在 HTML 里找 item_id
|
|
|
|
登录态失效抛 NotLoggedInError;找不到 item 抛 CartOperationError。
|
|
"""
|
|
async with self._lock:
|
|
await self._auth_session.require_logged_in("rakuten")
|
|
await self._refresh_context_if_stale()
|
|
|
|
per_task = self._per_task_state.get(task.task_id, {})
|
|
item_id = (task.intent or {}).get("item_id") or per_task.get("item_id")
|
|
if not item_id:
|
|
raise CartOperationError(
|
|
"无法确定 item_id:intent 未提供且 add_to_cart 未记录"
|
|
)
|
|
|
|
# 1. cart count API
|
|
_, count = await self._query_cart_count()
|
|
if count == 0:
|
|
raise CartOperationError("购物车为空,加购可能未生效")
|
|
logger.info("cart count=%s task_id=%s", count, task.task_id)
|
|
|
|
# 2. 渲染 cart 页确认 item_id 在里面
|
|
await self._verify_item_in_cart_html(item_id, label=f"task_id={task.task_id}")
|
|
|
|
async def cart_status(self) -> dict:
|
|
"""轻量查询购物车状态:调 cart count JSONP API,不渲染整页
|
|
|
|
返回 {logged_in, count, raw_status}。
|
|
count 是站点返回的购物车里商品总件数(含数量,非 SKU 数)。
|
|
raw_status 是站点的状态码字符串,"100" 表示正常。
|
|
"""
|
|
async with self._lock:
|
|
await self._auth_session.require_logged_in("rakuten")
|
|
await self._refresh_context_if_stale()
|
|
raw_status, count = await self._query_cart_count()
|
|
return {
|
|
"logged_in": True,
|
|
"count": count,
|
|
"raw_status": raw_status,
|
|
}
|
|
|
|
# ---- 已实现:clear_cart / remove_item(Playwright UI 点击)----
|
|
|
|
async def clear_cart(self) -> dict:
|
|
"""清空购物车:渲染 cart SPA → 反复点第一个「削除」按钮 → count API 校验
|
|
|
|
策略:每次循环都重新查 `button[aria-label="削除"]`,点完一个等 SPA 重渲染
|
|
再点下一个,避免索引漂移。最多 _CLEAR_CART_MAX_ITER 次防死循环。
|
|
|
|
未实测前已知边界:
|
|
- 确认 modal 不确定是否存在,按「先 try 找再 click,找不到就继续」处理
|
|
- 若 SPA 把按钮渲染在 iframe 里,selector 失败需实测后调整
|
|
- Rakuten cart item 卡片无 data-testid,本方法不依赖 DOM 结构定位
|
|
|
|
返回 {removed_count, cart_count};cart_count=-1 表示末尾 count API 调用失败。
|
|
"""
|
|
async with self._lock:
|
|
await self._auth_session.require_logged_in("rakuten")
|
|
await self._refresh_context_if_stale()
|
|
|
|
page = await self._context.new_page()
|
|
removed = 0
|
|
try:
|
|
await page.goto(_CART_PAGE, wait_until="domcontentloaded", timeout=30_000)
|
|
await self._wait_cart_rendered(page, label="clear_cart")
|
|
|
|
for i in range(_CLEAR_CART_MAX_ITER):
|
|
btn = page.locator(_DELETE_BUTTON_SELECTOR).first
|
|
try:
|
|
await btn.wait_for(state="visible", timeout=2_000)
|
|
except Exception:
|
|
logger.info("clear_cart:第 %s 次循环未找到删除按钮,结束", i + 1)
|
|
break
|
|
try:
|
|
await btn.click()
|
|
except Exception as exc:
|
|
logger.warning("clear_cart:点击删除按钮失败:%s", exc)
|
|
break
|
|
removed += 1
|
|
await self._handle_confirm_modal(page)
|
|
# 等 SPA 重新渲染:domcontentloaded 或 1s 兜底
|
|
try:
|
|
await page.wait_for_load_state("domcontentloaded", timeout=5_000)
|
|
except Exception:
|
|
await page.wait_for_timeout(1_000)
|
|
else:
|
|
logger.warning(
|
|
"clear_cart 触发安全上限 %s,可能有删除失败或 SPA 异常",
|
|
_CLEAR_CART_MAX_ITER,
|
|
)
|
|
finally:
|
|
await page.close()
|
|
|
|
# 末尾用 count API 校验
|
|
try:
|
|
_, cart_count = await self._query_cart_count()
|
|
except CartOperationError as exc:
|
|
logger.warning("clear_cart 后查 cart count 失败:%s", exc.message)
|
|
cart_count = -1
|
|
|
|
logger.info("clear_cart 完成:removed=%s cart_count=%s", removed, cart_count)
|
|
return {"removed_count": removed, "cart_count": cart_count}
|
|
|
|
async def remove_item(self, item_id: str) -> dict:
|
|
"""删除购物车里指定 item_id 的商品
|
|
|
|
策略:渲染 cart SPA → 在 DOM 里找 button[aria-label="削除"],向上 walk
|
|
parentElement 找 innerText 包含 item_id 的祖先 → click 那个按钮。
|
|
|
|
Rakuten cart item 卡片无稳定 data-* 属性,CSS modules hash class 易变,
|
|
只能靠「按钮祖先节点的 innerText 包含目标 item_id」做文本回溯定位。
|
|
item_id 在 Rakuten 是 8 位数字,正常页面其他位置误匹配概率低。
|
|
|
|
Raises:
|
|
InvalidRequestError: item_id 为空
|
|
CartOperationError: 购物车里找不到 item_id
|
|
NotLoggedInError: 登录态失效
|
|
|
|
返回 {removed, item_id}。removed=false 表示点击了但 SPA 没在末尾 HTML
|
|
里移除该 item_id(可能删除被站点静默拒绝)。
|
|
"""
|
|
if not item_id:
|
|
raise InvalidRequestError("item_id 必填")
|
|
async with self._lock:
|
|
await self._auth_session.require_logged_in("rakuten")
|
|
await self._refresh_context_if_stale()
|
|
|
|
page = await self._context.new_page()
|
|
try:
|
|
await page.goto(_CART_PAGE, wait_until="domcontentloaded", timeout=30_000)
|
|
await self._wait_cart_rendered(page, label=f"remove_item {item_id}")
|
|
|
|
# JS 在 DOM 里定位包含 item_id 的祖先节点的删除按钮,click 它
|
|
clicked = await page.evaluate(
|
|
"""(itemId) => {
|
|
const buttons = document.querySelectorAll('button[aria-label="削除"]');
|
|
for (const btn of buttons) {
|
|
let node = btn.parentElement;
|
|
for (let i = 0; i < 12 && node; i++) {
|
|
const text = node.innerText || "";
|
|
if (text.includes(itemId)) {
|
|
btn.click();
|
|
return true;
|
|
}
|
|
node = node.parentElement;
|
|
}
|
|
}
|
|
return false;
|
|
}""",
|
|
str(item_id),
|
|
)
|
|
if not clicked:
|
|
raise CartOperationError(
|
|
f"购物车里没有 item_id={item_id}(或 SPA 未渲染出来)"
|
|
)
|
|
|
|
await self._handle_confirm_modal(page)
|
|
try:
|
|
await page.wait_for_load_state("domcontentloaded", timeout=5_000)
|
|
except Exception:
|
|
await page.wait_for_timeout(1_000)
|
|
|
|
# 校验:item_id 不再出现在 cart HTML
|
|
html = await page.content()
|
|
if str(item_id) in html:
|
|
# SPA 可能还没刷新完,再等 2s 兜底
|
|
await page.wait_for_timeout(2_000)
|
|
html = await page.content()
|
|
removed = str(item_id) not in html
|
|
finally:
|
|
await page.close()
|
|
|
|
logger.info("remove_item 完成:item_id=%s removed=%s", item_id, removed)
|
|
return {"removed": removed, "item_id": str(item_id)}
|
|
|
|
# ---- 内部辅助:cart count API 与 cart 页渲染 ----
|
|
|
|
async def _query_cart_count(self) -> tuple[str, int]:
|
|
"""调 cart count JSONP API,返回 (raw_status, count)
|
|
|
|
解析失败、status 非 100 都抛 CartOperationError——这是站点在告诉我们
|
|
「请求被拒了」,常见原因是 Referer 错或 cookie 失效。
|
|
"""
|
|
resp = await self._context.request.get(
|
|
_CART_COUNT_API + "?sid=1010",
|
|
headers={"Referer": _CART_PAGE},
|
|
)
|
|
body = await resp.text()
|
|
status_match = re.search(r'"status"\s*:\s*"(\d+)"', body)
|
|
if not status_match:
|
|
raise CartOperationError(f"cart count 响应无法解析:{body[:200]!r}")
|
|
raw_status = status_match.group(1)
|
|
if raw_status != "100":
|
|
raise CartOperationError(
|
|
f"cart count API 异常:status={raw_status} body={body[:200]!r}"
|
|
)
|
|
count_match = re.search(r'"count"\s*:\s*"(\d+)"', body)
|
|
count = int(count_match.group(1)) if count_match and count_match.group(1) else 0
|
|
return raw_status, count
|
|
|
|
async def _verify_item_in_cart_html(self, item_id: str, *, label: str) -> None:
|
|
"""渲染 cart SPA,在 HTML 里找 item_id,确认商品确实进了购物车"""
|
|
page = await self._context.new_page()
|
|
try:
|
|
await page.goto(_CART_PAGE, wait_until="domcontentloaded", timeout=30_000)
|
|
await self._wait_cart_rendered(page, label=label)
|
|
|
|
html = await page.content()
|
|
# 旧 marker 出现一定是登录失效;新 SPA 不渲染 marker,所以这判据是单边的
|
|
if _LEGACY_LOGGED_OUT_MARKER in html:
|
|
raise NotLoggedInError(
|
|
site="rakuten",
|
|
detail="购物车页出现旧版未登录 marker",
|
|
)
|
|
if str(item_id) not in html:
|
|
raise CartOperationError(
|
|
f"购物车页未找到 item_id={item_id}(加购可能被服务端静默丢弃)"
|
|
)
|
|
logger.info("cart 校验通过:%s item_id=%s in cart HTML", label, item_id)
|
|
finally:
|
|
await page.close()
|
|
|
|
async def _wait_cart_rendered(self, page, *, label: str) -> None:
|
|
"""等 cart SPA 把商品列表渲染出来(shopUrlList 非空),最多 15s
|
|
|
|
失败时只记 warning 不抛——空购物车时 shopUrlList 本就为空,调用方根据
|
|
后续业务逻辑(看 HTML、看 count API)自行判断。
|
|
"""
|
|
try:
|
|
await page.wait_for_function(
|
|
"""() => {
|
|
const s = window.__INITIAL_STATE__;
|
|
return s && s.cart && Array.isArray(s.cart.shopUrlList) && s.cart.shopUrlList.length > 0;
|
|
}""",
|
|
timeout=15_000,
|
|
)
|
|
except Exception:
|
|
logger.warning(
|
|
"cart SPA 15s 内未渲染出 shopUrlList(可能购物车为空):%s", label,
|
|
)
|
|
|
|
async def _handle_confirm_modal(self, page) -> None:
|
|
"""点击「削除」后若弹出确认 modal,尝试找「はい」/「OK」按钮点击
|
|
|
|
站点是否弹 modal 未实测确认,按「先 try 找再 click,找不到就跳过」处理。
|
|
每个候选 selector 给 1.5s 等待,命中后立即返回。
|
|
"""
|
|
for sel in _CONFIRM_BUTTON_SELECTORS:
|
|
try:
|
|
btn = page.locator(sel).first
|
|
await btn.wait_for(state="visible", timeout=1_500)
|
|
await btn.click()
|
|
logger.info("点击确认 modal 按钮:%s", sel)
|
|
return
|
|
except Exception:
|
|
continue
|
|
|
|
# ---- 未实现:保留 NotImplementedError ----
|
|
|
|
async def enter_checkout(self, task: LeaseTask) -> str:
|
|
"""进入下单确认页。**未实现**:session upgrade 是硬墙"""
|
|
raise NotImplementedError(_NOT_IMPLEMENTED_MSG)
|
|
|
|
async def parse_checkout(self, html: str) -> CheckoutSummary:
|
|
"""从下单确认页解析应付金额、付款期限、订单号。**未实现**"""
|
|
raise NotImplementedError(_NOT_IMPLEMENTED_MSG)
|
|
|
|
async def submit_order(self, task: LeaseTask) -> str:
|
|
"""提交下单。**未实现**"""
|
|
raise NotImplementedError(_NOT_IMPLEMENTED_MSG)
|
|
|
|
async def pay(self, task: LeaseTask, site_order_id: str) -> None:
|
|
"""付款。**未实现**"""
|
|
raise NotImplementedError(_NOT_IMPLEMENTED_MSG)
|
|
|
|
async def monitor(self, task: LeaseTask, site_order_id: str) -> None:
|
|
"""付款后监控。**未实现**"""
|
|
raise NotImplementedError(_NOT_IMPLEMENTED_MSG)
|
|
|
|
|
|
# ---- 模块级辅助函数(纯函数,便于单测)----
|
|
|
|
|
|
def _parse_initial_state(html: str) -> dict | None:
|
|
"""从商品页 HTML 抽 window.__INITIAL_STATE__ 并解析为 dict"""
|
|
m = re.search(
|
|
r"window\.__INITIAL_STATE__\s*=\s*(.+?);\s*window\.",
|
|
html,
|
|
re.DOTALL,
|
|
)
|
|
if not m:
|
|
return None
|
|
try:
|
|
return json.loads(m.group(1))
|
|
except json.JSONDecodeError:
|
|
return None
|
|
|
|
|
|
def _extract_purchase_fields(state: dict, *, intent_override: dict | None) -> dict:
|
|
"""从 __INITIAL_STATE__ 抽加购所需字段
|
|
|
|
与 scraping/parsers/item.py::_purchase_info 共用基础字段构造
|
|
(app.shared.purchase_contract);本函数额外做:
|
|
- variant_id 自动选(多规格挑第一个非售罄;调用方覆盖优先)
|
|
- choice 自动填(必填选项拼「名:值」;调用方覆盖优先)
|
|
- 返回 basket_domain / min_price / purchase_condition / shop_name / item_name
|
|
等业务字段,便于日志与错误信息使用
|
|
"""
|
|
intent_override = intent_override or {}
|
|
purchase = state.get("purchase") or {}
|
|
sell_type = (purchase.get("sellType") or {}).get("normalPurchase") or {}
|
|
raw_sku = purchase.get("sku") or {}
|
|
item = state.get("item") or {}
|
|
shop = (state.get("shop") or {}).get("information") or {}
|
|
information = purchase.get("information") or {}
|
|
|
|
basket_domain = basket_domain_of(sell_type)
|
|
inventory_flag = inventory_flag_for(raw_sku.get("inventoryType"))
|
|
shop_id = shop.get("shopId")
|
|
item_id = item.get("itemId")
|
|
|
|
form_fields = base_form_fields(
|
|
shop_id=shop_id,
|
|
item_id=item_id,
|
|
inventory_flag=inventory_flag,
|
|
)
|
|
|
|
# variant_id 选择:调用方覆盖 > 多规格自动选第一个非售罄 > 单规格用 item.variantId
|
|
chosen_variant = None
|
|
if intent_override.get("variant_id"):
|
|
form_fields["variant_id"] = str(intent_override["variant_id"])
|
|
elif inventory_flag == INVENTORY_FLAG_MULTIPLE:
|
|
for v in raw_sku.get("variants") or []:
|
|
if not v.get("isSoldOut"):
|
|
chosen_variant = v
|
|
break
|
|
if chosen_variant is None and (raw_sku.get("variants") or []):
|
|
chosen_variant = raw_sku["variants"][0]
|
|
if chosen_variant:
|
|
form_fields["variant_id"] = str(
|
|
chosen_variant.get("variantId") or chosen_variant.get("id") or ""
|
|
)
|
|
elif inventory_flag == INVENTORY_FLAG_DEFAULT and item.get("variantId"):
|
|
form_fields["variant_id"] = str(item.get("variantId"))
|
|
|
|
# 必填选项:调用方覆盖 > 自动填第一个候选值
|
|
options = information.get("options") or []
|
|
required_options = [o for o in options if o.get("isRequired")]
|
|
has_required = bool(required_options)
|
|
if intent_override.get("choice"):
|
|
# 调用方给的可能是 list 或 str
|
|
c = intent_override["choice"]
|
|
form_fields["choice"] = ",".join(c) if isinstance(c, list) else str(c)
|
|
elif has_required:
|
|
pairs: list[str] = []
|
|
for opt in required_options:
|
|
values = opt.get("values") or []
|
|
if values:
|
|
pairs.append(f"{opt.get('name')}:{values[0].get('name')}")
|
|
if pairs:
|
|
form_fields["choice"] = ",".join(pairs)
|
|
|
|
return {
|
|
"basket_domain": basket_domain,
|
|
"form_fields": form_fields,
|
|
"quantity_field": "units",
|
|
"variant_field": "variant_id",
|
|
"options_field": "choice" if options else "",
|
|
"options": options,
|
|
"has_required_options": has_required,
|
|
"inventory_flag": inventory_flag,
|
|
"purchase_condition": sell_type.get("purchaseCondition"),
|
|
"min_price": sell_type.get("minPrice"),
|
|
"shop_name": shop.get("shopName"),
|
|
"item_name": item.get("itemName"),
|
|
}
|
|
|
|
|
|
def _extract_error_message(body: str) -> str:
|
|
"""从 Rakuten 错误页 HTML 抽可读的提示文案"""
|
|
msgs: list[str] = []
|
|
for m in re.findall(r">([^<>]{20,200})<", body):
|
|
s = m.strip()
|
|
# 过滤 CSS/JS 标识与版权之类
|
|
if not s or any(c in s for c in ["(", ")", "=", "{", "}", "."]):
|
|
continue
|
|
if "Rakuten Group" in s or "SSL" in s:
|
|
continue
|
|
msgs.append(s)
|
|
# 取前两条拼起来(实测错误页一般 1~2 条核心提示)
|
|
return " / ".join(msgs[:2])
|