"""子站解析器注册表与商品页分派 乐天部分官方旗舰店的商品页会 302 跳出 item.rakuten.co.jp,落到各自独立的站点。 这里按落地域名把页面分派给对应解析器;落到未登记的站点时抛 OffIchibaRedirectError, 让上游能明确区分「站点不支持」与「被反爬拦截」,而不是白白重试。 """ from __future__ import annotations from urllib.parse import urlsplit from app.scraping.core import site from app.shared.errors import OffIchibaRedirectError, ScrapeParseError from app.scraping.models.scrape import ItemDetailData from app.scraping.parsers.state import PageValidator, require_state_marker from app.scraping.parsers.subsites import biccamera, books, brandavenue from app.scraping.parsers.subsites.base import SubsitePage, SubsiteParser SUBSITE_PARSERS: dict[str, SubsiteParser] = { module.HOST: SubsiteParser( host=module.HOST, source=module.SOURCE, shop_name=module.SHOP_NAME, validate=module.validate, parse=module.parse, ) for module in (books, brandavenue, biccamera) } def host_of(url: str) -> str: return urlsplit(url).hostname or "" def build_item_page_validator(requested_url: str) -> PageValidator: """构造商品页校验器:按落地域名选用对应的页面校验规则 落到未登记的站点时直接抛错——换 cookie 或上浏览器都改变不了页面归属。 """ def validate(html: str, final_url: str) -> str | None: host = host_of(final_url) if host == site.ITEM_HOST: return require_state_marker(html, final_url) parser = SUBSITE_PARSERS.get(host) if parser is None: raise OffIchibaRedirectError(requested_url, final_url) return parser.validate(html) return validate def parse_subsite_item(page: SubsitePage) -> ItemDetailData: """把子站页面交给对应解析器""" parser = SUBSITE_PARSERS.get(host_of(page.final_url)) if parser is None: raise ScrapeParseError(f"没有匹配的子站解析器:{page.final_url}") return parser.parse(page)