This commit is contained in:
耿伦伦
2021-11-30 16:41:23 +08:00
parent febaf8599a
commit c5a38f00ae
10 changed files with 98 additions and 20 deletions

View File

@ -1,19 +1,26 @@
**抖音直播间(web)弹幕抓取**

抖音web直播间([live.douyin.com](https://live.douyin.com))弹幕抓取
--
**屏幕效果截图**
![enter image description here](https://github.com/gll19920817/tiktok_live/blob/main/WX20211129-144919@2x.png?raw=true)
> Pre Requirements
- Python3
- Charles
1. `git clone https://github.com/gll19920817/tiktok_live`
2. `pip install -r requirements.txt`
3. `Open Charles > Tools > Mirror > Mirrors Setting`
- `Enable Mirror`
- `Save to a folder, eg:/Users/douyin/feeds/`
- `Add location: https://live.douyin.com/webcast/im/fetch/`
4. `change main.py Watcher directory parameter to folder that Step 3 choose, eg: /Users/douyin/feeds/`
5. `python3 main.py`
**项目思路**
Final thoughts :
1. ~~Save data to mongodb~~
2. Charles alternative: maybe mitmproxy & scapy
1. Selenium 无窗口且detach模式打开live直播间
2. mitmproxy 捕获live.douyin.com http请求并保存响应为指定目录下文件
3. watchdog监控步骤2指定目录下文件变化后反序列化文件(application/protobuf格式)
4. 反序列化弹幕消息分类后terminal输出且入库(mongodb)
**使用方法**
1. 安装[python3](https://www.python.org/downloads/)
2. clone本项目,terminal进入项目目录,执行 `pip install requirements.txt`
3. 安装[mitmproxy](https://mitmproxy.org/) terminal执行`mitmproxy -s scripts/mitiproxy.py` (scripts/mitiproxy.py见项目)
4. terminal执行 `python3 main.py`
**注意事项**
1. 本源代码改动仅可作学习目的!!!
2. emm... 🤔