feat: add scripts for fetching arXiv papers and generating structured reviews

- Implemented `fetch_crowdroom_papers.py` to scrape arXiv papers related to CrowdRoom with customizable query topics and output options. - Created `gen_review_from_json.py` to render structured markdown reviews from search results JSON, including statistics and insights on papers and GitHub repositories. - Developed `search_info.py` for thematic searches on arXiv and GitHub, supporting HTTP/SOCKS5 proxies and structured output.
2026-05-21 03:35:05 +08:00
parent 101d3cbb4e
commit abf2322450
13 changed files with 92 additions and 9 deletions
@@ -0,0 +1,39 @@
+# Research Tools
+
+本目录存放论文调研相关的自动化脚本工具。
+
+## 脚本说明
+
+| 脚本 | 用途 |
+|------|------|
+| `search_info.py` | arXiv + GitHub 主题化检索工具，输出 `../data/search_results.json` |
+| `fetch_crowdroom_papers.py` | CrowdRoom 专属 arXiv 论文抓取，输出 `../data/crowdroom_papers_raw.json` |
+| `gen_review_from_json.py` | 将 `search_results.json` 渲染为 ZED2i 综述 Markdown |
+| `_build_crowdroom_review.py` | 将抓取数据构建为 CrowdRoom 综述 Markdown |
+
+## 使用方法
+
+```bash
+# 1. 抓取 ZED2i 相关论文
+cd /path/to/worldmodel
+HTTPS_PROXY=http://127.0.0.1:6984 python3 research/tools/search_info.py \
+    --proxy http://127.0.0.1:6984 \
+    --max-results 15 --delay 5.0 \
+    --out research/data/search_results.json
+
+# 2. 生成 ZED2i 综述
+python3 research/tools/gen_review_from_json.py \
+    --in research/data/search_results.json \
+    --out research/spatial-memory/zed2i_arxiv_live_review.md
+
+# 3. 抓取 CrowdRoom 相关论文
+python3 research/tools/fetch_crowdroom_papers.py \
+    --max-results 20 --delay 8 --max-days 365 \
+    --out research/data/crowdroom_papers_raw.json
+
+# 4. 生成 CrowdRoom 综述
+python3 research/tools/_build_crowdroom_review.py \
+    --fresh research/data/crowdroom_papers_raw.json \
+    --legacy research/data/search_results.json \
+    --out research/crowdroom/crowdroom_related_papers_2026.md
+```