OpenReview Review Analyzer
Fetch all public peer reviews for any paper on OpenReview and generate a structured synthesis report.
When to Use
- - User provides an OpenReview URL (e.g.,
https://openreview.net/forum?id=XXXXX) - User asks to analyze or summarize reviews for a conference paper
- User mentions a paper's OpenReview forum ID
- User wants to understand reviewer consensus, disagreements, or key concerns
- User asks about scores, ratings, or review content for any venue on OpenReview
Workflow
Step 1: Extract Forum ID
Parse the OpenReview URL or forum ID from user input. The forum ID is the id parameter in the URL:
- -
https://openreview.net/forum?id=xxxxxxx → forum ID = INLINECODE3
Step 2: Fetch Reviews via Script
Run the Python script to fetch all reviews and metadata:
CODEBLOCK0
The script has zero external dependencies — it uses Python's built-in urllib. If requests is installed it will use that instead, but it's not required.
The script outputs a JSON file at /tmp/openreview_<forum_id>.json containing:
- - Paper metadata (title, authors, abstract, venue, keywords)
- All official reviews with ratings, confidence, strengths, weaknesses, questions, and full review text
- All official comments (author responses, reviewer discussions)
- Meta-review if available
If the script fails (e.g., network restrictions, reviews not public, paper withdrawn), use these fallback methods in order:
Fallback 1 — web_fetch the API directly:
web_fetch https://api2.openreview.net/notes?forum=<forum_id>
Parse the JSON response to get the submission, then:
web_fetch https://api2.openreview.net/notes?forum=<forum_id>&trash=true
to get all replies including reviews. Filter replies where
invitations contains
Official_Review.
Fallback 2 — web_search for review content:
Search for "<forum_id>" review site:openreview.net or "<paper_title>" review <venue> to find discussions, blog posts, or cached review content.
Fallback 3 — inform the user:
If no review data is accessible, explain that reviews may not be public yet, or suggest the user check the OpenReview page directly.
Step 3: Generate Synthesis Report
Read the JSON output and produce a structured report following {baseDir}/references/report-template.md.
Key analysis points:
- 1. Score Distribution — list each reviewer's rating and confidence, compute average
- Consensus Points — identify strengths/weaknesses mentioned by multiple reviewers
- Key Disagreements — where reviewers diverge in opinion
- Critical Issues — weaknesses flagged as major by any reviewer
- Questions Raised — important unresolved questions
- Author Responses — summarize rebuttal if available, and whether reviewers updated scores
- Meta-Review — include AC recommendation if available
- Overall Assessment — synthesize into a clear verdict
Important Notes
- - OpenReview content is public for completed review cycles. Some venues keep reviews private until decisions are made.
- For withdrawn papers, reviews may or may not be visible depending on venue policy.
- Always attribute opinions to specific reviewers (e.g., "Reviewer 1 (rating: 5)") when citing specific claims.
- The script uses the OpenReview API v2 by default (for venues from 2024+) and falls back to API v1 for older venues.
- No authentication is needed for reading public reviews.
Output Language
Match the user's language. If the user writes in Chinese, output the report in Chinese. If in English, output in English.
OpenReview 审稿分析器
获取OpenReview上任意论文的所有公开同行评审，并生成结构化综合报告。
使用场景
- - 用户提供OpenReview链接（例如：https://openreview.net/forum?id=XXXXX）
- 用户要求分析或总结某篇会议论文的评审意见
- 用户提及某篇论文的OpenReview论坛ID
- 用户希望了解评审人的共识、分歧或关键关注点
- 用户询问OpenReview上任何会议/期刊的评分、评级或评审内容
工作流程
第一步：提取论坛ID
从用户输入中解析OpenReview链接或论坛ID。论坛ID是链接中的id参数：
- - https://openreview.net/forum?id=xxxxxxx → 论坛ID = xxxxxxx
第二步：通过脚本获取评审
运行Python脚本获取所有评审及元数据：
bash
python3 {baseDir}/scripts/fetchreviews.py id>
该脚本零外部依赖——仅使用Python内置的urllib库。如果已安装requests库则会优先使用，但并非必需。
脚本输出JSON文件至/tmp/openreviewid>.json，包含：
- - 论文元数据（标题、作者、摘要、会议/期刊、关键词）
- 所有正式评审（含评分、置信度、优点、缺点、问题及完整评审文本）
- 所有正式评论（作者回复、评审人讨论）
- 元评审（如有）
若脚本执行失败（例如网络限制、评审未公开、论文已撤回），按顺序使用以下备用方法：
备用方法1——直接通过web_fetch调用API：
webfetch https://api2.openreview.net/notes?forum=id>
解析JSON响应获取投稿信息，然后：
webfetch https://api2.openreview.net/notes?forum=id>&trash=true
获取所有回复（包括评审）。筛选invitations中包含Official_Review的回复。
备用方法2——通过web_search搜索评审内容：
搜索 review site:openreview.net或<论文标题> review <会议/期刊>，查找相关讨论、博客文章或缓存的评审内容。
备用方法3——告知用户：
若无法获取任何评审数据，说明评审可能尚未公开，或建议用户直接查看OpenReview页面。
第三步：生成综合报告
读取JSON输出，按照{baseDir}/references/report-template.md模板生成结构化报告。
关键分析要点：
- 1. 评分分布——列出每位评审人的评分和置信度，计算平均值
- 共识点——识别多位评审人共同提及的优点/缺点
- 主要分歧——评审人意见不一致之处
- 关键问题——任何评审人标记为重大的缺点
- 提出的问题——重要的未解决问题
- 作者回复——总结反驳意见（如有），以及评审人是否更新了评分
- 元评审——包含领域主席/程序委员会主席的建议（如有）
- 总体评估——综合得出明确结论
重要说明
- - OpenReview内容在完成评审周期后公开。部分会议/期刊在做出决定前保持评审不公开。
- 对于已撤回的论文，评审是否可见取决于会议/期刊政策。
- 引用具体观点时，务必注明具体评审人（例如：评审人1（评分：5））。
- 脚本默认使用OpenReview API v2（适用于2024年及以后的会议/期刊），对于较早的会议/期刊则回退至API v1。
- 阅读公开评审无需身份验证。
输出语言
与用户语言保持一致。若用户使用中文，则输出中文报告；若使用英文，则输出英文报告。
