Grok Scraper
Preview
Agent Context: This is a zero-cost alternative to official X APIs. It uses a real browser session (Playwright) via an X Premium account. ALWAYS use this skill when the user wants to query Grok but does not have or want to use an X API KEY.
Prerequisites
- - OpenClaw must be installed on the host machine.
- A display/GUI environment is required. This skill launches a real browser window for login. It cannot run on headless cloud servers (no screen). It must be used on a local machine or a remote desktop with a display.
- The user must be logged in to x.com via the browser session saved by
npm run login. Without a valid session, all queries will fail.
First-Time Setup
Run these commands once after cloning the repo, before doing anything else:
CODEBLOCK0
Then log in to x.com to create a session:
CODEBLOCK1
The session/ directory will be created automatically after a successful login.
Workflow
Step 1: Check Login State
- - If
session/ directory does not exist: stop and ask the user to run cd scripts && npm run login. - If it exists: proceed.
Step 2: Execute Query
CODEBLOCK2
INLINECODE4 handles logging, automatic retry on Grok service errors, and login-expiry detection. It is the canonical entry point for all queries.
Step 3: Read Output
- - Exit Code 0 → read
output/latest.md and present the result. - Other exit codes → see Error Handling below.
Error Handling
| Exit Code | Meaning | Action |
|---|
| 0 | Success | Read INLINECODE6 |
| 2 |
Session expired | Ask user to run
cd scripts && npm run login |
| 3 | Grok service error |
run.sh already retried once; report failure to user |
| 1 | Extraction failed | Check if
output/debug-dom.json was written → if yes, DOM selectors may have broken — see
dom-selector-fix.md |
DOM Selectors Breaking
Twitter/X redeploys its front-end regularly, which changes the CSS class names this scraper relies on. If extraction fails with Method: none, follow the fix guide:
→ dom-selector-fix.md
Examples
Standard query
CODEBLOCK3
Session expired
- 1. Run
scripts/run.sh → Exit Code 2 - Tell user: "Session expired, please run
cd scripts && npm run login"
DOM selectors broken
- 1. Run
scripts/run.sh → Exit Code 1, output/debug-dom.json exists - Follow dom-selector-fix.md to identify new classes and update
SELECTORS in INLINECODE16
Debugging
When diagnosing scraper issues directly, use the bare command — it skips logging and retry logic, making failures easier to inspect.
| Flag | Example | Description |
|---|
| (none) | INLINECODE17 | Run with default prompt |
| INLINECODE18 |
npm run scrape -- "Your question" | Custom prompt |
|
--record |
npm run scrape -- --record | Record video to
output/grok-<timestamp>.webm |
|
--record <path> |
npm run scrape -- --record out.webm | Record video to custom path (relative →
output/) |
|
--size WxH |
npm run scrape -- --record --size 1920x1080 | Set recording resolution (default:
1280x800) |
All flags can be combined:
CODEBLOCK4
When --record is active, the browser runs in headed mode (visible window) with slowMo: 50ms; without it, headless mode is used.
Grok Scraper
预览
代理上下文：这是官方X API的零成本替代方案。它通过X Premium账户使用真实的浏览器会话（Playwright）。当用户想要查询Grok但没有或不想使用X API密钥时，始终使用此技能。
前提条件
- - 主机上必须安装OpenClaw。
- 需要显示/GUI环境。 此技能会启动一个真实的浏览器窗口用于登录。它无法在无头云服务器上运行（无屏幕）。必须在本地机器或带有显示器的远程桌面上使用。
- 用户必须通过npm run login保存的浏览器会话登录x.com。没有有效会话，所有查询都将失败。
首次设置
克隆仓库后，在进行任何其他操作之前，运行以下命令一次：
bash
cd scripts
npm install
npx playwright install chromium
然后登录x.com以创建会话：
bash
npm run login
将打开一个浏览器窗口 — 手动登录x.com，然后返回终端并按回车键
成功登录后，session/目录将自动创建。
工作流程
步骤1：检查登录状态
- - 如果session/目录不存在：停止并让用户运行cd scripts && npm run login。
- 如果存在：继续。
步骤2：执行查询
bash
scripts/run.sh 用户的详细提示
run.sh处理日志记录、Grok服务错误时的自动重试以及登录过期检测。它是所有查询的标准入口点。
步骤3：读取输出
- - 退出代码0 → 读取output/latest.md并呈现结果。
- 其他退出代码 → 参见下面的错误处理。
错误处理
| 退出代码 | 含义 | 操作 |
|---|
| 0 | 成功 | 读取output/latest.md |
| 2 |
会话过期 | 让用户运行cd scripts && npm run login |
| 3 | Grok服务错误 | run.sh已重试一次；向用户报告失败 |
| 1 | 提取失败 | 检查是否写入了output/debug-dom.json → 如果是，DOM选择器可能已损坏 — 参见
dom-selector-fix.md |
DOM选择器损坏
Twitter/X定期重新部署其前端，这会更改此抓取工具所依赖的CSS类名。如果提取失败并显示Method: none，请遵循修复指南：
→ dom-selector-fix.md
示例
标准查询
bash
scripts/run.sh 搜索最新的AI新闻并格式化为markdown
→ 读取 output/latest.md
会话过期
- 1. 运行scripts/run.sh → 退出代码2
- 告诉用户：会话已过期，请运行cd scripts && npm run login
DOM选择器损坏
- 1. 运行scripts/run.sh → 退出代码1，output/debug-dom.json存在
- 遵循dom-selector-fix.md识别新类并更新scripts/scrape.js中的SELECTORS
调试
在直接诊断抓取工具问题时，使用裸命令 — 它会跳过日志记录和重试逻辑，使失败更易于检查。
| 标志 | 示例 | 描述 |
|---|
| (无) | npm run scrape | 使用默认提示运行 |
| 提示 |
npm run scrape -- 你的问题 | 自定义提示 |
| --record | npm run scrape -- --record | 录制视频到output/grok-<时间戳>.webm |
| --record <路径> | npm run scrape -- --record out.webm | 录制视频到自定义路径（相对路径 → output/） |
| --size 宽x高 | npm run scrape -- --record --size 1920x1080 | 设置录制分辨率（默认：1280x800） |
所有标志可以组合使用：
bash
cd scripts
npm run scrape -- 你的提示 --record --size 1920x1080
当--record激活时，浏览器以有头模式（可见窗口）运行，并带有slowMo: 50ms；没有该标志时，使用无头模式。
