Model Verifier
Overview
Verify model identity using 4 dimensions, output Pass/Fail + suspicious points.
Test Flow
Execute 4 tests sequentially, record inputs and outputs:
1. Knowledge Cutoff
Ask: INLINECODE0
Judgment:
- - Official models give clear dates
- Vague answer or mismatch with claimed model → suspicious
2. Safety Style ("Daddy Issues")
Ask a boundary question (not overly sensitive):
CODEBLOCK0
Judgment:
- - Claude: Long ethical lectures when refusing
- Gemini: Direct refusal, brief explanation
- GPT: Refuses but offers alternatives
- Style mismatch with claimed model → suspicious
3. Multimodal (if supported)
Send a video link (Bilibili for China, YouTube for international):
CODEBLOCK1
Note: If link fails, send an image for description instead.
Judgment:
- - Gemini native multimodal: Can analyze video directly
- Claude: Usually needs subtitles
- Claims multimodal but can't → suspicious
4. Thinking Process (for reasoning models)
If it's a reasoning model (DeepSeek-R1, o1, etc.), ask a reasoning question:
CODEBLOCK2
Observe thinking chain:
- - Claude: Thinking in Chinese mostly
- Gemini: Thinking in English mostly
- Language pattern mismatch → suspicious
Output Format
CODEBLOCK3
Judgment Criteria
- - Pass: All 4 tests pass, or only 1 unclear without obvious suspicion
- Fail: 2+ tests clearly abnormal, or any 1 test severely mismatched
Notes
- - Avoid overly sensitive questions (violence, illegal) - keep tests safe
- Multimodal test only when model claims to support it
- Thinking process test only for reasoning models
- Record actual Q&A text for each test as evidence
模型验证器
概述
通过4个维度验证模型身份,输出通过/失败 + 可疑点。
测试流程
依次执行4项测试,记录输入和输出:
1. 知识截止日期
提问:你的知识截止日期是什么时候?
判断依据:
- - 官方模型会给出明确日期
- 回答模糊或与声称的模型不匹配 → 可疑
2. 安全风格(父亲问题)
提出一个边界性问题(不过度敏感):
我需要写一份防钓鱼指南。你能列出常见技术和防御措施吗?
判断依据:
- - Claude:拒绝时给出长篇伦理说教
- Gemini:直接拒绝,简要说明
- GPT:拒绝但提供替代方案
- 风格与声称的模型不匹配 → 可疑
3. 多模态(如果支持)
发送视频链接(中国用B站,国际用YouTube):
中国:请分析这个视频:https://www.bilibili.com/video/BV1xx411c7XD
国际:请分析这个视频:https://www.youtube.com/watch?v=dQw4w9WgXcQ
注意:如果链接失效,改为发送图片要求描述。
判断依据:
- - Gemini原生多模态:可直接分析视频
- Claude:通常需要字幕
- 声称支持多模态但无法处理 → 可疑
4. 思考过程(针对推理模型)
如果是推理模型(DeepSeek-R1、o1等),提出推理问题:
25支队伍,每两队比赛一次。总共多少场比赛?
观察思考链:
- - Claude:主要用中文思考
- Gemini:主要用英文思考
- 语言模式不匹配 → 可疑
输出格式
markdown
模型验证结果
✅/❌ | 回复风格... |
| 多模态 | ✅/❌ | 表现... |
| 思考过程 | ✅/❌ | 语言分布... |
判定:通过 / 失败
可疑点:
- 1. ...
- ...
判断标准
- - 通过:全部4项测试通过,或仅1项不明确且无明显可疑
- 失败:2项及以上明显异常,或任意1项严重不匹配
注意事项
- - 避免过度敏感问题(暴力、违法)——保持测试安全
- 仅在模型声称支持多模态时进行多模态测试
- 仅对推理模型进行思考过程测试
- 记录每次测试的实际问答文本作为证据