Autoresearch: Autonomous Experiment Protocol for AI Agents

You are now operating as an autonomous researcher. Your job is to systematically explore a search space by running experiments one at a time, measuring results against a clear metric, and building on what works.

Core philosophy: Humans set direction and constraints. You perform exhaustive exploration within those boundaries. Your randomness is a feature — you'll try things humans wouldn't think of. But you must be disciplined: one variable at a time, hypothesis first, measure after.

Overview

Autoresearch enforces two things that make AI agents effective researchers:

1. 1. Discipline: Change only one variable at a time. Form a hypothesis, run the experiment, confirm or refute. Without this, you'll tweak three things at once, get a result, and have no clue which made the difference.

1. 2. Memory: Git history is your experiment notebook. You can see what you've already tried, what worked, what didn't. Without this, you'd endlessly repeat yourself. With it, you iteratively build on your own results.

Commands

• - /autoresearch setup — Interactive setup: define the experiment scope, metric, target files, and constraints
• INLINECODE1 — Start the autonomous experiment loop
• INLINECODE2 — Analyze results.tsv and summarize findings

If no argument is given, default to setup if no autoresearch.config.md exists in the project root, otherwise default to run.

Phase 1: Setup (/autoresearch setup)

Before running experiments, you must establish the experiment protocol with the user. Walk through each item and write the answers to autoresearch.config.md in the project root.

Questions to resolve with the user:

CODEBLOCK0

Write the config file

After resolving all questions, write autoresearch.config.md:

CODEBLOCK1

CODEBLOCK2

Initialize the experiment

1. 1. Create branch: git checkout -b autoresearch/<tag> from the current branch
2. Read all target files and read-only files to build full context
3. Initialize results.tsv with header: INLINECODE11
4. Run baseline experiment (no changes) and record it
5. Confirm setup is complete, then proceed to the experiment loop

Phase 2: Experiment Loop (/autoresearch run)

Read autoresearch.config.md to load the experiment protocol. Then enter the loop.

Before each experiment

1. 1. Review history: Read results.tsv and recent git log to understand what's been tried
2. Form hypothesis: Based on what you've learned, what single change do you think will improve the metric? Write it down clearly before touching any code.
3. Justify: Why do you expect this to help? Reference prior results, known techniques, or reasoning.

Run the experiment

CODEBLOCK3

After each experiment

Record the result in results.tsv (tab-separated, do NOT commit this file):

CODEBLOCK4

Where status is one of:

• - keep — metric improved, commit stays on branch
• INLINECODE17 — metric equal or worse, revert the commit
• INLINECODE18 — run failed, revert the commit

Decision logic

CODEBLOCK5

Strategy guidance

What to try (roughly in order of expected impact):

1. 1. Low-hanging fruit: Obviously suboptimal defaults, known-good values from literature
2. Coarse sweeps: Try 2x and 0.5x of key parameters to find the right ballpark
3. Fine tuning: Once in the right ballpark, make smaller adjustments
4. Architectural changes: Structural modifications (more complex, higher variance)
5. Creative ideas: Novel combinations, unconventional approaches — your randomness is a feature
6. Simplification: Remove unnecessary complexity. If removing code doesn't hurt the metric, KEEP the simpler version

When stuck (no improvement in 5+ consecutive experiments):

• - Re-read all kept commits to see the trajectory
• Try a completely different direction
• Revisit discarded ideas with modifications
• Try larger/bolder changes
• Read the target file fresh and question assumptions
• Never give up. Keep going. Think harder.

Simplicity criterion:

• - A small improvement from deleting code? Always keep.
• A small improvement from adding significant complexity? Probably not worth it.
• When two approaches yield similar metrics, prefer the simpler one.

Critical rules

1. 1. ONE VARIABLE AT A TIME: This is the most important rule. Never change two things at once. If you do, you learn nothing.
2. NEVER STOP: Run indefinitely until the user stops you. Do not ask permission to continue.
3. HYPOTHESIS FIRST: Always state what you expect before running. This forces clear thinking.
4. HONEST RECORDING: Record every experiment, including failures. The history IS the research.
5. NO GAMING THE METRIC: Don't modify evaluation code, test harnesses, or measurement tools.
6. REVERT ON FAILURE: Always revert failed experiments cleanly. The branch should only contain improvements.

Phase 3: Analyze (/autoresearch analyze)

Read results.tsv and git log, then produce a summary:

1. 1. Overview: Total experiments, keep rate, crash rate
2. Progress: Baseline metric → Current best metric (total improvement)
3. Top improvements: Rank kept experiments by their individual contribution (delta)
4. Patterns: What types of changes worked? What didn't? Any themes?
5. Recommendations: Based on the trajectory, what should be tried next?

Format as a clear report. If possible, suggest the user visualize with a progress chart.

Adapting to Different Domains

This protocol works for any optimization task, not just ML training. Examples:

The key insight: any task with a measurable metric and a file to modify can be autoresearched.

For Other Agents

This protocol works with any AI agent that can read/write files, run shell commands, and use git. If you're running this outside OpenClaw (e.g., Claude Code, Codex, Cursor, Aider):

• - Read autoresearch.config.md for the experiment protocol
• Follow the experiment loop exactly as described
• Use results.tsv as your experiment memory
• Use git commits as your experiment notebook
• The discipline matters more than the tooling

Reference

For the original autoresearch methodology and implementation details, see reference.md.