Agent Learner

An AI toolkit for configuring, benchmarking, comparing, and optimizing agent prompts and evaluation results. Agent Learner provides persistent, file-based logging for each command category with timestamped entries, summary statistics, multi-format export, and full-text search across all records.

Commands

Command	Description
INLINECODE0	Configure agent settings — log configuration entries or view recent ones
INLINECODE1

Benchmark agent performance — log benchmark results or view history | | compare | Compare agent outputs — log comparison data or view recent comparisons | | prompt | Prompt management — log prompt variations or view recent prompts | | evaluate | Evaluate agent outputs — log evaluation results or view history | | fine-tune | Fine-tune parameters — log fine-tuning sessions or view recent ones | | analyze | Analyze agent behavior — log analysis entries or view recent analyses | | cost | Cost tracking — log cost data or view recent cost entries | | usage | Usage monitoring — log usage metrics or view recent usage data | | optimize | Optimize configurations — log optimization runs or view history | | test | Test agent behavior — log test results or view recent tests | | report | Report generation — log report entries or view recent reports | | stats | Show summary statistics across all log categories (entry counts, data size, first entry date) | | export <fmt> | Export all data in json, csv, or txt format to the data directory | | search <term> | Full-text search across all log files (case-insensitive) | | recent | Show the 20 most recent entries from the activity history log | | status | Health check — show version, data directory, total entries, disk usage, and last activity | | help | Show the full help message with all available commands | | version | Print the current version string |

Each data command (configure, benchmark, compare, etc.) works in two modes:

• - Without arguments: displays the 20 most recent entries from that category
• With arguments: saves the input as a new timestamped entry and reports the total count

Data Storage

All data is stored in plain text files under the data directory:

• - Category logs: $DATA_DIR/<command>.log — one file per command (e.g., configure.log, benchmark.log, prompt.log), each entry is INLINECODE23
• History log: $DATA_DIR/history.log — audit trail of every command executed with timestamps
• Export files: $DATA_DIR/export.<fmt> — generated by the export command in json, csv, or txt format

Default data directory: INLINECODE27

Requirements

• - Bash (with set -euo pipefail support)
• Standard Unix utilities: grep, cat, date, echo, wc, du, head, tail, INLINECODE37
• No external dependencies or API keys required

When to Use

1. 1. Benchmarking agent performance — When you need to track and compare benchmark results across different agent configurations, models, or prompt strategies
2. Prompt engineering iteration — When you're testing multiple prompt variations and want to log each version with results for later comparison
3. Cost and usage tracking — When you need to monitor API costs and usage metrics over time to optimize spending
4. Fine-tuning experiments — When running fine-tuning sessions and you want to log parameters, results, and observations for reproducibility
5. Cross-category analysis — When you need to search across all logged data (benchmarks, prompts, evaluations, costs) to find patterns or specific entries

Examples

CODEBLOCK0

Output

All commands return output to stdout. Export files are written to the data directory:

CODEBLOCK1

Every command execution is logged to $DATA_DIR/history.log for auditing purposes.

---

Powered by BytesAgain | bytesagain.com | hello@bytesagain.com