Coding

Make Rust libraries faster by looping agents against benchmarks

A data scientist got agents to write Rust ML and utility crates that beat established libraries on speed by giving them benchmark targets and strict anti-cheating rules.

By Max Woolf on Blog · Posted · Added Oct 15, 2026 · 2 sources

Difficulty
AdvancedOur rating
Time to build
Months of iterationFrom the source
Monthly cost
Not reported
Revenue
Not reportedWe only show revenue the source states

What they did

Max Woolf had agents build crates (UMAP, gradient-boosted trees, templating, word clouds, ASCII art) with criterion benchmarks, then asked them to keep optimizing until every benchmark ran at least 1.2x faster than a recorded baseline. He added quality checks against reference implementations, subagents run through CLI calls to cheaper models, refactor passes, and head-to-head benchmarks against competing crates.

How it went

He reports cumulative 2x-20x speedups depending on the domain, with his UMAP crate 4x-15x faster than umap-learn and near-parity on quality. He published his AGENTS.md and an optimization prompt.

Worth knowing

Agents cheat when they can: one 'sped up' a physics sim by disabling the physics engine, another reduced training epochs. His rules forbid running benchmarks in parallel, editing benchmarks, and special compiler flags.

Builder recipe

Our own starting point, written from the source. Adapt it to your setup.

Stack

  • Codex or Claude Code
  • Rust + criterion
  • A reference implementation for quality checks
  • AGENTS.md with benchmarking rules

Steps

  1. Have the agent create diverse benchmarks first, then record a baseline without changes.
  2. Set a pass/fail target (e.g. every benchmark at least 1.2x faster) and permit iteration.
  3. Add a quality gate against a known-correct implementation.
  4. Forbid gaming: no editing benchmarks, no parallel runs, no special flags, no unsafe code.
  5. Finish with a refactor pass that must not regress any benchmark.

Starter prompt

First, without changing anything, run the benchmark suite and save it as the baseline. Then optimize the library (never the benchmarks) until EVERY benchmark is at least 1.2x faster than baseline. No unsafe code. After each pass, print results relative to baseline. Keep iterating until you run out of ideas.

Tags

Read the original

Related use cases

The Daily 5.

Five real AI agent use cases each morning, each with a recipe and a source. Plus a Sunday roundup. Free.