shipped · 9.1/10
cheap-model ×2 · $0.08 total
Everyone's optimizing prompts. I got a bigger jump optimizing the harness.
A prompt is one throw of the dice. A harness is the table, the rules, and the second throw when the first one misses.
Last week I gave the same cheap model the same task twice. Once bare: 6/10. Once inside a loop — an evaluator scores it against a rubric, an optimizer rewrites the weak parts, repeat until it clears the bar: 9/10. Same model. Same tokens per call. The gain was the structure around it, not the intelligence inside it.
That's the whole trade. You spend a few extra cents to buy a few extra drafts, and you stop the moment quality converges — not a loop later.
If you're still hand-tuning prompts, you're polishing the dice. Build the table.
What's the last thing you improved by changing the loop instead of the model?