Part 7: Iterating to Green: Real Bugs, and When You'd Actually Reach for a Framework

Part 7 (final) of a series building a support-ticket agent with no framework. Previous: Part 6 (observability). Repo: github.com/akash-pal/agent-from-scratch The other six parts described the finished design. This one is about what "finished" actually took — the real bugs the eval set caught, and the two questions every agent build eventually has to answer honestly: do you need more than one agent, and do you need a framework. Full detail on everything below: docs/iteration-log.md . The iteration log, condensed 1. Exact trajectory matching was the wrong check. First eval run: 12/21 passed. Most failures were the agent correctly sending a confirmation email where the eval only expected a lookup — correct behavior, wrong assertion. Fix: switched the harness from exact-array equality to ordered-subsequence matching (every expected tool must appear, in order; extra steps in between are fine). Still catches a missing, reordered, or wrong tool. Stops false-failing on benign non-determinism. 2. No retry/backoff meant a transient error crashed the whole run. A 503 — model overloaded on case 1 took down the entire eval harness. Fixed with exponential backoff on 429 / 503 specifically, plus inter-case pacing to stay under free-tier rate limits. 3. A -latest model alias silently rolled onto a stricter quota. gemini-flash-latest worked, then started failing with a 20 requests/day cap after quietly resolving to a newer model. Fixed by pinning an explicit model version instead of an alias, after checking the provider's live usage dashboard for actual quota — 25x more headroom on the pinned model. The takeaway generalizes past this one provider: "latest" aliases optimize for capability, not quota stability, and what they resolve to changes over time without your code changing at all. 4. A testing artifact that looked like a real bug. Piping multiple answers into the interactive CLI via printf "a\nb\n" | npm run agent intermittently hung after the first prompt. Root cause: a Node.j