The day I asked three LLM agents to rewrite legacy Java for me — and what actually happened

1. The question that started everything Three weeks into my internship, my supervisor sat down across from me and asked, very casually: "OK your NLP pipeline extracts intentions and rules from legacy Java. Nice. And then what? " I looked at him. I looked at my laptop. I looked back at him. The whole project — Pulsar Modernizer — was supposed to eventually turn legacy Java into modern Spring Boot code. My part was the "understand the old code" part. F1 = 0.857 on the annotated corpus, a shiny React UI, everything humming in Docker. But the "and then?" was doing a lot of work in that sentence. That evening I wrote in my notes: "Nobody has actually tried the generation part. Everyone assumes it'll be easy because LLMs. That is very obviously wrong." So I decided to try. 2. Why "just prompt an LLM to rewrite it" doesn't work The naive move — feed the old code and the extracted rules to an LLM and say "please modernize this" — has three problems and I hit all of them in the first hour: The model hallucinates. It happily invents helper classes that don't exist and calls methods with the wrong signature. You have no criterion for stopping. The model tells you "it's done ". OK. Is it? By what test? You have no criterion for equivalence. Even if it compiles, how do you know the new code actually does what the old one did? I needed something more constrained than "prompt it and pray". 3. The setup — a chain, not a monolith I ended up building three specialized agents in sequence: IntentCard + RuleCards │ ▼ [APIDesigner] ──► JSON contract (class, methods, DTOs, throws) │ ├───────────────┐ ▼ ▼ [CodeGenerator] [TestGenerator] │ │ ▼ ▼ .java *Test.java │ │ └────► verifier (mvn test) The key insight: each rule extracted from the legacy code should become a test that the generated code has to pass. This flips the whole thing. I don't trust the LLM. I trust javac and JUnit. I did all of this on a local model — Qwen 2.5 Coder 3B via Ollama. No cloud APIs, no data leaving my Mac. On a