I Built a Hands-Free AI Harness for Migrating Legacy Tests

Migrating a large legacy test suite is not a code-generation problem. It is a confidence problem. An agent can translate an old test into modern syntax very quickly. The difficult question is whether it preserved the behavior-and how you can prove that across hundreds or thousands of tests without manually supervising every generated change. I built an agent harness to solve that problem. You point it at a specification, and it works through every test hands-free: it plans the work, writes the migration, explores the live application when needed, runs verification, records evidence, and moves on only when the evidence is complete. The model does the work inside the loop. The harness owns everything between the steps: scope, retries, verification, evidence, and the final decision. A model may propose a migration. It never gets to declare that migration complete. This was built for a real migration; the repo is a safe public version The original harness was built around a Ruby Watir-to-Playwright migration in a private codebase. For the public reference repository , I recreated the pattern using a bundled Protractor-to-Playwright example. The framework names differ; the design problem does not. In both cases, the legacy test is a useful behavioral specification. It tells us what a user does and what the application must prove afterwards. # The original shape: a legacy browser test describes behavior. browser . goto ( "/login" ) browser . text_field ( id: "username" ). set ( "demo" ) browser . button ( id: "login-submit" ). click expect ( browser . url ). to include ( "/tasks" ) The goal is not a prettier translation. The goal is a Playwright test that demonstrably preserves that behavior. Ground truth changes the agent problem For many AI tasks, there is no single right answer. You need an evaluation framework to judge whether an answer is good enough. Test migration is different. We already have ground truth: the old test defines the behavior to preserve; the running