LLM Narrative Engines, Part 5: Integration Testing and Behavior Freezing

Before reading this : I'd recommend skimming Part 3's "Parser" section and Part 4's summary to understand how the parser outputs a domain.Contract . This post assumes you already know the parser can turn .meph into a struct. I. A Narrative Engine's Fourth Problem: How Do You Keep Behavior Stable? The parser is written. But it's code that gets maintained long-term — requirements change, formats expand, bugs get fixed. Every change risks breaking existing behavior. The tension here is: creators depend on stable behavior, while developers depend on freedom to change. If every code change requires manually testing every known scenario, the developer will fear refactoring. If you don't test, broken behavior reaches the creator — but the creator doesn't care that you refactored the parser. The solution is to "freeze" parsing behavior: use a fixed set of contracts as watchdogs. After every change, automatically compare parse results against expectations. This is what integration tests do: take a fixed set of .meph contracts as "watchdogs," run them after every change, and verify that behavior hasn't been accidentally altered. II. Golden File Testing: Freezing Parse Results The most straightforward approach: prepare a standard contract, parse it, serialize the result to JSON, and save it. On every subsequent test run, compare the current parse result against that JSON file. The project's testdata/sample.meph is that standard contract. Here's the test flow: func TestParseSample ( t * testing . T ) { got , err := ParseFile ( "testdata/sample.meph" ) if err != nil { t . Fatalf ( "parse failed: %v" , err ) } goldenPath := "testdata/sample.golden" var want domain . Contract if err := loadGolden ( goldenPath , & want ); err != nil { // Golden file doesn't exist — generate it automatically saveGolden ( goldenPath , got ) t . Log ( "Golden file generated. Please review and re-run the test." ) t . FailNow () } // Compare got and want if diff := cmp . Diff ( want , got ); diff != ""