Stop Begging Your LLM for Valid JSON: Self-Correcting Structured Output in Spring AI 2.0

Every developer who has worked with LLMs has been there. You ask the model for JSON. You describe the schema. You say "please only respond with valid JSON." And sometimes, it still breaks. Your application crashes because the model returned a string where you expected an integer. Or it wrapped the JSON in markdown code blocks. Or it omitted a required field. Spring AI 2.0 has a solution that treats this like a real engineering problem instead of a prayer. The Problem When you use structured output in Spring AI, the workflow goes like this: You define a Java type (a record, class, or enum) Spring AI generates a JSON schema from that type The schema gets appended to the prompt sent to the LLM The model returns a response Spring AI attempts to deserialize the response into your type This works well with frontier models like Claude and GPT-4. But smaller open-source models, like Llama 3.2 1B running locally via Ollama, fail more often. They might return null for a primitive field, omit required fields, or produce malformed JSON. When it fails, you get a deserialization exception. Your endpoint returns a 500 error. Spring AI provides no built-in recovery mechanism. The Old Approach: Hope Consider a conference talk submission system. Speakers submit messy, unstructured abstracts. You want to extract structured data: public record TalkSubmission ( String title , String abstractText , Level level , // BEGINNER, INTERMEDIATE, ADVANCED Track track , int duration , List < String > tags , String speakerHandle ) {} Here is what the basic typed response looks like: @PostMapping ( "/typed" ) public TalkSubmission typed ( @RequestBody String rawSubmission ) { return chatClient . prompt () . system ( systemPrompt ) . user ( spec -> spec . text ( "Extract the talk submission: {submission}" ) . param ( "submission" , rawSubmission )) . call () . entity ( TalkSubmission . class ); } You define your type. Spring AI generates the schema and appends it to the prompt. The model gets the in