Introduction Every post in this series has quietly touched a piece of the same problem. Building Agentic Workflows in Python said a tool's input is untrusted and must be validated before it reaches your code. Building Reliable LLM Applications in Python said the model will confidently invent facts, so ground it and get typed output instead of parsing prose. Neither post named the thing underneath both statements: anything that crosses from outside your code into the model, or from the model back into your code, is untrusted input — a request body from the network, not a trusted internal value. This post names that boundary directly and gathers the defenses in one place — prompt injection (direct and indirect), input validation, output validation, and PII redaction — with the SAFE pattern shown beside every unsafe one it replaces, since this is the security-forward capstone of the series. The Trust Boundary: Three Kinds of Untrusted Input An LLM application has three places where untrusted text enters: User input — anything a person types, uploads, or submits through an API. Retrieved content — Making RAG Accurate in Python built a pipeline that ranks and returns chunks from a document store; those chunks were written by whoever authored the source document, not by you, and a malicious or compromised document can carry text aimed at the model reading it, not at a human reader. Model output — untrusted the moment it's about to be used rather than displayed : passed to a tool, interpolated into a query, or fed into another LLM call as context. A model that just read attacker-controlled retrieved text can be manipulated into producing attacker-controlled output. The single rule under all three: text is data until your code has explicitly decided it's safe to use for anything more than display. Nothing below is executed against a live API — every snippet is illustrative, and none of it uses a real key or a real record. Direct Prompt Injection: Defending the System Prompt

Introduction Every post in this series has quietly touched a piece of the same problem. Building Agentic Workflows in Java said toolUse.input() is untrusted and must be validated before it reaches your code. Building Reliable LLM Applications in Java said the model will confidently invent facts, so ground it and get typed output instead of parsing prose. Neither post named the thing underneath both statements: anything that crosses from outside your code into the model, or from the model back into your code, is untrusted input — a request body from the network, not a trusted internal value. This post names that boundary directly and gathers the defenses in one place — prompt injection (direct and indirect), input validation, output validation, and PII redaction — with the SAFE pattern shown beside every unsafe one it replaces, since this is the security-forward capstone of the series. The Trust Boundary: Three Kinds of Untrusted Input An LLM application has three places where untrusted text enters: User input — anything a person types, uploads, or submits through an API. Retrieved content — Making RAG Accurate in Java built a pipeline that ranks and returns chunks from a document store; those chunks were written by whoever authored the source document, not by you, and a malicious or compromised document can carry text aimed at the model reading it, not at a human reader. Model output — untrusted the moment it's about to be used rather than displayed : passed to a tool, interpolated into a query, or fed into another LLM call as context. A model that just read attacker-controlled retrieved text can be manipulated into producing attacker-controlled output. The single rule under all three: text is data until your code has explicitly decided it's safe to use for anything more than display. Nothing below is executed against a live API — every snippet is illustrative, and none of it uses a real key or a real record. Direct Prompt Injection: Defending the System Prompt A di

Introduction https://pg-blogs.netlify.app/posts/10-building-reliable-llm-apps-in-python/ closed with a section on picking the right model per task and caching a shared prefix. That was the entry point into a bigger discipline: LLM spend is an engineering variable, not a fixed bill — one you can measure and reduce with the same rigor you'd apply to query latency or memory footprint. This post goes deeper on four levers: how input/output pricing actually works and why the prefix is usually where the money goes, the exact cache_control shape and how to prove a cache hit instead of assuming one, the Batches API for work that isn't latency-sensitive, and model routing — a cheap model triaging requests and escalating only the hard ones. The throughline is honest: measure before you optimize. Every lever here has its own cost; misapplied, it makes things slower or pricier, not cheaper. Token Economics: Why the Prefix Is the Bill LLM providers price input and output tokens separately, and output always costs more — generation is autoregressive (each token depends on every one before it), while input can be processed in parallel. Representative pricing from the current model catalog: Model Input Output Claude Opus 4.8 $5.00 / MTok $25.00 / MTok Claude Sonnet 5 $3.00 / MTok $15.00 / MTok Claude Haiku 4.5 $1.00 / MTok $5.00 / MTok Two things follow: A long system prompt, tool list, or RAG context is billed as input on every request , not written once. Send a 20K-token system prompt on 10,000 requests and that's 200M input tokens — at Opus 4.8 rates, $1,000 before the model has generated a single output token. The shared prefix , not the user's actual question, is usually the dominant cost. Verbose output costs twice — once directly (more output tokens billed at the higher rate), and again because the next turn's history carries that verbosity forward as input. Asking for concise output and setting a sane max_tokens is a cost control, not just a style choice. This is why the tw

Introduction We already covered picking the right model tier for the task and caching a large shared prefix in https://pg-blogs.netlify.app/posts/11-building-reliable-llm-apps-in-java/ . Those two lines were the tip of a bigger discipline: LLM cost is not a fixed line item, it's an engineering variable — one you can measure and shrink with the same rigor you'd apply to database query time or container memory. This post goes deeper: how input/output pricing actually works, the exact cache_control shape and how to prove a cache hit rather than assume one, the Batches API for work that isn't latency-sensitive, and model routing — using a cheap model to triage, escalating only the hard cases to a stronger one. The honest framing throughout: measure before you optimize. Every technique here has a cost of its own; applied to the wrong workload, "optimization" makes things slower or more expensive. Token Economics: Why the Prefix Is the Bill Anthropic (like every hosted LLM provider) prices input and output tokens separately, and output is always pricier — the model has to generate output autoregressively, one token informed by all the ones before it, while input can be processed in parallel. Representative pricing from the current model catalog: Model Input Output Claude Opus 4.8 $5.00 / MTok $25.00 / MTok Claude Sonnet 5 $3.00 / MTok $15.00 / MTok Claude Haiku 4.5 $1.00 / MTok $5.00 / MTok Two consequences follow directly: Long system prompts, tool definitions, and RAG context are read on every request , not written once. A 20K-token system prompt sent on every one of 10,000 requests is 200M input tokens — at Opus 4.8 rates, $1,000 before a single output token is generated. The shared prefix , not the user's question, is usually where the money goes. A verbose model wastes money twice — once on the extra output tokens themselves, and again because the next turn's messages history now carries that verbosity forward as input on every subsequent call. Trimming max_tokens an

Introduction Building Reliable LLM Applications in Python put it plainly: treat model output as a hypothesis to verify, not a fact to trust. Testing Best Practices in Python put the same discipline in pytest terms: a suite only earns trust by asserting the right things at the right level, unhappy paths included. This post is where those two ideas meet — a pytest assertion either passes or fails against a fixed expected value; an LLM's output is a paragraph of prose that might be right in spirit while differing token-for-token from anything you wrote down in advance. Evaluating it takes a harness, not an assert . That harness has three parts: a golden dataset of representative cases with known-good expected behavior, scoring that turns each case into a pass/fail or a number, and regression testing that runs the harness on every change and fails the build when the score drops. Making RAG Accurate in Python already gave you half of this story — recall@k, precision@k, MRR, nDCG measure whether retrieval found the right chunks. This post measures the other half: whether the generated answer built from those chunks is actually good, which is a genuinely different question a retrieval metric can't answer on its own. Everything below is illustrative, non-executed Python, grounded in the same Anthropic SDK shapes as posts 10/11. The Golden Dataset: Curating Cases, Not Just Inputs A golden dataset is a small, hand-curated set of (input, expected behavior) pairs that represents the ways your application is actually used — not a random sample, and not just the cases that already work. Each case needs enough structure to be scored automatically later: from dataclasses import dataclass , field @dataclass class EvalCase : id : str category : str # "extraction", "qa", "summarization", ... input : str # the prompt/question sent to the system under test expected_exact : str | None = None # non-None only for cases scorable by exact match must_contain : list [ str ] = field ( default_f

Introduction Building Reliable LLM Applications in Java put it plainly: treat model output as a hypothesis to verify, not a fact to trust. Testing Best Practices in Java put the same discipline in JUnit terms: a suite only earns trust by asserting the right things at the right level, unhappy paths included. This post is where those two ideas meet — a JUnit test either passes or fails against a fixed expected value; an LLM's output is a paragraph of prose that might be right in spirit while differing token-for-token from anything you wrote down in advance. Evaluating it takes a harness, not an assertEquals . That harness has three parts: a golden dataset of representative cases with known-good expected behavior, scoring that turns each case into a pass/fail or a number, and regression testing that runs the harness on every change and fails the build when the score drops. Making RAG Accurate in Java already gave you half of this story — recall@k, precision@k, MRR, nDCG measure whether retrieval found the right chunks. This post measures the other half: whether the generated answer built from those chunks is actually good, which is a genuinely different question a retrieval metric can't answer on its own. Everything below is illustrative, non-executed Java, grounded in the same Anthropic Java SDK shapes as posts 10/11. The Golden Dataset: Curating Cases, Not Just Inputs A golden dataset is a small, hand-curated set of (input, expected behavior) pairs that represents the ways your application is actually used — not a random sample, and not just the cases that already work. Each case needs enough structure to be scored automatically later: public record EvalCase ( String id , String category , // "extraction", "qa", "summarization", ... String input , // the prompt/question sent to the system under test String expectedExact , // non-null only for cases scorable by exact/programmatic match List < String > mustContain , // key facts a correct answer must mention (programma

Introduction Every agent needs tools, and every tool needs a way to reach the model. Building Agentic Workflows in Python built that connection by hand — a hand-written JSON schema, a loop that dispatches on block.name . LLM Frameworks vs. the Raw SDK in Python showed LangChain's @tool turning a plain function into that same schema via bind_tools . Both are still bespoke : the tool lives inside one process, wired to one agent, in one language. The Model Context Protocol (MCP) solves a different problem: it standardizes the wire format between an AI application and a tool server, so the server doesn't have to be rewritten per agent, per framework, or per language. This post covers what that buys you, builds a minimal MCP server and a client that consumes it — both on the official Python SDK — and gives an honest answer to when reaching for a protocol is worth it over a direct tool call. The Problem MCP Solves Without a shared protocol, every pairing of agent framework and tool needs its own glue code: a LangChain @tool wrapper, a hand-rolled schema for the raw SDK, a different wrapper again for whatever framework a teammate picks next — an integration per framework, per tool. That's an M×N problem. MCP flattens it to M+N. A server exposes tools, resources, and prompts once, over a standard JSON-RPC protocol. Any host application — Claude Code, Claude Desktop, VS Code, or your own agent — creates an MCP client that speaks that same protocol, regardless of which framework built the host. Write the server once; every MCP-aware host can use it without new integration code. The protocol itself is intentionally boring: JSON-RPC 2.0 messages for lifecycle negotiation, tool discovery, and tool execution. Discovery ( tools/list ) and execution ( tools/call ) are the two calls that matter for this post: // tools/list response (abbreviated) { "jsonrpc" : "2.0" , "id" : 2 , "result" : { "tools" : [ { "name" : "get_account_balance" , "description" : "Look up the balance for an ac

Introduction Every agent needs tools, and every tool needs a way to reach the model. Building Agentic Workflows in Java built that connection by hand — a hand-written Tool schema, a loop that dispatches on toolUse.name() . LLM Frameworks vs. the Raw SDK in Java showed LangChain4j and Spring AI turning an annotated Java method into that same schema via reflection. Both are still bespoke : the tool lives inside one process, wired to one agent, in one language. The Model Context Protocol (MCP) solves a different problem: it standardizes the wire format between an AI application and a tool server, so the server doesn't have to be rewritten per agent, per framework, or per language. This post covers what that buys you, builds a minimal MCP server and a client that consumes it — both on the official Java SDK — and gives an honest answer to when reaching for a protocol is worth it over a direct tool call. The Problem MCP Solves Without a shared protocol, every pairing of agent framework and tool needs its own glue code: a LangChain4j tool wrapper, a Spring AI @Tool method, a hand-rolled schema for the raw SDK — three integrations for one capability, repeated for every tool and every framework you add. That's an M×N integration problem. MCP flattens it to M+N. A server exposes tools, resources, and prompts once, over a standard JSON-RPC protocol. Any host application — Claude Code, Claude Desktop, VS Code, or your own agent — creates an MCP client that speaks that same protocol, regardless of which framework built the host. Write the server once; every MCP-aware host can use it without new integration code. The protocol itself is intentionally boring: JSON-RPC 2.0 messages for lifecycle negotiation, tool discovery, and tool execution. Discovery ( tools/list ) and execution ( tools/call ) are the two calls that matter for this post: // tools/list response (abbreviated) { "jsonrpc" : "2.0" , "id" : 2 , "result" : { "tools" : [ { "name" : "get_account_balance" , "description"

There was a period in the early aughts when Definitive Jux (nee: Def Jux) seemed like it was going to be the future of hip hop. While the label featured plenty of experimental, boundary-pushing, and politically minded acts, Lif stood out as the most "conscious rapper" in the traditional sense. It was clear though, that […]

They were annoying to use and bad 3D movies didn't help.

I created this app to accurately calculate light windows like Golden hour, blue hour, etc. for a particular city/location. It helps you plan your photography time by knowing the exact time and direction of sun. It uses swiss ephemeris on the backend to calculate all the path and timings for most accurate data. It also has a 3D view of the sun's path on any day and at any point of the day for the location. Looking at the sun's direction and angle can help you plan your shoot correctly. It is a si

submitted by /u/craigkerstiens [link] [留言]