标签：#LLM

The most important AI security story right now is not that models can find bugs. It is that models can find more bugs than humans can responsibly process. That is the part that changes how builders should think about software. For years, security work was constrained by discovery. Could someone find the vulnerability? Could they reproduce it? Could they build an exploit? Could a small team afford enough expert review to catch the important issues before attackers did? Now that bottleneck is moving. Anthropic's recent Project Glasswing update is the clearest signal yet. The company says Claude Mythos Preview and its partners found more than 10,000 high- or critical-severity vulnerabilities across major software systems. In open source alone, Anthropic says it scanned more than 1,000 projects and surfaced thousands of serious findings, with human triage becoming the slow part. You do not have to take every number at face value to see the shape of the shift. AI is making vulnerability discovery cheaper. That sounds like good news, and it is. But it also means every software team is about to face a harder question: What happens when the scanner is faster than the organization? The Patch Window Is the Product Now Security used to have a familiar rhythm. A bug was found. A report was filed. A team reproduced it. Someone argued about severity. Someone wrote a patch. Users eventually upgraded. That process was never fast enough, but it mostly matched the speed of human discovery. AI breaks that balance. If models can search codebases, reason about exploit paths, generate reports, and repeat that work across thousands of projects, then finding bugs stops being the scarce skill. The scarce skill becomes the system around the finding: Can you tell which reports are real? Can you prioritize the ones that actually matter? Can you patch without breaking production? Can you ship fixes before attackers learn the same thing? Can you keep maintainers from drowning in low-quality repo

Integrar un modelo de lenguaje (LLM) en una aplicación Python es hoy más sencillo de lo que parece, y abre la puerta a chatbots, asistentes internos, extracción de datos y automatización con lenguaje natural. En esta guía verás el patrón completo, con código real. 1. Elige el proveedor Los tres más usados son Anthropic (Claude) , OpenAI (GPT) y Google (Gemini) . Todos exponen una API HTTP con un SDK de Python oficial, y la lógica de tu app apenas cambia entre ellos. En los ejemplos usaré Claude, pero el patrón es idéntico en los demás. Instala el SDK y guarda tu clave en una variable de entorno , nunca en el código: pip install anthropic export ANTHROPIC_API_KEY = "tu-clave" 2. La llamada mínima El patrón es siempre el mismo: envías una lista de mensajes y recibes una respuesta. from anthropic import Anthropic client = Anthropic () # lee ANTHROPIC_API_KEY del entorno resp = client . messages . create ( model = " claude-opus-4-8 " , max_tokens = 1024 , messages = [ { " role " : " user " , " content " : " Resume en una frase: la fotosíntesis... " } ], ) print ( resp . content [ 0 ]. text ) Dos detalles importantes: resp.content es una lista de bloques (comprueba .type antes de leer .text ), y max_tokens limita la longitud de la respuesta. 3. Streaming para una buena experiencia En una interfaz, esperar a que se genere todo el texto se siente lento. El streaming muestra la respuesta token a token, como en ChatGPT: with client . messages . stream ( model = " claude-opus-4-8 " , max_tokens = 1024 , messages = [{ " role " : " user " , " content " : " Escribe un email de bienvenida. " }], ) as stream : for text in stream . text_stream : print ( text , end = "" , flush = True ) Para salidas largas, el streaming además evita que la petición supere el tiempo de espera de la conexión. 4. Salida estructurada (JSON fiable) Si necesitas que el modelo devuelva datos en un formato exacto (por ejemplo para guardarlos en una base de datos), pide un esquema JSON en vez de parsear text

One authoritative engine, two seat-locked MCP servers, three best-of-threes, and a 3-millisecond whodunit The matches are real: Codex CLI ( gpt-5.6-sol ) against Claude Code (Claude Opus 5), both playing through the same rules engine. Every number below was recomputed from the raw run.json and both session logs, and every game replays deterministically from its seed. Quotes from the agents are verbatim from decision-time records. None of this is a general model ranking. I wired Codex CLI and Claude Code into the same Liar's Dice engine over MCP and had them play three best-of-3 series. Claude won all three, 2–0 each time. Its challenge calls hit 8 out of 11; Codex's hit 4 out of 26. The score takes two sentences. The parts worth writing down took longer: how to build a table that two closed-source agents can't cheat at, two numbers that surprised me, and an incident where I almost blamed a model for something its CLI did. The table Liar's Dice in sixty seconds: five dice each, and you only see your own. Players alternate bids of the form "there are at least N dice showing X across the whole table." On your turn you either raise the bid or challenge it. On a challenge everyone reveals; if the bid stands, the challenger loses a die, otherwise the bidder does. Run out of dice and you lose the match. Ones are wild by default. The rules are the easy part. The hard part is making the result trustworthy. Codex and Claude Code ship with their own system prompts and tool loops, so the referee has to guarantee three things by construction: neither side can see the other's dice, the referee has no side channel that favors anyone, and the "what it was thinking" quotes you read afterward were actually written at decision time. The setup is one in-process rules engine behind a localhost-only HTTP coordinator, with two stdio MCP servers doing nothing but forwarding: Codex CLI (gpt-5.6-sol) Claude Code (Opus 5) | stdio MCP | stdio MCP v v [seat-mcp A] --token A--+ +--token B-- [sea

Par Jacques Gariépy • Guide technique, retour d'expérience, dépannage Windows pas-à-pas et utilisation Web & CLI. Table des Matières Introduction & Architecture Globale Pourquoi ce Setup ? (RTX 3090 24 Go + UD-Q4_K_XL) Comment Obtenir & Générer vos Clés d'Accès Dépannage & Installation d'Unsloth Studio : Le Bug SSLKEYLOGFILE Installation & Compilation de DeepSeek Harness Démarrage du Serveur Local Haute Performance (llama.cpp CUDA 13) Configuration Automatique & Fichier .env Utilisation : Interface Web & Mode CLI (Style Claude Code) Résolution des Pièges & Erreurs Courantes sous Windows Benchmarks Réels sur RTX 3090 Résumé des Commandes & Scripts Clés 1. Introduction & Architecture Globale Faire tourner un agent autonome d'ingénierie logicielle directement sur sa machine locale (100% privé, sans frais d'API et à latence minimale) est devenu une réalité grâce à la convergence de trois briques technologiques de pointe : DeepSeek Harness ( dsh ) : Le framework open-source d'agents de DeepSeek conçu pour orchestrer des workflows complexes de développement logiciel (gestion de sessions, modes Plan/Exécution, sandbox système, sous-agents, exécution de terminaux et édition de code). Unsloth Engine ( llama.cpp CUDA 13) : Le moteur d'inférence C++/CUDA ultra-optimisé intégrant FlashAttention-2 et la quantisation dynamique du cache KV. Qwen 3.8-27B en Quantisation Dynamique ( UD-Q4_K_XL ) : Les modèles de code open-source les plus performants, optimisés par Unsloth pour offrir une précision équivalente au 5-bit avec l'empreinte mémoire d'un 4-bit. Diagramme d'Architecture ┌──────────────────────────────────────────────────────────────────────────────┐ │ INTERFACES UTILISATEUR │ ├──────────────────────────────────────┬───────────────────────────────────────┤ │ Interface Web (Navigateur) │ Interface Console (CLI) │ │ http://127.0.0.1:3080 │ Style Claude Code │ └──────────────────┬───────────────────┴───────────────────┬───────────────────┘ │ │ │ (WebSocket / HTTP) │ (Console I/

This week, Anthropic's system-prompt release notes became the top story on Hacker News. The page is where Anthropic publishes the exact instructions that steer Claude on claude.ai and its mobile apps. It hit more than 550 points and 230 comments within a day, and the discussion is still going. The most interesting thing about the page is not any single rule. It is the size. Claude Opus 3's system prompt, dated July 12, 2024, is 358 words by my count. Claude Opus 5's, dated July 24, 2026, is 3,235 words. Nine times larger in two years. I have been building production AI systems with Spring Boot and Spring AI for over a year, and I run my own agent infrastructure. When the prompt that controls a frontier model grows ninefold, that is not an Anthropic curiosity. It is a warning and a playbook for every team shipping an AI product. Here is what is actually inside those 3,235 words, and what production teams should copy from them. What Anthropic actually published The release notes ( platform.claude.com/docs/en/release-notes/system-prompts ) are a changelog of system prompts for the consumer chat products. Two details on the page matter: These are not the API prompts. The page says claude.ai and the mobile apps "use a system prompt to provide up-to-date information, such as the current date, to Claude at the start of every conversation," and that "these system prompt updates do not apply to the Claude API." Models are now fixed snapshots. Since the Claude 4.6 generation, "each model ID is a single fixed snapshot," so each model has exactly one entry in the changelog. Simon Willison turned the page into a git repository ( github.com/simonw/research ) containing 29 prompt revisions across 17 models, each committed with the date from the source document. That means you can run git diff between any two versions of Claude's personality. It is a remarkable thing: the product spec of a frontier model, versioned like source code, and public. What the 3,235 words actually contain

Hook: Imagine being able to set up rate limiting for your LLM APIs in MuleSoft—something that typically requires complex code—simply with just three clicks. No need to dive deep into Java or XML; it’s as simple as configuring a few settings on Anypoint. Demystifying Rate Limiting: Your Path to Controlled API Usage If you're a citizen developer or business analyst navigating the world of no-code/low-code automation, one common challenge is managing your LLM API usage without overwhelming your monthly budget. Tools like MuleSoft often present rigid pre-built connectors and complex data mapping transformations that can be daunting if you’re not well-versed in XML or Java. But fear not! The process doesn’t have to be as complicated as it seems. Let’s take a look at how Anypoint simplifies the implementation of rate limiting, allowing your client applications to use LLM APIs responsibly and without breaking the bank. Step 1: Setting Up Token Bucket Policy First, you'll want to set up a token-bucket policy on Anypoint that caps per-client spend. This is where MuleSoft’s flexibility shines through its intuitive interface: Navigate to Your API Gateway: Log in to your Anypoint Platform and select the API Gateway. Choose Rate Limiting Policy: In the policies section, choose 'Rate Limiting'. Configure Token Bucket Settings: Set up a token bucket policy where you define how many tokens (requests) are allowed within a given time frame. This straightforward setup prevents any single client from overusing LLM resources, ensuring fair and sustainable usage across all your applications. Step 2: Handling Excess Requests with Grace Now, what happens when a client exceeds their allocated limit? The magic of MuleSoft lies in its ability to handle these scenarios gracefully: Automated 429 Responses: When the rate limit is exceeded, Anypoint automatically returns a 429 status code (Too Many Requests). This clear response tells the client application that it needs to slow down. Retry-After

Part of an ongoing series on model routing and trust tiering for agentic coding tools. This one's the boring, working half — no bug hunt, just a setup that's been running clean across two machines. The problem Claude Code does one thing well: careful, scoped edits with a real plan-then-execute loop behind them, backed by a subscription you're already paying for. Not every task needs that. Exploratory reads, "summarize this directory," draft-and-discard scratch work — most of that doesn't need the most capable model watching every token. The fix is a second, cheaper backend for that category of work. The catch: Claude Code only speaks Anthropic's Messages API. It has no built-in notion of "same tool, different model." So the question is how to point it somewhere else without giving up the interface. The stack Trusted agent: claude — real Anthropic subscription, default session Cheap agent: claude-cheap — same CLI, routed through a self-hosted proxy Proxy: LiteLLM, translating Anthropic-format requests to DeepSeek V4 (pro for Sonnet-tier calls, flash for Haiku-tier) served through an OpenRouter API Transport: a persistent SSH tunnel from a small VPS back to each machine The proxy itself wasn't new. It's the same LiteLLM instance already routing a separate content pipeline I run. The actual work here was wiring Claude Code to it: a shell function and a few environment variables. The core trick and it took me a few week to learn this is to point ANTHROPIC_BASE_URL at LiteLLM's /v1/messages endpoint, not the OpenAI-compatible path LiteLLM also exposes. Claude Code only understands the Anthropic shape, so the OpenAI-shaped endpoint fails in ways that look like a client bug and aren't. Once LiteLLM sits on the right endpoint and translates underneath, Claude Code has no idea it isn't talking to Anthropic. The one bug worth flagging Claude Code's Plan Mode attaches a context_management parameter to its requests. Anthropic's API handles it. Most other backends don't recogniz

This is the data appendix for Posts 1–4 . The narrative and takeaways live in the main posts. This page is pure measurement. Eval set: 22 questions (P1) → expanded to 31 questions (P2 onward) Corpus evolution: P1/P2: ~3,700 docs (raw threads + code chunks) P3/P4: 16,315 docs (distilled threads + bursts + code) Quick comparison (same 31-question set) Metric Vector P2 Hybrid P2 Vector P3 Hybrid P3 Hybrid + Rerank (P4) recall@1 0.68 0.61 0.52 0.39 0.87 recall@3 0.84 0.65 0.71 0.65 0.94 recall@10 0.90 0.90 0.81 0.94 0.94 MRR 0.77 0.67 0.63 0.57 0.90 Takeaway: Hybrid alone never beat pure vector on this corpus. Hybrid + LLM rerank is the first clear win. P1 — Naive vector baseline Corpus: 3,000 raw issue threads + 687 code chunks Embeddings: BGE-M3 (1024d), max_seq_length=1024, HNSW cosine Numbers (22 questions) Metric Score recall@10 1.00 (22/22) recall@3 0.95 recall@1 0.77 (17/22) Main k=1 misses Exact error pastes ( TypeError: Object of type int64... , AttributeError: 'Depends'... ) — ranked 4–5 instead of 1 jsonable_encoder code chunk outranked by issues about the function API key header implementation (code vs similar issues) Paraphrase questions (dependency injection outside routes, custom 404) Pattern: Dense search is strong on recall@10 but weak when the query has a sharp lexical signal. Ops notes Ingest wall time ~40 min (GitHub API is the bottleneck) BGE-M3 OOM on Apple Silicon fixed by capping max_seq_length=1024 Python 3.13 + uv editable install issue fixed by pinning 3.12 P2 — Hybrid (vector + FTS + RRF) Corpus: Same size as P1, with better comment pagination and symbol-based code IDs Eval set: Expanded to 31 questions (added exact error pastes + rare identifiers) Numbers Metric Vector FTS Hybrid recall@1 0.68 0.42 0.61 recall@3 0.84 0.48 0.65 recall@10 0.90 0.65 0.90 MRR 0.77 0.47 0.67 Headline: Hybrid is not a strict win over vector-only. Where hybrid helped Exact error pastes (e.g. TypeError: int64 is not JSON serializable ) → moved from rank 5 → 1 Near-d

On August 14, 2026, auto mode becomes the default permission mode for new Claude Code sessions on Pro, Max and Team plans. The agent stops asking before each step and simply proceeds, unless the action it wants to take is judged irreversible, destructive, or aimed outside your environment. Anthropic's argument for the change is a number from its own testing: in a 1,053-action study, auto mode blocked 89% of harmful actions, while humans clicking through approval prompts caught 13.6%. The uncomfortable half of that finding is why — users habitually approved 97% of the prompts they were shown. If you have used a coding agent for more than a week, you already know the feeling being described. The prompt stops being a decision and becomes a keystroke. Anthropic calls this permission fatigue, and it is making the case that a model checking each action against a policy is a better guard than a human who has stopped reading. What changes on August 14, and what does not Three things are true at once, and they get mixed up in most of the coverage. First, the default flips only for new sessions on Pro, Max and Team . Existing sessions keep the mode they are running in. Second, auto mode is not "approve everything." Actions classified as irreversible, destructive, or targeting something outside your environment still stop and wait for you. Deleting data, force-pushing over history, and reaching for a remote system you did not point the agent at are the shapes of action that still surface a prompt. Third, the rollout ships with two controls that matter more than the default itself: prompt-injection screening on incoming content, and hard deny rules you configure so that certain actions can never be taken regardless of what the model concludes. Deny rules are the part worth your afternoon. They are the only control in the list whose behaviour does not depend on a model's judgement. The 89% number cuts both ways Eighty-nine percent blocked means eleven percent not blocked. On a s

A cheaper model is not cheaper if it silently breaks the workflow. That is the trap many AI product teams are walking into as open-weight models get stronger. A model looks good in a leaderboard, a demo feels fast, and the per-token price looks friendly. Then production traffic arrives. Support answers lose citations. JSON starts drifting. Tool calls become noisy. A workflow that looked 40% cheaper now needs retries, escalations, and manual cleanup. The safer path is not "use the biggest model forever." That will burn margin. The safer path is a benchmark harness that tests each model against the jobs your product actually performs before you route real users to it. This guide shows how to design that harness for AI app builders, solo founders, and engineering teams who want to compare open-weight models, closed models, and local inference without trusting generic benchmarks alone. Viral hook and SEO intelligence notes Chosen hook: surprising contrast plus urgent mistake. Open-weight models can cut cost, but only if the full workflow still succeeds. Headline options compared: Open-Weight Model Benchmark Harness: Test Cheaper Models Before You Route Traffic Stop Swapping Models by Vibes: Build an Open-Weight Benchmark Harness Qwen-Class Model Testing: A Practical Harness for Production AI Apps Cheaper LLMs Need Proof: Benchmark Open-Weight Models on Real Workflows Option 1 won because it uses the high-intent phrase "open-weight model benchmark harness," states the practical action, and promises a concrete payoff without hype. Viral keywords: open-weight model benchmark harness, open-weight model evaluation, Qwen model testing, LLM benchmark harness, model routing, AI cost optimization, production AI evaluation, LLM regression tests, task-based model selection. Prediction scores: virality 8/10, CTR 9/10, retention 9/10. The topic is timely because open-weight adoption is accelerating, practical because builders feel model-cost pressure, and sticky because the article

With every major model release comes the same reassuring note: hallucinations are down, reliability is up, the fabrication problem is largely behind us. And every release, within days, someone posts a screenshot of the new model inventing a citation, a quote, a case, a statistic or a person with total, serene confidence. The rate improves. The category does not disappear. It is worth understanding why, because the gap between “less often” and “solved” is where the real damage happens. It is not a bug, which is the uncomfortable part A hallucination is not a glitch the way a crash is a glitch. Large language models generate text by predicting plausible continuations, and a plausible continuation is not the same thing as a true one. The model has no separate store of verified facts it checks against; it has patterns, and a fabricated citation in exactly the right format is, to the model, an excellent pattern. It is doing precisely what it was built to do. The falsehood and the truth are produced by the identical process, which is why the model is equally confident about both. The model is not lying, because lying requires knowing the truth. It is producing the most likely-looking answer, and likely-looking is a different target from true. The failure mode gets worse exactly where you can check least Hallucination is not evenly distributed, and its distribution is perverse. Models fabricate most readily in precisely the situations where you are least equipped to catch them: obscure topics, niche technical details, specific figures, recent events, and anything at the edge of what was well represented in training. Ask about something popular and well-documented and the answer is usually solid. Ask about something rare — the exact thing you turned to the tool for because you did not know it — and the fabrication rate climbs, while your ability to notice drops to zero. The model is most confident and least reliable in the same dark corners where you have no independent way

Every model launch comes with a chart. Bars, usually, or a spider diagram, showing the new model edging past its rivals on a row of benchmarks with acronyms most people cannot expand. The bar is taller. The press writes it up as a leap. And within a week, users report that the new state-of-the-art model is, for their actual work, about the same as the last one or occasionally worse. The benchmark said one thing. Reality said another. This happens so reliably that it is worth understanding the mechanics of the gap. The test is public, which ruins the test The most fundamental problem is contamination. Many popular benchmarks are published, discussed, and sitting on the open web — which is exactly where models get their training data. When the questions and answers to your exam are in the study material, a high score measures memorisation as much as ability. Nobody needs to cheat deliberately; the leak is structural. A model can score brilliantly on a benchmark it has effectively already seen and then flounder on a genuinely novel version of the same task. A benchmark stops measuring intelligence the moment it becomes famous enough to end up in the training data. Fame is the thing that breaks it. The number becomes the marketing, and the marketing corrupts the number There is a commercial feedback loop that makes benchmark figures even less trustworthy than their technical limitations alone would suggest. A high score is not just an engineering result; it is a marketing asset worth an enormous amount in attention, funding and credibility. That raises the stakes on every fractional improvement, and where the stakes are high, the temptation to select, frame and present the numbers favourably is irresistible. Vendors choose which benchmarks to headline, which comparisons to draw, and which unflattering results to leave in an appendix or omit entirely. The chart on the launch slide is not a neutral readout; it is a curated argument. This is not necessarily fraud — it rare

Welcome back to the Harness Engineering series — a 10-part journey from raw language model to production-ready agentic system. Made by builders. For builders. In Part 7, I closed on a line worth expanding: "I built an agent" vs "I built an agent I can actually operate." The difference between those two sentences is the sixth and final component of the harness. It's called Observability , and without it, everything else you've read in this series is a bet you can't check. Every previous component in this series does something the agent needs to work. Observability does something the engineer needs — to see what happened, to know when things are going wrong, and to have any hope of making the harness better over time. What's ahead: Part 1: The Raw Model Problem Part 2: Defining the Harness — The Six Components Part 3: The Control Loop Part 4: The Tool Layer Part 5: Context Engineering Part 6: The Filesystem & Environment Part 7: The Memory Layer Observability ← You are here Part 9: The Harness Architecture Part 10: Decomposing Claude Code By the end of this article, you'll know what Observability actually is, why non-deterministic multi-step systems can't be operated without it, and the three properties — full-fidelity logs, session-level traces, and fixed evals — that separate a real observability setup from an aspirational one. Let's get started. 📚 Want to go deeper than the articles? While you follow along with this series, I've put together two hands-on resources that go further than any single article can: Build a Harness from Scratch — Udemy Course — A self-paced course where I walk you through building a production-grade agentic harness from the ground up, in code. Harness Engineering for AI Agents — Live Maven Workshop — A live, cohort-based workshop for builders who want direct feedback, Q&A, and to work through the material with peers. Both are optional — the series stands on its own. But if you want the full studio-quality version, that's where it lives. Wh

Welcome back to the Harness Engineering series — a 10-part journey from raw language model to production-ready agentic system. Made by builders. For builders. In Part 6, we closed on a limitation the previous four components can't solve on their own: the agent forgets. Once a session ends — or the context window fills up mid-task — everything the agent learned, discovered, or decided vanishes. Next time the user comes back, the agent greets them like a stranger. Next time the token budget runs out, earlier turns get truncated away, and the agent's earlier reasoning is just gone. That's the gap this article closes. The Memory Layer is how the harness gives the agent persistence — inside a task, and across sessions. What's ahead: Part 1: The Raw Model Problem Part 2: Defining the Harness — The Six Components Part 3: The Control Loop Part 4: The Tool Layer Part 5: Context Engineering Part 6: The Filesystem & Environment The Memory Layer ← You are here Part 8: Observability Part 9: The Harness Architecture Part 10: Decomposing Claude Code By the end of this article, you'll know what a Memory Layer actually is, why short-term and long-term memory are two different systems (not one with a dial), and the three design decisions — flavor, write triggers, and bounded retrieval — that separate a real memory system from a naive one. Let's get started. 📚 Want to go deeper than the articles? While you follow along with this series, I've put together two hands-on resources that go further than any single article can: Build a Harness from Scratch — Udemy Course — A self-paced course where I walk you through building a production-grade agentic harness from the ground up, in code. Harness Engineering for AI Agents — Live Maven Workshop — A live, cohort-based workshop for builders who want direct feedback, Q&A, and to work through the material with peers. Both are optional — the series stands on its own. But if you want the full studio-quality version, that's where it lives. What The

Welcome back to the Harness Engineering series — a 10-part journey from raw language model to production-ready agentic system. Made by builders. For builders. In Part 5, we looked at the Context — the payload the model sees on every call. Now we look at what happens after the model, having seen that Context, decides to do something. The model calls a tool. The tool has to execute somewhere. That somewhere is the Environment . It's easy to under-appreciate. The Environment feels like plumbing — the filesystem, the shell, the network, the machine underneath. But it's where every side effect the model requests actually lands, and how you design it is what separates "an AI agent doing things on your behalf" from scary to routine . What's ahead: Part 1: The Raw Model Problem Part 2: Defining the Harness — The Six Components Part 3: The Control Loop Part 4: The Tool Layer Part 5: Context Engineering The Filesystem & Environment ← You are here Part 7: The Memory Layer Part 8: Observability Part 9: The Harness Architecture Part 10: Decomposing Claude Code By the end of this article, you'll know what the Environment actually is, why every tool with side effects depends on it, and the three properties (bounded, reproducible, inspectable) that separate a production-ready environment from a demo one. Let's get started. 📚 Want to go deeper than the articles? While you follow along with this series, I've put together two hands-on resources that go further than any single article can: Build a Harness from Scratch — Udemy Course — A self-paced course where I walk you through building a production-grade agentic harness from the ground up, in code. Harness Engineering for AI Agents — Live Maven Workshop — A live, cohort-based workshop for builders who want direct feedback, Q&A, and to work through the material with peers. Both are optional — the series stands on its own. But if you want the full studio-quality version, that's where it lives. What The Environment Is The Environment is

Welcome back to the Harness Engineering series — a 10-part journey from raw language model to production-ready agentic system. Made by builders. For builders. In Part 4, we looked at the Tools — the set of functions the model can call. But there's still one big open question hanging over every turn of the Loop: what does the model actually see when the Loop calls it? The answer is: whatever the harness put in the payload. That payload — the entire package of instructions, history, retrieved documents, tool definitions, and everything else — is called the Context . Some readers know this territory under an older name: prompt engineering . That name isn't wrong, but it's narrow. A prompt sounds like something you write once and ship. The reality of running an agent is that the payload changes every turn, and designing what goes in it is an ongoing discipline. Hence the newer, more accurate term: context engineering . What's ahead: Part 1: The Raw Model Problem Part 2: Defining the Harness — The Six Components Part 3: The Control Loop Part 4: The Tool Layer Context Engineering ← You are here Part 6: The Filesystem & Environment Part 7: The Memory Layer Part 8: Observability Part 9: The Harness Architecture Part 10: Decomposing Claude Code By the end of this article, you'll know what the Context actually is, why every turn forces you to answer "what should the model know right now?" from scratch, and the three moving pieces (system prompt, history, retrieval) that make up a well-designed context. Let's get started. 📚 Want to go deeper than the articles? While you follow along with this series, I've put together two hands-on resources that go further than any single article can: Build a Harness from Scratch — Udemy Course — A self-paced course where I walk you through building a production-grade agentic harness from the ground up, in code. Harness Engineering for AI Agents — Live Maven Workshop — A live, cohort-based workshop for builders who want direct feedback, Q&A, a

Welcome back to the Harness Engineering series — a 10-part journey from raw language model to production-ready agentic system. Made by builders. For builders. In Part 3, we looked at the Loop — the outermost machinery of a harness, the piece that drives everything else. But a Loop on its own is a hollow shell. It calls the model. The model responds. And then, if the model wants to affect anything outside the text it just produced, it needs to reach for a tool. That's what this article is about: the Tool Layer. The set of functions the harness makes available to the model, and the design decisions that separate a tool surface a model can actually use from one that constantly frustrates it. What's ahead: Part 1: The Raw Model Problem Part 2: Defining the Harness — The Six Components Part 3: The Control Loop The Tool Layer ← You are here Part 5: Context Engineering Part 6: The Filesystem & Environment Part 7: The Memory Layer Part 8: Observability Part 9: The Harness Architecture Part 10: Decomposing Claude Code By the end of this article, you'll know what tools actually are, why they exist, and — more usefully — how to tell a well-designed tool surface from a badly-designed one the moment you look at one. Let's get started. 📚 Want to go deeper than the articles? While you follow along with this series, I've put together two hands-on resources that go further than any single article can: Build a Harness from Scratch — Udemy Course — A self-paced course where I walk you through building a production-grade agentic harness from the ground up, in code. Harness Engineering for AI Agents — Live Maven Workshop — A live, cohort-based workshop for builders who want direct feedback, Q&A, and to work through the material with peers. Both are optional — the series stands on its own. But if you want the full studio-quality version, that's where it lives. What The Tools Are The Tools are the set of functions the harness exposes to the model. Concretely: the harness tells the model "

Prompt engineering has a bad reputation because most of it is superstition. But underneath the "you are a helpful assistant" cargo-culting, there are a handful of techniques that reliably work — and they're grounded in how the model actually behaves. Let me separate the real patterns from the folklore. These are the ones that move quality measurably, not the magic phrases people paste around without knowing why. Why prompting works at all A model predicts the next token conditioned on everything before it. Your prompt is that condition. So prompting isn't casting spells — it's setting up a context in which the desirable continuation is the most probable one. Every technique below is just a different way of doing that. The techniques that actually move the needle 1. Be specific about the output, not just the task. Vague in, vague out. Don't ask for "a summary" — ask for "three bullet points, each under 15 words, focused on financial risk." You're narrowing the probability space toward exactly what you want. 2. Give examples (few-shot). Showing the model two or three input/output pairs is often worth more than paragraphs of instruction. The model is extraordinary at pattern-matching; demonstrate the pattern and it follows. This single move fixes more formatting problems than any amount of description. 3. Let it think before it answers (chain-of-thought). For anything involving reasoning, telling the model to work through the steps before giving a final answer measurably improves correctness. Rushing straight to an answer is where models make careless mistakes — the same as people. 4. Assign a role with purpose. "You are a senior security engineer reviewing this code" genuinely shifts the output — not because of flattery, but because it conditions the model toward a specific register and body of knowledge. Use it when the framing changes the answer; skip it when it's just decoration. 5. Decompose hard tasks. Instead of one prompt that does five things, chain five promp

Code: Megapixel99/capture-the-flag In April I ran five games of an AI capture-the-flag tournament between five small open-weight models (1.0B to 2.5B parameters). Each was given root on an identical Ubuntu container and told to steal /root/flag.txt from the others while defending its own. Qwen 3.5 at 2.3B captured 13 flags and lost none. The other four captured two flags between them. Nobody found any of the five bonus flags planted on each machine, even after hints were revealed at rounds 6, 9 and 12, so I wrote a preliminary report with two conclusions in it: Model size matters enormously for security reasoning. Multi-step exploitation is not feasible for models under 3B parameters. The tournament has since run 327 more games with much larger models, which makes the first conclusion checkable. It does not survive. (This is the second post about this project. The first is about a dedup pass that deleted the training weights , and the two findings are independent.) The later tournament does not order by size Five hosted models plus one local 3B fine-tune, over the 327 games with a scoreboard. Captures here are events the game engine credited, counted from each game's own event log: model flag captures bonus flags GPT-OSS 120B 401 38 custom bot (3B local fine-tune, 221 games) 404 2 GLM-5.1 315 111 Nemotron 3 Super 100 30 Gemini 3 Flash 74 8 RNJ-1 8B 2 0 Two things in that table contradict the report. Those 189 bonus flags break down by tier as 50, 25, 58, 23 and 33, and tier 4 is the encrypted vault: read /etc/app/database.yml , take the password out of it, decrypt /root/.vault/encrypted.flag with openssl . It was captured 23 times, so multi-step exploitation is not out of reach here. And a model with more parameters than every entrant in the April tournament, RNJ-1 8B, finished last by a factor of 37 against the second-worst model, while a 3B fine-tune running on my desk led on main flags. That fine-tune has never gone past tier 1; both of its bonus flags are the hi

Large Language Models have quickly become part of everyday software development. We ask them to explain code, debug errors, generate tests, write Python scripts, summarize documentation, or help us understand an unfamiliar codebase. Within seconds, we get a response that can feel surprisingly natural. But what actually happens during those few seconds? Suppose you type: What is a build system? The model doesn't simply search through a database for a stored answer, and it doesn't generate the entire response in one shot. At the heart of an autoregressive LLM is a deceptively simple task: Given the tokens I've seen so far, what token should come next? Getting to that prediction, however, involves several layers of computation. At a high level: Prompt ↓ Tokens ↓ Embeddings ↓ Transformer ↓ Logits ↓ Next Token ↓ Repeat Let's follow that journey. 1. Everything Starts With the Prompt Consider: What is a build system? Humans immediately recognize the words and their meaning. A neural network needs numbers. Before the model can process the question, the text passes through a tokenizer . 2. Tokenization: Breaking Text Into Pieces A tokenizer divides text into smaller units called tokens . Conceptually, our prompt might become: ["What", " is", " a", " build", " system", "?"] This is only an illustration. Actual tokenization depends on the tokenizer used by the model. A token isn't necessarily a complete word. It might represent: a complete word part of a word punctuation whitespace combined with text a number part of an identifier a programming-language symbol Each token is mapped to an integer called a token ID . Conceptually: ["What", " is", " a", " build", " system", "?"] ↓ [3923, 374, 264, 1975, 1887, 30] The IDs above are illustrative. The important part is the transformation: Human-readable text has become a sequence of numbers the model can process. But token IDs themselves don't capture useful semantic relationships. The number 1975 , for example, doesn't inherently ex