标签：#llm

Coding agents draft specs, architecture docs, changelogs, and README updates in seconds — but a human still has to judge the quality of all that output. The bottleneck shift A year ago, the typical workflow was: you write a spec, you get comments, you revise, then you implement and get code review. Humans did most of the writing and coding. Now, agents produce first drafts of design docs, API references, runbooks, and onboarding guides — and they do it in seconds. Code implementation and code review can now be handled by agents, so those are no longer the bottleneck. What surfaced instead is the step right before: document review. A human has to read 2,000 lines of generated markdown and decide what's wrong. The writing part got dramatically faster. LLMs can assist with document review too, but compared to code implementation and code review, the human judgment required is still larger. This asymmetry compounds fast. Every agent-assisted project now has a stack of "needs human review" documents growing in a shared folder. If you're running multiple agent loops in parallel — one for the spec, one for the implementation plan, one for the test strategy — review becomes a pipeline stall. GitHub PRs remain the right tool when you need third-party review. But the step before that — the fast local self-review loop where you and your agent iterate on a draft — doesn't belong in a PR. Branching, diffing, and assigning reviewers is a lot of process for a first draft the agent wrote in seconds. Why prose feedback is lossy The most common workaround today is to have the agent read the document and then fix things based on natural-language feedback: "The error handling in section 3.2 is too vague — be specific about what happens on timeout." This looks reasonable. The agent reads it, searches for something about error handling, and makes a change. But several things go wrong: Position is ambiguous. If section 3.2 has three paragraphs about error handling, which one did the revie

We’ve all heard "it works on my machine," but when it comes to AI-driven features, that phrase is a recipe for disaster. You can have a perfectly tested agent today, but if you upgrade your base model or change your quantization strategy tomorrow, you might inadvertently kill your agent's reliability. You can’t afford to wait for production to find out your agent is hallucinating or failing its tool calls. This is why we built the headless QuantaMind CLI—to shift AI evaluation left into your CI/CD pipeline. By integrating custom eval JSON collections into your build process, you can now treat your AI agent like any other piece of code. If a model upgrade or a quantization tweak causes your agentic reliability to dip below your required threshold, your CI pipeline should block that merge. It’s not just about testing; it’s about enforcement. If you aren’t gating your deployments based on real, repeatable model performance, you aren’t shipping software—you’re shipping a guessing game.

The most important design decision in an email classifier isn't the model — it's the label set, and here's the one I keep coming back to: You triage email into one of four categories: URGENT — production incidents, executive requests; reply within 1 hour ACTION — code reviews, meeting follow-ups; reply same day FYI — informational, no response needed NOISE — newsletters, marketing, automated notifications From: {sender} Subject: {subject} Snippet: {snippet} Return ONLY the category name. Nothing else. That's the working prompt from the Nylas email triage recipe , and almost every line encodes a taxonomy-design lesson worth unpacking. Most people building email agents obsess over model choice and prompt phrasing. The recipe's quiet thesis is that the label set itself does the heavy lifting — get the taxonomy right and a cheap model classifies well; get it wrong and no model saves you. Why four is the magic number The recipe states it flatly: four is the right number. Three loses fidelity — everything important collapses into one overloaded bucket and you've built a binary classifier with extra steps. Five and the model starts confusing categories, because the boundaries between adjacent labels get too thin to express in a definition. Notice what makes these four work. They aren't topics — they're response obligations . URGENT means "reply within the hour," ACTION means "reply today," FYI means "no response needed," NOISE means "archive." Each label maps to exactly one behavior. That's the test I'd apply to any email taxonomy: if two labels lead to the same action, merge them; if one label leads to two different actions depending on content, split it. The same principle shows up in the sales context. The Agent Accounts overview describes an outreach agent classifying replies as interested / not now / unsubscribe — three labels, because the workflow has exactly three branches: book the meeting, schedule a follow-up, stop emailing. The taxonomy is the decision tree, fla

Day 1, 4:02 p.m.: a customer asks your agent a billing question and gets an answer. Day 6, 9:30 a.m.: they reply "actually, that didn't work." If your agent lives in a chat widget, that second message starts from zero — the session died with the tab, the context is gone, and the customer gets to repeat themselves. If your agent lives in a mailbox, the reply arrives inside the conversation , with the full history attached by the protocol itself. That's the argument in one before/after: chat sessions evaporate; email threads persist. And for agents that work across days rather than minutes, the thread is the most underrated memory substrate available. The protocol already built your memory layer Email threading runs on three headers, as the threading docs lay out. Every message carries a globally unique Message-ID . A reply adds In-Reply-To (the ID it's answering) and References (the full chain of IDs, oldest to newest). By the time a thread is five messages deep, References holds five Message-IDs in order — a complete, tamper-evident record of the conversation's shape, maintained by every mail client on earth. Compare that to what we hand-roll for chatbots: session stores, conversation tables, context windows we serialize and rehydrate. Email gives you the equivalent for free, federated across organizations, and — this is the part I find most compelling — human-auditable . Anyone with mailbox access can read exactly what the agent's memory contains, because the memory is the correspondence itself. No vector store inspection tools required. With Nylas Agent Accounts (in beta), the agent owns the mailbox where this accrues, and you never parse headers by hand. The Threads API groups messages by their header chain; each thread object gives you ordered message_ids , participants , and activity timestamps. When a reply fires message.created , the payload includes a thread_id — fetch the thread, walk its messages, and the agent has its full conversational past before decid

Stop letting the prompt be your state machine You shipped an LLM feature six months ago. Now the same user input produces wildly different outputs depending on... nothing you can point to. Something in the sampling? The time the context filled up and a chunk got dropped? Nobody knows. This is what happens when the prompt becomes your runtime. The trap: the prompt as an accidental runtime Here is what the trap looks like in TypeScript: async function handleUserRequest ( input : string ): Promise < string > { const prompt = ` You are a helpful assistant. The user said: ${ input } Previous context: ${ someGlobalContext } Decide what to do, gather any information you need, format the response, and return it. ` ; return llm . complete ( prompt ); } The model is doing everything here: deciding the intent, gathering data, formatting output, choosing what to persist. That is a footgun. You handed the runtime to a stochastic function. Gartner attributes many failed agentic AI projects to unclear value and inadequate risk controls. Deterministic, testable workflows address both. The fix is not a better prompt. The fix is to stop using the prompt as an architecture. What "deterministic" can and cannot mean here Be honest about what you can and cannot control. You cannot control: the model's exact output. It is probabilistic by design. You can control: The shape of the output (structured output plus schema validation) The steps that run before and after the model call What data enters the model What happens when the output fails validation Whether a human reviews the result before it commits to anything irreversible Determinism here means: the same inputs, the same workflow steps, the same guardrails every time. Not the same tokens every time. That is a realistic and achievable target. It is also the thing teams skip when they are moving fast. Typed workflow steps around the model call Break the work into discrete typed steps. Each step has a clear input type and a clear output

Separately, neither could compete. Now they hope they can.

TL;DR Provider invoices aggregate by model and billing period. They cannot tell you which team, product, or agent caused a cost spike. Request-level AI cost attribution links every API call to structured owner metadata (team, product, environment, trace ID) so investigations take minutes, not days. Three approaches exist: provider dashboard, gateway log enrichment, and application trace attribution. They differ sharply in setup cost and query granularity. Gateway log enrichment is the highest-leverage first step for most teams. It requires no changes to application code and covers all traffic behind the gateway. Real example: a platform team at a 60-person AI company discovered that 31% of their $18k/month spend came from a misconfigured retry loop in a background job, identified in under 20 minutes once request-level logs were searchable. Why Your Invoice Is Lying to You Your OpenAI invoice for last month shows $22,400. Your Anthropic invoice shows $6,800. Total: $29,200. Your CFO wants to know which business unit owns each line. You forward the invoices to your finance partner, who forwards them to three engineering managers, who reply with estimates that sum to $24,000 and do not match any real allocation. This is the standard state of LLM spend governance at companies between $5k and $50k per month in AI API costs. The invoices arrive, the spend is real, and attribution is a spreadsheet exercise done with guesses. The problem is structural. Provider billing aggregates by model and by billing period. It has no concept of your internal ownership model, your product boundaries, your tenant hierarchy, or your agent topology. A single gpt-4o line in your invoice might represent spend from a customer-facing chat feature, an internal summarization service, a nightly batch job, and three developers running experiments against production endpoints. You get one number. You have four or more owners. Request-level AI cost attribution is the practice of enriching every API c

There is a persistent myth that to build a worthy code assistant, you absolutely must use GPT or Claude. This is false. You don't need a 1-trillion parameter model. You need a small local model and extremely rigorous engineering around it. This is the direction history is taking for companies. As Mark Zuckerberg mentioned, the future isn't a single omniscient model, but "every company having its own specialized AI" . And this specialization necessarily involves fine-tuning and local deployment (or on sovereign servers) to guarantee data security. The thesis behind the construction of Vibrisse Agent can be summed up in one sentence: Small models, Great tools. In this article, I will detail the technical stack and concrete engineering solutions I implemented to tame a local model and make it reliable in production: LangGraph, Ollama, FastAPI, React (no build step, with embedded custom CSS) , all running on a machine with 32 GB of RAM. For the curious who want to run the agent on their machine right now: // MacOs / Linux curl -sSL https://agent.vibrisse-studio.dev/install.sh | bash // Windows irm https://agent.vibrisse-studio.dev/install.ps1 | iex Architecture: Why a State Machine (LangGraph)? At first, when building an LLM application, we tend to think in sequential chains: Input -> Prompt -> Tool -> Output . The problem is that if one node fails, the whole chain stops without us being able to catch the error or understand the context of the crash. That's where LangGraph comes in. Vibrisse's architecture isn't a chain, it's a state machine . Every node in the graph has a very precise responsibility, shares a global conversation state, and uses conditional transitions to move to the next node. I implemented the Supervisor / Worker pattern: The Supervisor analyzes the user's intent. It does nothing else but route. It dispatches the task to specialized Workers (the RAG Worker, the Search Worker, the Ghost Worker...). If a Worker fails or needs more information, it can se

SearchLeak exploit shows why the industry's approach to LLM security fails over and over.

Automating workflows has always been a priority for me, especially for repetitive and error-prone manual processes. Recently, integrating AI capabilities into these automations offers a great opportunity for those, like me, who seek practical solutions. However, this integration often requires coding or complex API integrations. This is where "low-code/no-code" tools like n8n come into play. I used n8n to set up AI-powered agent flows on my own servers, without compromising data privacy and control. In this post, I will share my experience and explain how to do it step-by-step. Why n8n and Self-Hosted AI Automation? A few years ago, I experimented with different automation tools, especially for simple data transfers and notification flows. But when AI capabilities became involved, I either found them too expensive or avoided cloud-based solutions due to data security concerns. Especially in a client project where we needed to set up an automation handling sensitive financial data, a self-hosted solution became inevitable. n8n offers a wide range of integrations and, thanks to Docker support, I can easily host it on my own server. For me, self-hosting is not just about cost advantage; it also means having complete control over my data. Especially when using AI agents, the question of how much of the data you send to LLMs is logged or used for training is always a concern. With an n8n setup under my control, I minimized these worries. Moreover, n8n's flexible structure gives me the freedom to add as many custom integrations or LLM providers as I want. This is a significant advantage for someone like me who enjoys testing different LLMs. ℹ️ Data Privacy Priority Especially when working with sensitive data, it's crucial to carefully review the data usage policies of cloud-based AI services. Self-hosted n8n offers more control in this regard, making it my preferred choice for critical workflows. Setup Steps: Quick Start with Docker Compose The easiest way to run n8n on y

Local Inference Powers Browser Sign Language, Open-Source Agent Infra, & AI Engineering Guides Today's Highlights This week highlights practical advancements in local AI, featuring a browser-based sign language reader running entirely on-device, new open-source infrastructure for building and evaluating AI agents, and a comprehensive guide to AI engineering from scratch, focusing on building and shipping models efficiently. I Built a Webcam Sign-Language Reader in the Browser (No Cloud) (Dev.to Top) Source: https://dev.to/dev48v/i-built-a-webcam-sign-language-reader-in-the-browser-no-cloud-11hg This article details the creation of a real-time sign language reader that operates entirely within a web browser, without relying on cloud services or model uploads. The developer showcases how to achieve genuinely useful AI functionality, traditionally associated with heavy research labs and GPU clusters, using client-side processing. This approach emphasizes privacy, reduced latency, and accessibility by making advanced AI applications runnable on consumer hardware, specifically within the browser environment. The implementation leverages lightweight models optimized for on-device inference, demonstrating the power of WebAssembly or WebGPU for local execution of machine learning. Such a system offers significant advantages for applications requiring immediate feedback or handling sensitive user data, aligning perfectly with the principles of local AI and empowering developers to deploy sophisticated multimodal solutions without external dependencies. This project serves as an excellent example of practical, self-hosted AI and multimodal processing on consumer hardware. Comment: Running a vision model this complex purely client-side with decent performance is impressive. It really pushes the boundaries of what's feasible for local, privacy-preserving multimodal AI in the browser. trycua/cua — Open-source infrastructure for Computer-Use Agents (GitHub Trending) Source: https

A solo developer with a $200/month budget can now access the same AI coding power that cost enterprises $50,000/month just two years ago. The secret isn't one tool — it's knowing how to mix and match three different access models to get frontier output at budget prices. I've been running this exact stack for months. Here's the breakdown. The Three Ways to Access AI Coding Models Before we talk strategy, understand your three options. Each has a wildly different cost profile. Option 1: Self-Hosted Open Models With models like GLM-5.2 hitting near-Claude Opus quality under MIT license, self-hosting is finally viable. The math is straightforward. Hardware cost: A dedicated GPU server (RTX 4090 or A100) runs $300–$800/month. An H100 rental starts at $1.99/hour on platforms like RunPod. Break-even point: According to cost analysis from multiple providers, self-hosting becomes cheaper than APIs at roughly 5–10 million tokens per month for premium-tier models [1]. Below that volume, you're paying for idle hardware. The catch: You need DevOps skills. Model deployment, quantization, monitoring, failover — it's real infrastructure work. If you save $500 on compute but burn out managing GPUs on weekends, you lost money. Best for: Teams with predictable, high-volume workloads and existing DevOps capability. Think 100M+ tokens/month where savings hit $5M+ annually [2]. Option 2: Pay-Per-Token APIs The default starting point. You pay exactly for what you use. Current pricing (early 2026, per 1M tokens): GPT-4o: $2.50 input / $10.00 output Claude 3.5 Sonnet: $3.00 input / $15.00 output Gemini 1.5 Pro: $1.25 input / $5.00 output DeepSeek V3: $0.27 blended (yes, really) Together AI (Llama 70B): $0.88 blended [1] The pricing floor crashed when DeepSeek V3 arrived at $0.27/M tokens with GPT-4-class quality. Open-source models routed through providers like Together AI or Cerebras ($6–12/M tokens at 969 tok/s) give you more options than ever. The trap: Pricing scales linearly forever. A

A new repository-level dataset, published on GitHub under CC0-1.0, helps researchers and developers discover multilingual developer content across READMEs, issues, and pull requests. The post Accelerating researchers and developers building multilingual AI with a new open dataset appeared first on The GitHub Blog .

AI Builder Notes - Week of June 14, 2026 My thoughts and my twitter’s feeds thoughts This week was all about the ‘loop’ and Fable . The Loop The best way I can describe it is: design the flowchart. Think of the deterministic flowchart on how you want your agents to work. Aim to have: more deterministic bits - this keeps things more predictable more verification bits - this is agent feedback more agent tool calls - this, on a frontier LLM, makes it perform better. The ‘loop’ is essentially: goal -> agent acts -> verifier checks -> state/memory updates -> policy decides next action -> repeat/stop/escalate now the specific implementation of this - will differ based on what you’re working on. Fable Fable capabilities are absolutely insane, I tried it myself and it is entirely worth it for you to spend 2 minutes looking at this. There are a few projects that I fire up a new model into to see what’s it gonna do. A project I wanted to build was a way to teach and demonstrate ‘spin’ in table tennis, every frontier model before Fable fumbled hard. But Fable outshined them with ease: https://srijanshukla.com/artifacts/spin-lab/ If you personally did not experience a big shift in capability, you are probably not asking it a complex enough or ambitious enough task. Fable came, and Fable was taken away. The United States Government(USG) was reported with a jailbreak or sorts - which Anthropic considers not significant. The USG anyway banned Fable just after few days of release. Big drama. Fable was very pricey $$$$ Hence, people developed some patterns of work on those few golden days of Fable being available. - use Fable as planner/architect/taste/spatial/front-end judge. - use GPT-5.5/DeepSeek/Kimi as executor/worker. Other things Openrouter released their Fusion feature as a model on their platform, accessible via API. Fusion is basically council-of-LLMs pattern - providing results that can rival the frontier Fable 5 solo. Google Open Knowledge Format - https://github.com/Goo

When I shipped Trooper , a privacy-aware LLM proxy written in Go, I didn't have a marketing plan. I had GitHub traffic analytics and a habit of checking them obsessively. Seven weeks later, I have something more useful than a viral moment: a ranked table of every traffic spike, what caused each one, and proof that the exact same playbook that worked at launch still works when you have something new to say. What is Trooper? Trooper sits between your app and your LLM provider. When your cloud quota runs out, it automatically falls back to a local Ollama instance with zero code changes on your end. It also tracks session context, so your agents don't go blind between calls. It's not a chatbot wrapper. It's plumbing. Which makes the distribution story more interesting, because plumbing doesn't go viral the way demos do. The Data GitHub gives you 14-day rolling windows for clones and views. I screenshotted them obsessively and tracked every spike. Here's the full ranked table: Rank Date Clones Unique Cloners Views Unique Visitors Driver 🥇 1 May 13 375 173 1,113 ~140 Reddit wave peak 🥈 2 May 10-12 312 137 974 133 Reddit launch spike 🥉 3 Jun 10 289 124 749 101 "Escalate the model" r/ollama post 4 Jun 11 268 112 840 95 Decaying from Jun 10 spike 5 Jun 12 240 99 739 74 Decaying from Jun 10 spike 6 Jun 9 175 102 802 100 Organic 7 Apr 25 174 71 664 113 Early Reddit posts 8 Jun 7 171 110 876 110 Organic recovery 9 Jun 6 163 104 755 102 Organic recovery 10 May 29-30 122 73 610 83 LinkedIn post 11 May 25 76 48 495 53 Claude Code integration chat What I learned 1. Reddit is the only thing that moved the needle, and community fit matters more than size The #1 and #2 peaks were both Reddit-driven. On May 10-11, I posted across r/ollama, r/LocalLLM, r/ClaudeCode, and r/Gemini simultaneously. Total views across those posts: ~7,000. r/ollama alone drove nearly 4,000 of those views. Not r/LocalLLM. Not r/ClaudeCode. r/ollama , the smallest of the four communities. The reason: Trooper so

On June 9, Anthropic shipped Claude Fable 5 — the most capable coding model the industry had ever seen. Three days later, the U.S. government ordered it offline for every user on Earth . No warning. No transition period. One directive, and the frontier vanished overnight. 📖 Read the full version with charts and embedded sources on ComputeLeap → The same week, Z.ai (Zhipu AI) released GLM-5.2 — a 744-billion-parameter coding model with a one-million-token context window, MIT-licensed open weights arriving within days. The timing was not lost on the developer community. ℹ️ The message landed clearly on Hacker News: as user Reubend put it, they're "grateful to Chinese labs for being open with their work" — especially after "the Fable 5 fiasco." Open weights aren't just a cost play anymore. They're insurance. This guide walks you through actually running GLM-5.2 on your own hardware — the VRAM you need, the quantization that fits, and the exact commands for llama.cpp, Ollama, and LM Studio. No API keys. No cloud dependency. No one can pull the plug. What GLM-5.2 Actually Is GLM-5.2 is the third major iteration in Z.ai's GLM-5 line, purpose-built for agentic coding and long-horizon software engineering . Here is what you are working with: Spec Value Architecture Mixture-of-Experts (MoE) Total Parameters 744 billion Active Parameters ~40 billion per token Context Window 1,000,000 tokens Max Output 131,072 tokens Training Data 28.5 trillion tokens License MIT (open weights) Thinking Modes High and Max The MoE architecture is the key to local viability. Only ~40 billion parameters fire per token — the rest sit idle. That is what makes aggressive quantization work: you are compressing 744B weights, but inference only touches a fraction of them at any given time. GLM-5.2 supports two thinking-effort presets: High and Max. Z.ai recommends Max as the default for coding work — it produces longer reasoning chains before generating output. The model launched on June 13 on Z.ai's C

Originally published at llmkube.com/blog/making-self-hosted-llm-agents-trustworthy . Cross-posted here for the dev.to audience. Running a single local LLM node is a solved problem. You write an InferenceService, the operator schedules it, llama.cpp or MLX serves it, and you get an OpenAI-compatible endpoint. We have been doing that for months. Running a fleet of them is where it stops being easy. My fleet is heterogeneous on purpose: CUDA pods in the cluster, and Apple Silicon Macs sitting off-cluster on the homelab network, each one running two separate agents (one for inference, one for the agentic coding harness). The day I shipped 0.8.4 to that fleet, I learned exactly how it does not scale. I updated each Mac by hand. The control plane had no idea what version any agent was running. And the launchd reload I used to restart an agent was a silent no-op on an already-loaded service, so the old binary kept running while I believed I had updated it. I found that out by hand-inspecting a process tree. Three machines made it annoying. Thirty would make it impossible, and the whole pitch for sovereign, on-prem AI is that you run a lot more than three. So the last stretch of work on LLMKube was not about a faster runtime or a bigger model. It was about making the fleet trustworthy : able to update itself safely, and unable to lie to the control plane about its own state. Here is what that took. Helm and brew for the edge The fix is a new cluster-scoped CRD, AgentRelease , and a self-update path in the agents themselves. You describe the release you want once, the operator rolls it out, and the agents pull and apply it. The design borrows directly from prior art that already solved this for Kubernetes nodes: Rancher's system-upgrade-controller, k0s autopilot's per-platform SHA-256 staging, and Teleport's outbound-only poll model. The properties that make it safe to leave running: Declarative and approved. An AgentRelease names the agent, the version, and the per-platform

You have 50 documents and you're running them through an LLM in a loop. The first one finishes at the 2-second mark. The fiftieth finishes at the 100-second mark — not because it's harder, but because it waited in line behind the other 49. Pool runs all 50 at the same time. The Problem With Loops Every developer who works with LLMs writes this code eventually: import os from yait_aichain.models import Model from yait_aichain.skills import Skill skill = Skill ( model = Model ( " claude-sonnet-4-6 " , api_key = os . getenv ( " ANTHROPIC_API_KEY " )), input = { " messages " : [{ " role " : " user " , " parts " : [ " Summarise in two sentences: \n\n {text} " ]}]}, ) documents = [{ " text " : f " Document { i } content... " } for i in range ( 50 )] results = [] for doc in documents : result = skill . run ( doc ) results . append ( result ) It works. It's readable. And it's painfully slow. Each LLM call takes roughly 2 seconds. Multiply that by 50 documents and you're staring at your terminal for almost two minutes. The calls are completely independent — document 37 doesn't need the result of document 12. Yet document 37 sits idle, waiting its turn. That's a scheduling problem, not a computation problem. I ran into this directly while building a task that pulled N files or links and produced a consolidated report. The sequential version was logically fine but just hemorrhaged time. I needed to fire everything at once without rewriting the Skill logic — no new prompt templates, no restructured code, just a different execution model. That's what Pool is. Pool: Parallel Map for LLM Calls Pool takes one Skill (or Chain) and a list of inputs , then launches all of them concurrently. Think of it as Array.map() where every element runs in parallel against an LLM. import os from yait_aichain.models import Model from yait_aichain.skills import Skill from yait_aichain.pool import Pool , DONE , FAILED skill = Skill ( model = Model ( " claude-sonnet-4-6 " , api_key = os . getenv ( "

You see words. A language model sees tokens — chunks of text, usually a few characters each. Everything starts here. Day 2 of my AIFromZero series. Text gets shattered into tokens "unbelievable" → ["un", "bel", "iev", "able"] (4 tokens, not 1 word, not 12 letters) Before any "thinking", your text is chopped into tokens and each becomes a number the model processes. Why not words or letters? Letters : too fine — the model would relearn spelling everywhere. Whole words : too many — millions, plus every typo and name. Subword tokens : the sweet spot. Common words = 1 token; rare words split into reusable pieces. A fixed ~100k-token vocabulary covers any text. The ~4-chars rule (and why it costs you) In English, ~4 characters ≈ 1 token , or ~0.75 tokens per word. This is how everything is priced and limited: API bills are per token (prompt + reply). A "context window" (how much it can read at once) is measured in tokens — 1,000 tokens ≈ 750 words. Verbose prompts and long chat history burn tokens. Concise prompting is a real cost lever. The strawberry problem The model never sees s-t-r-a-w-b-e-r-r-y. It sees a token like straw + berry . The individual letters are buried inside tokens, so counting characters is genuinely hard for it. It's not dumb — it just doesn't read letters. Tokens are step 1 of everything Tokenize → turn each token into a vector (embeddings, next) → run through the transformer → predict the next token. Every LLM starts exactly here. 🔤 Type anything and watch it tokenize live: https://dev48v.infy.uk/ai/days/day2-tokens.html Day 2 of AIFromZero.

The earlier posts in this series were about what the gateway lets you call (cache-aware spawning across five providers, the Codex review gate, the CLI-versus-API argument) and the one before this was about the parts that do not show up as a tool, the upstream-tracking and the website that became the project's front door. This one is about a different kind of front door: the gateway can now listen over HTTP, behind real authentication, and serve more than one caller without those callers being able to read each other's work. That sounds like a small toggle. It is not. Moving an MCP server off a local pipe and onto a network port changes the trust boundary completely, and 2.9.0 is the release where I sat down and remediated all seventeen findings from a multi-LLM red-team of exactly that surface before telling anyone the remote path was ready. Short version: llm-cli-gateway is one Model Context Protocol server that wraps five vendor CLIs (Claude Code 2.1.177, Codex 0.139.0, Google's Antigravity agy 1.0.8, Grok 0.2.51, and Mistral Vibe 2.14.1) behind a single, uniform tool surface, so one orchestrating agent can fan a task out to several models, collect independent opinions, run a red-team or a consensus check, and keep durable session and job state across all of it. Until recently that only made sense on localhost over stdio. As of 2.9.0 the same server runs over HTTP with a static bearer token or a built-in OAuth 2.0 authorisation server (PKCE on by default, an opt-in human-consent gate, and a trusted-principal-header seam for when you front it with your own identity-aware proxy), every session and job and stored request is stamped with an owner principal and access is enforced per principal, remote provider calls are refused unless a workspace is registered, and the whole thing fails closed rather than open when the configuration is dangerous. Long version is below, same shape as always: what it enables, what the remote options actually are, a stack of worked scenar