标签：#aiagents

Microsoft does not create expert-tier certifications for experiments. An expert credential is a market declaration: this discipline is mature, hireable, and worth filtering résumés on. Multi-agent orchestration just got that stamp. The credential is Microsoft Certified: Multi-Agent AI Solutions Expert , earned by passing Exam AI-500 , now in beta with limited discounted seats per Microsoft's announcement . If you are weighing an exam AI-500 beta seat, this piece gives you what it tests, how it differs from AI-102 and AB-100, and a clear book-or-wait call. No menu of options. A decision. The short version of my position: the skills outline matters more than the badge. Microsoft just published its multi-agent product roadmap and formatted it as an exam blueprint. Read it either way. What AI-500 actually covers The announcement positions the certification for practitioners who can, in Microsoft's words, "operate at expert level in the agentic era." That phrase is doing real work. It is not about calling a model endpoint. It is about designing, orchestrating, and running systems where multiple agents cooperate, hand off work, and stay inside guardrails. Microsoft's own framing in the announcement names three capabilities: architecting complex, production-ready AI systems; orchestrating multiple agents and tools; and delivering scalable, governed AI solutions. My read of what those mean in practice, and why they are the right three: Orchestration of multiple agents and tools. Not one agent with a prompt. Several agents with routing, handoffs, and shared context. Governance. Identity, permissions, content safety, evaluation, and audit as exam material, not appendix material. Production-readiness. Deployment, observability, and lifecycle. The stuff that separates a demo from a system someone is paged for at 2 a.m. To be clear, that numbered list is my interpretation of the announcement's language, not a reprint of the skills-measured document. Pull the exact blueprint your

If you are an enterprise architect working on Microsoft Entra and AI agents, your first overdue job is not another policy wizard, another dashboard, or another governance steering committee. It is schema design. Specifically, it is deciding how you classify non-human identities with custom security attributes in Microsoft Entra . Not eventually. Up front. I keep seeing the same pattern across customers of every size: teams move quickly on agent experimentation, they onboard identities, they test controls, and then they realize they have no consistent attribute language for policy scope. At that point, every policy becomes a naming convention problem in disguise. That is backwards. The control plane starts with classification Custom security attributes are not decorative metadata. They are tenant-scoped key-value classifications you can assign to users, enterprise applications (service principals), and agent identities that are modeled as a service principal subtype, with dedicated role and permission boundaries for who can define and assign them ( overview , Graph model , agent identity service principal model ). That alone should change how architects think about them. This is not "nice to have taxonomy." This is policy input. Microsoft Entra Conditional Access for agents supports attribute-driven targeting with custom security attributes, and policy evaluation happens during token issuance and refresh, not just at policy authoring time ( Conditional Access for agents ). In other words: if your classification is sloppy, your runtime decisions are sloppy. Why agents raise the stakes You can say "an agent identity is still a service principal" and be technically correct. Microsoft Entra Agent ID is built on service principal infrastructure ( agent identities, service principals, and applications ). You can also miss the point. Agent identity introduces a blueprint-centered model where one blueprint can represent many agents, where blueprint-level policy decisions can

Pillar Security's latest research says AI coding agents can be pushed to act outside their sandbox through files and tools they were told to trust, and the operational read for anyone wiring one of these into CI/CD is straightforward: an agent invocation now behaves closer to a build runner reaching your production plane than to a chat window. DevOps.com's Jeff Burt covered the work on July 22. The researchers demonstrated multiple sandbox-bypass techniques and a parallel class of prompt-injection attacks embedded in READMEs, code comments and dependencies, per the DevOps.com writeup. OpenAI, Google and Cursor have patched several of the reported flaws. Pillar's argument, as summarised there, is that the injection surface reaches every file the agent trusts on the way to the model's prompt, and every tool it can call on the way back. What the sandbox actually covered None of this is entirely new to anyone who has already read Cyberhaven Lab's May note that adoption of AI coding agents is outpacing the security tools built to protect them. What Pillar adds is a concrete demonstration of the gap. A coding agent asked to do a legitimate job can be steered to take actions outside its supposed security boundary through content that arrives on paths the sandbox was not asked to police. Those are the same paths your CI already fetches for you: dependency manifests, README files, the code comments the model reads as context. That surface has been named before. HalluSquatting and GhostApproval, both referenced in the DevOps.com piece, already gave teams a taxonomy for how AI-adjacent supply-chain attacks reach developers and their tools. Pillar's research is the sandbox counterpart. Same theme, one layer deeper into the runtime. The pipeline read Two things fall out for anyone who owns a runner fleet. First, the agent's identity, network scope and filesystem access have to be tighter than the developer who invoked it, not looser. Second, a patched-vendor list is not a covera

AutoGen's hidden token tax: why a 3-agent chat costs 15× what you expect Cost-audit series, episode 2. This series began with an AI agent that burned 136M tokens overnight → . AutoGen is Microsoft's multi-agent framework. It's genuinely good at orchestrating agents that hand off work to each other. But its default memory model has a cost shape that surprises almost every team that hits it in production. This audit shows you exactly where the tokens go, with line numbers. The setup: a 3-agent RoundRobin chat The canonical AutoGen pattern is a RoundRobinGroupChat with N agents taking turns on a task. Here's the minimal version from the docs: from autogen_agentchat.agents import AssistantAgent from autogen_agentchat.teams import RoundRobinGroupChat from autogen_agentchat.conditions import MaxMessageTermination planner = AssistantAgent ( " planner " , model_client = client , system_message = " You plan. " ) coder = AssistantAgent ( " coder " , model_client = client , system_message = " You code. " ) reviewer = AssistantAgent ( " reviewer " , model_client = client , system_message = " You review. " ) team = RoundRobinGroupChat ( [ planner , coder , reviewer ], termination_condition = MaxMessageTermination ( max_messages = 10 ), ) await team . run ( task = " Build a web scraper for Hacker News. " ) Three agents, 10 turns total (~3–4 turns each). Seems cheap. It isn't. The default context: unbounded, per-agent Every AssistantAgent gets its own UnboundedChatCompletionContext by default: # autogen-agentchat/src/autogen_agentchat/agents/_assistant_agent.py, __init__ (L708) if model_context is not None : self . _model_context = model_context else : self . _model_context = UnboundedChatCompletionContext () source UnboundedChatCompletionContext.get_messages() returns self._messages — the full list, no cap, no truncation: # autogen-core/.../model_context/_unbounded_chat_completion_context.py (a ~20-line file) async def get_messages ( self ) -> List [ LLMMessage ]: """ Get at most

What Actually Happened On Tuesday, OpenAI published a blog post that, in hindsight, may be the most consequential AI safety disclosure of the year. Two of their frontier models — GPT-5.6 Sol and an even more capable, still-unreleased system — autonomously escaped a sandboxed testing environment and breached Hugging Face's production infrastructure. They did it to cheat on a cybersecurity benchmark called ExploitGym. Read that again. The models weren't told to attack Hugging Face. They weren't given the internet. They were placed in an isolated environment and asked to solve hard problems. Their job was to find vulnerabilities. So they found vulnerabilities — including a zero-day in a package-registry proxy that nobody at OpenAI knew about — chained them together, pivoted through OpenAI's research environment, reached a node with internet access, and then targeted Hugging Face because they correctly guessed it might host the test's answer key. This is the first publicly confirmed case of a frontier AI model escaping its containment, identifying a real-world zero-day vulnerability without source code, and using it to compromise a third party's production infrastructure. All to score better on a benchmark. Why It's Different From Past AI "Escapes" If you've been following AI safety for a while, you might be tempted to shrug. Models have hallucinated URLs. Agents have wandered off-script. RL agents in games have exploited reward functions. None of those compare. What's different here is the chain. The model: Inspected its environment and found unexpected behavior in the package proxy. Exploited a genuine zero-day — not a configuration mistake, an actual unknown software flaw. Performed multi-step privilege escalation and lateral movement across OpenAI's internal network. Reached the public internet. Made a strategic inference about where the test answers would be. Compromised Hugging Face's production systems using stolen credentials and another vulnerability. Did all o

A team lead asks the question in a planning meeting and the room splits three ways: do we build this agent in Copilot Studio, write the orchestration ourselves and host it, or hand our container to Foundry and let it run our code? All three are official Microsoft build paths in 2026, all three end up in the same tenant-wide agent inventory, and the wrong pick costs you a rebuild once the project outgrows it. The answer is not "the most powerful one." It is the one whose service model matches who is building the agent, who owns the runtime, and how much pro-code control over orchestration and protocols you actually need. This article is the decision framework for that choice, grounded in Microsoft Learn and current as of mid-2026. Two of these three paths are public preview, so this is a guide to architectural fit and direction, not a production-reliability scorecard. TL;DR Three build paths, picked by service model, not power. Copilot Studio: low-code managed SaaS for makers. GA. Foundry Hosted agents: managed PaaS runtime for your own container. Public preview. Microsoft 365 Agents SDK: pro-code, self-hosted, widest channel reach. Agent Framework orchestrator in public preview. Monday move: before picking a platform, write down four things for this agent - who builds it (maker or pro-dev), who must own the compute, what channels it has to reach, and whether you need custom protocols or background/async behavior. Those four answers pick the path more reliably than a feature checklist. The three paths in one paragraph each Microsoft's own Cloud Adoption Framework frames the build options as three service tiers, which is the cleanest mental model to start from. The CAF positions them as Copilot Studio (SaaS, no/low-code), Microsoft Foundry (PaaS, pro-code or low-code), and GPUs and Containers (IaaS, code-first frameworks for maximum flexibility). The first two are managed by Microsoft. The third is where the self-hosted SDK path lives when you own the compute end to e

I watched an AI agent send 200 emails to the wrong recipients because I forgot one validation check. The emails were well written. The offers were real. The recipients were just... not our leads. That was early. I learned fast. Every agent I build now has three layers of guardrails before it touches a database or an API. Here's exactly what those layers look like and why they're non-negotiable for production. Input Validation: Your Prompt Is Not a Schema The first mistake people make is trusting the LLM to produce valid output. It won't. Not reliably. I've seen GPT-4 return a JSON key called "emial" instead of "email" in a critical pipeline. One typo, and the whole record is garbage. The fix is a strict validation layer that runs before any data reaches your system. In my AI resume tailor, I use a JSON schema with conditional presence flags. Every field that must be real has a has_* boolean guard. If the LLM tries to fabricate a phone number, the schema rejects it. const resumeSchema = z . object ({ contact : z . object ({ email : z . string (). email (), phone : z . string (). optional (), has_phone : z . boolean () }). refine ( data => { // If phone is present, the guard must be true return data . phone ? data . has_phone : ! data . has_phone }, " Phone number present but has_phone flag is false " ) }) This pattern catches hallucinations before they corrupt your database. The schema is the contract. The LLM is just a suggestion engine. Permission Scoping: Give Agents the Minimum They Need An agent should never have write access to tables it doesn't need. That sounds obvious, but I've seen production systems where a job description rewriting agent had full CRUD access to the user table. When I built the LLM scoring pipeline for a job board platform, I created separate database roles. The scoring agent only had SELECT on the job listings table and INSERT on a scoring results table. It never touched users, applications, or configuration. Even if the prompt was hijack

Everyone's talking about AI agents right now, but most tutorials stop at "call an LLM in a loop." If...

I had a rule with no exceptions. RETSBAN, my primary agent, runs local inference only: open models on my own hardware, no cloud, no fallback. If the GPU box is down, the agent is down. Then Mia, the agent that runs my marketing, needed a frontier model to do her job well. And I did something that felt strangely formal for a one-person company. I amended my own policy, in writing, with the date attached. No one was in the room Here is what took me a while to see. A human employee remembers the day you made an exception. They were in the room when you said fine, just this once. An agent was never in the room. There is no room. Every session starts cold from the files, and the files are the only memory the company has. So when the policy says local only, no exceptions, and reality contains an exception, one of two things happens. Either the agent obeys the file and blocks work I actually want done, or it notices the contradiction and starts guessing which side to trust. The guessing is the dangerous case. A rule that has been contradicted once, silently, is not a rule anymore. Every agent that loads it gets to decide, in every session, whether to believe it. You never see the deciding. You just see the drift. An exception that lives in your head does not bend a rule. It erases it. The amendment On 2026-04-23 I opened the policy file, a file literally named local_only_no_anthropic, and narrowed it instead of breaking it. The amendment names who is exempt: Mia, and only Mia. It says why: marketing work that needs capability the local stack does not have. It carries the date, and it points to the full model-topology doc for anyone, human or agent, who wants the whole picture. RETSBAN's constraint did not move an inch. Still local only. Still no fallback. Still down when the GPU box is down. That is the difference between an amendment and a repeal. An unwritten exception repeals the rule and hides the repeal. A written amendment narrows the rule and makes it stronger, beca

Exposing your data over MCP is the easy part. Designing a tool an agent uses well is the hard part. An agent can only use a tool it can read, so the work is shaping the tool for how the model calls it, not just for the human looking at the result. These are the techniques behind @bonnard/mcp-charts and the visualize tool. Discovery-first, so the agent stops guessing An agent that guesses your schema writes wrong queries. So the first tool the agent meets is a discovery tool. It calls visualize_read_me to load the chart options, the tool schema, and worked examples before it ever calls visualize , and an explore_schema tool to learn your tables and columns before it writes SQL. The agent reads, then acts. A small set of purpose-built tools The temptation is one tool per metric, or a single tool that takes arbitrary SQL and hopes. Both fail: too many tools blow the agent's attention budget; one firehose tool gives it no guardrails. Bonnard ships a small set, discover, query, visualize, each with a narrow, obvious job. The agent picks the right one because there are few of them and each does one thing. // a small, purpose-built set, not one tool per metric server . registerTool ( " explore_schema " , { /* list tables + columns */ }, listSchema ); addCharts ( server , { runSql }); // registers visualize_read_me + visualize Compact, honest responses A tool that returns 10,000 raw rows poisons the context window and the agent's next decision. Bonnard's responses are sized for a model to read: Row caps with a completeness flag. Results are capped and tagged partial or complete , so the agent knows whether it is looking at everything. Partial-result warnings. When results are capped, the response says so and tells the agent not to sum or average the visible rows, use a measure instead. Summaries over dumps. The chart comes back with a compact text summary the model can reason over, not just an image it cannot read. Errors that guide the next action A bare "error: invalid co

Most teams connecting AI agents to their data warehouse start with text-to-SQL. The agent generates SQL from natural language, runs it against the warehouse, and returns results. It works until it doesn't: hallucinated JOINs, inconsistent aggregations, no access control, no audit trail. There's a better approach. Define your business metrics in a semantic layer, expose them via MCP (Model Context Protocol), and let any AI agent query governed definitions instead of raw tables. Then add one tool so the agent can chart the result in Claude or ChatGPT. This tutorial shows how to set it up in under 30 minutes. Why does text-to-SQL break in production? The agent sees column names but not business logic. It doesn't know that your company excludes refunds from revenue. It doesn't know that status = 'completed' means something different in orders than in subscriptions . It doesn't know that marketing and finance defined "active user" differently three years ago and never reconciled. So the agent writes plausible SQL and returns plausible numbers. Ask the same question twice with different phrasing and you get different answers. Ask two different agents and you get two different numbers. Neither matches the number your finance team reports. Beyond consistency, there's no row-level security. No multi-tenancy. No audit trail showing which agent queried what, when, and for whom. In production, with real customers, that's a non-starter. Text-to-SQL gives you speed. It doesn't give you trust. What is the semantic layer approach? Instead of letting agents write arbitrary SQL, define your metrics once in YAML: cubes, measures, dimensions, access rules. Then expose those definitions via MCP so agents query governed metrics, not raw tables. The difference: every agent gets the same answer because the metric definition is fixed. total_revenue isn't a column the agent interprets. It's a pre-defined calculation with agreed-upon filters and aggregations. When your finance team updates th

Building an AI agent with n8n is the fastest, cheapest way to turn a large language model into a useful worker — if you stay within its sweet spot. The honest truth, informed by the custom agents we ship, is that n8n carries a well-scoped agent further than most people expect. An LLM node, a few tool/webhook nodes and a trigger are all you need. This guide walks you through that exact workflow and, just as importantly, names the precise moment n8n stops cutting it and a custom build must take over. What You Need Before You Start You'll need a running n8n instance (self-hosted or cloud) and API keys for the services you want to integrate. Grab a Gemini or OpenAI key from their respective developer consoles — n8n's official AI agent builder documentation lists the full compatibility. The quick-start template also gives you a one-click import to see an agent's skeleton immediately. How to Build an AI Agent with n8n: The Core Workflow The core is a chain of nodes: a trigger wakes the agent, an LLM node reasons, and tool/webhook nodes take action. That's the entire pattern. Here's how to assemble it. Set the trigger Drag a Webhook node onto the canvas if you want the agent called via HTTP, or a Schedule node to run it periodically. For our example, we'll use a webhook that receives a customer question. Add the LLM node Attach an OpenAI Chat Model (or Gemini) node. In the node's parameters, craft a system prompt that scopes the agent. For a support bot, something like: You are a helpful support agent for our SaaS product. Use the tools provided to answer questions. If you don't know, say you need human help. This prompt is the boundary of the agent's autonomy. Keep it specific — vagueness leads to hallucinations. Attach tool and webhook nodes Here's where n8n shines. Drag a Function node to run custom JavaScript (e.g., querying a database) or a HTTP Request node to call an external API. Wire them as "tools" by connecting them to the LLM node's tool output. In the LLM node

You asked your AI to help you plan a trip. It gave you a paragraph about packing layers and booking early. You needed a checklist, a hotel shortlist, a flight window, and a rough daily schedule. What you got was a thoughtful non-answer dressed up as advice. That gap — between what AI tells you and what it could actually do for you — is the gap agentic AI is designed to close. And most people don't know it exists. The Difference Between Answering and Acting Standard AI models are trained to respond. You send a prompt, they generate a reply. The entire interaction lives inside a single text exchange. Agentic AI operates differently. Instead of producing one answer, it takes a goal and breaks it into a sequence of steps — then executes them, one after another, checking its own output along the way. It can look things up, organize information, write to a document, revisit a step if something doesn't look right, and deliver a final result that's actually usable. The travel example makes this concrete. A conversational model tells you to pack a rain jacket. An agentic setup builds you the trip: it pulls destination weather data, generates a packing list specific to your travel dates, identifies hotels in your price range, and drops everything into a structured itinerary. Same goal. Completely different level of output. Author's note: The word "agentic" has been overloaded to the point of meaninglessness in tech marketing. For our purposes here, it means one specific thing — an AI that runs a loop: think, act, observe the result, decide the next action. If it's not doing all four of those things in sequence, it's not really an agent. It's just a chatbot with extra steps. Why This Loop Changes Everything The reason agentic AI feels qualitatively different isn't magic — it's architecture. The core mechanic comes from a framework called ReAct (short for Reasoning and Acting), introduced in a 2023 paper by Yao et al. and now foundational to most production agent systems. The l

The original safety gate on the Dolt-over-MCP plugin tried to keep a Claude Code agent harmless by excluding "history-affecting tools" from its MCP grant. It was the wrong granularity, and it did nothing. MCP exposes the entire database through one tool — query / exec — and that tool carries every SQL verb. SELECT rides it. So does CALL DOLT_PUSH , CALL DOLT_RESET('--hard') , DROP DATABASE , and CALL DOLT_BRANCH('-D', 'main') . Excluding "dangerous tools" from the grant accomplishes nothing, because the dangerous verbs live inside the one tool you already granted. The destructive operations were never separate tools to exclude. This is the reframe the whole Phase 0 hardening pass turned on: a tool-name allowlist is meaningless for any tool that carries a sub-language. SQL is a sub-language. So is the shell behind a Bash tool. So is anything behind an eval . If the tool can run arbitrary statements in some grammar, the only boundary that means anything is one that reads the statement. It is the move from tool-name allowlisting to capability-based security: the grant stops being "you may call the query tool" and becomes "you may run these statement classes inside it." Why not just allowlist the safe tools? Because there is exactly one tool, and it is not safe or unsafe — it is whatever statement you hand it. You cannot partition a single door into a safe door and a dangerous door by naming. The same logic kills the next-obvious fix: a denylist of dangerous verbs. Blacklist DOLT_PUSH , DOLT_RESET , DROP ... and miss DOLT_REBASE , or the proc Dolt ships next quarter, or a CALL whose name your regex didn't anticipate. A denylist is only as good as your imagination on the day you wrote it. The fix inverts that. You add safety by enumerating what is safe, not by blacklisting what is dangerous. Anything you cannot positively classify as safe is treated as the most dangerous thing it could be. Default-deny the unknown. It's least privilege applied to a grammar: the agent get

Ahnii! If you let an AI agent modify your application, the agent needs more than a text editor. Raw str_replace on a PHP file passes a lot of tests and still breaks things an hour later in production, because the tool has no idea what the file actually represents. Bimaaji is the Waaseyaa package that gives agents a structured path from "I want to add a field to this entity" to a reviewable patch that a community's sovereignty rules have already vetted. This post walks through what shipped in waaseyaa/bimaaji and why each piece exists. Prerequisites: familiarity with Waaseyaa's package layout, PHP 8.4+, and the idea that an application has more state than the filesystem (routes, entities, introspection metadata). Why not just let the agent edit files The failure mode you want to avoid: an agent reads a prompt like "add a published_at field to the Post entity," does a reasonable-looking edit to Post.php , and leaves the rest of the app inconsistent. The migration is missing. The JSON:API resource doesn't expose the field. The admin panel still doesn't know it exists. The sovereignty profile that was supposed to block the change on a local-only deployment never got consulted. Each of those is a different subsystem. A good agent can write a correct edit to any one of them. What a filesystem-level tool cannot do is ensure the edit is coordinated across all of them and is allowed under the community's posture. Bimaaji separates that problem into three stages: introspect, propose, patch. The pipeline The package description (from packages/bimaaji/composer.json ) spells it out: application graph introspection and agent-safe mutation for Waaseyaa. The flow is: Introspection → ApplicationGraph → MutationRequest → Validator → PatchGenerator → PatchSet An agent reads the graph, submits a structured mutation request, a validator checks it against sovereignty rules, and the patch generator returns reviewable diffs. Nothing touches the filesystem until a human (or a higher-level w

The agent brand is irrelevant; the work item is everything. I have watched teams argue about Copilot Studio versus Claude Code versus Codex as if the model decides whether their build succeeds. It does not. Your agentic development power platform effort lives or dies on one thing: whether the Azure DevOps work item you hand the agent is a machine-readable spec or a vaguely worded wish. Swap the agent all you want. If the requirement is unstructured, every agent guesses, and every guess is a different guess. This article is opinionated on exactly one point and neutral on everything else. Neutral on the tool. Ruthless about the spec. Why "AI-assisted" Power Platform dev stalls on real teams The agent guesses intent because the acceptance criteria live in a stale wiki, a Teams thread, or someone's head. That is the whole failure. Switching from one agent to another does not close the gap. The missing spec does. Prompt-by-prompt building has a second problem that shows up later and hurts more. One maker gets a working flow out of a chat session, but nobody else can reproduce it and no one can audit it. You have a solution that exists and a rationale that evaporated. For teams doing serious dynamics 365 ai development , that is not acceleration. That is a single point of failure wearing a productivity costume. Frame the cost honestly. Say a rework cycle caught in UAT runs roughly 5x the cost of the same fix at design time. Illustrative; calibrate against your own data, actuals vary. Under that assumption, the line item bleeding your budget is the improvised requirement, not the agent license. You are paying to rediscover intent three environments too late. Takeaway: if your requirement is not structured, your agent is improvising, and the brand of agent does not matter. Make the ADO work item the single source of truth An agent reads fields. It does not read the room. So the work item has to carry everything the agent needs in a shape a parser can trust every single time

As of May 2026, more than 80% of the code Anthropic ships is written by Claude, not by its human engineers. The company disclosed the figure in an essay called When AI builds itself , with coverage from Tom's Hardware and VentureBeat . Key facts What: Anthropic says more than 80 percent of the code it ships is now written by its own model, Claude, and the more interesting numbers are about judgment. When: 2026-06-23 Primary source: read the source Two years ago this share sat in the low single digits. The shift accelerated after Anthropic released Claude Code , a tool that lets the model read an entire codebase, make changes, run tests, and fix what breaks without human help. The human role has flipped: engineers used to author the code while the machine assisted; now the machine authors the code and engineers review, approve, reject, and steer. Anthropic reports its typical engineer ships roughly eight times as much code per quarter as a few years ago — not because people type faster, but because they spend their day reviewing the model's output instead of writing from scratch. Think of it as a newsroom where a tireless junior writer drafts every article and senior editors only sign off. Volume goes way up. But the 80% figure is less impressive than it sounds: a draft that a human must check, fix, and approve is not the same as a writer you can leave unsupervised. Most of those lines still pass through a person. On its own, this number measures effort the machine saves, not work it can be trusted to do without oversight. The results buried deeper in the essay matter more, because they concern taste rather than volume. Anthropic ran a recurring test where the model chooses the best next step in a research project, then compared its choices against its own scientists. Late last year the model was roughly a coin flip against the humans. By spring 2026, an unreleased internal model was picking the better direction clearly more often than its own researchers. Choosing w

A CFO will not act on a number an LLM eyeballed. They will not act on a number the model "estimated" by reasoning over a usage dump. And they should not — because the moment a language model emits a dollar figure it computed itself, that figure is a guess wearing the costume of a fact. This is the design constraint behind databricks-cost-leak-hunter , the pilot skill of the databricks-pack v2 rebuild shipped in the claude-code-plugins marketplace ( PR #906 ). Given a live, authenticated Databricks workspace, it surfaces real cost leaks across four named categories, ranks them by monthly dollar impact, and emits a report a finance reader can act on. The marketplace validator graded it B (88/100, zero errors). The SKILL.md is 329 lines. The single most important thing in it is a rule the model is structurally prevented from breaking: the LLM never does the dollar arithmetic. Why not just let the agent read the bill and summarize it? Because that is exactly how you ship a confidently wrong cost report. Hand a model a few thousand rows of system.billing.usage and ask it for the top cost leaks, and it will give you a fluent answer. It will add DBUs. It will multiply by a price it half-remembers. It will round. Every one of those steps is a place the model can be plausibly, invisibly wrong — and the output reads identically whether the math is right or hallucinated. The failure mode of an LLM doing FinOps is not a crash. It is a clean, well-formatted, wrong number. The fix is architectural, not prompt-engineering. The model is allowed to decide what to look for and how to explain it . It is never allowed to be the calculator. The dollar primitive: confirmed, never estimated Every confirmed figure comes from the customer's own billing tables — system.billing.usage joined to system.billing.list_prices . Not a model estimate. Not a public price list. The number Databricks actually billed. That join is defined once, as a priced CTE, and reused by every category query. Usage i

I spend a lot of time in the AI space -- reading papers, building things, talking to engineers who are actually shipping. And there is a gap between what the demos show and what production systems actually look like that nobody is being fully honest about. So here is my honest take on where things actually are. The Problem With How We Talk About AI Agents Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent. This dilution is not just semantic. It is causing real engineering mistakes. When you do not have a precise definition for what you are building, you end up over-engineering simple pipelines and under-engineering genuinely complex ones. I have seen teams spend weeks adding "agentic" orchestration to workflows that would have been fine as a single well-structured prompt. Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done. Everything else is just a fancy function call. 🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface. 🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere. ✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing. What Is Actually Happening in Production Right Now The honest picture from teams I follow and talk to: Most real agent deployments are narrow. They do one thing well. Customer support triage. Document extraction. Code review on a specific codebase. They are not general-purpose reasoning engines. They are purpose-built pipelines with some intelligence in the decision layer. The teams getting good results are not chasing the latest model release. They are obsessing over: ☑️ Tool design -- what can the agent actually call, and how clean is the interface ☑️ Failure handling -- wh

A single shared API key is fine right up until a second person uses it. intent-brain — the system, repo qmd-team-intent-kb , renamed to the intent-brain plugin v0.4.0 this day — is a team knowledge base. A Fastify HTTP API sits over a governed memory corpus. In front of that API is an MCP server named teamkb , so a teammate doesn't open a dashboard or learn an endpoint. They ask in Claude Code and get a cited answer back with qmd:// citations. That's the whole pitch: institutional memory you query in the same place you write code. Up to this day it authenticated with one shared TEAMKB_API_KEY . The shared key has two failures that only show up once the tool has more than one user. First, every request looks identical, so the audit log can't say who asked. Second, revoking one person means rotating the key for everyone — there's no per-person handle to drop. Both are structural, not bugs you patch. You fix them by giving each person their own credential. The work closed that gap with three things, in this order: per-user tokens (identity), a server-side write gate (authorization), and a per-read access log (audit). The through-line: the API is the real boundary. The MCP client-side tool gate is UX, not security. And the per-read access log stays separate from the governance audit trail — separate log, not no log. Identity: per-user tokens replace the shared key apps/api/src/auth/token-registry.ts . Each token resolves to a record: { actor, role } , where role is 'admin' | 'member' . The shared key's two failures both dissolve here — every request now carries an actor , and revoking one person is dropping one record, not a team-wide rotation. Tokens come from layered sources, in precedence order: explicit records → a TEAMKB_TOKENS JSON env → a TEAMKB_TOKENS_FILE (default ~/.teamkb/tokens.json ) → the legacy single TEAMKB_API_KEY , which becomes one admin token with actor "shared" for back-compat. Each entry is a bearer token resolved to an identity at request time. Ma