标签：#LLM

In this post I'll talk about my brief experience with muse , Meta's LLM harness for developers, as well as Muse Spark 1.3, their latest frontier-level model. The Bad I'll start with the bad, just because I like to end with the positive :) Skill Usage It's not that good following skills. If the skill has disable-model-invocation , sometimes it refuses to launch it, even if you manually call it. I think it happens when you call the skill mid-sentence, but it's not consistent. It's also not as good as other models at following skill instructions. It seems to get confused more often. For example, I have one skill that will address an issue from GitHub to PR. In Claude (Opus 5) and Cursor (Grok 4.6) it works perfectly. The first step is grilling the issue, after that's finished, the next step is autonomous, plan, implement with TDD, review and open PR. With Muse Spark 1.3, sometimes the skill will not continue and I have to nudge it for the next step, just saying something like "continue" is enough, but surely is annoying. Formatting The output is not great. Sometimes it will show me raw markdown, sometimes not. It's not consistent. Sandbox Having a sandbox is good, but in this case, it's a bit too restrictive. For example, I'm working with a Firebase project and I want to use the emulators. Well, too bad. The sandbox doesn't allow you to run files outside your workspace or use external ports. That would be great if I could add exceptions or some kind of configuration, but you can't. You are basically forced into --yolo mode if you don't want to be prompted on repeat for the same things over and over. What's sad is that even if you want to give them access, the models will just get stuck asking for permissions for the same thing over and over again and eventually they will just be stuck doing nothing. The Good Not everything is bad, of course. With a bit of effort I think it's actually quite usable. Price The main reason I decided to try the model. The subscription plan

Disclosure up front: Mnemoverse publishes this post, and Mnemoverse is one of the seven tools on it, so read every row knowing the author holds a position. With that on the table, the honest answer to the question in the title has not changed all year: there is no single best AI agent memory in 2026. There is a best answer to one prior question, and it decides more than any feature list: how much of your application should the memory system own? This post turns that question into a decision map. The deep, dated per-system read lives in Mem0 vs Zep vs Letta vs Cognee vs Supermemory ; head-to-head pages live on the comparison hub . TL;DR No single best exists. The boundary question (how much of the app the memory system owns) sorts the field faster than any benchmark. Seven systems, seven different jobs: embeddable SDK, temporal fact graph, self-editing runtime, ingestion pipeline, managed context engine, framework primitive, cross-tool managed memory. A tool chosen by ranking gets replaced; a tool chosen by job stays. Every claim here was checked against the vendors' public pages in July and August 2026, and these products change fast: verify against their own docs before you commit. The decision map The boundary question is the one-sentence filter this map runs on: how much of your application should the memory system own? Answer it first, and most of the table collapses to one or two rows. Your job Start with The cost you accept Embed an open-source memory SDK inside one application you fully own Mem0 You wire it into each app yourself; Apache-2.0 self-hosting is real Track facts that change over time, with valid-from and valid-to history Zep You operate Graphiti with a Neo4j backend, or take the managed cloud Build an agent that curates and edits its own memory as first-class behavior Letta You adopt a full runtime from the MemGPT line, not just a memory API Turn documents and data sources into a queryable knowledge graph Cognee Pipeline thinking: Extract, Cognify

Untrusted retrieval is now a more common production failure than a weak prompt, because agents ingest memory they never score. A seventy-minute workshop can add a cheap scoring gate, a replayable log, and a reject path before generation. Students leave with a runnable Python harness, a four-row decision table, and a timing plan they can repeat. The method stays useful if every product name is removed and the scoring host is only a free server. What you will build This workshop treats retrieved snippets as untrusted input, not as ground truth the model should quote. You will capture a retrieval batch, score each chunk against a written rubric, and allow only passing chunks into the prompt. A JSONL replay log records the fingerprint, score, and decision so later failures can be diffed. The generation model never sees dropped text, which keeps stale or planted memory out of the answer. Timing box 00:00–00:10 — install dependencies, copy the harness, and load the sample corpus 00:10–00:30 — Exercise 1: capture retrieval payloads and stable fingerprints 00:30–00:50 — Exercise 2: score chunks with a rubric and an optional free model 00:50–00:65 — Exercise 3: gate the prompt and replay one rejected case 00:65–00:70 — debrief against the decision table and list remaining holes The schedule is a teaching box, not a production SLA, and it assumes one laptop plus one HTTP scoring endpoint. If the endpoint is slow, freeze Exercise 2 after five scored chunks and continue with the logged samples. Do not expand the window to chase a perfect judge; the learning goal is a gate you can rerun. Why a scoring pass belongs in front of generation Cheap code generation has made it easy to wire a retriever into a chat loop in an afternoon. The failure mode that follows is quieter than a crash: the model answers fluently from a chunk that is expired, off-topic, or injected. Architecture diagrams rarely show that hop as a trust boundary, so teams skip scoring and jump to a larger generator. A

Master Prompts in 2026: Stop Prompting Like It's 2023 I still see people paste a 40-line “act as a senior expert with 20 years of experience” block into ChatGPT and call it engineering. That stopped working as a strategy a while ago. Models got better. Context windows got bigger. Agents started calling tools. And the failure mode shifted. It’s rarely “the model is dumb” now. It’s “your system has no contract.” This is a long, practical write-up on master prompts — the stable policy layer above individual tasks. How to write them. How to force planning. How to run Plan → Act → Observe → Verify without theater. How to make the same prompt useful to a tired human at 11pm and to an agent loop that only understands schemas. I’ve broken enough production prompts across GPT-4o, Claude 3.5 Sonnet, and Gemini-class stacks to have opinions. Some of them are uncomfortable. TL;DR / Key Takeaways A master prompt is not a clever sentence. It’s the policy layer : role, success criteria, process, constraints, output contract, failure handling. Production reliability comes from LLM orchestration patterns — plan JSON, single-task executors, and explicit done_when checks — not from longer personality blocks. JSON contracts + verification beat free-form answers. Agents that can’t prove completion will invent it. Treat prompts like code: version them, eval them, and put a real verify step after generation (including SEO/quality checks when you publish). Table of Contents What a master prompt actually is The 7-part anatomy that doesn’t collapse under pressure Frameworks worth keeping (and which ones to ignore) Planning is the real skill From plan to agent loop Context engineering beats clever wording Few-shot, JSON contracts, and the anti-hallucination rule Copy-paste masters you can actually deploy A real publish pipeline (including the verify step people skip) Eval or you’re guessing Failure patterns I keep seeing PromptOps: treat prompts like code One universal master prompt Ship chec

Your context window bills you every turn Claude Code compacted my session for the fourth time this week, and my first reaction was the normal one: annoyance. It just erased everything and I have to re-explain half of it. Then I pointed Claude Code at its own transcript files and did the arithmetic instead of the complaining. The transcripts are just JSONL on disk — every request, every token count, timestamped. Three sessions, 5,288 requests, a few minutes of parsing. The reframe that came out of it: compaction isn't the tax. It's the tax getting paid off. The tax is every turn before that. TL;DR Every turn re-sends the entire conversation so far. Nothing is "remembered" for free — a 900K-token context gets re-read, in some discounted form, on turn 901. Across three real sessions: 1.99 billion tokens read from cache, 62 million tokens written to it. A 32:1 ratio — each token you put in context gets paid for roughly thirty-two more times before it leaves. Four auto-compactions fired at 968K, 996K, 999K, and 771K tokens. Each one took 108–140 seconds of wall-clock time doing nothing but summarizing. Immediately after, cache-read dropped from ~990K to 0. At list-price API rates, cache discounting saved an estimated 86% versus paying full input price every turn — which is the whole mechanism working as intended, and also the reason a 1M-token context doesn't bankrupt anyone by turn 50. None of this is "Claude Code is expensive." It's "the meter is per-turn, not per-token-ever-seen," and almost nobody reasons about a session that way while they're in it. What actually happens on turn 500 There's no persistent working memory across a conversation. Each API call is stateless — the model sees whatever text is in the request, and nothing else. So a coding session's "memory" is an illusion built entirely out of re-sending: every prior file read, every tool result, every message, concatenated and shipped again, every single turn. Prompt caching is the thing that makes this sur

AI apps have moved past simple chat boxes. Today's AI systems need agents, tools, memory, data, security, monitoring, and scale. The hard part is not calling an LLM API. The hard part is building a reliable system around that API call. 1. From LLM Demo to Production System A demo is simple: flowchart LR A[Prompt] --> B[Model] --> C[Response] A real production system looks very different: flowchart TD U[User] --> API[API] API --> APP[Application Layer] APP --> ORCH[AI Orchestration] ORCH --> LLM[LLM] ORCH --> TOOLS[Tools] ORCH --> RAG[RAG] ORCH --> MEM[Memory] ORCH --> GUARD[Guardrails] ORCH --> DATA[Data + Infrastructure] DATA --> OBS[Observability] Each box matters. If you skip Guardrails , bad input can hijack your system. If you skip Memory , every message re-explains itself and costs more tokens. If you skip Observability , you won't know why the system failed until a user tells you. The rest of this article walks through each box. 2. Where AWS Fits Instead of listing AWS services, let's match each one to a real problem. Problem AWS Service Why Need a foundation model Amazon Bedrock Managed access to multiple LLMs, no infra to run Store documents and files S3 Cheap, durable, scales easily Store app data RDS / Aurora / DynamoDB Structured data, users, sessions, transactions Search by meaning (retrieval) OpenSearch / pgvector Vector search for RAG Run code Lambda / ECS Serverless or container compute for your app logic Handle async work SQS / EventBridge Queue jobs, decouple slow tasks, avoid lost requests Watch the system CloudWatch Logs, metrics, alarms Keep it secure IAM / Secrets Manager Access control and safe storage of keys flowchart LR subgraph Compute L[Lambda / ECS] end subgraph Data S3[(S3)] DB[(RDS / DynamoDB)] VEC[(OpenSearch / pgvector)] end subgraph AI BR[Bedrock] end subgraph Ops CW[CloudWatch] SEC[IAM / Secrets Manager] end L --> BR L --> S3 L --> DB L --> VEC L --> CW L --> SEC 3. AI Agents Change the Architecture An agent doesn't just answer — i

Let me start with a question. Why did I fear development done by artificial intelligence? The answer is plain. AI can build software, and on top of that, it never rests. AI has no labor law People rest. There are labor laws. We sleep at night. We need weekends. Work too many days in a row and the body breaks. So there is a ceiling on how much work a person can move forward in a day. For a long time, we treated that ceiling as a given. But AI has no labor law. It works at night. It works on weekends. Give it an instruction once, and it does not stop until morning. It never says it is tired. It takes no breaks. It keeps working for hours at the same quality. This difference did not fit inside the word "convenient." What I felt was fear. This was not a story about one more handy tool. It was a story about the ground under the speed of work changing at the root. Claude Code came out about a year and a half ago. That is when I understood. The company that runs it 24 hours takes the first-mover advantage. And the company that can punch with money wins. This is not a cynical take. It is the obvious consequence. The first mover wins — that story is not new. Whoever enters a market early takes the ground. They set the standard. Everyone after them chases the gap. AI widens that gap by the day. A company that moved ten hours forward overnight and a company that stood still overnight are ten hours apart by morning. The gap compounds daily. Can you catch up by hiring more people? You cannot. Hiring takes time. Post the opening, interview, teach, wait for people to settle in. That takes months. Meanwhile, the other side's AI keeps moving through the night. The speed of adding people cannot match the speed of adding AI. So the moment a small company steps into a contest of headcount, it loses. It was a ring we should never have entered. Companies that can punch with money win — obviously Why can I say it becomes a contest of money? Because there is no ceiling on how fast you can

Previously: 9 Bugs That All Looked Like a Working System · I Built an AI That Rewrites Its Own...

The first number I saw on Qwen3.6-35B-A3B was 12 tok/s . I almost hit publish on "Qwen regressed at generation speed" and moved on. The 3.5 baseline on the same RTX 4070 was 34.6 tok/s. A new generation running at a third of the old one would have been a hell of a headline. It was also completely wrong. The culprit was not the model. Another process on the box was sitting on 9-11 GB of VRAM, so the layers that were supposed to live on the GPU were spilling to system RAM. The tell was that my sanity-check run of Qwen3.5 slowed down too. When two independent models degrade together, the model is not the variable. I killed the offending process, re-measured, and got numbers that told a completely different story. Model Generation speed tg128 (tok/s) Runs Qwen3.6-35B-A3B 38.76 ± 0.82 avg of 3 Qwen3.5-35B-A3B 36.7 ± 1.4 avg of 3 (range 34.9-38.6) Both models sit inside the ±1.5 tok/s band on the same RTX 4070. On the tokens-per-second axis, "the new generation" is not a story. Same architecture, same activated-parameter count (3B active out of 35B), same MoE routing pattern. The half-speed regression was a measurement bug, and it lived for about half a day before its own inconsistency killed it. The lesson I keep re-learning: when the number you got is dramatically convenient for your narrative, measure it again before you write anything. The moment I could sell 12 tok/s as a regression, I should have been suspicious. The version of me that ran the second test earned the version of me that got to keep his self-respect. So where did the generation move to? If speed did not change, does the 3.5-to-3.6 bump mean anything? It does. The move lives on a different axis. The official Qwen3.6-35B-A3B model card publishes benchmarks with a very lopsided shape: Benchmark Qwen3.5 Qwen3.6 Lift Terminal-Bench 2.0 40.5 51.5 +27% QwenWebBench (frontend generation) 978 1,397 +43% SWE-bench Pro 44.6 49.5 +11% LiveCodeBench v6 74.6 80.4 +8% SWE-bench Verified 70.0 73.4 +5% AIME26 91.0 92.7

Previously: 9 Bugs That All Looked Like a Working System · I Built an AI That Rewrites Its Own Prompts · The Edit That Fixed 4 Tasks and Broke 1 · The Gate Is the Product · The Doctor Who Diagnosed Every Patient · 4 Models, 0 Promotable Edits In v0.1.0, an edit fixed 4 tasks and broke 1. Net +3 on 26 tasks. p=0.23. Gate rejected. The ceiling was clear: if you do not move enough tasks, the gate should say no. In v0.2.0, we expanded the A/B corpus to 40 tasks. We fixed the pipeline bugs. We added rejection context. We tested stronger models. The math got cleaner, not kinder. The ceiling shifted. It did not disappear. The v0.1.0 Result: 26 Tasks, 5 Movable, p=0.23 The edit was real. It fixed 4 tasks and broke 1: Task Prompt A Prompt B Expected Change classify-015 technical urgent urgent FIXED classify-023 security urgent, security urgent, security FIXED classify-024 feature feature, billing feature, billing FIXED classify-029 feature other other FIXED classify-014 technical feature technical BROKEN Net: +3. Mean delta = 0.115. p=0.23. The permutation test computes this by shuffling task labels 1,000 times and counting how often random chance produces a delta ≥ 0.115. 23% of the time — above the 5% threshold. The sign-test floor with 5 discordant pairs out of 26 is ~0.031 one-sided. Even a flawless edit that fixed all 5 would barely clear p<0.05 two-sided. The v0.2.0 Result: 40 Tasks, Still Nothing We expanded to 40 tasks. We should have more power. Here's what Mistral 24B, our strongest analyzer, produced: Iter p-value Mean delta Accuracy 1 0.55 +0.025 64% 2 0.52 +0.025 64% 3 1.0 0.0 64% 4 0.52 +0.025 64% 5 0.77 -0.025 64% Mistral produced positive deltas in 3 of 5 iterations. That is real signal. But the delta is +0.025 — 2.5% improvement on 40 tasks. At p~0.5, there is roughly a coin-flip chance this is noise. The ceiling did not disappear. It moved: with 40 tasks, the sign-test floor for a flawless edit that moves 5 tasks is ~0.016 one-sided — clearable. But Mistral

If your AI application suddenly becomes 300–500ms slower after adding an AI gateway, the first...

I spend a lot of time in the AI space -- reading papers, building things, talking to engineers who are actually shipping. And there is a gap between what the demos show and what production systems actually look like that nobody is being fully honest about. So here is my honest take on where things actually are. The Problem With How We Talk About AI Agents Everyone is calling everything an "agent" right now. A function that calls a tool? Agent. A chatbot with memory? Agent. A script with a loop? Agent. This dilution is not just semantic. It is causing real engineering mistakes. When you do not have a precise definition for what you are building, you end up over-engineering simple pipelines and under-engineering genuinely complex ones. I have seen teams spend weeks adding "agentic" orchestration to workflows that would have been fine as a single well-structured prompt. Here is the definition I keep coming back to: an agent is a system that has an objective, not just an instruction. It decides what to do next. It handles failure. It knows when it is done. Everything else is just a fancy function call. 🟢 If your system needs a human to tell it each step, it is not an agent. It is a chat interface. 🔵 If your system can recover from a failed tool call and try a different approach, you are getting somewhere. ✅ If your system can decompose a goal into subtasks and delegate them, that is the real thing. What Is Actually Happening in Production Right Now The honest picture from teams I follow and talk to: Most real agent deployments are narrow. They do one thing well. Customer support triage. Document extraction. Code review on a specific codebase. They are not general-purpose reasoning engines. They are purpose-built pipelines with some intelligence in the decision layer. The teams getting good results are not chasing the latest model release. They are obsessing over: ☑️ Tool design -- what can the agent actually call, and how clean is the interface ☑️ Failure handling -- wh

Here's the moment a lot of us had this year. You built an agent. It was genuinely impressive — it...

Stop drawing the graph: reactive agents over versioned artifacts Most agent frameworks make you draw the graph : connect nodes, wire memory, declare control flow. But a knowledge problem is not a workflow. Take a realistic question: "Why did infrastructure costs increase in Q2?" The answer may need Confluence docs, GitLab merge requests, CSV spend data, a calculation, source verification — and a clarifying question. The next question needs a different path. There is no universal graph here, and asking a developer to draw one for every possible question is asking them to predict the future. So we built an agent runtime where you don't describe execution at all . You describe what artifacts exist and what agents can do with them; the runtime derives what runs next from state changes. Agents react to events. There is no graph and no node pipeline. This is ctxloom — a reactive, artifact-driven agent runtime, now open source. What it looks like The whole loop is: create an artifact → agents react → one atomic patch → context advances . A knowledge question — say, "how much does GPU inference cost?" — becomes a chain of typed artifacts: UserQuery → TypedDoc → Evidence → Claim → Answer . Each is produced by an agent that reacts to the previous artifact. No graph describes this chain; it falls out of what each agent consumes and produces. ARTIFACT CREATED / UPDATED │ ▼ AGENTS REACT ──self.effects──► Effects ──compile──► Patch ▲ │ └──────────────────────────────────────────────────────┘ Context v+1 The event that wakes an agent is derived from that same change — the causal chain can never drift from the actual state. from pydantic import BaseModel from ctxloom import Budget , Consume , Context , Runtime , RuntimeResources , create_agent , produce , structured_llm class Question ( BaseModel ): text : str class FindingBody ( BaseModel ): text : str class Finding ( BaseModel ): text : str source : str class Conclusion ( BaseModel ): text : str @produce ( Finding ) async def ana

LLM LLM is a model, which means an equation. Example: y = mx + c y = m1x^3 + m2x^2 + m3x + m4 A model is actually made up of weights . In any model, e.g., ChatGPT model or Gemini model, they would have used a large amount of input to train the model. Input means a large amount of text/image data that is available on the internet. The input would have been fed into the Transformer architecture to get the output, which is the model. Weights are floating-point numbers that represent the model's learned parameters. A 10B or 100B parameter model means how many parameters (weights) are present inside the model. We cannot store a large-parameter model on our computer due to inadequate storage and computational power. Storage and CPU/GPU power decide what size of model can be run on a computer. To run a model locally, we can use one of the following tools: Llama.cpp Ollama LM Studio Open Weight Model vs Open Source Model An open-weight model shares its model weights. So, we can run them, fine-tune them, and host them on a local system. Here, the training code, data, and full methodology are not shared. Whereas, in an open-source model , the weights, training code, data, and sometimes the dataset are shared. Why Do We Need to Use LLMs? LLM is a next-word predictor . Suppose we ask: "Hi, how..." The answer can be: How are you? How do you do? How is your life? etc. These are possibilities. Here, most of the time, the answer will be "How are you?" because if a word has more presence, it has a higher possibility of occurring. Each possibility will have a score between 0 and 1 . We have 3 controlling parameters to control the output generated by the LLM. 1. Temperature Usually set from 0–1 . It controls the randomness of the model. If the value is 0–0.3 , which is low, it means generating the most likely words, i.e., facts or commonly occurring words. If the value is high, the model will choose less likely words. We use this high value in storytelling and creative writing . 2. To

In this Expo & React Native tutorial, you’ll learn how to run a large language model (LLM) directly on a user’s device: no server, no API key needed. We’ll start from scratch with a simple chat exchange, and progressively introduce more advanced features: multimodal input, speech-to-text, text-to-speech, voice activity detection, tool calling and RAG. Each concept is explained before the code, so you can follow along whether you're new to on-device AI. Why run AI On-Device? Most AI features rely on a cloud API: you send a request to a remote server, it runs the model, and sends a response back. That works well, but it comes with tradeoffs. Running the model directly on the device avoids all of them: Works offline — no internet connection required Privacy by design — user data never leaves the device Low latency — no network round-trip No cloud costs — inference is free The tradeoff is raw capability: on-device models are smaller and less powerful than frontier cloud models. But for many use cases like summarization, chatbots, or local search, they're more than good enough. About NobodyWho We'll use the NobodyWho library throughout this tutorial. It wraps llama.cpp in Rust and exposes a clean React Native API for running locally any model in .gguf format. Install it with npm install react-native-nobodywho or npx expo install react-native-nobodywho for Expo. Loading a Model NobodyWho can download a GGUF model for you directly from Hugging Face, cache it, and reuse it on every subsequent launch. That means you don't need to bundle anything into your app or manage downloads yourself: import { Chat } from " react-native-nobodywho " ; const chat = await Chat . fromPath ({ modelPath : " huggingface:NobodyWho/Qwen_Qwen3-0.6B-GGUF/Qwen_Qwen3-0.6B-Q4_K_M.gguf " , }); The first time this runs, the model is downloaded to the app’s cache directory. Every call after that loads the model directly. modelPath accepts a few different forms: Form Example Notes HuggingFace reference hf

Last week a colleague pasted a production config.yml into an AI chat, asked why the connection kept dropping, and got a working fix in three minutes. The file also contained a client secret. Now that secret sits in a model provider's logs. Maybe training data, too. You don't know. That's the problem. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The opinions are mine. I've written here about repo quarantine and dependency triage. This post is narrower: where do you draw the line between your code and a free model? Free model access and a free server are real options, but they shift trust boundaries. MonkeyCode, the open-source platform, offers both. I'm not going to quote quotas or hardware specs — they change faster than blog posts. The question is what you should send in the first place. The Trust Boundary Nobody Draws Think of your AI-assisted workflow as four zones: Zone 0: your terminal / IDE Zone 1: the agent or CLI process Zone 2: the platform API and its logs Zone 3: the model provider's infrastructure Every hop expands the attack surface. Zone 0 is yours. Zone 1 is mostly yours — unless the tool phones home. Zone 2 is someone else's server. "Free server" means Zone 2 is external by default. "Free model access" means your prompt leaves your network and lands in Zone 3. The trust boundary isn't the API call. It's the paste. Three Things That Should Never Cross the Boundary Secrets and credentials. API keys, passwords, tokens, private keys. Obvious, still happens daily. File paths and internal IPs. A stack trace like /srv/customer-42/checkout.py:314 reveals product structure, hostnames, and environment info. Unreleased code structure. AI models may memorize and regurgitate patterns. If your code is patent-sensitive or under NDA, don't feed it the source. Describe the logic instead. A Reproducible Gate: boundary_check.sh Stop relying on discipline. Add a mechanical gate. #!/usr/bin/env bash # boundary_check.sh - blocks high-ris

Proof over claim Last time I described the hour a platform shipped the feature I had spent months...

Nathan E. Sanders and I are writing a series of essays on real-world examples of democratic technologies for The Renovator . I haven’t been posting the full text on the blog because they’re a bit long, but here are links. Part 1 is about the Japanese digital democracy party, Team Mirai. Part 2 is about the Swiss Public AI model, Apertus. Part 3 is about the civic technologists of Open Knowledge Brazil. And the new one, Part 4 , is about civic AI in Scotland.

AgentSelfEdit is an open-source sidecar that rewrites its own system prompt from execution feedback....