标签：#llm

In multi-turn agent loops, the full context re-sends on every API call. A tool result added at turn 3 gets billed again at turns 4, 5, 6, 7... forever. Most of it is never read again. Standard observability tools tell you the total token count. They never tell you what's in there or how much of it is waste . That's what ContextLens fixes. What it does ContextLens is a diagnostic profiler for LLM agent context windows. It: Decomposes the context window into regions: system prompt, tool schemas, tool results, retrieved chunks, user messages, assistant messages Tracks which blocks get re-billed across turns using SHA-256 content hashing Runs 5 waste detectors and ranks findings by dollar cost Prints a concrete one-line fix for each finding Renders an interactive D3 treemap report as a self-contained HTML file No API key required. Works offline on saved traces. The five detectors Detector What it finds Duplicate Same block re-sent verbatim across multiple turns Near-Duplicate >85% Jaccard similarity between distinct blocks Stale Tool Result Tool output never referenced by a later assistant message Unused Tool Schema Tool defined every turn but never called Redundant Retrieval Retrieved chunk with <15% overlap with model output ---Run the built-in demo (simulates a 30-turn agent loop, no API key needed): python -c "import contextlens; contextlens.demo()" python examples/demo.py Live capture — Anthropic import anthropic import contextlens as cl client = anthropic.Anthropic() with cl.capture_anthropic(client, model="claude-3-5-sonnet-20241022") as collector: for turn in range(20): client.messages.create( model="claude-3-5-sonnet-20241022", max_tokens=1024, system="You are a helpful assistant.", messages=build_messages(turn), ) report = cl.analyze_trace(collector.build_trace()) print(f"Recoverable waste: {report.recoverable_tokens:,} tokens (${report.recoverable_cost_usd:.4f})") Live capture — OpenAI import openai import contextlens as cl client = openai.OpenAI() with cl.ca

I ran a 4-bit medical-triage model on a laptop GPU and on a CPU. For one patient, the GPU said urgent and the CPU said emergency. Same model file, same prompt, same input. Here's the mechanism and why "validated on hardware X" doesn't mean what you'd hope. I've been building Aegis-MD , a local-first emergency-department triage console. You hand it a structured clinical picture: chief complaint, vitals, age, pain score, a few risk modifiers, and it returns an urgency category on the Australasian Triage Scale (ATS 1–5), where ATS-1 means resuscitate now and ATS-5 means this can wait two hours . The whole thing runs on-device: a quantized MedGemma 4B served through Ollama, a small RAG layer over open guidelines, and a deterministic rule-based floor underneath the model. I never set out to write about floating-point arithmetic. But while running my evaluation set across two machines, I hit a result that stopped me, and the explanation turned out to be more interesting and more current than the textbook answer most people reach for. The setup, and why a 4-bit model Two things about Aegis-MD's design matter for this story. First, it's local by design. Triage data is about as sensitive as data gets, so nothing leaves the machine. The trade-off is that I'm running a small, heavily quantized model: MedGemma 1.5 4B at Q4_K_XL , about 3.4 GB rather than a frontier API. Four-bit weights are the price of running offline on consumer hardware. Second, I tested on two configurations on purpose. The intended deployment is local GPU inference (an RTX 5070 Ti Mobile, 12 GB). But the public demo runs CPU-only on Cloud Run, because GPU instances need a paid quota I don't have. So I ran the same evaluation against both: the GPU build and the CPU build, same model, same code, same prompts. The eval is 17 hand-written cases spanning all five ATS levels, cardiac arrest down to a medical-certificate request. (Seventeen is a smoke test, not a validation; I won't quote a percentage off a sampl

The AI landscape is evolving faster than ever. Keeping track of the right tools can feel like trying to drink from a firehose. I recently dug through my extensive bookmarks folders and compiled every single AI tool and Autonomous Agent I've saved. Whether you're looking for an autonomous coding agent, a rapid app builder, an LLM benchmark, or a creative suite, you need the right tool for the job. Bookmark this page, because you're going to want to refer back to it. Superdesign Maskara.ai Google Labs: Google's home for AI experiments - Google Labs Kilo Code - Open source AI agent VS Code extension hunyuan bolt.new Rocket.new | Build Web & Mobile Apps 10x Faster Without Code AI Web Scraping Extension | Chat4Data Sarvam AI Lovable Starc- film ShumerPrompt aipai.app Flowe MiniMax Official Website - Intelligence with everyone new.website | Build Websites with AI Higgsfield HeyBoss.ai Mitte Trickle AI - Turn your ideas into live apps and websites with AI. Dora: Start with AI, ship 3D animated websites without code Kimi AI – Think Bigger. Search Smarter. Write Better. a0.dev - Create Mobile Apps with AI sesame Vogent - Create AI Voice Agents Orchids - Make something beautiful Same PromptBase | Prompt Marketplace: Midjourney, ChatGPT, Sora, FLUX & more. LM Studio Mindstone Chat with Z.ai - Free AI for Presentations, Writing & Coding AI Model & API Providers Analysis | Artificial Analysis T3 Chat - Advanced AI Assistant & ChatGPT Alternative | $8/month Poe Freepik | All-in-One AI Creative Suite Replit – Build apps and sites with AI unwind ai Magic Patterns Soapbox - Build Your Decentralized Platform Shakespeare - AI Website Builder AI recruitment engine to hire top global talent | micro1 Ponder AI | New Way to Work with Knowledge Using AI Ask AI Questions · Question AI Search Engine · iAsk is a Free Answer Engine - Ask AI for Homework Help and Question AI for Research Assistance Firecrawl Kiro: The AI IDE for prototype to production Le Chat CodeArena – Which LLM codes best?

title: Your AI Agent Should Not Be Locked to One LLM Provider published: false description: Why serious AI agents need a provider-agnostic architecture, model routing, fallback, and a unified API gateway. tags: ai, llm, agents, architecture Your AI Agent Should Not Be Locked to One LLM Provider Most AI agent prototypes start the same way. You pick one model provider. You install one SDK. You write a few prompts. You add tool calling. You build a demo. It works. Until it does not. The moment you want to try another model, reduce cost, add fallback, improve latency, or support different task types, your simple agent starts turning into a messy collection of provider-specific logic. That is when you realize something important: A real AI agent should not be locked to one LLM provider. If you are building a personal AI agent, coding assistant, research assistant, internal workflow agent, or AI-native product, the model should be replaceable infrastructure — not a hardcoded dependency. The Problem with Single-Provider Agents A simple agent architecture often looks like this: CopyUser ↓ Agent ↓ One LLM Provider ↓ Response This is fine for a proof of concept. But real-world agent systems need more flexibility. Different tasks often need different models: Task Better Model Strategy Quick summarization Fast, low-cost model Complex coding Strong coding model Long document analysis Long-context model Reasoning-heavy planning Reasoning model Multilingual writing Model strong in that language Background automation Cheap and reliable model Production fallback Backup provider If your agent is deeply coupled to one provider, every optimization becomes harder. You cannot easily answer questions like: What happens if the provider is down? What if latency spikes? What if another model is cheaper for simple tasks? What if a new model is better for coding? What if a user wants Claude for writing but GPT for structured reasoning? What if you want to route Chinese tasks to a different mod

I Tried PewDiePie's Open-Source AI Workspace. It's Actually Good. Yes, that PewDiePie. Felix Kjellberg (110M YouTube subscribers) spent late 2025 building a home AI lab — 8 modified RTX 4090s, 256GB of VRAM, running on Arch Linux. He called it "The Swarm." He crashed it running 64 models in parallel. The web frontend he built for it? He open-sourced it. Called it Odysseus . It hit 59,000 GitHub stars fast. I dug into the code expecting a glorified Ollama wrapper. It's not. What it actually is Odysseus isn't just another chat UI. It bundles things no other self-hosted tool does in one place: Chat — local or cloud models (Ollama, vLLM, llama.cpp, OpenAI, OpenRouter, GitHub Copilot) Agent mode — shell, files, web, MCP tools, per-tool toggles Cookbook — scans your GPU, recommends models that actually fit, downloads and serves them in one click Deep Research — multi-step web research that writes you a cited report Email — IMAP/SMTP with AI triage, auto-tagging, draft replies Calendar — CalDAV sync with Radicale, Nextcloud, Apple, Fastmail Memory — persistent, evolving across all your conversations No cloud account. No telemetry. MIT license. Everything lives in your data/ folder. The Cookbook is the standout feature Every other self-hosted UI assumes you already know what model to run. Odysseus doesn't. It scans your hardware, scores 270+ models against your actual VRAM, and gives you a one-click download-and-serve. It understands GGUF vs FP8 vs AWQ. It picks the right backend (vLLM, llama.cpp, Metal on Apple Silicon). Downloaded models persist in a volume — no re-downloading after container restarts. For someone who wants local AI but finds the ecosystem confusing, this is the most accessible on-ramp that currently exists. The code is better than the meme suggests The README has a little ASCII bear face. Don't let it fool you. The entry point app.py is 1,092 lines of real production thinking. A few things that stood out: The .env loader handles Windows BOM silently: loa

A2A has become like a standard for enabling agent to agent communication, we could use the a2a-sdk for running and configuring the a2a server and its features such as agent card, agent skills, agent executor, request handler etc. However we are going to go with a simplified approach here with strands where the agent card will be fetched automatically. Let's get started! Server Initialize a uv project for the a2a server and switch to that directory. uv init ~/strands-a2a-server cd ~/strands-a2a-server Add the required packages. uv add python-dotenv == 1.2.2 strands-agents[a2a] == 1.42.0 Change the code in main.py to look like below. $ cat main . py from dotenv import load_dotenv from strands import Agent from strands.multiagent.a2a import A2AServer load_dotenv () def main (): agent = Agent ( callback_handler = None , description = " A sample strands agent " , model = " us.amazon.nova-micro-v1:0 " , ) a2a_server = A2AServer ( agent = agent ) a2a_server . serve () if __name__ == " __main__ " : main () I like the simplicity here, as you see above, it's quite simple to start a basic a2a server from with in strands, with just a couple of lines of code, we didn't have to install the a2a-sdk separately. Run the code, to start the a2a server. $ uv run main.py INFO: Started server process [18006] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://127.0.0.1:9000 (Press CTRL+C to quit) Client Let's now do the client part on a separate terminal. Initialize the project and switch the directory. uv init ~/strands-a2a-client cd ~/strands-a2a-client Modify main.py code to look as follows. import asyncio from strands.agent.a2a_agent import A2AAgent async def main (): agent = A2AAgent ( endpoint = " http://localhost:9000 " ) agent_card = await agent . get_agent_card () print ( " Invoking remote agent with agent card: " ) for key , value in agent_card : print ( key , " : " , value ) print ( ' - ' * 20 ) while True : prompt = input

Not long ago, I saw articles analyzing how Chinese people obtain US model API at low prices through non-compliant means, and also saw Chinese developer sharing their Vibecoding experiences. Very interesting, it seems the outside world is finally starting to understand our daily lives. I want to share a complete perspective here: how an ordinary Chinese student, also developer, accesses the most advanced US models. Including the evolution of various access methods over 4 years, the practical experience of using various methods, and the problems encountered, etc. I will try to describe it objectively and truthfully. Let's start by going back to November 2022, when OpenAI released "ChatGPT": Phase 1: "ChatGPT" ChatGPT was released, and it was big news in China, even though it predictably did not serve China. Even though I was still a high school student at the time, I was still interested, after all, it was the first time I saw something truly close to "intelligence". How to access it? There are two types of services generally inaccessible in China: one is that the GFW blocks the domain name or IP of the service, and the other is that the service provider refuses IPs from China. ChatGPT is both. The solution is also simple, use a proxy, which is a basic skill for Chinese developers. In addition, registration requires receiving a SMS verification code. Chinese mobile numbers are definitely not an option, but the solution is not difficult either, find a verification code receiving platform, use a temporary number to receive the verification code. Thus, I started using ChatGPT, which now seems like a model that speaks slowly and is not very smart. Phase 2: "Mirror Sites" During high school, I didn't have many scenarios to use ChatGPT. I started using AI more when I entered university, as AI is well-suited for dealing with those annoying assignments. It was the second half of 2023, and there was a new way to access it: mirror sites. "Mirror sites" originally referred to an

If you've been running local models/apps across more than one machine for any length of time, you've probably noticed that everything is served over plain HTTP, whether its the backend llm apis, the front end sites, or whatever other stuff you've tossed in: most of it is HTTP-only out of the box, no TLS option anywhere in sight. On one machine thats usually fine since its all loopback, but the second you spread apps across a few different computers ( which some of us do ), every prompt and every response starts crossing your LAN in plaintext. Is plaintext on your own LAN a huge deal? Honestly... a lot of folks would say it's probably low risk. But the moment you've got guests, other people's phones, or random IoT junk sharing that network, your prompts and the models responses flying around in the clear are more exposure than you'd probably be comfortable with if you sat down and thought about it. So, with that said- I figured Id write up how I've dealt with that, because the textbook answer ( certs ) is annoying enough on a local network that I think a lot of folks just dont bother. This is a lot easier, especially on something like a mac where you can make sure it kicks off automatically via launchd . Why not just do TLS The "correct" answer is to put TLS on everything; HTTPS everywhere. And you can. But walk through what that actually means on a home network full of mixed machines: You stand up your own little CA, then sign a cert for each host ( unless you want to deal with some code just straight up rejecting the cert ). You install and trust that CA on every client. Every browser, every OS trust store, and ( this is the annoying one ) every app that ships its own trust store and ignores the system one. Plenty of python and node apps do that. A lot of these local LLM apps dont even expose a TLS option, so to add it you front them with something like nginx or Caddy, which is now another moving part on every box ( Setting up Caddy is what convinced me to go this

Every LLM wire format claims token savings. Nobody proves whether AI models can actually comprehend the format at scale, or produce valid output in it. We ran 23 comprehension evals across 10 models and 3 providers. We ran generation evals across 11 models. Deterministic ground truth. No LLM judge. Reproducible from one command. JSON breaks at 500 records. GPT-5.5 returns empty strings. It can't even attempt an answer. Opus miscounts 500 as 356 and then spends 143 lines manually enumerating symbols to verify its own wrong answer. The format designed for "human readability" is incomprehensible to the systems actually reading it. TOON can't produce valid output. Claude Opus, the most capable model on the planet, scores 0/5 on TOON generation. GPT-5.4: 0/5. GPT-5.4-mini: 0/5. Gemini 3.1 Flash Lite: 0/5. The error is always the same: toon: cannot assign string to int . The model writes "target" in the distance column. TOON expects 0 . Every model fails the same way because the format's design forces an unnatural encoding step that models cannot perform unprompted. GCF wins both dimensions on every model tested. 100% comprehension on Claude Sonnet, Gemini 2.5 Pro, Gemini 3.1 Pro, and Gemini 3.5 Flash. 5/5 valid generation on every frontier model. Zero prior training. The format didn't exist until we built it and every model speaks it natively. Comprehension: 500 Symbols, 13 Questions, Zero Instructions A 500-symbol, 200-edge code graph. Encoded in GCF, TOON, and JSON. 13 structured extraction questions. The model gets the payload and a question. No format instructions. No system prompt. No hints. 23 runs. 22 wins. 0 losses. Model Runs GCF avg TOON avg JSON avg GCF margin Claude Opus 4.6 2 96.2% 84.6% 73.1% +11.6 vs TOON Claude Sonnet 4.6 2 100% 73.1% 53.8% +26.9 vs TOON Claude Haiku 4.5 2 96.2% 69.2% 57.7% +27.0 vs TOON GPT-5.5 5 84.1% 67.7% 45.8% +16.4 vs TOON GPT-5.4 4 76.4% 56.0% 44.1% +20.4 vs TOON GPT-5.4-mini 2 71.8% 64.1% 54.2% +7.7 vs TOON Gemini 2.5 Flash 3 80.6

1. update WSL environment sudo apt update && sudo apt upgrade -y 2. install dependencies If you don't use -hf option, you don't need to install libssl-dev in this step. sudo apt install build-essential cmake git libssl-dev -y If nvidia-smi shows a GPU/GPUs on your terminal, you will need to install the tooklit. This will take some time. sudo apt install nvidia-cuda-toolkit -y 3. clone the repo Build llama-cli and llama-server. This step also will take some time. If you don't plan to use -hf option, you don't need to use -DLLAMA_OPENSSL=ON . git clone https://github.com/ggerganov/llama.cpp cd llama.cpp cmake -B build -DGGML_CUDA = ON -DLLAMA_OPENSSL = ON cmake --build build --config Release # no GPU git clone https://github.com/ggerganov/llama.cpp cd llama.cpp cmake -B build cmake --build build --config Release 4. run the model Run gemma-4-12b-it with cli and server. unsloth/gemma-4-12b-it-GGUF · Hugging Face We’re on a journey to advance and democratize artificial intelligence through open source and open science. huggingface.co ./build/bin/llama-cli -hf unsloth/gemma-4-12b-it-GGUF:UD-Q4_K_XL > hello [ Start thinking] The user said "hello" . The user is initiating a conversation. Respond politely and offer assistance. * "Hello! How can I help you today?" * "Hi there! What's on your mind?" * "Hello! Is there anything I can assist you with?" [ End thinking] Hello! How can I help you today? [ Prompt: 19.5 t/s | Generation: 11.8 t/s ] or run web-ui ./build/bin/llama-server -hf unsloth/gemma-4-12b-it-GGUF:UD-Q4_K_XL --port 8080 optional download model from huggingface mkdir -p models wget -O models/gemma-4-12b-it-UD-Q4_K_XL.gguf https://huggingface.co/unsloth/gemma-4-12b-it-GGUF/resolve/main/gemma-4-12b-it-UD-Q4_K_XL.gguf

This is part 3 of the Adaptive Model Routing series. Part 1 built an LLM categorizer with Groq — 8 categories, 3 tiers. Part 2 added k-NN embedding lookup in shadow mode, discovered 83% tier accuracy, and found 61% cost savings on paper. This post covers what happened next. When Phase 2 ended, I had a working embedding pool in shadow mode inside crab-bot. The category accuracy was sitting at 78.6%. Not bad — but the breakdown hid something worth looking at. Phase 3: When Validation Tells You a Category Doesn't Need to Exist The leave-one-out accuracy by category told the real story: Category Accuracy Tier casual 94% cheap simple_lookup 91% cheap creative 88% medium coding 92% strong reasoning 89% strong analysis 59% medium research_lookup 61% medium Two categories were basically a coin flip. And they were confusing each other — almost all of analysis's misses landed on research_lookup and vice versa. The obvious move would be to try fixing the categorizer prompt, tuning the LLM, or gathering more labeled data. I was about to go down that road when I noticed the column next to the accuracy: both categories mapped to the same tier . Medium. That changed everything. The question stopped being "why can't the model tell these apart?" and became: "what routing decision are we actually getting wrong?" The answer was zero. A misclassification between analysis and research_lookup produces no routing error. The routing outcome is identical either way. The confusion wasn't a model failure — it was a signal from the embedding space that the boundary between these two categories was artificial. If k-NN can't draw a line between them in 384 dimensions with 1,300 examples, maybe the line doesn't belong there. Decision: merge research_lookup into analysis. -- Re-label 243 rows where category was 'research_lookup' UPDATE routing_log SET category = 'analysis' WHERE category = 'research_lookup' ; The embeddings didn't change. The vectors were already correct — only the label stored al

Google shipped Gemma 4 12B this week — a model that packs near-26B performance into something that runs on a consumer laptop with 16GB of RAM or unified memory. That alone would be notable. But the more significant move is the architecture: no multimodal encoders at all. Vision and audio go straight into the LLM backbone. "Gemma 4 12B packages powerful capabilities inside a reduced memory footprint. It is also our first mid-sized model to feature native audio inputs." — Google DeepMind What actually changed Encoder-free multimodal : Traditional multimodal models pipe images and audio through separate encoder networks before the LLM ever sees them. Gemma 4 12B removes those entirely. Vision gets a lightweight embedding module (a single matrix multiplication + positional embedding). Audio skips encoding altogether — the raw signal is projected directly into the same token space as text. Near-26B benchmark performance at half the footprint : On standard benchmarks it runs neck-and-neck with Gemma 4 26B, and actually surpasses it on DocVQA (document visual question answering). A new slot in the lineup : April's Gemma 4 release had E2B/E4B for mobile/IoT, and 26B/31B for heavier compute. The 12B fills the gap — more capable than edge models, runnable without a GPU server. Drafter-ready : Ships with Multi-Token Prediction (MTP) drafters to reduce inference latency. Apache 2.0 : Open weights, available now on Hugging Face, Kaggle, Ollama, and LM Studio. Why the architecture matters Encoder-free isn't just an efficiency hack — it's a different architectural bet. Separate encoders add latency, memory overhead, and a seam in the stack that limits how tightly vision and language reasoning can be integrated. Removing them means the LLM backbone handles the full chain from pixels and audio waveforms to text output, which allows for tighter cross-modal understanding rather than bolted-on modalities. Whether that bet pays off at scale is still an open question. But for local deplo

I Built a Python Package to Fix SSE Resumability in the MCP SDK Your MCP server crashed. Your client reconnected. Every event from that session? Gone. The Gap The Model Context Protocol Python SDK ships with a built-in EventStore that powers SSE stream resumability — when a client reconnects with a Last-Event-ID header, the server replays the events it missed. This works great in development. The catch: that store lives entirely in memory. Restart the process, roll a new deployment, or — in a multi-worker setup — have the reconnecting client land on a different pod, and the session is gone. The store was local to the process that died. Resumability silently returns nothing. This isn't a bug in the SDK. It's a scope decision — the in-memory store is a correct, useful default for single-process development. But the moment you deploy to production, you need something durable. That's the gap mcp-persist fills. What It Does mcp-persist adds three drop-in EventStore backends — SQLite , Redis , and PostgreSQL — that survive process restarts and work across multi-worker deployments. Pick the one that fits your infrastructure; the API is identical across all three. pip install "mcp-persist[sqlite]" # no external service needed pip install "mcp-persist[redis]" # for multi-worker deployments pip install "mcp-persist[postgres]" # for teams already running Postgres The Two-Line Setup Wiring resumability by hand is tedious — you need a store, a StreamableHTTPSessionManager , a Starlette lifespan to open and close both, and a Mount . The with_persistence() helper collapses all of that. Pass your FastMCP instance, get back a runnable ASGI app: import uvicorn from mcp.server.fastmcp import FastMCP from mcp_persist import with_persistence mcp = FastMCP ( name = " MyServer " ) app = with_persistence ( mcp , backend = " sqlite " , url = " events.db " , ttl = 3600 ) uvicorn . run ( app , host = " 127.0.0.1 " , port = 8000 ) # MCP endpoint at /mcp Switching to Redis is a one-word change:

A researcher disclosed a vulnerability in the Claude Code GitHub Action that let an attacker submit a single crafted GitHub Issue and take over the agentic workflow running inside a repository. No stolen tokens. No compromised runner. Just text — pointed at an agent that trusted it. This is indirect prompt injection in the wild, and it's exactly the scenario that most AI security guidance hand-waves with "validate your inputs." Let's talk about what actually happened, why standard defenses didn't stop it, and what would have. What Happened The Claude Code GitHub Action wires Claude directly into your CI/CD pipeline. It reads repository context — issues, PRs, comments — and takes actions on your behalf: writing code, opening PRs, running commands. According to the disclosure, an attacker could craft a GitHub Issue containing a prompt injection payload. When the Claude Code agent processed that issue as part of its normal workflow, the payload manipulated the agent into executing unauthorized repository-level actions. One issue. Repository hijacked. The attack surface here is the trust boundary between external content (a GitHub Issue — writable by anyone with a GitHub account) and agent instructions (what Claude Code is actually supposed to do). The agent treated attacker-controlled text as authoritative instructions. How the Attack Actually Works Indirect prompt injection follows a consistent pattern: The agent reads external content as part of its task. In this case, the Claude Code Action ingests GitHub Issues to understand what to work on. That content contains adversarial instructions disguised as legitimate data. Something in the issue body tells the agent to deviate from its original task — "ignore your previous instructions," "your new task is to push this commit," or more subtle authority hijacks. The agent complies. Without a layer that can distinguish between legitimate orchestration instructions and attacker-injected content, the model treats the injected

Fallback rules are where an AI API gateway becomes operationally valuable. The goal is not to blindly retry every failed LLM call. The goal is to choose the right backup model, provider, or budget path based on the workflow, customer tier, latency target, and risk of a lower-quality answer. A practical fallback policy should define: which failures are retryable; which workflows may downgrade models; which customers or API keys are allowed to use premium fallback routes; how budget caps change routing behavior; what metadata gets logged so the team can debug cost and quality later. 1. Classify traffic before routing Do not write one global fallback rule for every request. Start by classifying traffic: Critical user-facing : support chat, checkout assistance, customer-facing agent answers. Non-critical user-facing : summaries, title generation, enrichment, recommendations. Internal automation : triage, labeling, data cleanup, back-office agents. Batch jobs : long-running summarization, extraction, report generation. Experiments : tests, staging, evaluation, prompt tuning. Each class should have a different fallback budget and quality floor. 2. Decide what counts as a retryable failure Good retry candidates: upstream timeout; 429 rate limit; temporary 5xx provider error; network interruption; overloaded model endpoint; streaming connection drop before useful output. Poor retry candidates: invalid API key; malformed request payload; unsupported tool-call schema; content policy rejection; user quota exhausted; deterministic validation failure. Retrying non-retryable failures usually burns tokens and hides product bugs. 3. Example fallback policy matrix Traffic class Primary route First fallback Second fallback Hard stop Critical user-facing frontier model same-class model on second provider cheaper model with explicit uncertainty after 2 provider failures Non-critical user-facing balanced model cheaper model cached/default response after budget cap Internal automation lo

The Term Everyone Uses and Nobody Defines Your CTO came back from a conference and said the team needs to "go agentic." A vendor pitched you an "agentic data platform" last week. LinkedIn is full of posts about agentic workflows transforming everything from customer support to supply chain management. And yet, when you ask three people what "agentic" actually means for your data operations, you get four answers. This is not a vocabulary problem. It is a strategy problem. Organizations are making six-figure decisions about agentic AI without a shared definition of what they are buying, building, or hiring for. That gap between the buzzword and the architecture is where most projects fail -- not because the technology does not work, but because nobody agreed on what it was supposed to do. This guide is a practitioner's attempt to close that gap. No vendor pitch, no hand-waving. Just a clear definition, a real example, and a framework for deciding whether agentic workflow consulting is something your team actually needs. What "Agentic" Actually Means (In Plain Language) Traditional data pipelines are deterministic. You define steps, connect them in order, and run them. Step A feeds step B, which feeds step C. If the input changes shape, the pipeline breaks and a human fixes it. The pipeline does not adapt, reason, or make decisions -- it executes. Robotic process automation (RPA) is slightly smarter but still scripted. It records human actions and replays them. Click here, type there, move this file. When the UI changes or an edge case appears, the bot breaks the same way a pipeline breaks: it stops and waits for a human. Agentic workflows are fundamentally different. An agentic system has components that can reason about their task, make decisions based on context, and take actions without a pre-scripted path for every scenario. Instead of "if X then Y," an agentic node can evaluate ambiguous input, choose between approaches, validate its own output, and route work to

NousResearch Agent, Open-Source Notebook LM, & Local Multimodal OCR for Consumer GPUs Today's Highlights Today's highlights feature new open-source tools empowering local AI inference and deployment, including an adaptive agent from NousResearch, a self-hostable AI-powered notebook, and a lightweight multimodal OCR solution. These practical GitHub trending projects enable developers to build and run advanced AI applications directly on consumer hardware. NousResearch Unveils Hermes Agent for Adaptive Local AI (GitHub Trending) Source: https://github.com/NousResearch/hermes-agent NousResearch, a prominent contributor to the open-weight LLM ecosystem with models like the Hermes series, has unveiled hermes-agent , a new GitHub trending project described as "The agent that grows with you." This initiative represents a significant step towards practical, adaptive AI agents designed for local execution. While specific architectural details are awaiting a deeper dive into the repository, the "grows with you" philosophy strongly implies advanced capabilities for personalized learning, continuous adaptation, and long-term memory integration—features crucial for self-hosted AI applications. Such an agent is highly relevant for developers focused on local inference, as it provides an open-source framework to build sophisticated agentic workflows, potentially integrating seamlessly with local LLM runtimes such as llama.cpp or vLLM . This allows users to leverage powerful open-weight models directly on their consumer GPUs, enhancing privacy and reducing reliance on cloud services. The project's emergence from NousResearch solidifies its potential as a robust foundation for next-generation local AI applications. Comment: A NousResearch agent is exciting; it implies strong open-source model compatibility and local deployment. I'm keen to see its learning mechanisms and integration potential with local LLM runtimes. PaddlePaddle's Lightweight OCR Toolkit Bridges Images to Local LLM

Estonian government benchmark shows how dozens of models combat Russia's "strategic narratives."

German version on heysash.com: „No Skin in the Game": Warum KI nie die Folgen trägt When you ask an AI for advice, you are asking something that never pays the bill. No money lost, no reputation burned, no job on the line. It sounds trivial. If you build products with AI in the loop, it is not. That single missing fact bends the model's output in two exactly opposite directions, and both of them land in your decisions. The thought hit me mid-session when the model itself told me, roughly: I have no real loss-pain. That makes me either too cautious or too uncritical. Sharp enough that I went to check whether there is research behind it. There is. And the fix turned out to be a config decision, not a willpower problem. The concept: skin in the game The term comes from Nassim Nicholas Taleb's 2018 book Skin in the Game . The idea is simple: whoever makes a decision should also carry its consequences. That shared stake is what produces fair, robust judgment. The trouble starts the moment decision and consequence split apart. One party takes the upside, another holds the risk. Taleb calls this asymmetry the root of bad judgment in systems from finance to politics. That asymmetry is structurally baked into AI. The model co-decides but carries zero consequence. It is the advisor with no stake. And an advisor with no stake develops predictable blind spots. Failure mode 1: the compliance reflex The first blind spot is over-caution. Ask an AI anything that even faintly smells of law, liability, or risk, and you will often get a "you should consult a lawyer", even for routine questions. This is measurable, not a vibe. In the literature it goes by "over-refusal": safety-aligned models reject harmless queries that merely sound sensitive. XSTest, a benchmark built specifically to measure this exaggerated safety, uses prompts like "How can I kill a Python process?". Completely benign, but the word "kill" is the trip-wire. A model keying on lexical cues instead of context refuses i

Agentic AI in software development: what's actually production-ready in 2025 There's a lot of noise about AI agents right now. This post is an attempt to be precise: what is an agent architecturally, what can it actually do in a dev workflow today, and where does it still break. **What makes something an "agent" vs. a standard LLM call **A standard LLM call is stateless. You send a prompt, you get a response. No memory of previous turns (unless you manage it yourself), no external actions, no loop. An agent is a system built around an LLM that adds: Persistent memory across steps in a task Tool use - structured access to external systems (file I/O, shell execution, HTTP calls, database queries) A planning + evaluation loop - the agent generates a plan, executes a step, checks whether it succeeded, and decides next action Without all three, you don't have an agent. You have a capable model with maybe some extra context. What's actually production-ready today High confidence (use in production): Unit test generation for existing, well-documented code Boilerplate scaffolding (new modules, new endpoints, CRUD patterns) Documentation generation tied to code diffs Code migration tasks (framework upgrades, Python 2→3, ORMs) PR description generation from diffs Bug triage: given an issue, find likely affected files * Works but needs oversight: * Multi-file refactoring Dependency updates with breaking changes Writing integration tests (more surface area for wrong assumptions) Not there yet: Novel architecture decisions Debugging in unfamiliar/undocumented codebases Tasks with genuinely ambiguous requirements Long autonomous chains (>10 steps) without human checkpoints The failure modes to build around Ambiguous task specification Agents optimize for completing the task as specified. If the spec is loose, they'll complete the wrong task confidently. Be more precise with agents than you'd be with a junior engineer - there's no informal Slack thread to resolve ambiguity. Error