标签：#m

Most teams blame their model provider when the inference bill spikes. They're looking at the wrong line item. The real leak is architecture — and it's the difference between a token bill that scales with value and one that scales with chaos. Here's what we see shipping agentic systems in production. The hidden multiplier: agent loops A "2-minute task" is never one call. An agent fires 30–60 tool calls per run, and most frameworks stuff the entire conversation history into every prompt. So a job you'd estimate at ~4K tokens becomes 40 calls × 8K context = 320K tokens — billed at frontier rates. Frontier pricing per call looks cheap. Multiplied by agent-loop iterations, it quietly becomes the largest line in your cloud bill. The 80/20 of inference Not every call needs a frontier model. ~80% of agent traffic is routing, extraction, formatting, classification, summarization. Trivial. Leading efficient models — including top China models — handle these at near-parity. ~20% is genuine reasoning, open-ended generation, ambiguous planning. That's where frontier earns its price. Route the 80% to efficient models and reserve frontier for the 20%. Same output quality. A fraction of the bill. A unified gateway beats a drawer of API keys The trap most teams hit: they wire 4 providers with 4 clients, then let a naïve router "roam" between them. On failover it loses cache affinity, re-embeds context, and your 1.5x cost target drifts back toward ~1x — or worse. A single OpenAI-compatible endpoint across OpenAI + Gemini + leading China models fixes this: One client, one code path. Provider pinning holds cache locality; it only fails over on hard error, not price drift. Your application code never changes when you swap a model. In SEA, "PDPA-aligned" is the baseline, not a premium For Malaysia and SEA teams, inference isn't just a cost question — it's a compliance one. PDPA requires 72-hour breach notification and a designated DPO. In-region data residency (SG-hosted) is now the defa

As part of the Förderverein AWS Community DACH e.V. I joined the selection committee for talks again this year. Third year in a row (I wrote about the previous editions and my broader community journey in my 2025 year in review ). The process is always inspiring, fair, and full of incredibly well-crafted abstracts. How the evaluation works Sessionize uses a Comparison Evaluation Mode based on the Elo rating system (from chess). Three sessions are shown at the same time and you rank them relative to each other. You don't assign absolute scores; you just decide "this one is better than that one." Each comparison produces three "games" (A vs B, B vs C, C vs A) and the algorithm adjusts ratings accordingly. After a first pass through all sessions, the system gets smarter about which triplets to show next, targeting sessions with similar ratings for more precise differentiation. It's fast, focused and surprisingly fair, because you only ever think about the three in front of you, not all others. You can also use "Strong opinion" options: Top (this is excellent), Doesn't fit (wrong conference), or Ignore (conflict of interest, can't judge). These signal the algorithm without distorting the ranking. In my case: I marked 2 as "Doesn't fit" (not because they lacked quality; they were actually interesting but too inspirational for a technical community day, better suited for a different audience). The other 2 ignored were my own submissions : Vibecoding in Between Meetings and Serverless vs Kubernetes - The Final Showdown . The full process The behind-the-scenes post by Philipp Garbe explains the whole pipeline in detail. In short: Round 1 (Screening): Board members filter out spam, incomplete proposals, marketing pitches, non-AWS topics, and speakers outside EMEA. Round 2 (Content Evaluation): Every association member can participate. This is the Elo-based comparison round. Members must disclose personal connections and skip sessions they can't rate objectively. Round 3 (Fin

As the US sees its highest number of measles cases in decades and vaccination rates fall, researchers are developing drugs to help those who contract the virus or who are particularly vulnerable.

After you deploy half a dozen AI agents across a team, something weird happens. Friday afternoon, release day. The PM says their AI summarized the change impact. The dev says their AI reviewed the code and found nothing. QA says their AI ran the test suite and everything passed. Then production breaks. You dig through the logs and all you see is "system call." No way to tell which agent made which call, when, on what context, acting on whose behalf. Three agents sharing one service account, one API key, zero accountability. The humans end up taking the fall, and you can't even figure out which human to talk to. This isn't a thought experiment. It's what happens when you bolt AI assistants onto existing infrastructure without thinking about identity. Buy a batch of API credits, create a service account, share it across the team, done. That works fine when one person uses one assistant for their own work. It falls apart the moment multiple agents run in parallel across different roles. Permissions break first. A competitive research agent needs access to all project channel discussions. A code review agent should only see PRs and repository messages. That distinction doesn't exist in the service account model, which has a single binary switch: can access or cannot access. Teams work around it by manually creating groups, forwarding messages, and setting permission boundaries by hand. Add more agents and this manual isolation starts to crack. Some teams we've talked to ended up with over a dozen separate groups just to control agent visibility, with humans acting as message routers between them. At that point the AI is making things slower. The work history problem is more concrete. An engineer who's been on the team for three months, you know what they're good at, what they're sloppy at, which module they crushed last sprint. Next time you assign work, you use that information. An agent that's run a hundred tasks? Completion rate, rejection count, which task types it

Clear a corner of your workbench and gather a handful of parts, because this palm-sized television is an afternoon build, not a semester project. Here is the shopping list for a Scanwheel of your own: A Raspberry Pi Pico to run the show An A4988 stepper driver A 21-02485 stepper motor (or a similar small NEMA-style unit) Five LEDs, plus current-limiting resistors A 3D-printed case and spinning disk The Scanwheel, built by a maker who goes by [Ancient], is a mechanical TV that fits in your hand. Instead of a glowing panel, it leans on persistence of vision: your eye holds each flash of light for a fraction of a second, so a row of blinking LEDs seen through a moving slit reads as a solid picture. Spin the disk fast enough and the flicker melts into an image. How the picture actually forms The disk sitting on top of the case carries 20 small holes spaced evenly around its edge, each drilled at a slightly different height. As the motor turns, only one hole passes in front of the LEDs at a time, so light escapes in a scanning line rather than a wash. The Pico drives the stepper up to roughly 900 RPM through the A4988, then fires the LEDs in a precise order timed to the disk position. Get that timing right and the holes trace out a grid. The payoff is a 20x20 pixel color display in the center, flanked by two more 20x20 black-and-white panels that can each show a different image. Five LEDs feed all three. The whole coordination job lives on the Pico's GPIO pins, which is why the wiring stays simple enough to manage on a breadboard before you commit anything to a soldered protoboard. Small light baffles in the base keep the LEDs from bleeding into each other, a detail worth copying if your first image looks smeared. Give it a spin The full build guide, firmware, and disk files are on the project's GitHub repository , so you can match the hole spacing and LED timing exactly. If your image drifts or tears, start by trimming the RPM and re-checking when each LED switches rela

Hello everyone, I am currently building a high-performance API gateway that integrates business logic components—implemented via WASI and running within Wasmtime—as HTTP/TCP/QUIC handlers. I am exploring the best design approach to achieve a zero-copy data path from the upstream network layer—specifically hyper for HTTP/1.x and HTTP/2, and h3-quinn (based on Quinn) for HTTP/3—to the wasi:http guest environment. Given that: hyper and h3-quinn each manage their own internal buffer pools (e.g., bytes::Bytes ), asynchronous read/write streams, and frame decoders. Wasmtime's wasmtime-wasi-http implements the wasi:http (WASIp2) specification, which relies on resource types such as InputStream and OutputStream . I aim to minimize memory copying and CPU overhead when passing large request bodies or streaming responses across the sandbox boundary. For those experienced with bridging I/O between the host and guest in Wasmtime, I have a few architectural questions: Buffer ownership and memory mapping: How can host-side bytes::Bytes (from hyper or h3-quinn ) be mapped or bridged into Wasm linear memory (and vice versa) without requiring a CPU-based memcpy ? Does Wasmtime's resource streaming support direct memory views, or are we essentially limited to copying data chunks via guest memory pointers? Adapting stream abstractions: hyper uses http_body_util::combinators / http_body::Body , h3 uses its own stream primitives, while wasi:http uses wasi:io/streams . What is the idiomatic way to efficiently adapt these asynchronous streams on the host side (i.e., within the wasmtime-wasi-http handler implementation) without blocking the tokio runtime? Backpressure propagation: How can backpressure signals be correctly propagated from the Wasm guest (e.g., when the guest's InputStream is consuming data slowly) all the way back to the Quinn congestion controller or Hyper connection pool, thereby avoiding unbounded buffering on the host side? If anyone has built similar high-performance ga

இன்னும் ஜோஷ்! 🔥 முதல் நாள் St. Joseph's Institute of Technology, சென்னையில ஜோர்தான் இருந்தது. இரண்டாம் நாள் வந்ததும் என்னன்னா, க்ரவுட் இன்னும் அமைதியா, ஆனா உள்ள ஆர்வம் இன்னும் ஜாஸ்தியா இருந்துச்சு. எல்லாரும் "இன்னிக்கி ரொம்ப tech-ஆ போகணும்" னு மனசுல வெச்சிட்டு உட்கார்ந்திருந்தாங்க. இண்டு "தமிழன் நினைச்சா முடியாதது இல்ல" ங்கிற வார்த்தை என் மனசுல ஓடிக்கிட்டே இருந்தது — ஒரு சின்ன அறையில கூட, கம்ப்யூட்டர் screen-ல open source code-ஐ பார்த்துக்கிட்டே இருந்தா, அது எவ்ளோ பெரிய புரட்சின்னு தெரியும். FOSS-ன்னு சொல்ற ஒவ்வொரு லைனும், நம்ம மொழியில நம்ம கம்யூனிட்டியால எழுதப்படுற ஒவ்வொரு code-உம் ஒரு சிறிய வெற்றி தான். Session 1: வேகமா App கட்டணுமா? Meet Framework இருக்கே! 🚀 முதல் session-ல Meet Framework அறிமுகமானது — Python + JS ரெண்டையும் சேர்த்து ஒரே கூரையின் கீழ கொண்டு வர்ற ஒரு full-stack framework. இது என்ன பண்ணுது தெரியுமா? "Setup fatigue" ங்கிற பெரிய பிரச்சனையை ஒரே அடியில தீர்க்குது. Backend, frontend, database எல்லாத்தையும் தனித்தனியா தேடி, ஒட்டி, configure பண்ணி — இதெல்லாம் இல்லாம, ஒரே framework-ல எல்லாமே ready-ஆ இருக்கும். Speaker live-ஆ ஒரு app-ஐ கட்டி காமிச்சாங்க — routing, models, ஒரு simple UI எல்லாம் நிமிஷங்களில ready! அது பார்க்கும்போதே ஒரு எனர்ஜி கிடைச்சது. "Idea இருந்தா போதும், tool நம்ம கூட இருக்கு" ங்கிற நம்பிக்கை தான் FOSS-ன்ற அழகே. சின்ன Motivation: ஒரு framework கத்துக்கிறதும், ஒரு புது மொழி கத்துக்கிறதும் ஒண்ணுதான். ஆரம்பத்துல கஷ்டமா தான் தெரியும், ஆனா ஒரு அடி எடுத்து வெச்சா, மொத்த பாதையும் தெளிவா தெரியும். "தொடங்குறது தான் பாதி வெற்றி!" Session 2: NPM vs NixOS — ஒரு "Love-Hate" Relationship 😅 இரண்டாவது session ரொம்ப relatable-ஆ இருந்தது — NixOS-ல npm use பண்றது! அறையில இருந்த பலருக்கும் இது தெரிஞ்ச பிரச்சனை தான், எல்லாரும் தலையாட்டிட்டே இருந்தாங்க. பிரச்சனை என்னன்னா — Nix ரொம்ப strict-ஆ இருக்கும், file system-ஐ read-only-ஆ வெச்சிருக்கும். அதனால npm சாதாரணமா install பண்ற மாதிரி இங்க straight-ஆ வேலை செய்யாது. Speaker மூணு வழிகள் சொன்னாங்க: Local user prefix வெச்சு — npm-ஐ ஒரு writable இடத்துல install பண்ண வைக்கிறது. node2nix use பண்ணி — npm dependencies-ஐ

A Beijing startup just out-shipped every US lab's open-weight strategy On July 16, Moonshot AI — the Alibaba-backed startup behind Kimi — put Kimi K3 behind an API. Today, July 27, the full weights land on Hugging Face. No waitlist, no "responsible scaling" essay, no six-month delay between "we built something scary" and "here, run it yourself." Just 2.8 trillion parameters, open, on the day they said it would happen. That's not a small model with a big number attached. It's the largest open-weight model ever released, full stop. And unlike most "open" releases that quietly underperform their closed competitors, K3 is winning on the benchmarks developers actually care about. Let's get into what's real and what's marketing. The numbers K3 is a mixture-of-experts model: 2.8T total parameters, but it only activates 16 of 896 experts per token. That's the trick that makes a model this size runnable at all — you're not paying compute for the full 2.8T on every forward pass. The architecture story is Kimi Delta Attention (KDA), a hybrid linear attention mechanism Moonshot claims delivers 6.3x faster decoding, plus "attention residuals" that improve token efficiency by 25% for roughly 2% extra compute. Whether that holds up under independent scrutiny is still TBD, but the direction — make huge models cheap to serve — is the correct one, and it shows up in the token counts: K3 uses 21% fewer output tokens than its predecessor, K2.6, for comparable tasks. Context window: 1,048,576 tokens. Flat pricing, no context-length tiering — a real advantage over providers who quietly double your rate past 128K. Benchmarks that matter: Benchmark K3 Comparison Frontend Code Arena 1679 Elo (#1) Claude Fable 5: 1631, GPT-5.6 Sol: 1618 GPQA Diamond 93.5% Best open-weight score ever published GDPval-AA v2 1687 (#3) Behind Claude Fable 5 Max (1815), GPT-5.6 Sol Max (1747.8) — ahead of Claude Opus 4.8 (1600) Artificial Analysis Elo 1547 +732 over K2.6 Read that middle row again: an open-weight

AI coding tools have crossed a real threshold in the past year. Cursor and Windsurf count millions of active users. VS Code and JetBrains ship with built-in completion. Tencent Cloud demoed CodeBuddy NPC last month, where an agent takes a task spec, writes the code, opens a PR, runs CI, and fixes failures autonomously until everything goes green. You type a function signature and the model fills in a dozen lines before you finish thinking. Drop a comment saying "add unit tests" and a test skeleton appears in seconds. Individual coding speed is up somewhere between 3x and 5x by most team accounts. Pull the lens back from the editor to the team level and the picture changes. More code ships faster, but review queues grow longer, test environments get locked more often, and wait times between handoffs actually stretch out. A developer spends 20 minutes writing a feature, waits two hours for review, fixes comments, pushes again, and the staging environment is busy. QA posts failures in the group chat and the developer misses the message while on something else. Hours pass. Actual coding time might account for a tenth of the total delivery cycle. The rest is coordination, waiting, messaging, and context switching. The faster code gets written, the more congestion piles up behind it. The single-agent loop that products like CodeBuddy NPC demonstrate works cleanly for isolated work: one agent plans, codes, tests, and fixes until it passes. But any non-trivial feature in a real team crosses multiple roles. PMs confirm requirements. Tech leads do architecture reviews. QA runs regression in staging. Ops checks resource configs before deploy. Between each handoff today, someone pings Slack saying "PR up for review," drags a ticket from In Dev to Ready for QA in Jira, or scrolls through doc history trying to remember what came up in the last review. The agent can write the code, but it has no idea who to ping for review, how to provision a test environment, or why the last vers

The Go pod is running in production. CPU limit set to 2, metrics look reasonable. But under load, P99 latencies spike intermittently with no obvious cause. No errors, no goroutine leaks, just latency blowing up on traffic bursts. The root cause is usually invisible: GOMAXPROCS equals the number of CPUs on the physical node, not the container limit. Your Go app thinks it has 32 CPUs when it only has 2. The Linux kernel handles the gap in its own way — CFS throttling. What GOMAXPROCS reads (and what it ignores) By default, the Go runtime computes GOMAXPROCS via runtime.NumCPU() , which reads the number of CPUs available at the OS level. On a 32-core Kubernetes node, that returns 32 — regardless of what resources.limits.cpu says in your pod spec. Kubernetes CPU limits are enforced through Linux cgroups (v1 or v2). Cgroups are transparent to processes: a pod with limits.cpu: "2" doesn't see two virtual CPUs, it sees all the node's CPUs and gets suspended when it consumes too much. The Go runtime, historically, never read cgroups. It trusted the physical core count. package main import ( "fmt" "runtime" ) func main () { // Inside a pod with limits.cpu: "2" on a 32-core node fmt . Println ( runtime . NumCPU ()) // → 32 fmt . Println ( runtime . GOMAXPROCS ( 0 )) // → 32 } CFS throttling: how the kernel slows you down The Linux CFS (Completely Fair Scheduler) enforces CPU limits via two cgroup parameters: cpu.cfs_quota_us (allowed CPU time) and cpu.cfs_period_us (measurement window, 100 ms by default). A pod limited to 2 CPUs gets at most 200 ms of CPU time per 100 ms window. When Go spawns 32 OS threads for 32 parallel goroutines, those threads compete for physical CPUs. Once their combined usage exceeds the cgroup quota within the current window, the kernel suspends all threads in the cgroup until the next window starts. That's throttling: a complete application freeze lasting anywhere from a few milliseconds to several tens of milliseconds. A handful of these per second

Black holes eject powerful energy jets that blow away the surrounding gas to great distances. So how can they continue to grow?

A place to trade small favors instead of money. Post a skill ("I can fix spreadsheet formulas," "I'll practice beginner Spanish with you"), say how many minutes it takes, and swap it for someone else's time. No payments anywhere in this app, on purpose - the whole point is that minutes are the currency. If you guys like this app please follow https://github.com/Timinesh/ submitted by /u/comradetiminesh [link] [留言]

How to design, ship, and operate an AI agent that is reliable, efficient, performant, scalable, and secure enough to serve real companies — from a 5-person startup to a 50,000-person enterprise. This guide distills hard-won lessons from production agents (Claude Code, OpenHands, SWE-agent, GoClaw, Hermes, nanobot, PicoClaw, ZeroClaw, Multica, Paperclip) and grounds them in current engineering guidance from Anthropic and OpenAI plus the security and compliance standards you'll actually be audited against (OWASP Top 10 for Agentic Applications, NIST AI RMF, the EU AI Act, and 2025–2026 prompt-injection research). It focuses on the parts most articles skip: the enterprise tax — governance, security, compliance, integration, cost control, and the operating model — that separates a demo from a system a CISO will sign off on. 📖 How to use this guide Read Parts 0–2 to decide whether and what to build. Most failed agent projects die here. Read Parts 3–7 for the architecture and reliability engineering. Read Parts 8–10 for the enterprise gates: security, compliance, multi-tenancy, observability, cost. Read Parts 11–15 for delivery, scale & rollout: deployment topologies (SaaS/self-hosted/hybrid), how to adopt from pilot to org-wide, how to handle thousands of concurrent requests, the operating model, and a 30/60/90 plan. Every part ends with an ✅ Actionable checklist . Skim those for a design review. 📋 Table of Contents 🧮 Part 0 — The Core Equation 🧭 Part 1 — Decide Before You Build: Workflow vs Agent, Build vs Buy 🏛️ Part 2 — The Enterprise Tax: What Actually Changes 🏗️ Part 3 — Reference Architecture: The Layered Stack 🔄 Part 4 — The Reliable Kernel: The Agent Loop 🛠️ Part 5 — Tools & Enterprise Integration 🧠 Part 6 — Context & Memory: The Cost Center 🛟 Part 7 — Reliability Engineering 🔐 Part 8 — Security, Compliance & Governance 🧱 Part 9 — Multi-Tenancy & Isolation 📊 Part 10 — Observability, Evals & Cost Governance 🚀 Part 11 — Deployment & Delivery Models 📈 Part 12 — The

Hey everyone! I bring you my development journey on what I have discovered, accomplishments for this...

submitted by /u/fagnerbrack [link] [留言]

Netflix has described the production lessons behind bringing LLM inference into its internal serving platform, including the challenges of supporting different model sizes, hardware requirements, and rapidly evolving inference engines. By Matt Foster

Don't fight their strength — take away their advantage. Don't fight the boiling water — remove the...

Most teams building AI agents treat manual override queues as a last resort, a safety net for when the agent goes off the rails. That view keeps agents locked in read-only mode on anything risky. The real insight is the opposite: a well-designed AI agent manual override queue enables you to trust your agent with high-value actions you would otherwise never automate. An override queue is not a punishment for poorly trained models. It is a design tool that widens the feasible autonomy boundary by giving you a structured escape hatch for the 5% of cases the agent cannot confidently handle. Table of Contents What Is an AI Agent Manual Override Queue? What Makes a Manual Override Queue Different from Other Queue Types The Lifecycle of an Action in a Manual Override Queue Four Steps to Implementing a Manual Override Queue in Your Agent Workflow How to Evaluate a Manual Override Queue Solution: Key Dimensions Three Pitfalls That Sabotage Manual Override Queues When to Use a Manual Override Queue, and When to Skip It Why We Built AwaitHuman: Escalation-as-a-Service for Agentic Workflows Frequently Asked Questions What Is an AI Agent Manual Override Queue? An AI agent manual override queue is a structured holding area where an autonomous agent's action is paused and routed to a human operator for review, approval, or rejection before execution. This is the formal definition from our guide to safe autonomous workflows (internal page, but we'll keep the reference general). The queue sits between the agent's decision and its execution, intercepting only the actions that cross a configurable risk threshold. For example, an agent managing cloud deployments might trigger a terraform apply command. Without a queue, that action happens immediately. With one, the agent serialises the full reasoning trace, the proposed diff, and the affected resources into a queue item. A human operator receives a notification, reviews the packet, and approves or rejects. The agent then proceeds or ba

📺 Prefer to watch? 90-second YouTube Short · 💬 Telegram Originally published on software-engineer-blog.com . You already know the trick: add "think step by step" to your prompt and the model's answer gets better. Almost nobody explains why — and the real reason has nothing to do with motivation or effort. Mental model: A transformer spends a fixed stack of layers per token, so adding reasoning tokens doesn't make the model smarter — it buys it more compute passes and an external scratchpad to read from. The Problem: Fixed Compute per Token Here's the floor. When a transformer generates a token, it runs through the same neural network layers every time. The stack depth is fixed at model-creation time. Whether you ask it "2+2" or "Sara has 3 packs of 8 markers, gives 5 to each of 4 friends, how many left?", the model gets the same amount of layered computation to produce each output token. That compute budget never grows with problem difficulty. Now imagine you ask for just the answer: "Sara has 3 packs of 8 markers, gives 5 to each of 4 friends, how many left? Answer only the number." The model has to solve a three-step problem (multiply 3 × 8 = 24, multiply 4 × 5 = 20, subtract 24 − 20 = 4) in a single forward pass. It needs to hold "24" and "20" somewhere while computing the final step. But it's only got one forward pass, one set of layer outputs, and nowhere internal to stash intermediate values. So it guesses. It might say 19. It didn't get the math wrong because it's bad at math. It got it wrong because you handed it the wrong compute budget for the job. The Mechanism: Three Small Shifts Now ask the same question and let it write the steps: "Sara has 3 packs of 8 markers, gives 5 to each of 4 friends, how many left? Think step by step." Three mechanical things happen: 1. The model becomes a loop. Every token the model emits is appended to the input context and fed back in on the next forward pass. So if it writes "First, 3 × 8 = 24", that token sequence gets rea

Regression and regularization are both important concepts in machine learning and statistics, but they solve different problems. Regression is primarily used to model relationships and make predictions. Regularization is used to improve a model's ability to generalize by controlling its complexity. Regression This is a statistical and machine learning technique used to predict a continuous numerical outcome based on one or more input variables. For example, we might want to predict: A house's price based on its size and location A student's exam score based on study hours A company's sales based on advertising spending Simple Linear Regression In simple linear regression, we model the relationship between an input variable (x) and an output (y): $$ y = \beta_0 + \beta_1x + \epsilon $$ Where: (y) is the predicted outcome (\beta_0) is the intercept (\beta_1) is the coefficient or slope (x) is the input variable (\epsilon) represents the error The model learns values for (\beta_0) and (\beta_1) that make its predictions as close as possible to the actual values. Multiple Linear Regression In multiple linear regression, several predictors are used: $$ y = \beta_0 + \beta_1x_1 + \beta_2x_2 + \cdots + \beta_px_p + \epsilon $$ The goal is typically to minimize the sum of squared errors (SSE) : $$ \text{SSE} = \sum_{i=1}^{n}(y_i - \hat{y}_i)^2 $$ This approach is known as Ordinary Least Squares (OLS) . Regularization Regularization is a technique used to prevent a machine learning model from becoming too complex. A model can perform extremely well on training data but poorly on new, unseen data. This problem is called overfitting . Regularization addresses overfitting by adding a penalty for large model coefficients to the model's objective function. Instead of minimizing only the prediction error, the model minimizes: $$ \text{Prediction Error} + \text{Complexity Penalty} $$ The penalty discourages the model from relying too heavily on individual features. The Main Types o