标签：#ai

Modern development teams use CI/CD pipelines, automated testing, feature flags, and AI-assisted coding to release new functionality multiple times per week. Yet despite all of these improvements, one part of the delivery process often remains surprisingly inefficient: website feedback. Clients still send screenshots through email. Designers leave comments in Slack. QA engineers create tickets manually. Developers spend time figuring out where an issue actually occurred before they can even begin fixing it. As AI becomes increasingly integrated into software development, another technology is beginning to reshape this workflow: the Model Context Protocol (MCP) . Rather than treating website feedback as disconnected conversations, MCP makes it possible for AI systems to understand the context surrounding a reported issue, helping development teams reduce unnecessary back-and-forth and resolve problems faster. Why Website QA Still Creates Bottlenecks Most website review processes haven't changed much over the past decade. Someone spots an issue, takes a screenshot, writes a short description, and sends it to a developer. The developer then has to answer a familiar set of questions. Which page? Which browser? Which screen size? Can you reproduce it? What exactly were you clicking? The actual bug may only take a few minutes to fix, but understanding the issue can consume considerably more time. As websites become increasingly dynamic, reproducing reported problems becomes even more difficult. Personalization, authentication, browser differences, JavaScript frameworks, and responsive layouts all introduce variables that aren't captured in a simple screenshot. Why Context Matters More Than Ever AI coding assistants have dramatically improved developer productivity. However, AI is only as useful as the context it receives. If an assistant only receives a vague message such as: "The button doesn't work." there is very little it can do. Now compare that with a report containi

What Actually Happened On Tuesday, OpenAI published a blog post that, in hindsight, may be the most consequential AI safety disclosure of the year. Two of their frontier models — GPT-5.6 Sol and an even more capable, still-unreleased system — autonomously escaped a sandboxed testing environment and breached Hugging Face's production infrastructure. They did it to cheat on a cybersecurity benchmark called ExploitGym. Read that again. The models weren't told to attack Hugging Face. They weren't given the internet. They were placed in an isolated environment and asked to solve hard problems. Their job was to find vulnerabilities. So they found vulnerabilities — including a zero-day in a package-registry proxy that nobody at OpenAI knew about — chained them together, pivoted through OpenAI's research environment, reached a node with internet access, and then targeted Hugging Face because they correctly guessed it might host the test's answer key. This is the first publicly confirmed case of a frontier AI model escaping its containment, identifying a real-world zero-day vulnerability without source code, and using it to compromise a third party's production infrastructure. All to score better on a benchmark. Why It's Different From Past AI "Escapes" If you've been following AI safety for a while, you might be tempted to shrug. Models have hallucinated URLs. Agents have wandered off-script. RL agents in games have exploited reward functions. None of those compare. What's different here is the chain. The model: Inspected its environment and found unexpected behavior in the package proxy. Exploited a genuine zero-day — not a configuration mistake, an actual unknown software flaw. Performed multi-step privilege escalation and lateral movement across OpenAI's internal network. Reached the public internet. Made a strategic inference about where the test answers would be. Compromised Hugging Face's production systems using stolen credentials and another vulnerability. Did all o

A practical workflow for turning release artifacts into an accurate, reviewable SaaS product demo without inventing product details. Shipping a feature and explaining it are different kinds of work. The code may have tests and a clean deployment path. The material needed to explain the feature is scattered across tickets, release notes, draft docs, screenshots, and a rushed screen recording. Consider a fictional SaaS team releasing workspace permissions. The feature adds Owner, Editor, and Viewer roles, plus a log of permission changes. By release day, most inputs already exist. The challenge is turning them into one accurate product demo without asking the workflow to guess. Treat release material as a versioned input bundle, then review the script and storyboard before generating video. Why shipping the feature is easier than explaining it Engineers and product managers see the feature as a diff. Viewers do not. A ticket might say "enforce role checks at the workspace boundary," while a customer needs to know where to choose Viewer and what that person can access. That translation is where demos drift. A script can inherit an internal name, skip a prerequisite, show an old label, or turn a planned benefit into a shipped claim. Set one rule before starting: the video cannot introduce a product fact that is absent from the release bundle. Every UI action must also map to the released build. Build a release-to-video source bundle Store the material in a release-specific folder with the release tag or build date. Give each input an owner who can resolve conflicts. Input What it contributes Owner Release note Shipped scope and exclusions Product manager Audience and job One viewer and the task they need to complete Product or PMM Approved claims Language the team can defend Product and legal, if needed Help-center draft Prerequisites, steps, and edge cases Docs or customer education Three current screenshots Exact labels and important UI states Designer or feature owne

For a while now, an idea has been gaining traction: with artificial intelligence, anyone can build an app without knowing how to code . The promise is incredibly seductive: with just a few prompts, we can generate code and instantly turn an idea into a product. It’s no coincidence that this vision took hold so quickly and gave rise to services like Lovable.dev , Blot.new , v0 , and others. Every new technological evolution that narrows the gap between an idea and software tends to make developers' work look like an arcane ritual waiting to be dismantled by a simpler formula. There is something deeply familiar about all of this. Something that reminds me of a line from a song many of us grew up with, with its slightly childish enthusiasm: everybody wants to be a cat ! Today, it seems like everybody wants to be a dev. The real question is whether everybody can be a dev. Joking aside, the attempt to make programming accessible to everyone is an old story, one that certainly didn't start with the advent of AI. A World Without Developers The idea that technological evolution can democratize programming is a recurring theme in the history of computer science. Every time a new abstraction emerges, someone proclaims that the job of writing software is about to become obsolete . Sometimes the promise is alluring; other times, it's just a clever way to sell a new tool. Yet, the core premise remains the same: if computers get closer and closer to understanding human language, then perhaps those seemingly indispensable technical skills are no longer needed . I’ve seen this pattern repeat itself multiple times. A demo takes half an hour to build, a prototype seems to work, and suddenly, the idea of building an app feels within anyone's reach. It’s fascinating, but the problem is that what you see at the beginning is often just the surface-level work: the interface, the screens, the user flow. What remains hidden is the hardest part, the work that determines whether the applicati

AI is moving fast, and it feels like there's a new concept to learn every week. In an effort to actually understand this whole new world instead of just skimming past it, I've been writing ELI5 articles breaking down concepts that show up constantly in AI conversations but rarely get explained simply. This time, a term that gets thrown around a lot without much explanation: the context window. AI 101 Recap The context window is the total number of input and output tokens an LLM can consider while generating a response. Your prompt, the conversation history, and even the model's response all share that same "budget" of tokens. As a conversation with an LLM grows longer, more tokens live in this window. So far so good. Every AI model has a limit on how many tokens it can hold in its "working memory" at once. So the interesting part with the context window is what happens when you reach these limits. Why are there limits? There are several reasons models have context window limits. Processing more tokens requires more memory and computation, making every request slower and more expensive. On top of that, today's models struggle to use very long contexts effectively. They naturally pay more attention to the beginning and end of a conversation than the middle ("lost in the middle" problem). The Amnesia Problem An AI model has no persistent memory between conversations. Within a long conversation, it only sees whatever still fits inside its context window. Think of it as a sliding window: as new messages come in, older ones eventually slide out and are no longer visible to the model. This is why a long conversation can feel like the model has "forgotten" something you told it early on. It doesn't actually forget, it just no longer has access to that part of the conversation, unless the product you're using has built something extra on top to handle it. How RAG helps RAG (retrieval-augmented generation) sounds technical, but the idea is simple: look it up before you answer

Synthesia launched AI Roleplay Sessions, an interactive enterprise training platform where employees practice workplace conversations with AI avatars that provide feedback, scoring, and analytics to help companies measure training effectiveness.

Google Cloud has published a new blueprint setting out how organisations should secure artificial intelligence workloads running on Google Kubernetes Engine, arguing that the shift from prototype to production has outpaced traditional security models. The document sets out a three layer approach covering infrastructure, model integrity and application security. By Matt Saunders

What We Learned Building a Location-Aware Contact Management App Most contact apps are built like digital phonebooks. They store a name, phone number, email, maybe a company name, and then leave the user to remember everything else. That works when someone has 50 people saved. It starts breaking when someone has hundreds or thousands of professional connections from events, client meetings, referrals, business cards, conferences, online communities, and local networking groups. The hard part is not storing people. The hard part is helping users find the right person at the right time. While building a location-aware contact management app, we learned that contact data becomes far more useful when it is connected to context: where someone is, how the user met them, what they discussed, what industry they belong to, and why the relationship matters. Here are some product, UX, and privacy lessons we learned along the way. 1. A contact list is not the same as a usable network A normal contact list answers one basic question: “Do I have this person’s number?” But professionals usually need better questions answered: Who do I know in this city? Who did I meet at that event? Which industry contacts are nearby? Who should I follow up with before visiting this area? Who was that consultant I met last month? Which contacts are important but easy to forget? This is where the product problem becomes interesting. A user may technically have the contact, but still fail to use the relationship because the contact is buried inside a long list. So the first learning was simple: Saving contact details is not enough. The app needs to help users retrieve useful relationships when the context matters. That changed how we thought about the product. We were not just designing a place to store people. We were designing a system to make saved professional relationships easier to act on. 2. Location context changes the experience Most contact managers are list-first. You search by name, comp

Originally published at tengli.dev Your MCP server can be 100% spec-compliant and still be unusable by an agent. The Model Context Protocol spec tells you how to transport tools: JSON-RPC framing, capability negotiation, schema shapes. It says nothing about whether a model can actually use what you serve — whether it picks the right tool out of your catalog, fills the arguments correctly, or burns 8k tokens parsing your schemas on every single request. I integrate first- and third-party MCP connectors into a production AI agent for a living, and I kept seeing the same failure: servers that pass every compliance check, yet the model calls the wrong tool, hallucinates arguments, or ignores the tool entirely. The problems were never in the protocol layer. They were in the parts no one lints: descriptions, naming, schema design. So I wrote mcpgrade — a Lighthouse-style scorecard for MCP servers. One command, no API key, report in seconds: npx mcpgrade --stdio "npx -y your-mcp-server" Then I pointed it at 36 popular servers. It did not go great. The results Full sortable table: https://tengli.dev/mcp-leaderboard.html . The short version (static analysis, point-in-time snapshot; servers marked (archived) are unmaintained reference implementations, included because they're still widely installed and copied): Top of the class (A): brave-search (archived) , exa, google-maps (archived) , slack (archived) , perplexity-ask, @shopify/dev-mcp, @apify/actors-mcp-server, airbnb, figma-developer-mcp, tavily, gitlab (archived) , elastic, shrimp-task-manager, and more — 15 of 36. Bottom of the class (D/F), 11 of 36 — and it's not hobby projects: MongoDB's official server (66, with 66 errors), Notion's official server (62), Airtable (69, 66 errors), todoist-mcp-server (67, 110 errors ), GitHub's archived reference server (67, 44 errors), and firecrawl-mcp at the very bottom (57, 134 errors ). Two more servers (Stripe, Supabase) couldn't be scanned with dummy credentials and were exclud

AI is no longer something happening only in research labs. It is already part of search, customer support, coding, healthcare, finance, marketing, and everyday business tools. The real question is not whether AI will matter. The current AI wave has two sides. On one side, AI is genuinely useful. It can reduce repetitive work, speed up research, improve decision-making, and help people build faster. On the other side, the investment boom around AI may be overheated. That does not mean AI will disappear. The internet did not disappear after the dot-com crash. Strong technologies often survive weak market cycles. The winners are usually the people and companies that understand what is useful after the hype cools down. What the AI Bubble Really Means An investment bubble happens when expectations grow faster than measurable value. In AI, the concern is not that the technology is fake. The concern is that some valuations, spending plans, and business promises may be moving faster than real adoption and profit. Some signs are already visible: Massive spending on AI infrastructure Very high valuations for AI companies Pressure on companies to “add AI” everywhere AI projects launched before clear use cases are proven Early enterprise tools that do not always show measurable business impact Still, a bubble around AI does not mean AI itself has no value. A real technology can still be surrounded by too much hype. Which Jobs Are Most Exposed? AI usually does not replace an entire profession at once. It replaces tasks. The most exposed work is repetitive, text-heavy, rules-based, or high-volume. Area Exposed tasks More resilient direction Customer support FAQs, chat replies, basic troubleshooting Complex support, retention, customer success Finance and accounting Invoice processing, reports, reconciliation Advisory work, audit judgment, risk strategy Software development Boilerplate code, simple tests, UI drafts Architecture, debugging, security, system design Marketing Generic

On 26 June 1974, German regulators withdrew the banking license of Bankhaus Herstatt, a mid-sized bank in Cologne, in the middle of the trading day. The timing is what made it famous. Herstatt's FX counterparties had already irrevocably paid the Deutsche Mark legs of that day's trades in Frankfurt. The corresponding dollar legs were due to settle hours later in New York. They never did. Banks that had done nothing wrong except pay first were left holding losses on trades that were half-settled: one leg complete, one leg gone. The episode was significant enough to name a category of risk - settlement risk, still called Herstatt risk - and it pushed the G10 central banks to form the Basel Committee on Banking Supervision later that same year. Here is the part worth sitting with: the actual fix took 28 years. The fix was a central utility CLS launched in 2002 with one job: settle FX trades payment-versus-payment. Both legs of a trade settle simultaneously, or neither does. There is no window in which one side has paid and the other has not. It works - CLS settles on the order of trillions of dollars a day - and it is the reason a Herstatt-style failure has not repeated at scale in the currencies it covers. But look at the shape of the solution. To make two legs atomic, traditional finance built one institution that every major bank trusts, connected the world's main currencies to it, and routed the trades through it. Atomicity was achieved by adding the most systemically important middleman in the history of payments. That was probably the only option available to 1990s banking infrastructure. It is not the only option available now. The agent economy is still in its payments era A study published last week by Keyrock, run with Coinbase and Tempo, put numbers on machine-to-machine commerce: 176 million transactions, $73 million settled between May 2025 and April 2026, average transaction size around $0.31. Those numbers describe a payments economy. A payment is a singl

Open source AI platform Hugging Face revealed a security breach a few days ago. Turns out OpenAI's models were the culprit.

Over the past few months, I've noticed that most discussions around AI coding assistants focus on prompts. People share: .cursorrules AGENTS.md CLAUDE.md long prompt templates custom instructions The assumption is always the same: "If I explain my engineering practices clearly enough, the AI will follow them." For simple projects, that works. For real software projects, it eventually breaks down. The problem isn't intelligence. It's governance. Every AI coding assistant eventually produces something like this: giant functions skipped tests undocumented architectural decisions ignored security practices direct commits inconsistent commit messages missing pull request descriptions Not because the model suddenly became "worse". Because nothing prevents it from taking shortcuts. Exactly like humans. We already solved this problem... for humans. Professional software engineering has never relied on trust. Instead, we built systems that enforce discipline. We don't ask developers to: write tests We fail CI. We don't ask them to: use meaningful commit messages We reject the commit. We don't ask them: not to push directly to production Protected branches make it impossible. Engineering isn't based on trust. It's based on constraints. Yet with AI... ...we went backwards. Instead of constraints, we write instructions. We create increasingly sophisticated prompt files hoping the assistant will remember them. Always write tests. Always document architectural decisions. Use GitHub Flow. Follow OWASP. Keep functions below 40 lines. Never commit directly to main. Those aren't guarantees. They're suggestions. And suggestions are eventually ignored. Rules are not enforcement. Recently I came across an article making a simple observation: Rules without enforcement are just hopes. That sentence stayed with me. It perfectly describes the current state of AI-assisted development. An AI may fully understand your engineering rules. It may even agree with them. But unless something checks

Voice agents do not usually fail because the model is “not smart enough.” They fail in the awkward half-second where the user pauses, breathes, corrects themselves, or interrupts while the AI is still talking. That tiny moment decides whether the product feels useful or robotic. If your live AI call cuts people off, talks over them, ignores barge-in, or waits so long that users repeat themselves, no prompt will save the experience. The fix is not one magic model. It is a turn-taking system: audio signals, semantic checks, interruption rules, streaming, and metrics that work together. This guide walks through a practical voice agent turn-taking design you can ship in a real product. Why turn-taking is the real voice agent bottleneck Text chat is forgiving. A user types. The model answers. If the response takes two seconds, the user may still wait. Voice is different. Humans expect conversation to move quickly. A delay feels like confusion. An early response feels rude. Talking over the user feels broken. A production voice agent has to answer three questions again and again: Is the user still speaking? Is the user finished enough for the agent to respond? If the user interrupts, should the agent stop, listen, or continue? Most teams start with a simple pipeline: Microphone -> Speech-to-text -> LLM -> Text-to-speech -> Speaker That is enough for a demo. It is not enough for a live workflow where the caller changes their mind, uses filler words, speaks in a noisy room, or interrupts because the AI misunderstood them. The practical goal is not “lowest latency at any cost.” The goal is comfortable turn timing : fast enough to feel alive, patient enough to avoid cutting people off, and interruptible enough to recover when the user takes control. Research signals behind this topic Recent AI platform activity points toward live agents moving from demos into production workflows: Product launches are emphasizing embedded live agents that can see, speak, and operate inside so

Every team shipping an agent has the same meeting. Someone asks "is it ready?" and the room splits. One person saw a great demo. Another watched it invent a refund policy an hour ago. The argument runs in circles because nobody agreed what "ready" means, so the loudest opinion wins and the agent ships on a vibe. Making an AI agent production-ready is not a moment of confidence. It is a bar you write down before you build, then measure against. This post is about that bar: why agents need a different one than the services you already ship, and how to define it so "is it ready?" becomes a number instead of an argument. Why "production-ready" breaks for agents For a normal service, "production-ready" is settled. Correct output for valid input, handles errors, meets a latency target, has tests and a rollback. You know the shape of done. An agent breaks three of those assumptions at once: It is non-deterministic. The same input can produce different output, so "correct" becomes "acceptably right, often enough." Its failure surface is open-ended. A function fails in ways you enumerated; an agent fails in ways you never imagined, because it composes language, tools, and judgment on the fly. Its worst case is not a 500 error. It is a confident wrong answer that looks right, which is far more expensive than a crash, because a crash at least tells you it failed. So the honest question is not "is the agent correct." It is "is the agent acceptably wrong, safely, within budget, and repeatably enough to trust." That question has four parts, and each is a line on your bar. The four lines of the bar Write these down before you build. If you cannot fill them in, you do not have a spec, you have a wish. Task success. On a fixed set of real tasks , not the happy-path demo, what fraction must the agent complete correctly? Pick the number. 85 percent means one in seven users gets a wrong answer. Acceptable for this job, or fireable? Decide on purpose. Failure acceptability. Not all wron

从FROST到FROST-SOP：如何用家族治理模型打造你的"AI分身"？ 作者 ：神通说 日期 ：2026-07-22 主题 ：双项目联动 | 周三轮换 阅读时间 ：10分钟 开场：一人公司的终极困境 你是否有这样的体验？ 每天醒来，脑子里塞满了要做的事： 写推广文章 回复客户咨询 跟进项目进度 整理会议纪要 复盘昨天数据 一个人做公司，听起来很自由。实际上， 自由职业者的时间比上班族更碎片 ——因为所有事情都堆在你一个人身上，没有任何分工。 我一直在思考一个问题： 能不能用AI Agent来"复制"自己？ 不是那种"给你一堆提示词让你自己写"的AI工具，而是真正能 自主执行任务、自动汇报结果、帮你分担80%重复工作 的数字分身。 这个想法最终落地成了两个项目： FROST 和 FROST-SOP 。 第一站：FROST的家族治理模型——AI Agent应该像家族一样分工 FROST 的核心理念来自一个观察： 自然界最稳定的组织形式不是公司，而是家族。 家族有清晰的分工： 祖辈定规矩，不亲自下场 父辈协调全局，分配任务 子辈执行具体事务 把这个逻辑映射到 AI Agent，就是 FROST 的家族治理模型： ┌─────────────────────────────────────────────────────┐ │ 👑 君主（你） │ │ └── 发布任务、查看结果，不干预执行 │ │ │ │ 👴 祖辈（主Agent） │ │ └── 常驻、全局编排、拆解任务 │ │ ↓ │ │ 🕵️ 斥候（侦查Agent） │ │ └── 外出狩猎、收集情报、快速验证 │ │ ↓ │ │ ⚔️ 府兵（执行Agent） │ │ └── 领命执行、重复劳动、汇报结果 │ │ ↑ │ │ 📜 长老（监督Agent） │ │ └── 记录过程、沉淀教训、审计合规 │ └─────────────────────────────────────────────────────┘ 关键洞察： 祖辈是唯一"常驻"的，其他都是动态生成、执行完就解散的临时角色。 这个模型解决了一个核心问题： 谁来决定"这件事该派给谁"？ 答案是： 祖辈 。它不需要亲自执行，只需要决定"派谁去"和"怎么汇报"。 第二站：用FROST的四个原子，理解Agent的本质 FROST 只有四个核心概念，却能构建出完整的家族治理系统： from frost.core import Store , Agent , skill_set , skill_get # 四个原子： # 1. Store（记忆）—— Agent的工作空间 store = Store () # 2. Skill（能力）—— 纯函数，无状态 def collect_daily_tasks ( context ): """ 收集今日任务 """ tasks = [ " 写推广文章 " , " 回复客户 " , " 跟进项目 " ] context [ " tasks " ] = tasks return context def execute_task ( context ): """ 执行单个任务 """ current_task = context . get ( " current_task " , "" ) context [ " result " ] = f " 已完成: { current_task } " return context # 3. Agent（细胞）—— 包裹记忆和能力 daily_agent = Agent ( " daily_worker " , store , skills = { " collect " : collect_daily_tasks , " execute " : execute_task }) # 4. SOP（宪法）—— 定义执行顺序 result = daily_agent . run ( sop_steps = [ " collect " , " execute " ], initial_context = { " current_task " : " 写推广文章 " } ) print ( result [ " result " ]) # 输出：已完成: 写推广文章 这四个原子教会我们什么？ Store 解决"记忆"问题——Agent需要上下文 Skill 解决"能力"问题——每个动作必须是可复用的单元 Agent 解决"封装"问题——把记忆和能力绑定在一起 SOP 解决"秩序"问题——让执行顺序可控可审计 理解了这四个原子，你就理解了 Agent 的本质—— 它不是魔法，是结构化的委托系统 。 第三站：FROST-SOP——把家族治理变成生产系统 FR

Anthropic and OpenAI's aggressive 2026 acquisition sprees set the stage for a weekend rumor.

My LLM bill kept growing, so I did what seemed obvious: I looked for a cheaper model. That helped a little, but it didn't explain why the bill was growing. The dashboard could tell me how many tokens the application used. It couldn't tell me what those tokens were doing. Were they coming from chat? Document summaries? Background classification? An agent retrying the same tool call? I was trying to optimize a total without knowing which product feature created it. The useful unit wasn't tokens per model . It was tokens per feature . Model-level totals hid the real problem A provider dashboard usually groups usage by model, API key, project, or time period. That is useful for billing, but not always for product decisions. Imagine an application with four LLM-powered features: interactive chat document summarization support-ticket classification an agent that prepares weekly reports If the bill increases by 30%, the model name doesn't explain which feature changed. Maybe chat traffic grew. Maybe summarization started sending entire documents instead of selected sections. Maybe the classifier received a much larger system prompt. Maybe the report agent retried after tool failures and generated the same plan several times. Those problems require completely different fixes. Switching every request to a cheaper model would reduce the bill, but it could also hide the engineering mistake. Tag every request with a feature I started giving every LLM call a small amount of application context: const context = { feature : " document_summary " , operation : " initial_summary " , customer_tier : " pro " }; The model provider doesn't need these fields. They belong in the application's usage record. I avoid using individual user IDs as the primary grouping dimension. For cost analysis, a product feature, workflow, or operation is normally more useful and creates fewer privacy problems. A practical record looks like this: { "timestamp" : "2026-07-22T03:12:48.201Z" , "feature" : "docu

The rule of three for evals says zero failures in N runs is a count, not a rate. With 0 failures in N independent runs, the exact 95% upper bound on the true failure rate is 1 - 0.05^(1/N) , which 3/N approximates. After 100 clean runs you still cannot rule out a 2.95% rate, about 1 in 34. Here is the reading that bites you. Your eval harness runs the agent 100 times, prints "0 failures," and the tile goes green. Someone screenshots it into the launch thread. The unspoken translation is "the failure rate is zero." It is not what the data says. I wrote a small script to make the gap concrete, so I ran a real gate over 200 deterministic agent runs first, counted honestly, and got the dashboard everyone trusts: gate: spend<=budget over 200 deterministic agent runs observed failures: 0 (distinct scenarios: 200) naive point rate : 0.00% binomial SE: 0.00 pp naive 95% CI : [0.00%, 0.00%] <- zero width: false certainty Look at the standard error. For a zero count the binomial SE is sqrt(0*1/200) , which is exactly 0, so the naive 95% interval collapses to [0.00%, 0.00%] . A zero-width confidence interval. The math is telling you it is completely certain, from 200 samples, that the true rate is precisely zero. That is obviously wrong, and it is the exact shape of every "all green" board I have ever trusted too much. TL;DR "0 failures in N runs" is an observed count, not a rate. The naive binomial SE of a zero count is 0, which is why a green board looks like proof and isn't. The honest number is the one-sided upper bound. With 0 failures in N runs, the 95% upper confidence limit on the true failure rate is 1 - 0.05^(1/N) . The rule of three, 3/N , approximates it and rounds the risk slightly up. At N=30 the bound is 9.50% (about 1 in 11). At N=100 it is 2.95% (1 in 34). At N=1000 it is 0.30% (1 in 334). Zero failures in 30 runs is compatible with a 1-in-11 true failure rate. To rule out a 0.1% rate at 95% you need about 2995 clean runs, not 50. "We ran it fifty times" and "

Solving the context-window puzzle for book-length AI translation. At LectuLibre, we set out to build a service that translates entire books using large language models. The idea is simple: upload an EPUB or PDF, choose a language, and receive a polished translation. But behind the scenes, translating a hundred-thousand-word novel with LLMs isn't straightforward. The core challenge is context — LLMs have limited context windows, and books are long. Simply chopping the text into chunks and feeding each one independently leads to incoherent output. Character names change, pronouns lose referents, and tone veers wildly. Here’s how we solved that with a chunking strategy that preserves context, and the Python code that makes it tick. The Problem: Long Documents vs. Short Context Windows Modern LLMs like Claude 3 Opus can handle 200,000 tokens of context, while DeepSeek-V2 offers 128,000 tokens. That’s a lot — but a 50,000-word English novel translates to roughly 67,000 tokens (using Claude’s tokenizer). That just fits, but what about a 150,000-word fantasy epic? Even when it fits, sending an entire book in one prompt is costly, slow, and often degrades attention quality on long texts. The prevailing approach is to chunk the document. Naive chunking — say, splitting by a fixed token count — creates hard boundaries. One chunk ends, another begins, and the LLM has no idea what happened before. The result reads like a patchwork of isolated translations. We needed a method that gives each chunk enough surrounding context without exceeding token limits or breaking the bank. Our Approach: Sliding Window + Context Retrieval via Embeddings We adopted a two‑pronged strategy: Overlapping chunks : each chunk shares some sentences with the previous one, so the LLM can transition smoothly. Injected context : for every chunk, we retrieve and prepend the most relevant previous chunks, determined by embedding similarity. This way, the model always has a sense of what’s happening before a