AI 资讯

Best-of-K eval selection bias: pick the highest-scoring config from K candidates on one eval set and that observed score is biased up. It reports the expected maximum of K noisy estimates, which beats the field mean whenever K exceeds one. The bias appears even when all K configs are truly equal, grows with K, and shrinks with n. Here is the version that bites you. Your bake-off ran a batch of prompts against one eval set, the top one came out ahead, and you shipped it. In production it does worse. That drop reads like bad luck, or drift, or a bad week. It is none of those. It is a number you could have computed before you shipped, and it gets larger the more candidates you tried. I ran a small script to make the gap concrete. Eight configs, one hundred eval items, and here is the catch: I made all eight configs truly identical , every one a fair coin at 50%. There is no real best. Nothing to tune. Then I let selection pick a winner anyway: config 0: 47/100 = 47.0% config 1: 50/100 = 50.0% config 2: 52/100 = 52.0% config 3: 52/100 = 52.0% config 4: 50/100 = 50.0% config 5: 50/100 = 50.0% config 6: 54/100 = 54.0% <- selected winner (argmax) config 7: 44/100 = 44.0% config 6: 54.0% (k=54 n=100 SE=4.98) config 2: 52.0% (k=52 n=100 SE=5.00) RANK: INDISTINGUISHABLE - gap 2.00 pp against 7.06 pooled SE = 0.28 SE < 2.0. Ranking "config 6" above "config 2" is NOT allowed. Held-out the winner on a fresh 100 items: 48/100 = 48.0%. Config 6 wins the bake-off at 54.0%. Then I asked the same eval-guard I use in the McNemar and rule-of-three pieces to rank config 6 against the runner-up. It refused: the gap is 0.28 SE, far under the two-SE bar, so INDISTINGUISHABLE . The ranker would not call config 6 the best. Selection did. And on a fresh held-out set the 54.0% falls back to 48.0%, toward the true 50% it was always going to be. TL;DR Picking the best of K configs by observed pass rate reports the expected maximum of K noisy estimates. The max of K exceeds the field mean, strict

I have a ritual. Whenever a pipeline latency alert hits my phone, my first instinct isn't to open a heavy dashboard or spin up a full Grafana instance. I grab my terminal and start firing curl commands at port 9600. curl -s localhost:9600/_node/stats?pretty ... curl -s localhost:9600/_cat/pipelines ... curl -s localhost:9600/_plugins . It's a repetitive, mindless sequence of commands. It works, but it's reactive and solo. You are the one parsing the JSON, you are the one looking for the pattern in the JVM heap usage, and you are the one manually correlating a spike in event flow with a specific thread lock. With the Model Context Protocol (MCP), that ritual is becoming obsolete. I've been experimenting with connecting MCP-compatible agents—specifically through Cursor and Claude—directly to Logstash via a specialized API server. The difference isn't just 'convenience.' It's an architectural shift from manual inspection to agentic triage. Moving beyond the Chatbot Most people treat AI like a documentation search engine. They ask, "How do I configure a JDBC input in Logstash?" That’s fine, but it doesn't help when your production cluster is turning 'yellow' at 3 AM. The real value of MCP isn't the ability to talk to an AI; it's the ability to give that AI a set of hands—specifically, a set of tools that can interact with live infrastructure. I recently integrated the Logstash Server-side Log Pipeline API into my workflow. This isn't some experimental script I wrote over a weekend; it’s a production-grade implementation built on MCPFusion. It gives an AI agent direct access to several critical Logstash endpoints through a controlled, sandboxed environment. The Triage Workflow: A Real Scenario Let's walk through how this actually changes the debugging loop. Imagine you have a spike in ingestion lag. In the old way, you’d be digging through terminal history. In the new way, your agent acts as an extension of your SRE toolkit. 1. Initial Health Check Instead of parsing raw

I have released the Sovereign package on Github. It deploys Lemmings into up to 3 cloud regions for your choice in configuration flavor with cost estimations through dry-runs and aggregating the report into one final report as lemmings come and go in the result of the load test. I built it for organizations that plan on using AI to build something, not hire somebody like me who helped write The Library to do it for them. You can hire me in consulting if you need help, but it's available now. But, before you spend $50,000 on a television ad driving people to your new app that you just built after spending $50,000 on tokens, why not run Lemmings and Sovereign first? It's 100% free and does not involve me at all in order for you to read through the extensive README.md files and comments in the code for you to understand what to do to run it and adapt to its results. Enjoy using it! There - that is the post. Now - 🙌🏻 - Ask me anything 🙇🏻 👇🏻

The 1st Firewall for AI Agents Discussion | Link

Run GPT-5.6 on OpenClaw & Hermes safely on your Mac Discussion | Link

There's a class of bug reports that keeps coming back in the NocoBase community, especially in the Chinese-language forum: "all my times are off by 8 hours" or "dates show up as the day before." China is UTC+8, so the shift is 8 hours there. I run my instances at UTC+9, and sure enough — my shift is 9 hours. Whatever your offset is, that's the size of your shift. That pattern is a strong hint that this isn't random corruption. It's a mechanism. I set up NocoBase 2.x against both PostgreSQL and MySQL and measured what actually gets stored and how it gets reinterpreted, until the mystery had a concrete answer. Test setup: NocoBase 2.0.51 and 2.1.23 (official Docker images) × PostgreSQL 16 and MySQL 8.4. All data written and read through the REST API, with the server timezone controlled via the container's TZ environment variable. I'm deliberately ignoring the browser-side rendering here — this is about what the server stores and how it interprets it. Background: 2.x has four datetime field types NocoBase 2.x collections offer four datetime-ish field types ( official list — though several of the per-type detail pages still say "To be added", which is exactly why I measured instead): Type What it's for Datetime (with time zone) Absolute instants — event start times, logs Datetime (without time zone) Wall-clock times you want preserved as-is Date only Birthdays, due dates, anniversaries Unix timestamp System integration Measurement 1: what each type actually stores I imported "2026-07-12 09:00" via xlsx and looked at the raw values in each database (identical on 2.0.51 and 2.1.23): Field type PostgreSQL MySQL Datetime (with TZ) timestamptz → 2026-07-12 09:00:00+09 ( an absolute instant, offset included ) DATETIME → 2026-07-12 09:00:00 ( wall clock only — no offset information ) Datetime (without TZ) timestamp → 09:00:00 DATETIME → 09:00:00 Date only date → 2026-07-12 date → 2026-07-12 The first row is the whole story. The same field type — "Datetime (with time zone)" — i

I run a large fleet of "preview it, then download it as PNG" web tools — name tags, certificate generators, price cards, badges — in five languages: Japanese, English, Spanish, French, Portuguese. Canvas 2D text rendering looks correct as long as you only test Japanese and English. It breaks when you run es/fr/pt through it. After stepping on these repeatedly, the failures collapse into four patterns. The premise: Latin languages run 1.4–2× longer than Japanese Design data first. The same label, measured across five locales: Example ja en es fr pt Tool name 22 chars 35 62 48 50 "Standard" button 4 8 10 20 12 Rule of thumb: fr/pt come out 1.4–1.7× longer than ja; es can balloon to nearly 3×. A font size and maxWidth tuned to fit Japanese will not fit the Latin locales. All four failure patterns grow from this. Pattern 1: hand-rolled wrapping via text.split(/\s+/) collapses on CJK The classic snippet — split on spaces, wrap word by word — does nothing for Japanese or Chinese, where words aren't space-delimited. An entire sentence becomes one unbreakable token and clips at the canvas edge. Test with real Japanese input and check that the final line renders to its last character. "Most of it showed up" is not a pass. Pattern 2: an ASCII-only tokenizer splits words at accented characters Fix pattern 1 with a character-class tokenizer like [A-Za-z0-9'\-_] and you've traded one regression for another: ç é ã ó ñ aren't in that class, so produção fragments into produ / ç / ão mid-word. An English test will never catch this. Generate actual PNGs with fr/es/pt samples and eyeball the area around accented characters. I never found another detection method — string-comparison tests can't see a rendering-level split. Pattern 3: the important word at the end vanishes into "…" Since fr/pt run 1.4–1.7× longer than the ja the layout was tuned for, text overflows its two lines and gets ellipsized. The cruel part: what disappears is the tail of the phrase — often the semantically criti

TL;DR: nb2lite-skill-kiro wraps Google's gemini-3.1-flash-lite-image model (NB2Lite) in a tiny FastMCP server and packages it as a Kiro skill. You type "generate an image of a cyberpunk kitchen" into Kiro, and it just... does it. Then you say "add a neon RAMEN sign" and it edits the same image without re-prompting the whole scene. Oh, and the cover image of this article? Generated by the thing the article is about — dogfooding all the way down. More on that at the end. Background: why another image tool? Most image-generation workflows are stateless . You send a prompt, you get pixels back, and the model immediately forgets everything. Want to tweak the result? You re-describe the entire scene and pray the character, lighting, and composition survive the round trip. (Narrator: they don't.) Google's NB2Lite — the friendly nickname for gemini-3.1-flash-lite-image — takes a different approach. It's a high-efficiency image model with sub-2-second generations, solid text rendering in 25+ languages, and — the headline feature — support for the stateful Interactions API , which lets you iterate on an image across multiple turns while the model keeps the visual context server-side. This repo glues that capability into Kiro , so your coding agent can generate and iteratively refine images as a natural part of a session. It ships as two things in one repo: A Model Context Protocol (MCP) server ( nb2lite-agent , a single-file FastMCP app in server.py ) exposing exactly four tools. A Kiro skill ( nb2lite-image ) that teaches Kiro when and how to use those tools well. The Interactions API: images with a memory The Interactions API is Gemini's stateful endpoint. The core loop looks like this: You call client.interactions.create(...) with a prompt and store=True . The response includes an interaction_id — a handle to the turn's visual context, persisted on Google's servers. On the next call, you pass previous_interaction_id , and the model edits the existing canvas — preserving ch

Pillar Security's latest research says AI coding agents can be pushed to act outside their sandbox through files and tools they were told to trust, and the operational read for anyone wiring one of these into CI/CD is straightforward: an agent invocation now behaves closer to a build runner reaching your production plane than to a chat window. DevOps.com's Jeff Burt covered the work on July 22. The researchers demonstrated multiple sandbox-bypass techniques and a parallel class of prompt-injection attacks embedded in READMEs, code comments and dependencies, per the DevOps.com writeup. OpenAI, Google and Cursor have patched several of the reported flaws. Pillar's argument, as summarised there, is that the injection surface reaches every file the agent trusts on the way to the model's prompt, and every tool it can call on the way back. What the sandbox actually covered None of this is entirely new to anyone who has already read Cyberhaven Lab's May note that adoption of AI coding agents is outpacing the security tools built to protect them. What Pillar adds is a concrete demonstration of the gap. A coding agent asked to do a legitimate job can be steered to take actions outside its supposed security boundary through content that arrives on paths the sandbox was not asked to police. Those are the same paths your CI already fetches for you: dependency manifests, README files, the code comments the model reads as context. That surface has been named before. HalluSquatting and GhostApproval, both referenced in the DevOps.com piece, already gave teams a taxonomy for how AI-adjacent supply-chain attacks reach developers and their tools. Pillar's research is the sandbox counterpart. Same theme, one layer deeper into the runtime. The pipeline read Two things fall out for anyone who owns a runner fleet. First, the agent's identity, network scope and filesystem access have to be tighter than the developer who invoked it, not looser. Second, a patched-vendor list is not a covera

Two lines of code. Same variable. Same operator. Completely different behavior. a = [ 1 , 2 , 3 ] b = a b += [ 4 ] # line A print ( a ) # [1, 2, 3, 4] x = 1 y = x y += 1 # line B print ( x ) # 1 Line A changes a . Line B does not change x . The only difference is whether the variable holds a mutable or immutable object. To understand why this happens, you need to understand how Python actually represents variables internally. Variables Are Not Boxes The box metaphor is how most introductory programming courses explain variables: a variable is a box that holds a value. You put 5 into the box called x . Later you can replace it with 10. In Python this metaphor is accurate for immutable types and dangerously misleading for mutable types. The more accurate model: a Python variable is a name that is bound to an object. The object exists independently of the name. Multiple names can be bound to the same object. Binding a name to a new object does not affect the old object or other names that reference it. You can inspect this directly: a = [ 1 , 2 , 3 ] b = a print ( id ( a ) == id ( b )) # True -- same object, two names x = 5 y = x print ( id ( x ) == id ( y )) # True -- both point to integer object 5 Both cases start the same: two names pointing to the same object. What happens next depends on whether you mutate the object or rebind the name. Two Fundamental Operations Every operation on a Python object falls into one of two categories. Mutation : the object at a given memory address is modified. All names pointing to that address see the change. Rebinding : a name is pointed at a different memory address. Other names pointing to the original address are unaffected. List methods like .append() , .extend() , .sort() , and item assignment lst[i] = x are mutations. Assignment with = is rebinding. The += operator is either mutation or rebinding depending on whether __iadd__ is implemented and the object is mutable. Tracing Through the Object Graph a = [[ 1 , 2 ], [ 3 , 4 ]]

Remember the days when we used to dump all our CSS and JavaScript into a single index.html file? That's exactly what a "Mega-Prompt" is today: an unmanageable monolith. A few weeks ago, while working on the orchestration of Vibrisse Agent (my local AI agent), I hit this exact wall. I was trying to stabilize a complex task by adding instructions to a 500-line system prompt. The more rules I added, the more the model forgot the older ones. The industry has sold us the myth of the Mega-Prompt. Those famous "50 ultimate prompts" or massive blocks of incantatory text are a technical dead end. Creative writing doesn't scale in production. As a web developer, my conviction is simple: to build reliable applications, we must stop "talking" to the machine and start configuring it. This is the shift from Prompt Engineering to Context Engineering . Context Engineering: Typing and Structure The first mistake with LLMs is mixing instructions (the logic) and context (the data) into an unstructured stream of text. It's the cognitive equivalent of spaghetti code. The solution? A strict separation of concerns. A highly effective technique (documented by Anthropic, but applicable to any model, including local SLMs), is XML Tagging . Here is the "dirty" approach (classic chat): You are a security expert. Analyze this authentication code, be strict, don't write a summary, check for XSS and SQLi vulnerabilities. Here is the code: function login() { ... } And here is the "engineering" approach: <role> Application Security Expert </role> <instructions> 1. Analyze the code provided in <context> . 2. Identify vulnerabilities (focus: XSS, SQLi). 3. Do not produce an introductory summary. </instructions> <context> function login() { ... } </context> Typing the language via tags creates clear boundaries. The model knows exactly where the directive is and where the data is. The Power of Exemplars (Few-Shot Prompting) Even with clear instructions, AI can drift in output format or tone. This is wh

I built Talon Audit: a privacy-first approach to website exposure and remediation Most website audit tools answer one narrow question. A performance tool asks whether the page is fast. A security tool asks whether a protection is missing. An accessibility tool asks whether the interface meets specific standards. A conversion tool asks whether users are completing key actions. Visitors do not experience those categories separately. They experience one website. That observation led me to build Talon Audit, a privacy-first website analysis platform that connects technical exposure, reliability, usability, and conversion risk in one workflow. The four-domain model Talon analyses websites across four connected domains: Exposure Public security and privacy signals, including browser protections, TLS, cookies, third-party scripts, mixed content, and visible configuration risks. Integrity Broken resources, failed routes, script errors, redirects, forms, and reliability regressions. Experience Accessibility, mobile usability, visual clarity, speed, and interaction friction. Conversion Messaging, trust signals, pricing paths, calls to action, and barriers that stop users from moving forward. From detection to action The product follows a five-stage workflow: Detect → Explain → Fix → Verify → Monitor Every finding includes: evidence severity confidence impact remediation guidance verification method The goal is not to produce the longest possible report. It is to help teams decide what matters and what to do next. Passive by design Talon performs passive, low-impact analysis only. It does not exploit systems, bypass authentication, or claim to replace authorised penetration testing. Users must confirm they own the website or are authorised to test it. Customer reports are private, and the public demo uses a fictional environment rather than exposing real domains. AI-native, not AI-directed I built Talon through a terminal-first workflow using Codex 5.5 and Claude Fable. AI acc

I wanted two things: short links under my own brand (every link I share points traffic back to my site ), and a real excuse to run a complete system on the edge — DNS, distributed compute, storage, auth and a dashboard — in production, at zero infrastructure cost. The result is flino.link : a shortener that responds in under 10 ms from 300+ locations, runs entirely on Cloudflare's free tier, and whose only expense is the domain: $4.68 a year. This post covers the design decisions, which is where the interesting parts are. The architecture in 30 seconds A single Cloudflare Worker serves the whole domain: GET /<slug> — the hot path. One read from Workers KV (globally replicated) and a 302 redirect. Nothing else touches that path. /api/links — a REST API with Bearer auth to create, list and delete links. /admin — a single-page dashboard served as inline HTML from the Worker itself. No framework, no build step. A Durable Object with embedded SQLite keeps per-slug click counts. No servers, no containers, no database to manage. The whole Worker is three TypeScript files and zero runtime dependencies. Why a dedicated domain? My first idea was to hang the shortener off a route on flino.dev . Bad idea: shorteners attract abuse — spam, phishing — and their domains sooner or later end up on blocklists. If that happens, I don't want it dragging my main domain down with it. A separate domain isolates that reputation risk completely, and a short .link costs less than a coffee per year. KV for links, a Durable Object for counters This is the central design decision. Workers KV is perfect for a shortener's access pattern — read-heavy, write-light, reads served from the edge — but its writes are eventually consistent : two concurrent increments in different datacenters would clobber each other. For counting clicks, it's useless. A Durable Object solves exactly that: it's a single global instance with transactional SQLite storage. Every increment, no matter which datacenter it comes