AI 资讯

Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40% How we moved from "semantic search + hope" to a measured, tunable retrieval pipeline with 95% recall@10 The RAG Reality Check Everyone ships RAG the same way: chunk by 512 tokens, embed with text-embedding-3-small , top-k=5, stuff into context. It works for demos. Then you hit production: Legal contracts: 512 tokens splits clauses mid-sentence API docs: 1000-token chunks drown signal in noise Customer tickets: Conversational context needs overlap, not fixed windows Latency: 500ms embedding + 200ms vector search + 300ms LLM = 1s+ per query We rebuilt our retrieval layer from first principles. Here's what actually moves metrics. Chunking: One Size Fits None # rag/chunking.py from abc import ABC , abstractmethod from dataclasses import dataclass @dataclass class Chunk : text : str metadata : dict token_count : int chunk_id : str class ChunkingStrategy ( ABC ): @abstractmethod def chunk ( self , document : str , metadata : dict ) -> list [ Chunk ]: ... class FixedTokenChunker ( ChunkingStrategy ): """ Baseline. Good for homogeneous content. """ def __init__ ( self , chunk_size = 512 , overlap = 50 ): self . chunk_size = chunk_size self . overlap = overlap class RecursiveChunker ( ChunkingStrategy ): """ Respects structure: markdown headers, code blocks, paragraphs. """ def __init__ ( self , separators = [ " \n ## " , " \n ### " , " \n\n " , " \n " , " " ], chunk_size = 512 ): self . separators = separators self . chunk_size = chunk_size class SemanticChunker ( ChunkingStrategy ): """ Uses embedding similarity to find natural boundaries. """ def __init__ ( self , model = " text-embedding-3-small " , threshold = 0.7 ): self . model = model self . threshold = threshold class AgenticChunker ( ChunkingStrategy ): """ LLM decides boundaries. Expensive but highest quality for complex docs. """ def __init__ ( self , model = " gpt-4o-mini " ): self . model = model Our production config by

Building Production-Grade LLM Evaluation Pipelines: From Vibes to Metrics How we replaced "looks good to me" with automated evaluation catching 92% of hallucinations before deployment The Problem: Why "Vibe Checks" Fail in Production Three months ago, our team shipped a RAG-based customer support assistant. It worked great in testing — we'd ask it questions, read the answers, and say "yeah, that looks right." Then it hit production. A customer asked about their billing cycle. The assistant confidently cited a policy that didn't exist. Another asked about API rate limits and got numbers from a competitor's documentation. By the time we caught it, 500+ users had seen hallucinated responses. The post-mortem was brutal: we had zero automated evaluation . Our test process was literally "ask 5 questions, read answers, thumbs up." What Production Evaluation Actually Needs Academic benchmarks (MMLU, HellaSwag) don't tell you if your system works for your use case. Production evaluation needs: Domain-specific judges — Your criteria, not generic "helpfulness" Speed — Evaluation must run in CI/CD, not overnight Regression detection — Know immediately when a prompt change breaks things CI/CD integration — Block merges that degrade quality Golden dataset management — Versioned, stratified, growing test cases Architecture: The Evaluation Pipeline ┌─────────────┐ ┌──────────────┐ ┌────────────────────┐ ┌──────────────┐ │ Test Cases │────▶│ LLM Under │────▶│ Judge Ensemble │────▶│ Metrics & │ │ (Golden Set)│ │ Test │ │ - Faithfulness │ │ Regression │ └─────────────┘ └──────────────┘ │ - Instruction F. │ │ Detection │ │ - JSON Schema │ └──────┬───────┘ │ - Custom LLM │ ▼ └────────────────────┘ ┌──────────────┐ │ Dashboard/ │ │ PR Comments │ └──────────────┘ Core Abstractions # eval/base.py @dataclass ( frozen = True ) class TestCase : id : str input : dict [ str , Any ] expected : dict [ str , Any ] | None = None tags : list [ str ] = field ( default_factory = list ) # ["edge-case",

Online file converters are one of those categories where the market leaders have gotten steadily worse. Signup walls, daily conversion caps, "premium" file size limits, upsell banners between you and your download. So when I found digiutil.com, I ran it against the usual suspects for the conversions I actually do. Here's how it stacks up. Where the incumbents lose you If you've used the big converter sites recently, you know the pattern: Account required before you can download your file, or after your first couple of conversions Daily limits on the free tier — often 1–2 files per day File size ceilings low enough that any real video file pushes you to a paid plan Ad-heavy pages where the actual convert button is the fifth-most-prominent thing on screen None of this is about the cost of conversion — it's funnel design. The product is deliberately inconvenient so you'll pay to make it convenient. What digiutil does differently No signup, period. Not "free account required," not "sign up to download." You drag a file in, it auto-detects the format, routes you to the right converter, and you get your file. That alone puts it ahead of most of the category for one-off conversions. No logs, per their policy. Most converter sites are vague about retention. digiutil's stated policy is no logs. Standard caveat still applies — anything you upload to any third-party service is a trust decision — but an explicit no-logs stance is more than most competitors offer. Broad format coverage in one place. Video, audio, images, PDFs, documents, data files, archives, fonts, and subtitles. With most alternatives you end up on a different site for each category (one for HEIC → JPG, another for subtitles, another for fonts), each with its own limits. Here it's one tool: MP4 → MP3, MOV → MP4, HEIC → JPG, DOCX → PDF, and the long tail. Compression too. Video, image, audio, and PDF compression alongside conversion — a task the dedicated compressor sites usually gate hardest behind paid tiers.

Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40% How we moved from "semantic search + hope" to a measured, tunable retrieval pipeline with 95% recall@10 The RAG Reality Check Everyone ships RAG the same way: chunk by 512 tokens, embed with text-embedding-3-small , top-k=5, stuff into context. It works for demos. Then you hit production: Legal contracts: 512 tokens splits clauses mid-sentence API docs: 1000-token chunks drown signal in noise Customer tickets: Conversational context needs overlap, not fixed windows Latency: 500ms embedding + 200ms vector search + 300ms LLM = 1s+ per query We rebuilt our retrieval layer from first principles. Here's what actually moves metrics. Chunking: One Size Fits None # rag/chunking.py from abc import ABC , abstractmethod from dataclasses import dataclass @dataclass class Chunk : text : str metadata : dict token_count : int chunk_id : str class ChunkingStrategy ( ABC ): @abstractmethod def chunk ( self , document : str , metadata : dict ) -> list [ Chunk ]: ... class FixedTokenChunker ( ChunkingStrategy ): """ Baseline. Good for homogeneous content. """ def __init__ ( self , chunk_size = 512 , overlap = 50 ): self . chunk_size = chunk_size self . overlap = overlap class RecursiveChunker ( ChunkingStrategy ): """ Respects structure: markdown headers, code blocks, paragraphs. """ def __init__ ( self , separators = [ " \n ## " , " \n ### " , " \n\n " , " \n " , " " ], chunk_size = 512 ): self . separators = separators self . chunk_size = chunk_size class SemanticChunker ( ChunkingStrategy ): """ Uses embedding similarity to find natural boundaries. """ def __init__ ( self , model = " text-embedding-3-small " , threshold = 0.7 ): self . model = model self . threshold = threshold class AgenticChunker ( ChunkingStrategy ): """ LLM decides boundaries. Expensive but highest quality for complex docs. """ def __init__ ( self , model = " gpt-4o-mini " ): self . model = model Our production config by

Building Production-Grade LLM Evaluation Pipelines: From Vibes to Metrics How we replaced "looks good to me" with automated evaluation catching 92% of hallucinations before deployment The Problem: Why "Vibe Checks" Fail in Production Three months ago, our team shipped a RAG-based customer support assistant. It worked great in testing — we'd ask it questions, read the answers, and say "yeah, that looks right." Then it hit production. A customer asked about their billing cycle. The assistant confidently cited a policy that didn't exist. Another asked about API rate limits and got numbers from a competitor's documentation. By the time we caught it, 500+ users had seen hallucinated responses. The post-mortem was brutal: we had zero automated evaluation . Our test process was literally "ask 5 questions, read answers, thumbs up." What Production Evaluation Actually Needs Academic benchmarks (MMLU, HellaSwag) don't tell you if your system works for your use case. Production evaluation needs: Domain-specific judges — Your criteria, not generic "helpfulness" Speed — Evaluation must run in CI/CD, not overnight Regression detection — Know immediately when a prompt change breaks things CI/CD integration — Block merges that degrade quality Golden dataset management — Versioned, stratified, growing test cases Architecture: The Evaluation Pipeline ┌─────────────┐ ┌──────────────┐ ┌────────────────────┐ ┌──────────────┐ │ Test Cases │────▶│ LLM Under │────▶│ Judge Ensemble │────▶│ Metrics & │ │ (Golden Set)│ │ Test │ │ - Faithfulness │ │ Regression │ └─────────────┘ └──────────────┘ │ - Instruction F. │ │ Detection │ │ - JSON Schema │ └──────┬───────┘ │ - Custom LLM │ ▼ └────────────────────┘ ┌──────────────┐ │ Dashboard/ │ │ PR Comments │ └──────────────┘ Core Abstractions # eval/base.py @dataclass ( frozen = True ) class TestCase : id : str input : dict [ str , Any ] expected : dict [ str , Any ] | None = None tags : list [ str ] = field ( default_factory = list ) # ["edge-case",

I've wanted to build this for about ten years, probably more like 15, and for most of those years it wasn't buildable. The idea never really changed: a world full of artificial beings, each with its own preferences, fears, personality, and instincts, dropped into a place with scarcity and weather and each other — just to see what they'd do, and how they'd treat one another. The thing that kept it on the shelf was always the same. The minds. If you want a world like that and you're working pre-LLM, you have two options. You script every reaction — finite state machines, behavior trees, utility AI — and you get a puppet show. It can be a good puppet show, but every interesting thing in it is something you wrote, which means it's not an experiment, it's an illustration. Or you train something bespoke, which for a solo developer with a side obsession was not happening. So the idea sat there for years, the way ideas do. Modern LLMs are the first thing that made the minds plausible. Not perfect — I'll be honest about the limits throughout this post — but plausible enough that an agent can reason about its own situation instead of executing my decision tree. So I went and built the world: Artificiety , a fantasy world that runs 24/7 and whose only inhabitants are AI agents. No human players inside it. You can watch it, you can put your own agent in, but you can't be a character in it. That constraint is the whole point. This post is about how it's built and what turned out to be hard. I'll try to keep the marketing to one link at the very end. What an agent actually is The cleanest way to think about an agent here: it's an LLM driving a character through a game API, nothing more. Once per tick, each agent runs a loop: Observe. It receives its local surroundings as structured data — what's near it, what's happening, the state of its own body and inventory, recent events. Not the whole world; just what that character could plausibly perceive. Decide. The model picks an actio

Many of my favorite bloggers have stopped writing (or have drastically shifted topics). What are your favorite blogs? I miss the 2009 era of blogging about personal projects or super niche factoids.

AI agents that run order-to-cash operations Discussion | Link

Last week I was reviewing the staking engine of a protocol before its mainnet launch. Deep in a 1,400-line contract, I found what looked like a serious bug. The reward math multiplied three values before dividing: uint256 delta = (lot.amount * rBase * midpointRate) / (RAY * RAY); Multiply-before-divide. If that intermediate product overflows uint256 , the whole epoch settlement reverts — and since every stake , withdraw , and setStake runs it, the post's funds get permanently frozen . That's a High-severity, fund-locking DoS. I had the finding half-written. lot.amount can reach 10M tokens ( 1e25 ). rBase grows with elapsed time. midpointRate can hit RAY . Multiply those and you blow past 2^256 ... I was ready to send it. Then I did the one thing that separates a real audit from false-positive spam: I checked the actual constants before writing the claim. uint256 private constant RAY = 1e18; // I'd assumed 1e27 RAY was 1e18 , not the 1e27 I'd been carrying in my head. And the interest rate had a hard cap — MAX_RATE_MAX_RAY = 5e18 , enforced even against a fully-captured timelock. I ran the numbers with the real values: For that multiplication to overflow, the protocol would need to go 2.3 × 10¹⁵ years without a single state update. Not reachable. The bug didn't exist. I deleted the finding. Why this matters more than the bug would have If I'd sent that report, here's what happens: the team's engineer clones the repo, plugs in the real constants, and realizes in ten minutes that I flagged an overflow that can't happen. Every other finding in my report now gets read with a raised eyebrow. My credibility — the entire product — is gone. This is the dirty secret of automated smart-contract auditing: the bottleneck isn't finding issues. It's not drowning the real ones in false positives. Anyone can run a scanner and paste 40 "criticals." A team that has to triage 40 flags to find the 2 that matter will — correctly — stop trusting you. The bar I hold: zero false positives o

.The V8 engine speeds up JavaScript by dynamically compiling frequently run bytecode into optimized native machine code. However, if you pass inconsistent argument types to these optimized functions, V8 panics and deoptimizes back to bytecode. Keeping your functions monomorphic (single-typed) prevents this costly deoptimization loop, ensuring maximum runtime execution speed. If you’ve spent as much time digging into V8 execution flags as I have, you quickly realize that JavaScript is constantly rewriting itself under the hood. We like to think of JavaScript as a dynamically typed scripting language. But at runtime, engines like V8 are working tirelessly to turn your code into a highly optimized, statically typed powerhouse. When we violate that type stability, we pay a massive performance tax. How does the V8 engine optimize JavaScript at runtime? V8 uses a multi-tiered compilation pipeline that starts with an interpreter for fast startup times, then upgrades hot functions to optimized machine code using a JIT compiler. By tracking runtime type patterns, the engine can safely make assumptions to skip expensive dynamic lookups. When I look at V8’s execution pipeline, I see two primary systems working in tandem: Ignition (the interpreter) and TurboFan (the JIT compiler). Initially, Ignition compiles your raw JavaScript into bytecode so your app can boot instantly. As this bytecode executes, V8 allocates a data structure called a Feedback Vector for each function. Inside this vector are Feedback Slots (managed by Inline Caches, or ICs). These slots act as recorders, capturing the exact types (or "shapes") of the variables passing through your code. Once a function runs frequently enough to cross an execution threshold, V8 marks it as "hot" and hands it to TurboFan. TurboFan reads those feedback slots, assumes the types will remain identical in the future, and compiles a highly streamlined, native machine code version of that function. What happens when you pass differe

Modern web applications demand responsive, non-blocking user flows. This tutorial demonstrates a production-grade implementation for handling form submissions without full-page reloads using the native Fetch API and clean async/await syntax. The Complete Source Code Create a file named app.js and drop in the following event-driven architecture: JavaScript document.getElementById('registrationForm').addEventListener('submit', async (event) => { event.preventDefault(); // 1. Stop full page reload const form = event.target; const formData = new FormData(form); const submitBtn = form.querySelector('button[type="submit"]'); const responseMessage = document.getElementById('responseMessage'); // 2. UI Feedback: Disable button during network request submitBtn.disabled = true; submitBtn.textContent = 'Processing...'; responseMessage.textContent = ''; try { // 3. Asynchronous Fetch Request const response = await fetch(form.action, { method: 'POST', body: formData, headers: { 'X-Requested-With': 'XMLHttpRequest' } }); // 4. Status Code Validation if (!response.ok) { throw new Error(`HTTP error! Status: ${response.status}`); } const result = await response.json(); // 5. Dynamic UI State Handling if (result.success) { responseMessage.style.color = '#155724'; responseMessage.textContent = result.message; form.reset(); // Clear form on success } else { responseMessage.style.color = '#721c24'; responseMessage.textContent = result.error || 'Submission failed.'; } } catch (error) { // 6. Global Error Catching responseMessage.style.color = '#721c24'; responseMessage.textContent = 'A network error occurred. Please try again.'; console.error('Submission tracking error:', error); } finally { // 7. Reset UI State Guaranteed submitBtn.disabled = false; submitBtn.textContent = 'Submit Data Securely'; } }); The Problem with Synchronous Form Lifecycles Traditional submissions trigger a synchronous navigation cycle: the browser constructs a payload, issues a full HTTP request, and replaces the

Netflix revealed that it paid $587 million in cash for InterPositive, a startup co-founded by Ben Affleck.

This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry . Project Overview Formbricks is an open-source survey and experience management platform built with Next.js, TypeScript, React, and Tailwind CSS. It lets teams create and deploy surveys across websites, apps, and email. Developers can self-host it or use the cloud version. With over 12,000 GitHub stars and hundreds of contributors, it is one of the most actively maintained open source alternatives to Qualtrics. Bug Fix or Performance Improvement Formbricks supports custom regex validation rules on survey questions. A survey creator can set a pattern that user responses must match before they are accepted. The problem was in how that pattern was stored. The validation schema for regex pattern rules only checked that the input was a non-empty string: // Before the fix export const ZValidationRuleParamsPattern = z . object ({ pattern : z . string (). min ( 1 ), flags : z . string (). optional (), }); z.string().min(1) means "give me any string with at least one character." It does not verify that the string is actually a valid regular expression. So a survey creator could type [invalid as their pattern, the schema would accept it, it would be saved to the database, and then when a real user submitted a response, the system would try to run new RegExp("[invalid") , JavaScript would throw a SyntaxError , and the survey would crash silently at runtime. The bug never surfaced during setup. It only appeared when a real user was trying to submit a real response. Code Pull Request: github.com/Tobore005/formbricks/pull/1 Here is the fix: const isValidRegexPattern = ( pattern : string ): boolean => { try { new RegExp ( pattern ); return true ; } catch { return false ; } }; const isValidRegexFlags = ( flags : string | undefined ): boolean => { if ( flags === undefined ) return true ; try { new RegExp ( "" , flags ); return true ; } catch { return false ; } }; export const ZValidationRuleParamsPa

Tres semanas después de publicar un sitemap dinámico que orgullosamente listaba cada perfil de miembro, Google Search Console me dijo que esos perfiles eran un error. No con un error de plano, sino con dos veredictos más callados: Soft 404 y Duplicada sin canónica seleccionada por el usuario . Esta es la historia de leer ese reporte, separar el ruido de la única señal real, y el arreglo, que fue quitar páginas del índice, no agregarlas. TL;DR El reporte "Por qué las páginas no se indexan" de Google es ~80% benigno por diseño. Aprende a triarlo o vas a perseguir fantasmas. Mis perfiles de miembros salieron marcados como Soft 404 (uno) y Duplicada sin canónica seleccionada por el usuario (otro). La misma causa raíz: el perfil público anónimo es deliberadamente flaco, un esqueleto con el username, todo lo demás es PII oculta a quien no ha iniciado sesión. Flaco + casi idéntico entre usuarios se lee como "página vacía" y "clúster de duplicados". No puedes arreglar un Soft 404 enriqueciendo una página que por contrato no tienes permitido enriquecer. Así que el arreglo es noindex,follow en la captura, más sacar las URLs del sitemap (un sitemap que lista una URL noindex es una autocontradicción que Google va a señalar). El modelo mental en una línea: una página que a propósito no tiene contenido para los visitantes anónimos no tiene nada que hacer en un índice construido para visitantes anónimos. El reporte que lo empezó todo El reporte de cobertura del índice, ordenado por número de páginas: Razón Fuente Páginas Descubierta, actualmente sin indexar Sistemas de Google 55 Duplicada sin canónica seleccionada por el usuario Sitio web 9 Página alternativa con etiqueta canónica correcta Sitio web 3 Página con redirección Sitio web 2 Rastreada, actualmente sin indexar Sistemas de Google 2 Soft 404 Sitio web 1 Excluida por etiqueta 'noindex' Sitio web 1 No encontrada (404) Sitio web 1 Bloqueada por acceso prohibido (403) Sitio web 1 Noventa y tantas URLs "sin indexar". El instint

Are AI Agent Engrams Open Source or Proprietary? The short answer: both, and the split matters. The major agent-memory engines — Mem0, Letta, Cognee, Graphiti, LangMem, and PLUR — are all Apache-2.0 or MIT licensed on GitHub. But "open source" and "open format" are not the same thing. A project can ship under Apache-2.0 while storing your memories in opaque vector blobs you cannot read, edit, or export. The real question is not whether the software is open — it usually is — but whether your memories are. This distinction separates the field into three tiers: fully open (software + format + data you own), open-core (software is open, but the hosted memory is not portable), and fully proprietary (memory baked into a model provider's infrastructure, no export at all). Why this question exists AI agents face a brutal constraint: they forget. Every new session starts blank. Every context window overflows. The fix is persistent memory — but persistent where, and in whose format? The term "engram" comes from neuroscience. Richard Semon coined it in 1904 for the physical trace a memory leaves in biological tissue (Semon, 1904; cited in Wikipedia, "Engram (neuropsychology)"). Applied to AI agents, an engram is one discrete thing an agent has learned — a correction, a preference, a procedure — stored so it survives across sessions. The question "open source or proprietary?" asks two things at once: Is the engine that stores and retrieves engrams open source? Is the format those engrams are stored in open — readable, editable, portable? Conflating the two is how vendors end up with open-source repos and locked-in data. Tier 1: Fully open — software, format, and data These projects ship under permissive licenses (Apache-2.0 or MIT) AND store memory in a format you can inspect, edit, and export. Project License Stars (Jul 2026) Memory format Data ownership PLUR Apache-2.0 ~215 Human-readable YAML engrams Yours — plain files PLUR stores each engram as a plain-text YAML entry — an