标签：#EV

Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40% How we moved from "semantic search + hope" to a measured, tunable retrieval pipeline with 95% recall@10 The RAG Reality Check Everyone ships RAG the same way: chunk by 512 tokens, embed with text-embedding-3-small , top-k=5, stuff into context. It works for demos. Then you hit production: Legal contracts: 512 tokens splits clauses mid-sentence API docs: 1000-token chunks drown signal in noise Customer tickets: Conversational context needs overlap, not fixed windows Latency: 500ms embedding + 200ms vector search + 300ms LLM = 1s+ per query We rebuilt our retrieval layer from first principles. Here's what actually moves metrics. Chunking: One Size Fits None # rag/chunking.py from abc import ABC , abstractmethod from dataclasses import dataclass @dataclass class Chunk : text : str metadata : dict token_count : int chunk_id : str class ChunkingStrategy ( ABC ): @abstractmethod def chunk ( self , document : str , metadata : dict ) -> list [ Chunk ]: ... class FixedTokenChunker ( ChunkingStrategy ): """ Baseline. Good for homogeneous content. """ def __init__ ( self , chunk_size = 512 , overlap = 50 ): self . chunk_size = chunk_size self . overlap = overlap class RecursiveChunker ( ChunkingStrategy ): """ Respects structure: markdown headers, code blocks, paragraphs. """ def __init__ ( self , separators = [ " \n ## " , " \n ### " , " \n\n " , " \n " , " " ], chunk_size = 512 ): self . separators = separators self . chunk_size = chunk_size class SemanticChunker ( ChunkingStrategy ): """ Uses embedding similarity to find natural boundaries. """ def __init__ ( self , model = " text-embedding-3-small " , threshold = 0.7 ): self . model = model self . threshold = threshold class AgenticChunker ( ChunkingStrategy ): """ LLM decides boundaries. Expensive but highest quality for complex docs. """ def __init__ ( self , model = " gpt-4o-mini " ): self . model = model Our production config by

Building Production-Grade LLM Evaluation Pipelines: From Vibes to Metrics How we replaced "looks good to me" with automated evaluation catching 92% of hallucinations before deployment The Problem: Why "Vibe Checks" Fail in Production Three months ago, our team shipped a RAG-based customer support assistant. It worked great in testing — we'd ask it questions, read the answers, and say "yeah, that looks right." Then it hit production. A customer asked about their billing cycle. The assistant confidently cited a policy that didn't exist. Another asked about API rate limits and got numbers from a competitor's documentation. By the time we caught it, 500+ users had seen hallucinated responses. The post-mortem was brutal: we had zero automated evaluation . Our test process was literally "ask 5 questions, read answers, thumbs up." What Production Evaluation Actually Needs Academic benchmarks (MMLU, HellaSwag) don't tell you if your system works for your use case. Production evaluation needs: Domain-specific judges — Your criteria, not generic "helpfulness" Speed — Evaluation must run in CI/CD, not overnight Regression detection — Know immediately when a prompt change breaks things CI/CD integration — Block merges that degrade quality Golden dataset management — Versioned, stratified, growing test cases Architecture: The Evaluation Pipeline ┌─────────────┐ ┌──────────────┐ ┌────────────────────┐ ┌──────────────┐ │ Test Cases │────▶│ LLM Under │────▶│ Judge Ensemble │────▶│ Metrics & │ │ (Golden Set)│ │ Test │ │ - Faithfulness │ │ Regression │ └─────────────┘ └──────────────┘ │ - Instruction F. │ │ Detection │ │ - JSON Schema │ └──────┬───────┘ │ - Custom LLM │ ▼ └────────────────────┘ ┌──────────────┐ │ Dashboard/ │ │ PR Comments │ └──────────────┘ Core Abstractions # eval/base.py @dataclass ( frozen = True ) class TestCase : id : str input : dict [ str , Any ] expected : dict [ str , Any ] | None = None tags : list [ str ] = field ( default_factory = list ) # ["edge-case",

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry . 🎭 The Recurring Villain If you've built WordPress sites long enough, you know this bug. It doesn't show up as an error in your console. It doesn't throw a fatal exception. It just quietly sits there, page after page, client after client — and it's called video bloat . Nine years into web development, I lost count of how many times a client sent me a site with a message like "it's just... slow." And nine times out of ten, when I opened DevTools and checked the waterfall, the story was the same: three, four, sometimes ten embedded YouTube videos, each one dragging in its own iframe, its own set of scripts, its own chunk of megabytes — all loading the second the page rendered, whether the visitor scrolled past them or not. One video embed alone can pull in 500KB–1MB+ before a user even presses play. Multiply that by a landing page stacked with testimonials, product demos, and a hero video, and you've got a page that fights your Core Web Vitals before it's even finished loading. 🔁 The Manual Fix (Every. Single. Time.) For years, my solution was the same tedious ritual: Find every video embed on the page. Replace the iframe with a static thumbnail image. Write a bit of custom JavaScript to swap the thumbnail for the real embed on click or scroll. Repeat it for YouTube. Repeat it again for Vimeo, because Vimeo's oEmbed API works differently. Repeat it again for self-hosted <video> tags, because now you're dealing with <source> tags instead of iframes. Test it across whatever page builder the client was using — Elementor, Divi, WPBakery, or straight-up Gutenberg blocks — because every one of them nests the video markup slightly differently. It worked. But it was never reusable. Every project meant writing the lazy-load script again, half from memory, half copy-pasted from my own old projects, tweaked just enough to fit the new theme's markup. It was the same bug, wearing a different site's c

Back in July my scheduled DEV.to publishing run failed at the very first step — the quota check couldn't reach dev.to:443 at all. Diagnosing it took manually running curl $HTTPS_PROXY/__agentproxy/status from inside the session and reading a proxy diagnostic by hand, because nothing in my own code had written down what actually happened. Not which host, not which of my two HTTP helper functions made the call, not a timestamp, nothing. The failure was real and the fix (get dev.to added to the environment's egress allowlist) was correct, but I found it by treating my own server as a black box and probing it from outside, which is exactly backwards for code I wrote myself. eight tools, one shared blind spot server.py is an 8-tool FastMCP server: three GitHub tools, four DEV.to tools, one that shells out to claude -p . Every HTTP-calling tool routes through one of two helpers: def _gh ( path , method = " GET " , data = None ): req = urllib . request . Request ( f " https://api.github.com { path } " , method = method ) req . add_header ( " Authorization " , f " token { os . environ [ ' GITHUB_TOKEN ' ] } " ) req . add_header ( " Accept " , " application/vnd.github.v3+json " ) if data : req . add_header ( " Content-Type " , " application/json " ) req . data = json . dumps ( data ). encode () with urllib . request . urlopen ( req ) as r : return json . loads ( r . read ()) def _dev ( path , method = " GET " , data = None ): req = urllib . request . Request ( f " https://dev.to/api { path } " , method = method ) req . add_header ( " api-key " , os . environ [ " DEV_TO_API " ]) req . add_header ( " Content-Type " , " application/json " ) req . add_header ( " User-Agent " , " developer-presence-mcp/1.0 " ) if data : req . data = json . dumps ( data ). encode () with urllib . request . urlopen ( req ) as r : return json . loads ( r . read ()) Neither one logs anything. When urlopen raises, the caller — whichever @mcp.tool() function invoked it — sees a bare urllib.error.HTTPEr

Perplexity added "Skills" to its Agent API, letting developers compose built-in and custom skills for more complex agent outputs. More skills mean more actions, and more actions mean more ways to produce an irreversible change. Before you add a fifth skill to your agent, make sure the first four have an undo path. The undo problem An agent skill that writes, sends, publishes, or deploys is an irreversible action if there is no rollback. The more skills an agent has, the more irreversible actions it can take in a single session. A chain of skills (research → draft → format → publish) can complete before a human notices the first step was wrong. The undo contract Every agent skill should declare: { "skill_name" : "publish_to_blog" , "action_type" : "write" , "reversible" : true , "undo_method" : "set_published_false" , "undo_timeout_seconds" : 3600 , "undo_side_effects" : [ "SEO index will retain the URL for up to 24h" ] } If reversible is false , the skill should require explicit human approval before execution. A skill registry with undo support # skill_registry.py """ Registers agent skills with undo metadata. Blocks irreversible skills from running without explicit approval. """ from dataclasses import dataclass from typing import Optional , Callable import json @dataclass class Skill : name : str action_type : str # "read", "write", "send", "deploy" reversible : bool undo_fn : Optional [ Callable ] = None undo_timeout_seconds : int = 3600 side_effects : str = "" requires_approval : bool = False class SkillRegistry : def __init__ ( self ): self . skills = {} self . execution_log = [] def register ( self , skill : Skill ): # Irreversible write/send/deploy skills require approval if not skill . reversible and skill . action_type in ( " write " , " send " , " deploy " ): skill . requires_approval = True self . skills [ skill . name ] = skill def execute ( self , skill_name : str , inputs : dict , approved : bool = False ): skill = self . skills . get ( skill_name ) i

Visual Studio 2026's July update added organization-level custom agents for GitHub Copilot. An organization owner can now define a custom agent that every repository in the org automatically detects and offers in the agent selector. This is powerful and introduces a rollout risk: a single agent definition affects every developer in the organization simultaneously. The rollout problem When an org-level agent is published, every developer working in any repo in that org sees it in their agent selector. If the agent definition contains an incorrect instruction, a broken tool reference, or an overly permissive scope, it affects all developers at once. There is no canary by default. The agent definition review checklist Before publishing an org-level custom agent, review its definition against this checklist. 1. Instruction review # agent-definition.yml (example structure) name : org-code-reviewer description : Reviews PRs for security and style instructions : | Review each file change. Flag: - SQL injection in string concatenation - Missing input validation on public endpoints - Hardcoded credentials Do not modify files. Only comment. tools : - read_file - search_code - post_comment scope : organization Check: [ ] Instructions are specific enough to be testable ("flag SQL injection in string concatenation" not "review for security") [ ] Instructions do not conflict with existing repository-level AGENTS.md files [ ] The agent does not claim capabilities it does not have (e.g., "run tests" when no tools entry supports it) 2. Tool surface audit # Extract all tool references from the agent definition rg -n 'tools:' agent-definition.yml -A 20 | grep '^\s*-' For each tool: What resource does it access? (filesystem, network, repository API) Is it read-only or read-write? Does it require elevated permissions beyond the developer's own role? A post_comment tool can create noise across every PR. A write_file tool can modify code in every repo. Audit accordingly. 3. Canary test Be

Max Korbacher explains why successful internal development platforms cannot be built on tech alone. He discusses the pitfalls of infrastructure-first thinking, the importance of a clear product mindset, and how to measure real value using DevEx and SPACE metrics. Learn how to align your team, manage tech debt, and foster a thriving community to ensure lasting platform adoption. By Max Körbächer

Several Chinese businesses associated with the ebike brand Aipas settled a lawsuit with Amazon and the safety certification company UL in July.

Moving from Windows to macOS often creates a small workflow problem for developers: where did my familiar SSH tools go? PuTTY has been a classic SSH and Telnet client for Windows for many years. Many developers know its interface and workflow, so after switching platforms, the first thought is usually finding a macOS version. The reality is that the macOS ecosystem works differently. There are several practical options depending on how you manage remote systems. Option 1: Use the built-in SSH client on macOS For many developers, the SSH client that ships with macOS is already enough. A typical workflow looks like this: Use the terminal for server access Manage hosts through ~/.ssh/config Store aliases, usernames, ports, and key settings in one place Automate common connection tasks with scripts For example: ssh production-server can replace a long connection command when your SSH configuration is organized properly. This approach works especially well for developers who prefer command-line workflows and manage Linux servers regularly. Option 2: Use a GUI SSH client Some developers prefer a graphical interface because they manage many machines, protocols, or connection profiles. A good GUI client can help with: Organizing dozens or hundreds of servers Saving authentication settings Switching between SSH, RDP, VNC, and other protocols Reducing repetitive configuration work The macOS App Store has many SSH clients available. The right choice depends on whether you only need SSH or you need a complete remote management workflow. Tools like DartShell are designed around this multi-protocol scenario, where developers may need SSH for Linux servers, RDP for Windows machines, and other remote access methods in one place. Choosing the right workflow The decision usually comes down to how you work: A few Linux servers: macOS Terminal + SSH config is usually enough. Many servers and different protocols: a GUI management tool can save time. Team environments: centralized connec

AWS gives you three ways to run LLM inference in production. I've deployed all three for clients and the decision always comes down to the same variables: volume, team size, and how much you value your weekends. Here's the short version. The three paths Bedrock — Fully managed, pay-per-token. You call an API, you get tokens back. No GPUs, no cold starts, no 3am pages about OOM pods. SageMaker Endpoints - Semi-managed. You bring your model (or a fine-tuned one), deploy it on dedicated instances, and handle autoscaling. Pay per hour whether you're serving requests or not. Self-hosted on EKS — Full control. vLLM or TGI on GPU spot instances with Karpenter. Cheapest per token at scale, most operational overhead. The cost crossover that matters This is the table I keep coming back to with every client: Volume Bedrock (Haiku) SageMaker (g5.xlarge) EKS (g5.xlarge spot) 1K req/day ~$36/mo ✓ ~$1,015/mo ~$674/mo 50K req/day ~$1,800/mo ~$1,015/mo ~$674/mo ✓ 500K req/day ~$18,000/mo ~$6,090/mo ~$2,022/mo ✓ The crossover point where self-hosting beats Bedrock: 10,000–20,000 requests/day . Below that, Bedrock wins on simplicity alone. Above it, you're leaving serious money on the table. The hidden cost nobody models upfront Teams prototype on Bedrock (smart move — it's the fastest path to production). But the cost curve isn't linear. At 10K requests/day it's cheap. At 50K it's "we need to talk to finance." At 500K it's a rearchitecture project. The mistake is not choosing Bedrock at low volume. The mistake is not planning the exit path before you need it. Quick decision framework You should pick... When... Bedrock No ML infra team, <50K req/day, need frontier models (Claude, Llama) SageMaker Fine-tuned models, predictable traffic, need dedicated VPC EKS self-hosted >100K req/day, open-source models, dedicated platform team What I actually recommend Use a hybrid. Most production systems I've deployed use: Bedrock for complex reasoning and customer-facing chat (low volume, high qua

I'm excited to share a project I've been building: Letras Personalizadas — a free Unicode text styling tool that helps you create creative copy-and-paste text for social media, gaming, and messaging apps. Here's how it works: • Type your text • Instantly preview dozens of Unicode text styles • Copy your favorite with one click • Use it on Instagram, WhatsApp, TikTok, Discord, Free Fire, and more Although the website is designed primarily for Spanish-speaking users targetting Brazil, it works with any standard Latin text. One thing I've learned while building this project is that these tools are about much more than "fancy fonts." The real challenge is creating a smooth user experience—fast previews, mobile-friendly design, reliable copy buttons, platform compatibility, useful categories, favorites, and making the styled text effortless to use across different apps. I'm continuously improving the interface, expanding the style categories, and refining the mobile experience. If you have a few minutes to try it out, I'd love to hear your feedback and suggestions. Every comment helps make the tool better.

Part 2 of the Building create-notils series. In my previous article , I explained why I stopped copy-pasting repositories and started building my own project scaffolding tool. However, one major architectural problem remained: I wanted create-notils to support both of these primary project structures. A standalone Next.js application: my-app/ ├── src/ ├── public/ ├── package.json └── components.json And a Turborepo monorepo: my-app/ ├── apps/ │ └── app/ ├── packages/ │ ├── ui/ │ └── config/ ├── turbo.json └── package.json At first glance, the obvious solution is to maintain two separate templates—one for standalone and one for monorepo. Problem solved, right? Except... it isn't. The Hidden Cost of Multiple Templates Every starter template starts out identical. Then one day, you fix a subtle bug in one template and forget to update the other. A week later, you upgrade Next.js in one repository before getting around to the second. A month later, you improve your UI package and find yourself manually copying files back and forth between folders. Eventually, the templates slowly drift apart. The true cost isn't creating templates; the cost is maintaining them forever. What Actually Changes? When I sat down and compared the two project layouts side by side, surprisingly little was different. The actual application code, UI components, theming, and utility functions were 100% identical. The only real differences were the structural project boundaries: Concern Monorepo Standalone UI Package packages/ui src/components/ui Utilities @notils/ui/lib/utils @/lib/utils Configuration Shared workspace package Local configuration Package Manifests Multiple ( package.json files) Single root manifest Workspace Tooling Present ( turbo.json , workspaces) Removed entirely Everything else was effectively the exact same code. That single observation changed the entire architecture of create-notils . A Different Approach: The Canonical Source of Truth Instead of maintaining two templates, I

Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40% How we moved from "semantic search + hope" to a measured, tunable retrieval pipeline with 95% recall@10 The RAG Reality Check Everyone ships RAG the same way: chunk by 512 tokens, embed with text-embedding-3-small , top-k=5, stuff into context. It works for demos. Then you hit production: Legal contracts: 512 tokens splits clauses mid-sentence API docs: 1000-token chunks drown signal in noise Customer tickets: Conversational context needs overlap, not fixed windows Latency: 500ms embedding + 200ms vector search + 300ms LLM = 1s+ per query We rebuilt our retrieval layer from first principles. Here's what actually moves metrics. Chunking: One Size Fits None # rag/chunking.py from abc import ABC , abstractmethod from dataclasses import dataclass @dataclass class Chunk : text : str metadata : dict token_count : int chunk_id : str class ChunkingStrategy ( ABC ): @abstractmethod def chunk ( self , document : str , metadata : dict ) -> list [ Chunk ]: ... class FixedTokenChunker ( ChunkingStrategy ): """ Baseline. Good for homogeneous content. """ def __init__ ( self , chunk_size = 512 , overlap = 50 ): self . chunk_size = chunk_size self . overlap = overlap class RecursiveChunker ( ChunkingStrategy ): """ Respects structure: markdown headers, code blocks, paragraphs. """ def __init__ ( self , separators = [ " \n ## " , " \n ### " , " \n\n " , " \n " , " " ], chunk_size = 512 ): self . separators = separators self . chunk_size = chunk_size class SemanticChunker ( ChunkingStrategy ): """ Uses embedding similarity to find natural boundaries. """ def __init__ ( self , model = " text-embedding-3-small " , threshold = 0.7 ): self . model = model self . threshold = threshold class AgenticChunker ( ChunkingStrategy ): """ LLM decides boundaries. Expensive but highest quality for complex docs. """ def __init__ ( self , model = " gpt-4o-mini " ): self . model = model Our production config by

Building Production-Grade LLM Evaluation Pipelines: From Vibes to Metrics How we replaced "looks good to me" with automated evaluation catching 92% of hallucinations before deployment The Problem: Why "Vibe Checks" Fail in Production Three months ago, our team shipped a RAG-based customer support assistant. It worked great in testing — we'd ask it questions, read the answers, and say "yeah, that looks right." Then it hit production. A customer asked about their billing cycle. The assistant confidently cited a policy that didn't exist. Another asked about API rate limits and got numbers from a competitor's documentation. By the time we caught it, 500+ users had seen hallucinated responses. The post-mortem was brutal: we had zero automated evaluation . Our test process was literally "ask 5 questions, read answers, thumbs up." What Production Evaluation Actually Needs Academic benchmarks (MMLU, HellaSwag) don't tell you if your system works for your use case. Production evaluation needs: Domain-specific judges — Your criteria, not generic "helpfulness" Speed — Evaluation must run in CI/CD, not overnight Regression detection — Know immediately when a prompt change breaks things CI/CD integration — Block merges that degrade quality Golden dataset management — Versioned, stratified, growing test cases Architecture: The Evaluation Pipeline ┌─────────────┐ ┌──────────────┐ ┌────────────────────┐ ┌──────────────┐ │ Test Cases │────▶│ LLM Under │────▶│ Judge Ensemble │────▶│ Metrics & │ │ (Golden Set)│ │ Test │ │ - Faithfulness │ │ Regression │ └─────────────┘ └──────────────┘ │ - Instruction F. │ │ Detection │ │ - JSON Schema │ └──────┬───────┘ │ - Custom LLM │ ▼ └────────────────────┘ ┌──────────────┐ │ Dashboard/ │ │ PR Comments │ └──────────────┘ Core Abstractions # eval/base.py @dataclass ( frozen = True ) class TestCase : id : str input : dict [ str , Any ] expected : dict [ str , Any ] | None = None tags : list [ str ] = field ( default_factory = list ) # ["edge-case",

For a long time I ran product extraction off a database of custom selector configs. Hundreds of retailers, each with its own set of CSS selectors mapped field by field: price here, title there, image over there. It worked. It was also a treadmill. Every retailer that redesigned its frontend silently broke its config, and the maintenance tax grew with every retailer I added. Hundreds of configs is hundreds of things that can rot without telling you, usually right before someone downstream asks why a brand's prices went null. At some point I stopped feeding it. The scraping platform I ran at my last engagement fed a 20M+ product catalogue at millions of pages a day, and the thing that made that survivable wasn't better per-site config. It was leaning on almost no per-site config at all. The pattern Instead of one careful parser per site, run several cheap generic extractors on every page, in parallel: JSON-LD. A huge share of e-commerce pages ship a schema.org/Product block because Google rewards it. It has name, price, currency, availability, images. It's the closest thing to a public API hiding in the HTML. OpenGraph tags. og:title , og:image , product:price:amount . Lower quality than JSON-LD, but present on sites too lazy for structured data, because everyone wants pretty link previews. Microdata / RDFa. Older sites, still surprisingly common outside the US. Embedded state. __NEXT_DATA__ , window.__INITIAL_STATE__ and friends. A JSON blob the site's own frontend hydrates from. Heuristics. A price-shaped string near a currency symbol inside the main content region. The largest above-the-fold image. The h1 . Dumb, and dumb works more often than you'd think. None of these is reliable alone. OG price tags go stale, JSON-LD sometimes describes the wrong variant, heuristics grab the crossed-out "was" price. The trick is you don't pick an extractor. You pick per field, and you let agreement between sources carry the decision. Confidence merge Each extractor emits candida

Running a production database on a minimal two-core PostgreSQL instance presents unique engineering challenges. In resource-constrained environments, default configurations quickly lead to CPU exhaustion, memory thrashing, and high latency. To maintain stable performance, you must aggressively manage resource allocation and optimize query execution plans. The first critical area is connection management. PostgreSQL allocates a separate operating system process for each connection. On a two-core machine, allowing hundreds of concurrent connections will cause severe context switching overhead, degrading performance significantly. You should restrict maximum connections to a low double-digit number and implement a connection pooler like PgBouncer. A pooler ensures that incoming traffic queuing happens outside the database engine, allowing the CPU to focus on executing active queries rather than managing process states. Memory allocation parameters require precise tuning on limited hardware. The shared buffers parameter, which dictates how much memory PostgreSQL uses for caching data, should typically be set to twenty-five percent of total system RAM. However, work memory is where many developers run into trouble. The work memory setting determines the amount of memory used by internal sort operations and hash tables before writing to temporary disk files. Since this memory is allocated per query operation, a complex query with multiple joins can consume many times the configured value. On a two-core instance with limited RAM, setting this value too high can trigger the out-of-memory killer, while setting it too low forces slow disk-based sorting. You must analyze your heaviest queries and find a balanced value that prevents disk thrashing without exhausting system memory. Query optimization becomes a daily necessity when compute resources are scarce. You must regularly inspect query execution plans using the explain analyze command to identify sequential scans on large

We’ve all been there: you are busy coding, and suddenly you get a scary error saying your code conflicts with a teammate's work. Git won't let you pull their changes because it's afraid of overwriting your unsaved files. Instead of panicking, I used git stash. This command acts like a temporary clipboard—it safely hides your current work away so your workspace becomes completely clean. With a clean screen, I was able to safely update the project, fix the conflicting lines of code by hand, and make sure everything compiled perfectly. Once the team's updates were safely pushed, I ran git stash pop to bring my hidden work right back. Git wasn't trying to break my project; it was just protecting my code. Next time you get stuck, remember the golden rule: Stash your work, fix the conflict, and pop it back!

Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40% How we moved from "semantic search + hope" to a measured, tunable retrieval pipeline with 95% recall@10 The RAG Reality Check Everyone ships RAG the same way: chunk by 512 tokens, embed with text-embedding-3-small , top-k=5, stuff into context. It works for demos. Then you hit production: Legal contracts: 512 tokens splits clauses mid-sentence API docs: 1000-token chunks drown signal in noise Customer tickets: Conversational context needs overlap, not fixed windows Latency: 500ms embedding + 200ms vector search + 300ms LLM = 1s+ per query We rebuilt our retrieval layer from first principles. Here's what actually moves metrics. Chunking: One Size Fits None # rag/chunking.py from abc import ABC , abstractmethod from dataclasses import dataclass @dataclass class Chunk : text : str metadata : dict token_count : int chunk_id : str class ChunkingStrategy ( ABC ): @abstractmethod def chunk ( self , document : str , metadata : dict ) -> list [ Chunk ]: ... class FixedTokenChunker ( ChunkingStrategy ): """ Baseline. Good for homogeneous content. """ def __init__ ( self , chunk_size = 512 , overlap = 50 ): self . chunk_size = chunk_size self . overlap = overlap class RecursiveChunker ( ChunkingStrategy ): """ Respects structure: markdown headers, code blocks, paragraphs. """ def __init__ ( self , separators = [ " \n ## " , " \n ### " , " \n\n " , " \n " , " " ], chunk_size = 512 ): self . separators = separators self . chunk_size = chunk_size class SemanticChunker ( ChunkingStrategy ): """ Uses embedding similarity to find natural boundaries. """ def __init__ ( self , model = " text-embedding-3-small " , threshold = 0.7 ): self . model = model self . threshold = threshold class AgenticChunker ( ChunkingStrategy ): """ LLM decides boundaries. Expensive but highest quality for complex docs. """ def __init__ ( self , model = " gpt-4o-mini " ): self . model = model Our production config by

My upload CLI looked restart-safe until I tested two failures together: the process crashed at 63%, and the signed resume URL expired before restart. The checkpoint needs identity, not credentials: { "file" : "release.tar.gz" , "fingerprint" : "sha256:..." , "uploadId" : "up_123" , "localOffset" : 66060288 , "updatedAt" : "2026-07-19T12:00:00Z" } Do not persist the signed URL. On restart, exchange the durable upload ID for fresh authorization, query the server offset, then reconcile: const remote = await headUpload ( freshUrl ); if ( remote > file . size ) throw new Error ( " invalid remote offset " ); if ( remote !== checkpoint . localOffset ) { await saveCheckpoint ({ ... checkpoint , localOffset : remote }); } await sendFrom ( file , remote , freshUrl ); The server offset wins because a crash can occur after bytes are accepted but before the local checkpoint is renamed. Save checkpoints through write-to-temp plus atomic rename so a partial JSON file cannot destroy recovery. My fixture matrix includes: Failure Expected behavior crash after remote commit rewind/advance to remote offset expired URL refresh without creating a second upload changed local file stop on fingerprint mismatch missing remote upload ask before starting over checkpoint write interrupted retain previous valid checkpoint The tus resumable upload protocol specifies offset discovery and conflict handling that are useful even if the service uses a smaller custom protocol. The key idea is explicit reconciliation, not assuming client memory is authoritative. I also shipped upload status , upload forget , and an exportable checkpoint directory. Recovery is a user-facing feature; if users cannot inspect or remove state, “resumable” becomes hidden lock-in.

A SQLite service can keep answering health checks while its WAL grows without a successful checkpoint. One forgotten read transaction is enough to reproduce the risk. Run a controlled drill: Enable WAL mode and create a small table. Open connection A, begin a read transaction, and keep it open. From connection B, commit batches of writes for 60 seconds. Sample WAL bytes and checkpoint results every second. Release A and observe recovery. PRAGMA journal_mode = WAL ; PRAGMA wal_checkpoint ( PASSIVE ); Record the three checkpoint counters plus duration, WAL file size, oldest transaction age, write latency, and free disk bytes. A green SELECT 1 says nothing about checkpoint progress. My fault-injection gate looks like this: Given: one reader holds its transaction When: 10,000 writes commit Then: writes remain bounded And: WAL growth triggers an alert before the disk budget When: the reader closes Then: checkpoint progress resumes and WAL size converges Do not start with an automatic TRUNCATE loop. A busy result is evidence of contention; aggressive checkpoints can add latency without fixing the reader lifecycle. First identify long transactions, ensure result iterators close, and define a maximum transaction age. SQLite’s WAL documentation explains that a checkpoint must stop when it reaches pages beyond an active reader’s end mark. This is why the drill needs concurrency rather than a synthetic file-growth assertion. Operational thresholds should be tied to a disk budget: alert when projected WAL growth reaches the remaining safe window, not at an arbitrary file size. The runbook should name how to find the oldest reader, how to shed writes, and when a restart is safer than waiting. A health check is useful only when it measures the subsystem that is failing.