标签：#Performance

The web has changed . Applications are no longer simple HTTP servers. Today we build real-time dashboards, AI-powered services, multiplayer systems, APIs, microservices, and applications that need to handle thousands of connections with minimal overhead. But our frameworks are still mostly designed for yesterday's problems. So we asked a simple question: What if a * Go * framework was built from the ground up for modern workloads? Meet Breeze . A high-performance Go framework designed around one idea: Performance should not come at the cost of developer experience. Why Breeze ? Go already gives us incredible performance. But the framework layer often becomes the bottleneck. Too much abstraction. Too many allocations. Too much hidden complexity. Breeze takes a different approach: ⚡ High-performance networking powered by gnet 🔥 Real-time WebSocket architecture built in 🧩 Modular middleware system 📚 Automatic Swagger/OpenAPI generation 🎨 Built-in SPA template engine 🚀 Optimized worker pool architecture 🗄️ BreezeORM for efficient database operations Everything you need to build production-grade applications — without assembling dozens of unrelated tools. The Future Is Real-Time Modern applications are moving toward instant experiences: Live collaboration Trading platforms AI assistants Gaming backends Monitoring systems Real-time analytics Breeze is designed for this world. Instead of adding real-time capabilities later, Breeze treats them as a first-class citizen. Less Glue Code. More Building. A common problem in backend development: You start with a simple API... Then suddenly you need: Authentication Documentation WebSockets Background workers Database optimization Frontend integration Your stack becomes a collection of disconnected pieces. Breeze tries to bring these pieces together into one coherent ecosystem. Built With Go Philosophy Go was created around simplicity, performance, and reliability. Breeze follows the same principles: Simple APIs. Predictable behavi

Videos are one of the heaviest assets you can add to a web page. Loading videos too early can significantly impact your application's performance. The good news is that modern browsers are starting to support lazy loading for video elements , allowing you to defer loading until users are likely to watch them. However, there's one important thing to know: 👉 This feature is not yet part of the Baseline web platform , so browser support is still limited. At the time of writing, lazy loading for <video> elements is supported in Chromium-based browsers such as Google Chrome , Microsoft Edge , and Opera , while browsers like Firefox and Safari do not yet support it natively. In this article, we'll explore: What lazy loading videos is Why it's important for web performance How to implement it Browser support considerations Best practices for optimizing video loading Let's dive in. 🤔 What Is Lazy Loading for Videos? Lazy loading means delaying the loading of a resource until it's actually needed. Instead of downloading every video immediately during page load, the browser waits until the video is close to entering the viewport. This helps reduce: initial network requests bandwidth usage page load time memory consumption Especially on pages with multiple videos, the difference can be significant. 🟢 What Problem Does It Solve? Imagine an e-commerce page with several product videos. Without lazy loading: every video starts downloading immediately bandwidth is consumed even for videos users never watch page rendering may become slower This negatively impacts; Largest Contentful Paint (LCP), Time to Interactive (TTI), and overall user experience. Most visitors won't watch every video on the page. So why load them all? Lazy loading ensures videos are fetched only when they're actually needed. 🟢 How to Lazy Load a Video The easiest approach is using the loading="lazy" attribute. Example: <video controls loading= "lazy" poster= "/preview.jpg" > <source src= "/video.mp4" type= "vide

Headline: React Compiler — formerly React Forget — shipped stable with React 19 and automatically memoizes components, hooks, and callbacks by analyzing data flow at build time. No dependency arrays to write; the compiler infers them. Here is what it handles, when it opts out, and whether you should delete your useMemo calls. Key takeaways React Compiler inserts useMemo , useCallback , and React.memo automatically at build time — no dependency arrays to maintain. Enable it in Next.js 15/16 with experimental.reactCompiler: true in next.config.ts . The compiler is conservative: if it cannot prove memoization is safe, it emits the component unchanged. "use no memo" is the escape hatch for functions the compiler should not touch. Run npx react-compiler-healthcheck@latest before enabling to see coverage and violations. What does React Compiler actually do? React Compiler transforms component and hook code at build time to insert memoization automatically. Instead of useMemo(() => expensiveCalc(a, b), [a, b]) , the compiler analyzes data flow, determines which values are stable across renders, and emits equivalent memoized code. The compiled output uses React's memo infrastructure at runtime. The compiler is babel-plugin-react-compiler — it works with any Babel-based build pipeline. How do I enable it in Next.js? // next.config.ts const nextConfig = { experimental : { reactCompiler : true , }, }; export default nextConfig ; Before enabling, run the healthcheck: npx react-compiler-healthcheck@latest The healthcheck reports optimizable component count, files with violations, and blocking patterns. Fix violations first for more coverage on day one. What does the compiler memoize? Components — equivalent to React.memo ; re-renders only when props change. Values — equivalent to useMemo ; computed results, derived arrays, objects. Callbacks — equivalent to useCallback : event handlers, functions passed as props. Dependencies are inferred from escape analysis — n

Headline: Partial Prerendering (PPR) in Next.js serves a static HTML shell from the CDN edge instantly, then streams Suspense-wrapped dynamic children from the origin in the same HTTP response. No full-page ISR staleness, no full-page origin latency. I shipped it on two production routes — here is the model. Key takeaways PPR serves a static HTML shell from the CDN edge , then streams dynamic Suspense children from the origin in the same response. The static shell is built at build time — outside <Suspense> renders statically; inside renders dynamically per request. PPR replaces the ISR vs. dynamic tradeoff for pages that are mostly static with isolated personalized sections. No changes to Server Components or Suspense — just experimental.ppr: 'incremental' in config and export const experimental_ppr = true per route. PPR and use cache are complementary : CDN delivery for the shell, origin memoization for dynamic islands. What does PPR actually do? PPR splits a page into two rendering phases within the same HTTP response. At build time, Next.js freezes everything that does not read dynamic request data into a static HTML shell on the CDN edge. At request time, the CDN delivers the shell at edge latency while the origin streams each <Suspense> boundary's content into the same response. On a product page: navigation, title, and description arrive at CDN speed. The in-stock badge and personalized recommendations stream from the origin a fraction of a second later. The user sees a nearly-complete page immediately. How is PPR different from ISR and streaming Suspense? Strategy First byte Dynamic freshness Staleness ISR (revalidate: N) CDN edge Whole page up to N seconds stale Full page Dynamic rendering Origin 100% fresh; waits for slowest query None Streaming Suspense (no PPR) Origin Fresh; TTFB includes origin latency None PPR CDN edge Dynamic islands 100% fresh Static shell only How do I enable PPR? // next.config.ts export default { experimental : { ppr : ' inc

Local LLM inference has an expensive habit: It recomputes prefixes it has already seen. A system prompt. A reused RAG document. A few-shot block. A long static context. If the prefix is identical, why pay the prefill cost again? That's the problem I explored with EdgeSync-LLM. The idea The mechanism is simple: Prompt = shared prefix + new suffix On the first request, EdgeSync prefills the prefix and captures its KV cache state. On the next request sharing that exact prefix, it restores the state and decodes only the new suffix. No llama.cpp fork. No patch. The current validated path uses the public: llama_state_seq_get_data and llama_state_seq_set_data APIs. Measured on a real Android ARM64 phone Model: Qwen2.5-0.5B-Instruct Q4_K_M Shared prefix: 123 tokens 40 requests. 4 threads. Release build. Path Mean TTFT p50 p95 Cold 4828 ms 4752 ms 5297 ms KV state reuse 486 ms 476 ms 569 ms 9.9× lower TTFT on cache hits. The warm path was approximately: 363 ms to decode the 10-token suffix 123 ms to restore the state blob Fragment size: 1.64 MB I also measured the same mechanism on x86-64. Cold mean TTFT: 1395 ms Warm mean TTFT: 185 ms That's 7.5× on cache hits. But I almost published a fake 8.8× speedup This was the most important part of the project. My first implementation directly copied raw K/V tensors. It was fast. Very fast. The benchmark reported an 8.8× speedup. There was one problem. It was wrong. llama.cpp tracks more than the K/V tensor values. Cache cells also have position and sequence metadata used to construct the attention mask. Copying tensor values without restoring that bookkeeping produced an inert fragment. The model skipped prefix computation... ...but attention could not actually see the restored prefix. 14 of 24 cache hits reproduced, token for token, the output of a generation with no prefix at all. The “speedup” was dropped context. So I discarded it. Timing is not enough A broken cache can be fast. That's why EdgeSync now runs two correctness chec

The incident that taught me to respect pinning looked like nothing. A service freshly migrated to virtual threads, a load test that plateaued at about 420 requests per second no matter how much traffic we threw at it, CPU sitting at 9%, zero errors, zero warnings, nothing in the logs. The machine had 8 cores, and the one downstream HTTP call in the hot path took about 19 ms. Do the arithmetic: 8 × (1000 / 19) ≈ 421. The service that was supposed to scale to millions of virtual threads was serving exactly one request per CPU core. Loom had quietly handed us back a bounded thread pool, and the code looked perfectly innocent. That failure mode has a name — pinning — and this is the field guide I wish I'd had that night: what it is, the two (and only two) things that cause it, what JDK 24 changed, and how to catch it before your throughput graph does. What pinning actually is A virtual thread doesn't own an OS thread. It runs on a small pool of platform threads called carrier threads — concretely, the workers of a dedicated ForkJoinPool living in a thread group named CarrierThreads , with default parallelism equal to Runtime.availableProcessors() . When a virtual thread blocks — on I/O, a lock, a queue — it normally unmounts : it saves its stack, steps off the carrier, and frees that carrier to run another virtual thread. That unmount is the entire trick that lets a handful of OS threads serve millions of virtual ones. Pinning is when the unmount can't happen. The virtual thread blocks but stays mounted, and its carrier sits there doing nothing useful for the whole duration. One pinned carrier is a rounding error. But the default carrier pool is only as big as your core count, so if a hot path pins routinely, you pin every carrier at once — and then no virtual thread anywhere makes progress. That's not a slowdown; it's scheduler starvation, and from the outside it looks a lot like a deadlock. You can raise the ceiling with -Djdk.virtualThreadScheduler.parallelism=N , bu

Now it's an arms race between OEMs locking down chips and tuners trying to crack them.

One common issue on WordPress websites with a large number of posts is that the Rank Math XML Sitemap can become slow to load. This happens because the sitemap is generated dynamically every time a visitor or search engine bot requests it. A simple solution is to use a static sitemap cache , allowing the web server to serve pre-generated XML files directly without executing PHP for every request. This significantly reduces server load and improves crawling performance. Benefits of Using a Static Sitemap Using a static sitemap cache provides several advantages: Faster sitemap loading times. Lower CPU and PHP worker usage. Improved crawling efficiency for Google and other search engines. Ideal for websites with thousands or even millions of URLs. Reduced server load when search engine bots frequently request sitemap files. 1. Setting RankMath Sitemap Cache The first step is to enable static sitemap generation using the Rank Math Sitemap Tweak plugin. The plugin automatically creates static copies of your XML sitemaps and stores them in the following directory: /wp-content/uploads/rank-math/ Instead of generating the sitemap dynamically through WordPress, your web server can serve these static files directly. 2. Configure Apache (.htaccess) If your website is running on Apache , add the following rules to your .htaccess file. # ========================== # XML cache # ========================== RewriteCond %{REQUEST_METHOD} GET RewriteCond %{QUERY_STRING} ^$ RewriteCond %{HTTP:Cookie} !wordpress_logged_in RewriteCond %{DOCUMENT_ROOT}/wp-content/uploads/rank-math/%{HTTP_HOST}%{REQUEST_URI} -f RewriteRule ^(.*)$ /wp-content/uploads/rank-math/%{HTTP_HOST}/$1 [L] These rules check whether a cached sitemap file exists. If it does, Apache serves the static file immediately without loading WordPress. 3. Configure Nginx If your server is using Nginx , add the following configuration inside your server block. # # Static cache # location / { try_files \ /wp-content/uploads/rank-

Adding cloud regions changes latency and cost in ways simple math can't capture. This article presents a framework from multiple launches: decompose your latency budget before committing to infrastructure, choose deployment patterns by consistency and traffic profile, and optimize before expanding. A phased approach cut latency 35% through routing alone, before a new region brought it under 60ms. By Uttara Asthana

In a recent article, Datadog engineer Arnold Wakim shared what worked, what didn't, and the lessons they learned while evolving a critical production system using AI to overcome hard limits in its storage backend and significantly improve performance. By Sergio De Simone

One of the most important performance metrics for a WordPress website is Server Response Time, commonly measured as Time to First Byte (TTFB). While caching plugins like WP Rocket significantly improve performance, many server configurations still route every request through PHP before serving the cached page. In reality, cached HTML files can be delivered directly by the web server (Apache or Nginx), completely bypassing PHP and WordPress. This approach reduces CPU usage, lowers the PHP-FPM workload, and improves overall server response time. This guide explains how to optimize both Apache (.htaccess) and Nginx so they can serve WP Rocket's static HTML cache directly. Why Is This Optimization Important? By default, a typical WordPress request follows this flow: Visitor │ ▼ Apache/Nginx │ ▼ PHP │ ▼ WordPress │ ▼ WP Rocket Cache │ ▼ HTML Response Even when a page has already been cached, the request still passes through PHP before the cached content is returned. With the following configuration, the request flow becomes: Visitor │ ▼ Apache/Nginx │ ▼ WP Rocket HTML Cache │ ▼ HTML Response PHP and WordPress are only executed when a cached file does not exist. Benefits Lower Time to First Byte (TTFB) Reduced CPU usage Less PHP-FPM processing Better performance during traffic spikes Ideal for VPS and dedicated servers Improved scalability with minimal configuration changes Apache (.htaccess) Optimization If your server runs Apache, insert the following block inside the WordPress rewrite section, immediately after: RewriteBase / and before: RewriteRule ^index\.php$ - [L] The resulting configuration should look like this: # BEGIN WordPress # Die Anweisungen (Zeilen) zwischen „BEGIN WordPress“ und „END WordPress“ sind # dynamisch generiert und sollten nur über WordPress-Filter geändert werden. # Alle Änderungen an den Anweisungen zwischen diesen Markierungen werden überschrieben. < IfModule mod_rewrite.c > RewriteEngine On RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Autho

Users judge a mobile app in the first few seconds, and they judge it harshly. A slow launch, stuttering scroll, or a device that runs hot will sink an otherwise good app faster than a missing feature. Performance isn't one metric — it's four distinct areas, each with its own causes and fixes. Here's how to keep all of them healthy. Startup time — the first impression Time from tap to usable screen is the metric users feel most. Every extra second measurably increases abandonment. The usual culprits are doing too much before the first frame: heavy synchronous work at launch, loading data you don't yet need, and oversized bundles. Fixes: Defer non-essential initialization until after the first screen renders Lazy-load features and screens instead of loading everything upfront Show a real first screen fast, then hydrate data — don't block on the network Trim your dependency footprint; every library adds to startup cost Rendering — kill the jank Smooth means hitting the device's frame budget (about 16ms per frame for 60fps). Dropped frames show up as stutter during scrolling and animation. The main causes are doing heavy work on the UI thread and rendering more than you need. Virtualize long lists so only visible rows render (FlatList, RecyclerView equivalents) Move expensive work off the main thread Avoid unnecessary re-renders — in React Native, memoize and keep render functions cheap Optimize images: right-sized, cached, and in efficient formats Memory — don't get killed The OS terminates apps that use too much memory, and users read that crash as your bug. Leaks and oversized assets are the main offenders. Watch for retained references, unbounded caches, and full-resolution images held in memory. Load and decode images at display size, release resources when screens unmount, and cap in-memory caches. Battery and network — the invisible costs Users blame the app that drains their battery even if they can't name why. The big drains are aggressive polling, chatty netwo

We’re excited to announce our first ever Summer Bug Smash! Every app has its share of rockstar bugs....

Artillery هي مجموعة أدوات مفتوحة المصدر لاختبار التحميل مبنية على Node.js. تتيح لك توليد حركة مرور عالية التزامن على واجهة برمجة التطبيقات (API) من خلال ملف YAML بسيط: تحدد مراحل التحميل، تصف تدفقات الطلبات، تشغل artillery run script.yml ، ثم تقرأ نسب زمن الاستجابة المئوية، معدلات الطلبات، وعدد الأخطاء. يشرح هذا الدليل طريقة تثبيت Artillery v2، كتابة اختبار عملي، تشغيله، استخراج النتائج بالطريقة الصحيحة في v2، وربطه بمسار CI. جرّب Apidog اليوم ما هو Artillery ومتى تستخدمه؟ ينشئ Artillery مستخدمين افتراضيين (VUs) يرسلون طلبات إلى نقاط النهاية لديك ويقيسون قدرة النظام على تحمل الحمل المستمر. المستخدم الافتراضي هو عميل مُحاكى ينفذ سيناريو خطوة بخطوة، كما يفعل مستخدم أو خدمة حقيقية. استخدم Artillery عندما تريد إجابات عملية على أسئلة الأداء مثل: كيف يتغير زمن الاستجابة p95 عند 50 طلبًا في الثانية؟ عند أي معدل وصول تبدأ الأخطاء بالظهور؟ هل تبقى واجهة API مستقرة لمدة 5 دقائق من الحمل المستمر؟ هل يتدهور الأداء تدريجيًا مع استمرار الضغط؟ الميزة الأساسية في Artillery أن الاختبار تصريحي. بدل كتابة حلقات تزامن يدويًا، تصف شكل الحمل في YAML. وبما أنه يعمل فوق Node.js، يمكنك تشغيل نفس الاختبار محليًا وفي CI. إذا كنت تقارن الأدوات، راجع ملخص أفضل أدوات اختبار التحميل و مقارنة برامج اختبار التحميل لفهم الفروقات بين k6 وJMeter وGatling وغيرها. تثبيت Artillery v2 اسم الحزمة هو artillery ، والإصدار الرئيسي الحالي هو v2. ثبته عالميًا عبر npm: npm install -g artillery@latest artillery version تحتاج إلى إصدار LTS حديث من Node.js. يعمل Artillery على Windows وmacOS وLinux. إذا كنت لا تريد تثبيت الحزمة عالميًا، استخدم npx : npx artillery@latest run script.yml كتابة اختبار Artillery يتكون ملف الاختبار من قسمين أساسيين: config : يحدد الهدف ومراحل الحمل والمتغيرات. scenarios : يحدد ما يفعله كل مستخدم افتراضي. مثال كامل: config : target : " https://api.example.com" phases : - name : " Warm up" duration : 60 arrivalRate : 5 - name : " Ramp to peak" duration : 120 arrivalRate : 5 rampTo : 50 - name : " Sustained load" duration : 300 arrivalRate : 50 maxVusers : 500 variables : productId : - " 100

Tra performance, piattaforma e toolchain: i temi che contano davvero per chi costruisce frontend oggi. Il frontend nel 2026 è diventato una disciplina sempre più “di prodotto”: non basta far funzionare l’interfaccia, serve che sia veloce, stabile, accessibile e misurabile in produzione. E quando l’ecosistema Chrome parla di “connessione” tra developer e piattaforma, il messaggio utile per chi lavora sul web è semplice: capire dove investire tempo per ottenere impatto reale sugli utenti . Di seguito, una lettura pratica dei temi che continuano a emergere come prioritari per chi costruisce applicazioni e siti moderni. 1) Performance: meno benchmark, più realtà La performance non è più un esercizio di ottimizzazione a fine progetto. È un requisito continuo che va gestito con strumenti, metriche e processi. Cosa significa “misurabile” oggi Metriche di campo (real user monitoring) : le prestazioni che contano sono quelle che arrivano dai dispositivi reali, su reti reali. Metriche di laboratorio : restano utili per regressioni e CI, ma vanno interpretate come “segnali” e non come verità assolute. Implicazione pratica Imposta una pipeline dove: le metriche sintetiche bloccano regressioni evidenti (build/PR), le metriche reali guidano le priorità (release e backlog). 2) DevTools: dal debug al controllo qualità Gli strumenti di sviluppo non servono più solo a “trovare il bug”, ma a ridurre il rischio : regressioni di layout, memory leak, risorse inutili, dipendenze pesanti. Abitudini che fanno differenza Profilare prima di ottimizzare: CPU, rete e rendering hanno colli di bottiglia diversi. Isolare i cambiamenti: una variazione di bundling o di immagini può ribaltare il profilo prestazionale più di una micro-ottimizzazione in JS. 3) La piattaforma web continua a crescere (e chiede scelte più consapevoli) La Web Platform oggi offre API potenti, ma la parte difficile non è “usarle”: è scegliere quando usarle. Un criterio utile Se una feature riduce complessità (meno librerie,

In this blog post, we will see how to performance test a RAG (Retrieval-Augmented Generation) application properly, covering both speed and correctness, and how to wire both into a CI/CD pipeline so regressions get caught before they reach production. Performance testing a RAG application requires two separate testing gates: one for speed and one for answer quality. Traditional load testing tools measure response times but cannot detect hallucinations, where a model returns fast but factually incorrect answers grounded in fabricated context rather than retrieved documents. The guide demonstrates using k6 for load testing end-to-end latency and DeepEval for evaluating faithfulness and answer relevancy using an LLM-as-judge approach. Both gates are integrated into a GitHub Actions CI/CD pipeline so regressions in either performance or output quality are caught automatically on every pull request before reaching production. If you've come from a JMeter or k6 background like I have, your first instinct with a RAG endpoint is probably to point a load test at it and check response times. That gets you halfway there. A RAG app can return a fast, confident, completely wrong answer, and a plain load test will never tell you that. You need two testing surfaces, not one: performance and quality. This guide covers both, using a single running example throughout: a documentation assistant that answers "How do I run JMeter in non-GUI mode?" against a small knowledge base. Why RAG breaks traditional load testing assumptions A conventional API returns a complete response and you measure the round trip. A RAG endpoint does two expensive things before it answers: it retrieves context from a vector store or search index, then it streams a generated response token by token. That second part matters a lot. A single request can stream hundreds of tokens over several seconds, so "request duration" as a single number hides two very different problems: how long the model took to start answe

Todo dev brasileiro que já precisou consultar CNPJ conhece o dilema: ou você usa uma API que faz proxy da Receita (3 a 10 segundos por consulta, quando não cai), ou baixa o dump de dados abertos e monta a própria base — e descobre que "baixar um CSV" era a parte fácil. Eu montei a própria base. Este post é o diário honesto do que funcionou, do que quebrou e dos números reais — 217 milhões de linhas servidas em ~10ms de p50 dentro do datacenter, num Postgres de 1 vCPU. A arquitetura em uma frase Não consulte a Receita em tempo real. Ingira o dump mensal e sirva da sua infra. O resto é decorrência. Receita (dump mensal, ~6GB zip) ──▶ ingestão Go (COPY) ──▶ Postgres ──▶ API (chi) CGU (CEIS/CNEP, zip diário) ──▶ job diário ──┘ O dump da Receita: as pegadinhas que ninguém documenta O layout oficial existe, mas o que quebra parser de verdade é o que está fora dele: Encoding latin1 (ISO-8859-1) — acento vira lixo se você ler como UTF-8. Em Go: charmap.ISO8859_1.NewDecoder() num transform.Reader streaming. Decimal com vírgula ( "1000000,00" ) e datas YYYYMMDD onde 0 e 00000000 significam nulo. CNPJ quebrado em 3 colunas (básico 8 + ordem 4 + DV 2). A chave de junção entre empresas, estabelecimentos e sócios é o básico — errar isso custa um dia. As partições 0–9 não se alinham entre arquivos. O estabelecimento da partição 3 pode ser de uma empresa da partição 7. Foreign key rígida entre as tabelas = COPY quebrando no meio da carga. A solução: sem FK; a integridade vem da fonte. Bytes NUL ( 0x00 ) no meio dos dados. O Postgres rejeita NUL em text . Um strings.ReplaceAll(s, "\x00", "") no parser economizou três recargas. Desde jan/2026 o repositório é um Nextcloud do SERPRO+ com WebDAV público — dá pra listar meses com PROPFIND e baixar com o token do share como usuário. Adeus, scraping. COPY ou morte A diferença entre INSERT em lote e o protocolo COPY não é incremental — é outra categoria. Com pgx.CopyFrom e lotes de 50k: 28,1 milhões de empresas em 1m28s (~320k linhas/s) num

PostgreSQL provides several planner configuration parameters, such as enable_seqscan and enable_indexscan , that influence how execution plans are generated. These settings affect planning, not the execution of an already-generated plan. With prepared statements, this raises an interesting question. Should planner settings be applied before PREPARE, before EXECUTE, or both? Let's look at a simple example: a "tasks" table with a due date and a "done" status: \ c drop table if exists tasks ; -- a table of tasks with status (done or not) and due date create table tasks ( id bigint generated always as identity primary key , due timestamptz , done boolean ); -- insert 500 tasks, with 1% not done insert into tasks ( due , done ) select now () + interval '1 day' * n , 42 != n % 100 from generate_series ( 1 , 500 ) n ; -- index the todo (partial index) create index on tasks ( due , id ) where done = false ; vacuum analyze tasks ; With a partial index, I indexed only the tasks that are not yet done ( done = false ) because that's my most frequent query pattern: postgres =# explain select id , due , done from tasks where done = false and id > 0 order by due limit 1 ; QUERY PLAN --------------------------------------------------------------------------------------- Limit ( cost = 0 . 13 .. 3 . 60 rows = 1 width = 17 ) -> Index Scan using tasks_due_id_idx1 on tasks ( cost = 0 . 13 .. 17 . 47 rows = 5 width = 17 ) Index Cond : ( id > 0 ) ( 3 rows ) With partial indexes, the condition covered by the index is not even visible in the execution plan because the index itself enforces the condition. Prepared statement I decided to use a prepared statement with all values as parameters. It is probably not a good idea in this case. When a parameter can have only a few different values and you expect different cardinalities for each, you should probably define one query per value, using literals. I'm doing this to illustrate what can happen, with a simple, extreme example: postgres =# pr

Introduction We already covered picking the right model tier for the task and caching a large shared prefix in https://pg-blogs.netlify.app/posts/11-building-reliable-llm-apps-in-java/ . Those two lines were the tip of a bigger discipline: LLM cost is not a fixed line item, it's an engineering variable — one you can measure and shrink with the same rigor you'd apply to database query time or container memory. This post goes deeper: how input/output pricing actually works, the exact cache_control shape and how to prove a cache hit rather than assume one, the Batches API for work that isn't latency-sensitive, and model routing — using a cheap model to triage, escalating only the hard cases to a stronger one. The honest framing throughout: measure before you optimize. Every technique here has a cost of its own; applied to the wrong workload, "optimization" makes things slower or more expensive. Token Economics: Why the Prefix Is the Bill Anthropic (like every hosted LLM provider) prices input and output tokens separately, and output is always pricier — the model has to generate output autoregressively, one token informed by all the ones before it, while input can be processed in parallel. Representative pricing from the current model catalog: Model Input Output Claude Opus 4.8 $5.00 / MTok $25.00 / MTok Claude Sonnet 5 $3.00 / MTok $15.00 / MTok Claude Haiku 4.5 $1.00 / MTok $5.00 / MTok Two consequences follow directly: Long system prompts, tool definitions, and RAG context are read on every request , not written once. A 20K-token system prompt sent on every one of 10,000 requests is 200M input tokens — at Opus 4.8 rates, $1,000 before a single output token is generated. The shared prefix , not the user's question, is usually where the money goes. A verbose model wastes money twice — once on the extra output tokens themselves, and again because the next turn's messages history now carries that verbosity forward as input on every subsequent call. Trimming max_tokens an

Introduction https://pg-blogs.netlify.app/posts/10-building-reliable-llm-apps-in-python/ closed with a section on picking the right model per task and caching a shared prefix. That was the entry point into a bigger discipline: LLM spend is an engineering variable, not a fixed bill — one you can measure and reduce with the same rigor you'd apply to query latency or memory footprint. This post goes deeper on four levers: how input/output pricing actually works and why the prefix is usually where the money goes, the exact cache_control shape and how to prove a cache hit instead of assuming one, the Batches API for work that isn't latency-sensitive, and model routing — a cheap model triaging requests and escalating only the hard ones. The throughline is honest: measure before you optimize. Every lever here has its own cost; misapplied, it makes things slower or pricier, not cheaper. Token Economics: Why the Prefix Is the Bill LLM providers price input and output tokens separately, and output always costs more — generation is autoregressive (each token depends on every one before it), while input can be processed in parallel. Representative pricing from the current model catalog: Model Input Output Claude Opus 4.8 $5.00 / MTok $25.00 / MTok Claude Sonnet 5 $3.00 / MTok $15.00 / MTok Claude Haiku 4.5 $1.00 / MTok $5.00 / MTok Two things follow: A long system prompt, tool list, or RAG context is billed as input on every request , not written once. Send a 20K-token system prompt on 10,000 requests and that's 200M input tokens — at Opus 4.8 rates, $1,000 before the model has generated a single output token. The shared prefix , not the user's actual question, is usually the dominant cost. Verbose output costs twice — once directly (more output tokens billed at the higher rate), and again because the next turn's history carries that verbosity forward as input. Asking for concise output and setting a sane max_tokens is a cost control, not just a style choice. This is why the tw