标签：#scala

Netflix is moving toward the open-source Apache Flink Autoscaler for more than 30,000 streaming jobs across multiple AWS regions. The operator-level approach addresses limitations of Netflix’s cluster level autoscaler for complex, stateful pipelines. Netflix reports a 58% reduction in annualized Flink compute expenditure for one team, saving approximately $1.1 million annually. By Leela Kumili

Scalability and Technical Debt: The Hidden Trade-off That Determines Your System's Future Every successful startup faces a critical moment: the moment when "it works" stops being good enough and you have to ask yourself a harder question: "Will it continue to work?" I've lived this moment multiple times. At my last fintech company, we built a Spring Boot monolith that processed millions of transactions daily. It scaled beautifully for the first two years. Then it didn't. The problem wasn't the code quality. It wasn't poor architecture decisions. It was technical debt—accumulated during our scramble to achieve scalability fast enough to keep up with growth. And that debt had compounded. The relationship between scalability and technical debt is one of the most misunderstood trade-offs in software engineering. Most teams treat them as opposites when they're actually co-dependent . Build for scalability without managing debt, and you'll collapse under your own complexity. Obsess over code quality without scaling capability, and your brilliant system becomes irrelevant because it can't handle real-world load. The Scalability-Debt Paradox Here's the core paradox: The faster you scale, the more debt you accumulate. The more you eliminate debt, the slower you scale. Let me illustrate this with real numbers from a system I managed. The First Year (Speed Over Perfection) We had a customer acquisition target: 100,000 active users within 12 months. This wasn't negotiable. Our competitors were moving faster, and we needed to prove the market opportunity before our funding ran out. We made deliberate trade-offs: Built features using the simplest patterns that worked (mostly monolithic endpoints) Duplicated code instead of abstracting it (faster to ship) Used an ORM that wasn't optimized for high-throughput queries (easier to iterate) Skipped advanced caching layers (complexity tax wasn't worth it yet) Result : We hit 100,000 users in 11 months. We dominated our market segment in

Airbnb redesigned its authentication architecture around server driven flows and policy based challenge selection. The new Flexible Authentication system reduced authentication related code by 60%, cut the web client bundle by 100 KB, improved successful authentication by 2.6%, reduced duplicate account creation by 27%, and lowered OTP costs by 11%. By Leela Kumili

RAG Retrieval Gotchas at Scale: Insights and Solutions Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm in natural language processing (NLP), combining retrieval and generation to produce contextually relevant outputs. However, implementing RAG at scale introduces several challenges, or "gotchas," that can significantly impact performance and usability. In this article, we'll explore these pitfalls and provide concrete solutions, complete with code snippets and specific version numbers, to help you scale your RAG implementations effectively. Understanding RAG Architecture Before diving into the gotchas, it's essential to understand the architecture of RAG. The RAG model typically consists of two components: Retriever : This component fetches relevant documents from a large corpus based on a given query. Generator : This component generates a response based on the retrieved documents. In a typical RAG setup, you might use models from Hugging Face's Transformers library (version 4.21.1 or later is recommended) for both the retriever and generator. For instance, the RAG model can be set up as follows: from transformers import RagTokenizer , RagRetriever , RagSequenceForGeneration tokenizer = RagTokenizer . from_pretrained ( " facebook/rag-sequence-large " ) retriever = RagRetriever . from_pretrained ( " facebook/rag-sequence-large " ) model = RagSequenceForGeneration . from_pretrained ( " facebook/rag-sequence-large " ) Gotcha 1: Document Retrieval Latency Problem When scaling RAG systems, one common issue is the latency during document retrieval. If the retriever is querying a large corpus, the response time can significantly slow down the overall processing speed. Solution To mitigate this, consider optimizing your retrieval strategy. One approach is to use approximate nearest neighbor (ANN) search algorithms, such as FAISS (version 1.7.1), which can drastically reduce retrieval times. Here's a brief example of how to implement FAISS with your

Brian Martin discusses the real-world performance costs of metrics libraries and shares strategies for low-overhead, "fearless" instrumentation. Drawing from his work at IOP Systems, he explores atomic primitives, per-CPU sharding, lock-free histograms, and eBPF integration to help software architects and engineering leaders maintain full system visibility without sacrificing performance. By Brian Martin

Hello everyone. Scala's a programming language I've enjoyed learning on the side not only because I think it's stylish but because it's made me a better developer. Some of the gripes you encounter once you try to go intermediate or beyond, it's the Typelevel stack complexity. You just want to bootstrap a server and start writing some routes, and tbh sometimes the docs aren't that friendly. That's why I wrote Scala 3 Domain Design & Typelevel Stack Cookbook — a book that teaches some DDD in Scala and gives you some recipes to get you started on the stack (cats, cats-effects, fs2, http4s). It's a WIP currently at 40%. You can read a couple of chapters for free in leanpub: https://leanpub.com/scala3-domain-typestack-dev

Ian Nowland discusses why conventional CI/CD practices break down for stateful, core infrastructure. Drawing from his leadership at AWS and Datadog, he shares actionable techniques for safe progressive deployments, synthetic testing in production, and mitigating blast radius in complex software platforms. By Ian Nowland

API documentation has a reliability problem. The code gets updated; the OpenAPI spec gets forgotten. The spec gets updated; the TypeScript client doesn't regenerate. By the time an enterprise client asks for your API contract, the document you hand them describes a system that no longer exists. Baklava, an open-source library by Iterators , solves this structurally: documentation is generated from the tests that verify your actual API behaviour, so it cannot drift. The problem Documentation drift is the default state of any API that lives long enough. The causes are well-understood: docs and code are maintained separately, documentation updates require extra discipline at every PR, and no automated check catches a route signature change that wasn't reflected in the OpenAPI file. The consequence is real. Clients building against a stale spec hit integration errors in production. Internal teams onboarding to a service spend hours reconciling the documented contract with actual behaviour. TypeScript front-ends break when an API response field changes without a corresponding client update. The problem compounds as the API grows. The solution Baklava integrates into your existing test suite. When routing tests run, baklava observes each request and response, infers the API surface, and generates documentation as a test output, not as a separate build step, not as a manually-maintained file. In baklava, the test is the documentation spec. Instead of a standard assertion block, each route is defined with path() , supports() , and onRequest() scenarios that both verify the API behaviour and describe it for documentation output: ​`// The test IS the documentation spec class UserApiSpec extends AnyFunSpec with BaklavaPekkoHttp[Unit, Unit, ScalatestAsExecution] with BaklavaScalatest[Route, ToEntityMarshaller, FromEntityUnmarshaller] { path("/users/{userId}")( supports( GET, pathParameters = p Long , summary = "Get user by ID" )( onRequest(pathParameters = 1L) .respondsWith Use

Meryem Arik discusses strategies for designing low-cost LLM inference architectures for high-volume, non-real-time workloads. She explains how software architects and engineering leaders can achieve order-of-magnitude cost reductions by making critical trade-offs across hardware, inference runtimes, speculative decoding, and smart queue reordering. By Meryem Arik

This is Part 10 of my "From One User to One Million" series, where we'll build an understanding of System Design by following a simple application as it grows from a single user to millions. Instead of memorising technologies, we'll learn why they exist by solving real problems as they appear. In Part 9, we solved the problem of data that had grown too large for a single database. We split it across multiple shards, each holding its piece of the whole, so that no single machine ever had to carry everything. At that point, the architecture could scale in almost every direction we'd tried to push it. Traffic was distributed across application servers. Repeated database work was absorbed by the cache. Read traffic was spread across replicas. Data itself was partitioned across shards. And yet. We ended Part 9 by noticing something that none of those solutions addressed. Some user requests trigger a lot of downstream work. Saving an order is one thing. But saving the order, sending a confirmation email, generating an invoice, updating inventory, firing off a notification, recording an analytics event, triggering the recommendation engine: that's an entirely different conversation. Right now, all of that happens before the user gets a response. The question we left with was this: what if they didn't have to wait for all of it? -- Section 1: The User Doesn't Need Everything Right Now Before we look at any solution, it's worth asking a simpler question. When a user places an order, what do they actually need to know before they can move on? They need to know the order was received. They need confirmation that the important thing happened: their money was accepted, their items are reserved, the transaction is real. That's it. That's what they're waiting for. They do not need to wait for the confirmation email to land in their inbox. They do not need to wait for the invoice to be generated and stored somewhere. They do not need to wait for the analytics system to record that

Martin Spier explains how agentic workflows dramatically increase code change volume at OpenAI. He discusses the hidden systemic performance costs of rapid shipping beyond GPUs, and shares how deploying always-on AI agents automates profiling, regression detection, and continuous optimization to maintain product speed and scalability at massive global scale. By Martin Spier

JioHotstar explains the distributed architecture behind its real-time ad request workflow, covering ad decisioning, waterfall tiering, pacing algorithms, latency optimization, and service coordination required to select and deliver personalized advertisements during streaming playback at scale. By Leela Kumili

This is Part 7 of my "From One User to One Million" series, where we'll build an understanding of System Design by following a simple application as it grows from a single user to millions. Instead of memorising technologies, we'll learn why they exist by solving real problems as they appear. Last time, we ended on a question that sounded simple but isn't. Aisha updated her profile picture. Her new photo is now saved in the database. But the cache is still holding onto the old one, completely unaware that anything changed. So every request for Aisha's profile gets served the old data. Confidently. Instantly. Incorrectly. How does a cache know when the data it's holding is no longer correct? Think about what we've actually built at this point. We have an application that responds fast, scales horizontally, and avoids hammering the database with repeated identical queries. From a performance standpoint, it looks great. But Aisha's friends are loading her profile and seeing a photo she replaced five minutes ago. The system isn't slow anymore. It's wrong. Speed and correctness are two different things. We optimized hard for one, and quietly broke the other. Engineers have a name for this problem: cache invalidation . It refers to the challenge of keeping the data in your cache consistent with the data in your database, as that underlying data changes over time. It turns out to be one of the genuinely hard problems in building software systems. Not hard in a complicated-algorithm way. Hard in the way that every solution has a catch, and the right answer always depends on what you're willing to accept. Let's think through it together. -- Section 1: When Cached Data Lies It's worth sitting with the problem a little longer before rushing to fix it, because the damage stale data can cause varies enormously depending on what's being cached. Consider a few examples. Your application caches the list of trending articles. An hour later, the list has changed. New articles have ri

"We use Redis Cluster" can mean two very different things: Our dataset is distributed across Redis nodes. Every individual data structure is distributed across Redis nodes. The first can be true while the second is false. That distinction matters for leaderboards. In Podium , each leaderboard uses several Redis keys and atomic Lua scripts. Redis Cluster helps us scale a large fleet of independent leaderboards, but it cannot split one giant sorted set across primaries. We are sharing this architecture because "Redis Cluster scales horizontally" is true only after you define what the system actually shards. TeneficGames / podium High-performance, Redis-backed leaderboards for games and competitive applications. Podium High-performance, Redis-backed leaderboards for games and competitive applications. Podium provides ready-to-run HTTP and gRPC APIs for scores, ranks, seasons, and player-relative views. It is designed for backend teams operating large fleets of independent leaderboards without provisioning each leaderboard in advance. Fair, deterministic ordering when scores are equal. Single and bulk score updates, including multi-leaderboard fan-out. Standalone Redis and real Redis Cluster integration coverage. Deploy one multi-architecture OCI image with Docker, containerd, Kubernetes or another OCI-compatible runtime. Quickstart · Performance · API · Documentation · Helm chart · Docker Hub · GHCR Quickstart Start Redis 8.2 and the latest stable Podium image: docker network create podium docker run --detach --name podium-redis --network podium redis:8.2-alpine docker run --detach --rm --name podium \ --network podium \ --publish 8880:8880 \ --publish 8881:8881 \ --env PODIUM_REDIS_HOST=podium-redis \ --env PODIUM_REDIS_PORT=6379 \ trungdlp/podium:latest start Verify the service: curl http://localhost:8880/healthcheck WORKING Submit two equal scores: curl --request … View on GitHub Here is how the design works, why hash tags are necessary, and where the scaling bounda

The engineering team at Zalando recently described the design and implementation of an in-process, client-side load balancer for a high-throughput API handling around 1 million requests per second. The result was more predictable latency, a drop in infrastructure costs, and better visibility into where failures actually originate. By Renato Losio

In a recent article, Datadog engineer Arnold Wakim shared what worked, what didn't, and the lessons they learned while evolving a critical production system using AI to overcome hard limits in its storage backend and significantly improve performance. By Sergio De Simone

Both vulnerabilities allow untrusted users to gain root privileges.

Netflix engineers introduced dynamic partition splitting for Cassandra to address wide partitions in time series workloads. The metadata-driven approach detects oversized partitions, splits them smaller units, and routes reads across child partitions. Netflix reported lower read latency from seconds to milliseconds, reduced timeouts, and improved cluster stability while maintaining transparency. By Leela Kumili

The product drop had been planned for months. The direct-to-consumer subscription business had run three separate load tests, provisioned extra capacity for the launch window, and staffed a warroom across two time zones. The drop itself went cleanly. Two hours in, an unrelated video from a creator with a large following mentioned the product without warning, and the sign-up flow collapsed under a rush of new members for twenty-eight minutes. Customers were told the site was busy and to try again later. Some did. Most did not. The refund exposure was manageable. The customer acquisition exposure was not. What went wrong is not the interesting question. The system was under-provisioned for a specific traffic shape it had not seen before, and the team fixed it. The interesting question is what happened seven weeks later. A weather event redirected a wave of app traffic in an entirely different sector, at midnight on a Tuesday, without any warning. That system held, because a small group of engineers had spent those seven weeks quietly rebuilding assumptions about when peak load happens and what it looks like. The lesson from the product drop was not "provision more capacity for product drops." The lesson was that the mental model of peak load as a scheduled event had stopped being useful. This is another post in our series on the engineering layer underneath enterprise strategy. The previous post ( Sovereignty Versus Efficiency ) argued that sovereignty has become an architectural property that procurement cannot solve on its own. This post makes an analogous argument about load. Across banking, media, retail, travel, restaurant chains, and sport, the architectures built to survive named events are increasingly the wrong architectures for the traffic these businesses now routinely encounter. The discipline required has moved closer to what telecommunications engineers have always done, while the cost models have not caught up. What peak load used to mean For most of th

Quando comecei a trabalhar com aplicações descentralizadas há mais de uma década, lembro bem da frustração de pagar US$ 50 em taxas de transação para mover alguns tokens na rede Ethereum durante um pico de congestionamento. Era um problema técnico que ameaçava inviabilizar todo o ecossistema. Hoje, observo com entusiasmo profissional como as soluções de Layer 2 transformaram radicalmente esse cenário, abrindo portas para casos de uso que antes eram economicamente impraticáveis — especialmente aqui no Brasil, onde a tokenização de ativos e os pagamentos em stablecoins crescem em ritmo acelerado. O problema fundamental: o trilema da escalabilidade Para entender por que as soluções de segunda camada são tão importantes, precisamos compreender o trilema da blockchain proposto por Vitalik Buterin. Uma rede precisa equilibrar três pilares: descentralização, segurança e escalabilidade. O Ethereum, em sua arquitetura original, priorizou os dois primeiros, processando apenas cerca de 15 a 30 transações por segundo (TPS) na camada base. Para se ter dimensão, redes de pagamento tradicionais como a Visa processam milhares de transações por segundo. Quando o DeFi explodiu em 2020 e 2021, e novamente com o boom dos NFTs, a rede simplesmente não dava conta da demanda. As taxas de gas dispararam, e usuários comuns foram literalmente expulsos pelo custo. Em meus projetos de consultoria, atendi empresas brasileiras que desistiram de iniciativas Web3 justamente porque os custos operacionais inviabilizavam o modelo de negócio. A pergunta que sempre me faziam era: "Como cobrar R$ 5 de um cliente se a taxa da transação custa R$ 30?". A resposta estava — e está — nas camadas de segunda geração. Como funcionam as soluções de Layer 2 O conceito central das soluções de Layer 2 é elegante: em vez de processar todas as transações diretamente na blockchain principal (Layer 1), executamos a maior parte do processamento "fora da cadeia" e depois enviamos apenas uma prova compacta de volta para o