标签：#STEM

Long-running workflows look simple when you first build them. Something happens. A few systems exchange data. Everything completes. Done. At least that's the expectation. Reality is very different. The biggest thing I underestimated was time. Not execution time. Elapsed time. Because once workflows start running for hours, days, or continuously, strange things start happening. APIs become temporarily unavailable Data changes halfway through the process Retries arrive much later than expected Someone manually updates a record Another system processes things in a different order Nothing is broken. But everything is slightly different from when the workflow started. Early on, I assumed workflows were transactions. Start. Execute. Finish. Now I think of them as conversations between systems. And conversations can get interrupted. Another thing I underestimated: State changes. You might start processing an order that is "pending". Ten minutes later, another system marks it as "cancelled". An hour later, a retry comes in from an earlier step. If your workflow only thinks about data, weird things happen. Because the world has changed while the process was still running. Long-running workflows also expose assumptions you didn't know you made. Like: this API will always respond quickly data will arrive in order users won't modify records manually retries will happen immediately Those assumptions survive in testing. Production removes them quickly. One thing that changed how I build these systems: I stopped asking: "Will this workflow finish?" And started asking: "What state will the world be in when it finishes?" Because those are two very different questions. Most problems in long-running systems aren't caused by one big failure. They're caused by lots of small changes happening while the workflow is still alive. And if you don't account for that, eventually the workflow finishes successfully and still produces the wrong outcome. This is something we think about constantly

Introduction This is the second part of sharing my journey to create a microservices-based backend system, in the previous part, I introduced the system and its services. In this part I am going to mention the main things that happened with me from then until now. Transitioning to online DBaaS platforms The first main thing was to transition my databases from my local device to accessible over the internet. I used Neon and MongoDB Atlas . Although performance-wise it is better to deploy the database on the same server that the whole system so all services can access the databases without network overhead , This solution is better from a point, that is avoiding consuming the free VM instance that I got to host my main services, as with DBaaS platforms I will not require storage for databases or additional overhead to handle the DBMS on a free small resources-constrained VM. Observability Observability is one of the main concerns in any system, it gives the ability for the system to expose what happens inside it without any need to guess and manually trace the code to determine where the error may happened. I utilized OpenTelemetry , as I found it is most used and accepted tool to log, trace and collect metrics about the system and the traffic. Also, I loved that it is not related to specific framework, that is .NET by the way, I loved the idea that it is standalone tool. But logging and tracing and metrics collection will not be beneficial if we do not see it actually and can achieve monitoring from those telemetry data, so I used Grafana Cloud to export the telemetry data to some place that is accessible online and a tool that can generate dashboards and visualizing for the metrics ( Prometheus ), and UI that can show all the telemetry data easily. Deployment Until this point, we have a system that is fully functional and have consistent docker configuration for its services via Docker Compose, and it database is already there and available. Now this is the point at

Payment data pipelines fail in ways that ruin a payments engineer’s week, and the failures rhyme. The dashboards froze. Fraud scores arrived after the transaction had already cleared. Settlement reports came in stale. Nobody slept. The frustrating part is that the same data architecture had run fine for years. So, what changed? The honest answer is that batch thinking does not survive contact with real-time payments. A lot of banks built their data foundations in an era when nightly jobs were good enough. Load the warehouse overnight, run the reports in the morning, move on. That rhythm worked when money moved slowly. It does not work when a customer expects an instant confirmation and a fraud engine has milliseconds to make a call. Here is where things crack. Real-time payment rails push a constant stream of events instead of a tidy nightly dump. Your pipeline now has to ingest, transform, and serve data while transactions are still happening. Add ISO 20022 into the mix and the pressure climbs. ISO 20022 messages are rich. They carry far more structured detail than the old formats, which is wonderful for analytics and miserable for a pipeline that was never designed to parse that much context at speed. This is not a fringe concern either. Swift reported that by the time its MT/ISO 20022 coexistence period closed in November 2025, around 80% of daily traffic was already running on the ISO 20022 format, with more than 3.1 million of these messages exchanged every day. The rich-data era is the default now, not the roadmap. Then there is the fraud-scoring window. Fraud models need fresh features. Account behaviour over the last few minutes, velocity checks, device signals. If your pipeline takes thirty seconds to surface that data, the fraud decision is already too late. You are essentially detecting fraud after the loss. That gap between when data is created and when it becomes usable is the silent killer in most payment systems. And the cost of getting it wrong runs

If your LLM costs are climbing, the instinct is almost always the same: swap to a cheaper model. GPT-4 to GPT-4-mini. Claude Opus to Claude Haiku. Sometimes that helps a little. It rarely fixes the actual problem. The actual problem, in most workflows I've looked at, is that every step gets routed through the LLM, even the steps that don't need language reasoning at all. This post breaks down a simple mental model for deciding what should and shouldn't touch an LLM, with a working example you can adapt. The four components of any AI workflow Every automated workflow — whether it's a support ticket router, a fraud check, or a content pipeline — is built from some combination of four building blocks. They get treated the same once a workflow diagram is drawn flat, but they have wildly different cost and latency profiles. Component What it does Think of it as Typical cost Trigger Starts the workflow The doorbell ~$0 Deterministic ML Structured predictions — classify, score, rank The calculator Cents per 1,000 calls LLM / Generative Reads, writes, reasons in language The writer Dollars per 1,000 calls Tool / API Fetches or writes real data The hands Cents per 1,000 calls The gap between row 2 and row 3 is the whole article. A classifier and an LLM call can solve the exact same problem, but one costs roughly 100-1000x more than the other, depending on model and provider. If you're not deliberately deciding which one handles which step, you're probably defaulting to the expensive one — because in frameworks like LangChain or a quick custom agent loop, it's just easier to shove everything into a prompt. Where this actually shows up Here's a workflow I see constantly: an automated support ticket triage system. flowchart LR A[New support ticket] --> B{Classify intent} B --> C[Route to team] B --> D[Auto-draft response] D --> E[Update CRM] A naive build sends the entire ticket text to an LLM and asks it to do everything at once: classify the intent, decide routing, draft a re

Good architecture is not only about how a system is built. It is also about how well the team can understand that system once it is running. That is where observability belongs in the architecture conversation. It is common for observability to be treated as something that comes after the main engineering work. The service gets built. The API works. The deployment succeeds. Then, somewhere near the end, the team starts thinking about logs, dashboards, alerts, and operational visibility. That approach creates a gap. The architecture may look clean on paper, but once the system is in production, the team has to understand how it behaves under real conditions. Real users do not follow the happy path perfectly. Dependencies slow down. Queues back up. Data arrives in unexpected shapes. Deployments change behavior in ways that are not always obvious. If the system does not give the team a way to see those things clearly, the architecture is incomplete. Observability is not decoration around the system. It is part of the system design. Architecture Describes the System. Observability Shows the Truth. Architecture is built on assumptions. During design, teams make reasonable guesses about usage patterns, service boundaries, dependency behavior, data flow, scale, latency, and failure modes. Some of those assumptions are based on experience. Some are based on current requirements. Some are simply the best call the team can make with the information available at the time. That is normal. The problem is not that architecture contains assumptions. Every architecture does. The problem is when those assumptions cannot be tested once the system is real. A design might assume that an external dependency will be reliable enough. Production may show that it is the slowest part of the request path. A queue might look like a clean decoupling point during design. Production may reveal retry behavior, duplication, or ordering concerns that were not obvious upfront. A serverless function m

The Problem: Feature Velocity Was Creating Structural Debt The system originally started as a simple feature delivery backend: A Django API powering agricultural insights Celery workers handling asynchronous processing Independent endpoints for each new capability A growing set of Earth Observation computations (NDVI, NDWI, etc.) At first, it worked. But as more features were added, a pattern emerged: Each feature introduced its own pipeline logic Observability was inconsistent across services API contracts drifted between frontend and backend Debugging required tracing multiple disconnected systems We weren’t scaling functionality. We were scaling fragmentation. The Turning Point: Features vs Platforms The key realization was simple: Features solve user problems. Platforms solve system problems. We were repeatedly rebuilding: Authentication flows Data ingestion logic Processing pipelines API validation layers Monitoring hooks Each feature was solving its own version of these concerns. That is where platform engineering became necessary. The Shift: Introducing a Platform Layer We introduced a platform layer between feature delivery and infrastructure. Instead of building isolated pipelines, we standardized: 1. Unified API Surface All Earth Observation workflows (NDVI, NDWI, and future indices) were normalized into a consistent API contract. Shared request/response structure Versioned endpoints Schema validation through serializers Central routing logic This eliminated endpoint fragmentation. 2. Standardized Processing Pipeline Celery tasks were refactored into a reusable pipeline pattern: Ingestion Validation Computation Storage Publishing Instead of feature-specific workers, we moved toward composable tasks. This allowed new indices or processing logic to plug into the same execution flow. 3. Observability as a First-Class Layer One of the biggest failures in the original system was visibility. We introduced: Structured logging across all services Traceable job IDs

I bombed a system design interview last year — not because I didn't know the architecture, but because I spent the first 5 minutes fighting Excalidraw. So I built SystemDesignBoard — a free, keyboard-first whiteboard specifically for system design interviews. What it does You open it, press a key, and start drawing. No account, no onboarding, no drag-from-a-sidebar friction. R → place a Service node C → place a Database/Cache/Queue A → connect two nodes N → open the scratchpad for scale math The features I'm most proud of Animated connectors that show communication type Instead of just drawing arrows, connectors visually encode how services talk: ⇄ sync — paired dashes (request + ACK) ≋ stream — near-solid fast line with glow (continuous pipeline) This matters in interviews — your interviewer can glance at your diagram and immediately understand the communication pattern. Cloud provider badges Tag any node as AWS (EC2, Lambda, RDS, S3), GCP (GKE, Cloud Run, Firestore), or Azure. Each subtype has its own icon. Trade-off logging Right-click any node → Log Trade-offs → attach your CAP theorem stance, consistency level, and scaling strategy directly to the component. Diagram-as-Code Type: [Mobile App] -> [API Gateway] [API Gateway] -> [Auth Service] [Auth Service] -> [Users DB] [Feed Service] -> [Posts DB x3] [Feed Service] -> [Redis Cache] Hit Apply — it auto-lays out the whole architecture in seconds. Export to animated GIF Export your diagram as a GIF that shows live traffic flow animations. Great for sharing after an interview or in a design doc. Tech stack React + TypeScript + Vite @xyflow/react (ReactFlow v12) for the canvas Zustand + Immer for state with full undo/redo html-to-image + gifshot for PNG/GIF export It's free and open No signup required. Works entirely in the browser. Free during beta. 👉 systemdesignboard.com Would love feedback — especially from anyone who's done system design interviews recently. What's missing? What's annoying? Drop a comment below

As software engineers, we often architect solutions in a virtual ideal: fast networks, elastic resources, and servers that never physically degrade. But what happens when your carefully crafted systems need to interact with the messy, unpredictable physical world? Think factory floor monitors, real estate camera networks, or remote tracking devices. Suddenly, those cloud assumptions about infinite uptime and perfect connectivity crumble. My journey, particularly architecting and maintaining a continuous 24/7 camera livestream for a real estate group over six years, has been a masterclass in this reality. It's revealed that true reliability in the physical realm demands a hybrid approach – one that intelligently merges the power of edge computing with the scalability and data insights of the cloud. This isn't just about connecting devices; it's about building resilience into the very fabric of your architecture. In this article, I'll share the battle-tested strategies and design principles that enable systems to not just survive, but thrive, despite the harsh realities of physical deployment. 1. The Core Strategy: Smart Edge, Simple Cloud One of the most common pitfalls in hybrid architecture design is treating the edge device as a mere 'dumb' terminal, solely responsible for streaming raw data to a powerful cloud backend. This approach creates a critical single point of failure: if the network drops, the entire system grinds to a halt. Instead, I advocate for a Smart Edge, Simple Cloud architecture. This principle establishes a clear division of responsibility: The Edge : This is where the magic happens locally. The edge system should be robust enough to handle local processing , data filtering , buffering , and immediate hardware control . Critically, it must be capable of operating autonomously for extended periods without an active cloud connection. Think of it as a mini data center, designed for self-sufficiency. Benefits of a Smart Edge : Reduced bandwidth cost

The Counterintuitive Truth: Feature Flags Are Not Config Files Most engineers first encounter feature flags as a simple abstraction: a key-value lookup that returns true or false. That mental model works fine for a single service handling a few hundred requests per minute. It becomes actively dangerous at scale. A mature feature flag system isn't a config file with an API wrapper — it's a distributed control plane . The distinction matters architecturally. A control plane manages the real-time behavior of a running system across many nodes simultaneously, with its own consistency guarantees, failure semantics, and propagation latency. That's a fundamentally different design problem than reading a YAML file on startup. One constraint drives every downstream decision: user traffic must never block on a remote flag service call. If evaluation requires a synchronous RPC, you've coupled your request path to the availability and latency of an external system. Netflix's Archaius library enforces this by evaluating flags entirely in-process against a locally-cached configuration snapshot. A network round-trip per evaluation injects 10–50ms of tail latency at p99 — catastrophic when you're competing on streaming start times measured in hundreds of milliseconds. Google, Meta, and Netflix collectively evaluate flags against millions of requests per second with sub-millisecond overhead. That figure is only achievable through local evaluation backed by an async synchronization layer, not RPC. The other failure mode engineers underestimate is flag sprawl . Systems accumulate flags the way codebases accumulate dead functions — gradually, then all at once. I've seen services carrying thousands of flags where fewer than 10% were actively managed. The operational weight alone becomes a liability: which flags are safe to remove? Which ones are kill switches for production behavior that no one documented? Knight Capital's $440M loss in 45 minutes in 2012 remains the canonical cautionar

Failures are obvious. Retries are sneaky. When something fails, everyone notices. An alert goes off. A request errors out. Someone starts investigating. Retries are different. They look harmless. Most of the time, they save the system. But sometimes, retries create bigger problems than the original failure. Imagine an API call times out. No problem. The system retries. But what if the first request actually succeeded and only the response was lost? Now the retry creates: duplicate orders repeated emails inconsistent records workflows running twice The failure happened once. The retry multiplied it. Another thing I've seen: One slow dependency causes requests to pile up. Retries start firing. Those retries create even more traffic. Which slows things down further. Which triggers even more retries. Suddenly, the system is spending more effort retrying than doing useful work. Retries also hide problems. A temporary issue gets retried five times and eventually succeeds. Everything looks normal. Meanwhile: latency increases queues grow users experience delays Nothing technically failed. But the system is getting less healthy. What changed for me is that I stopped treating retries as free. Every retry has a cost. It consumes resources. It increases load. And if actions aren't designed carefully, retries can repeat side effects that should only happen once. Now when I build something, I don't ask: "What happens if this fails?" I ask: "What happens if this runs again?" Because in production, things almost always run again. And if the answer is "bad things happen," the retry mechanism isn't helping. It's making things worse. Failures are part of every system. Retries are too. The difference is that failures usually happen once. Retries can turn one problem into hundreds if you don't design for them. This is something we think about constantly at BrainPack when operating long-running workflows across multiple systems. AI and automation layers make retries even more common, wh

Single-model systems are simple. Multi-model systems are powerful. The challenge isn't choosing models — it's designing the architecture that orchestrates them. A multi-model system isn't about having more models. It's about having the right model for the right task at the right time. Architecture patterns Five patterns cover most use cases: Pattern Complexity When to use Tradeoff Single Model Lowest Prototyping, simple tasks Limited capability Sequential Low Multi-step workflows Higher latency Parallel Medium Independent tasks Higher cost Hierarchical High Complex reasoning Complex orchestration Ensemble Highest Critical decisions Highest cost Pick the simplest one that works. Complexity is real, and it compounds. Sequential architecture Process tasks through a chain of models, each specializing in a step. Pattern 1: Pipeline Pipeline pattern — each model's output feeds the next: class ModelPipeline : def __init__ ( self ): self . models = [ { " model " : " qwen2.5-1.5b " , " task " : " classify " }, { " model " : " qwen2.5-7b " , " task " : " extract " }, { " model " : " qwen2.5-32b " , " task " : " reason " }, ] def process ( self , input : str ) -> str : current = input for model_config in self . models : current = self . call_model ( model_config [ " model " ], self . create_prompt ( model_config [ " task " ], current ) ) return current Latency adds up. Three models in sequence means three times the latency. Only use this when each step actually needs a different model. Pattern 2: Router Router pattern — classify the task, route to the specialist: class ModelRouter : def __init__ ( self ): self . classifier = " qwen2.5-1.5b " self . specialists = { " code " : " qwen2.5-coder-7b " , " math " : " qwen2.5-32b " , " creative " : " claude-sonnet-4 " , " general " : " qwen2.5-7b " , } def route ( self , prompt : str ) -> str : task_type = self . classify ( prompt ) model = self . specialists . get ( task_type , self . specialists [ " general " ]) return self . call_m

Building a high-performance Polymarket trading bot requires mastering its unique hybrid architecture:...

pgrac is an open attempt to rebuild Oracle RAC's core machinery (shared-everything storage, multiple active nodes all writing one database, a cluster-wide change number) on top of PostgreSQL 16. Build log #1 laid out the four problems that fight back. This one is about the problem that turns a node failure into silent data corruption, and the first, deliberately modest, layer pgrac ships against it. The failure mode In a shared-nothing cluster an evicted node is mostly harmless: it owns its own disks, so the cluster routes around it. In a shared-everything cluster the same event is dangerous, because every node writes the same storage. Picture the classic split: node 2 misses heartbeats, the cluster declares it dead and remasters its work elsewhere, but node 2 is not actually dead. It is frozen on a long GC pause, or its interconnect NIC flaked, and it is about to wake up and finish the write it started. Now two nodes believe they own the same blocks, and shared storage will accept both writes. That is not a crash. It is corruption you find three days later. Oracle RAC's answer is I/O fencing: before remastering a dead node's resources, you make certain it can no longer touch the storage, with STONITH, SCSI-3 persistent reservations, or a hardware watchdog. The node is fenced at a layer below its own software, because the whole point is that you cannot trust the dead node's software to behave. That hardware layer is real work, and it is not what pgrac built first. What it built first is the layer above it: an in-process cooperative write-fence, now default-ON. The rest of this is precise about what that does and does not buy you, because "we have fencing" is the kind of claim that is worth less than nothing if it is overstated. A fence needs an authority everyone can agree on You cannot fence on local opinion, because the whole problem is that the dead node disagrees about being dead. Authority has to live on durable, shared, quorum-backed storage. pgrac writes a sm

One of the most common system design interview questions is: "How would you scale a web application from 100 users to 100 million users?" The answer is rarely a single technology. Instead, systems evolve through multiple stages, with each stage solving a specific bottleneck. This article walks through the typical evolution of a scalable system and explains why , how , and when each component is introduced. 1. Single Server Why Start Here? Every application starts simple. In the beginning: Traffic is low Development speed matters more than scalability Infrastructure costs should be minimal What Is It? A single machine handles everything: Frontend Backend Database Users | v Single Server ├── Application └── Database How Does It Work? User sends request. Application processes request. Database stores and retrieves data. Response is returned. Everything happens on one machine. Problem As traffic grows: CPU becomes overloaded Memory becomes insufficient Database competes with application for resources A single server becomes a bottleneck. Interview One-Liner A single server architecture is simple and cost-effective but becomes a bottleneck as traffic and resource usage increase. 2. Application and Database Separation Why Do We Need It? The application and database have different workloads. Application Server: Uses CPU Handles business logic Database Server: Uses memory and storage Handles queries Keeping them together causes resource contention. How Does It Work? Move the database to a separate machine. Users | v Application Server | v Database Server Benefits Independent scaling Better resource utilization Improved performance Example Suppose an e-commerce website receives thousands of requests. The application handles: Authentication Order processing API responses The database handles: Product data Orders User information Separating them prevents one workload from affecting the other. Interview One-Liner Separating the application and database allows each layer to scal

When organizations scale, a simple monolithic architecture often becomes difficult to maintain, deploy, and scale. This is where microservices come into the picture. However, moving to microservices introduces new challenges: How do we migrate from a monolith safely? How do we handle transactions across multiple services? How do we scale read-heavy applications efficiently? Three popular patterns solve these problems: Strangler Pattern – Monolith to Microservices Migration Saga Pattern – Distributed Transaction Management CQRS (Command Query Responsibility Segregation) – Read/Write Scalability 1. Strangler Pattern Why Do We Need It? Most companies cannot shut down a production monolith and rewrite everything from scratch. A complete rewrite is risky because: Development takes a long time. Existing customers are affected. Bugs can impact business operations. Rollback becomes difficult. The Strangler Pattern allows teams to migrate gradually with minimal risk. What Is It? The Strangler Pattern is a migration strategy where new microservices slowly replace parts of a monolithic application until the monolith is no longer needed. The name comes from the strangler fig tree, which gradually grows around another tree and eventually replaces it. How Does It Work? [Insert diagram here showing Client → API Gateway → Monolith + Microservices] Step 1 All requests go to the monolith. Client | v Monolith Step 2 Introduce an API Gateway (or Controller). Client | v API Gateway | v Monolith Step 3 Extract one module into a microservice. Client | v API Gateway |------> Order Service | v Monolith Step 4 Gradually move more modules. Client | v API Gateway |------> Order Service |------> Payment Service |------> Inventory Service | v Monolith Step 5 Eventually remove the monolith completely. Example Consider an e-commerce application. Initially, everything exists inside one application: Monolith ├── Orders ├── Payments ├── Inventory └── Users Over time: Orders become Order Service Payme

Amazon Fulfillment: The Three Tiers of Optimization Amazon processes billions of orders annually through a network of over 175 fulfillment centers globally. To maintain their 1-2 day (or same-day) delivery guarantees, they built a 3-tier optimization architecture: ┌─────────────────────────────────────────────────────────────┐ │ TIER 1: ANTICIPATORY SHIPPING (Long-term — weeks/months) │ │ → ML predicts demand → Moves inventory close to customers │ │ BEFORE they place an order │ ├─────────────────────────────────────────────────────────────┤ │ TIER 2: REGIONALIZATION (Medium-term — days/weeks) │ │ → Partitions the fulfillment network into autonomous zones│ │ → Ensures 70-80% of orders are fulfilled intra-region │ ├─────────────────────────────────────────────────────────────┤ │ TIER 3: CONDOR (Short-term — hours) │ │ → Continuously re-optimizes the fulfillment plan within │ │ a 5-6 hour window before pick-and-pack begins. │ └─────────────────────────────────────────────────────────────┘ Anticipatory Shipping — Shipping Before You Buy A Crazy but Effective Idea Amazon holds a patent (US Patent 8,615,473) describing a system that begins shipping items BEFORE a customer places an order . It sounds like science fiction, but it's a reality. Traditional Model: Customer orders → Warehouse processes → Ships → Delivered (2-5 days) Anticipatory Shipping: ML predicts: "Customers in Region X will buy 200 iPhone 16s in the next 3 days" → Amazon ships 200 iPhones from a central hub to local delivery hubs in Region X → Customer places order → The item is already locally staged → Delivered same-day! ML Model Input Features Input Feature Significance Purchase history What do they buy, and how often? Browsing behavior What are they looking at? Cart abandonment? Wishlists Explicitly desired items Seasonal patterns Winter coats in November, sunscreen in June Regional demographics High-income areas? Young families? College towns? Trending products Items going viral on social media Weathe

Every non-trivial business operation touches more than one system. An e-commerce order reserves inventory, charges a payment method, and schedules a shipment — three services, three databases. A bank transfer debits one account and credits another across two ledgers that may not even be in the same data center. A cloud VM provisioning workflow reserves a network port, allocates storage, starts the hypervisor, registers billing, and sends a notification — five services, five independent state stores. The question is: what happens when step four fails after steps one through three have already succeeded? In a monolith backed by a single database, the answer is simple: roll back the transaction. The database engine guarantees atomicity; either everything commits or nothing does. But when your workflow spans multiple services, each owning its own storage, there is no transaction boundary that wraps them all. There is no rollback button. Step one through three have already made durable changes to systems that do not know about each other, and step four's failure has left the system in an inconsistent state. This is not a pathological edge case. It is the default condition in any distributed architecture. And it gets worse: the failure might not be a hard error. The network might time out. The billing service might return a 503. You do not know whether step four applied its effect or not — you only know you did not receive a success response. Now what? This is the problem sagas were designed for. Client Inventory Svc Payment Svc Shipping Svc │ │ │ │ 1 │──reserve(item)──►│ │ │ │◄──── 200 OK ─────│ │ │ │ [reserved ✓] │ │ │ │ │ │ 2 │──────────── charge(card, $99) ────►│ │ │◄───────────────── 200 OK ──────────│ │ │ │ [charged ✓] │ │ │ │ │ 3 │─────────────────────── schedule(order) ─────────────►│ │◄─────────────────────────── 503 ──────────────────── │ │ │ │ [no record ✗] │ │ │ │ ╔══════════════════════════════════════════════════════╗ ║ ⚠ Inconsistent state ║ ║ Inventory: it

Your AI agent remembered the user's name. Then it forgot what it was doing. Here's the setup: User asks the agent: book the cheapest flight to NYC, search hotels under $150/night, then compare total trip cost. By step 3, the agent calls the LLM with 8,000 tokens of raw conversation history — and still answers as if it's turn 1. You need a memory architecture before this ships. Which one do you pick? A) In-context window only — full conversation stays in the system prompt. Simple. Breaks at ~15 turns or 8K tokens, whichever comes first. B) Vector memory store — embed past turns, retrieve the top-k by semantic similarity at query time. Works great until "NYC flight" pulls a memory about a past NYC trip instead of the current task. C) Episodic memory with summarization — compress old turns into structured event summaries, inject the relevant ones per request. More complex to build. Much harder to confuse. D) Redis session state — structured key-value store, explicit agent reads/writes. Deterministic. Requires the agent to know what to store and when. One of these collapses past 15 turns. One retrieves the wrong context at exactly the wrong moment. One is the right answer for task-oriented agents. Pick A, B, C, or D — and tell me where you've hit this in production. Full breakdown in the comments.

A common way to design an application is to begin with its nouns: User Product Order Payment Then each noun receives the standard architectural starter pack: UserController UserService UserRepository The controller receives users, the service services them, and the repository stores them somewhere responsible. This is noun-oriented architecture : treating every important thing in the domain as if it were automatically a useful software boundary. It works for simple CRUD systems. Unfortunately, most applications eventually do something. The noun becomes a drawer Consider a typical UserService : register() findByEmail() resetPassword() changeAddress() disableAccount() mergeAccounts() assignRole() calculateDiscount() These operations all involve a user. That is approximately where their similarity ends. They have different rules, dependencies, side effects, security concerns, owners, and reasons to change. They live together because User was the nearest available noun when the folders were created. As more behaviour accumulates, UserService becomes the official location for anything vaguely user-shaped. Other components depend on it. It gradually depends on authentication, email, permissions, billing, auditing, and several services added during incidents nobody wishes to revisit. The noun becomes both a dependency of everything and a consumer of everything. The folder remains impressively tidy. Name the capability, not the material A better starting question is not: What things exist in this system? It is: What must this system be capable of doing? That leads to components such as: UserRegistrar PasswordResetter AccountMerger OrderPlacer PaymentRefunder SubscriptionCanceller These are agentive names . They name the component responsible for performing a capability. Compare: UserService with: PasswordResetter UserService tells us which noun is nearby. PasswordResetter tells us what the component is for. That difference produces better architectural questions: What rules

Rate Limiting and Circuit Breakers in Distributed AI Systems Distributed AI systems are inherently complex, handling massive volumes of requests, variable latency from model inference, and dependencies on external services like GPU clusters, databases, or third-party APIs. Without proper safeguards, a single misbehaving component or a sudden traffic surge can cascade into system-wide failure. Two fundamental patterns— rate limiting and circuit breakers —provide essential protection. This post explores their roles, implementation strategies, and practical Python examples tailored for AI workloads. Why Distributed AI Systems Need These Patterns Consider a typical AI pipeline: a user sends a prompt, which hits a load balancer, then an API gateway, then an inference service (e.g., a large language model), which may call a vector database or a fine-tuning API. Each component has capacity limits: GPU inference servers can handle limited concurrent requests. External APIs (e.g., OpenAI, HuggingFace) impose rate limits. Database connections are finite. Without rate limiting, a single abusive client can exhaust resources. Without circuit breakers, a failing downstream service can cause cascading timeouts and resource exhaustion across the entire system. Rate Limiting: Controlling Request Flow Rate limiting restricts how many requests a client, user, or service can make in a given time window. It prevents resource starvation and ensures fair access. Common Algorithms Algorithm Pros Cons Token Bucket Smooth burst handling, easy to implement Memory per bucket Leaky Bucket Constant outflow rate, simple Less flexible for bursts Fixed Window Simple, low overhead Boundary spikes (reset issues) Sliding Window Smoother than fixed, accurate Slightly more complex For AI systems, token bucket is often preferred because it allows short bursts (e.g., a user sending a batch of prompts) while maintaining a long-term average. Python Implementation: Token Bucket Rate Limiter import time impor