标签：#STEM

The database platform Harper advocates for a single-runtime architecture that keeps application code and data together, with its benchmark against a Vercel-based stack reporting significantly better performance on live, personalized-data workloads. Harper recently released version 5.2, with a new record cache and more throughput per node. By Renato Losio

Many beginner developers believe software engineering is all about mastering programming languages, framework syntaxes, and clearing error logs. In reality, writing code is only a fraction of the actual job. The true core of software engineering lies in analyzing complex domain problems, evaluating deep trade-offs, and designing robust systems that stand the test of time. Let's explore what it genuinely takes to transition from a coder to a modern software engineer with the right engineering mindset. 1. Writing Code vs. Solving Problems Anyone with a healthy brain can learn syntax and write functional scripts after a few tutorials. However, the real engineering challenge begins long before you touch your IDE. Understanding the Domain: Breaking down business logic and user requirements. Evaluating Alternatives: Assessing whether a feature needs a complex custom hook or a simple native state. Long-term Value: Building solutions that won't break when requirements shift tomorrow. 2. The Importance of Maintainability Code is read much more often than it is written. When you are working on large-scale applications, you are never coding alone—even if you are solo for now, your future self is essentially a stranger six months down the line. Crafting clean, self-documenting code with meaningful, intention-revealing names. Enforcing single-responsibility functions to keep modules decoupled. Using predictable patterns so teammates can navigate and scale the application without getting buried in technical debt. 3. Pragmatic System Design and Trade-offs There is no silver bullet in software engineering. Every architectural decision—whether choosing a database, state management library, or caching strategy—comes with heavy trade-offs. Performance vs. Development Speed: Knowing when to optimize early and when to ship MVP code. Scalability vs. Complexity: Avoiding over-engineering simple features just because a shiny new tool exists. Balancing Constraints: A great engineer evaluate

Prompt engineering is a conversation; context engineering is system architecture. In the early days of working with Large Language Models (LLMs), optimizing the prompt was enough for simple text generation tasks. But when you are building autonomous systems—like a self-hosted automation server connecting cloud databases, webhooks, and reasoning nodes—prompts alone will not keep track of APIs, past decisions, and strict output constraints. Think of the LLM as the CPU, and the context window as the RAM. Context engineering is the discipline of treating that memory as a scarce resource, meticulously designing the pipeline that feeds the model the exact facts, instructions, and tools it needs at the precise moment it needs them. The Four Core Strategies To shift from vibe-coding a chatbot to architecting a resilient multi-agent system, you must manage what enters and stays in the context window using four primary techniques: Select: Decide exactly which external sources—like database schemas or specific API documentation—enter the context window to maximize the signal-to-noise ratio. Compress: Shrink the context payload only after the key facts are successfully structured. Write: Persist the task state and intermediate decisions outside the active context window so the agent can retrieve them later. Think of this as giving the agent its own local-first markdown vault for networked thought. Isolate: Separate contexts when domains collide. Instead of forcing one model to do everything, build multi-agent systems where each agent receives a strictly scoped slice of the context. Navigating the Failure Modes Stuffing a massive context window with raw JSON logs and unstructured data is a recipe for disaster. When building complex workflows, you must engineer guardrails against these critical failure modes: Context Poisoning: Hallucinated or incorrect information enters the context and compounds over time because the agent continually reuses it. Context Distraction: The agent g

In the previous part of this series, we built a tiny supervisor from scratch. When a worker crashed, the supervisor started a replacement. That replacement had a new PID: old worker -> #PID<0.102.0> new worker -> #PID<0.105.0> This reveals an important limitation of sharing PIDs as a public interface. A PID identifies one running incarnation of a process. It is excellent for sending a reply, setting up a monitor, or creating a link. It is not a stable address for a service that may stop and later be replaced. In this part, we'll use named processes to give a worker a discoverable address: :worker We'll build three small examples using: Process . register / 2 Process . whereis / 1 :global . register_name / 2 :global . whereis_name / 1 send / 2 No GenServer . No OTP Registry . The goal is to understand the lookup problem that registries solve before reaching for those abstractions. The PID-Sharing Problem Suppose one process starts a worker and gives its PID to a client: worker = spawn ( fn -> worker_loop () end ) send ( client , { :worker_started , worker }) The client can now send work directly: send ( worker , { :work , self (), "hello" }) This works while that particular worker process is alive. But process IDs are temporary. If the worker exits, the PID is no longer a route to the service: Client Worker holds #PID<0.102.0> #PID<0.102.0> | | | X exits | | send(#PID<0.102.0>, work) |------------------------------> no worker receives it Sending to a dead local PID does not raise an error and does not restart a process. The message is simply not delivered to a living worker. One answer is to tell every client about every new PID after a restart. That spreads lifecycle knowledge throughout the system. Another answer is to make clients depend on a name and resolve that name when sending. Registering a Local Name Our first worker waits for a stop message: defmodule Worker do def start do spawn ( fn -> receive do :stop -> :ok end end ) end end Starting it gives us a PID:

Sam Newman discusses the concept of progressive collapse in civil engineering and how it applies to distributed systems. Using real-world examples - from the 1968 Ronan Point tower failure to AWS outages - he shares crucial resilience engineering strategies for software leaders. Learn how to strengthen components, isolate failures, and reduce interconnections to prevent catastrophic cascades. By Sam Newman

With the Azure Kubernetes Services (AKS) platform, containerized workloads can be efficiently managed and scaled. However, the full potential of AKS is only realized when it is seamlessly integrated with other Azure services. This enables a complete cloud-native environment that is scalable, secure, and automatable, while providing maximum flexibility. In this final post of the series, we will show you how AKS can be integrated with other Azure services to create a robust and holistic platform for your applications. Why Integrating AKS into the Azure Cloud Is Crucial AKS provides a highly available and scalable infrastructure for managing containerized applications. However, integrating it with other Azure services like Azure DevOps, Azure Monitor, Azure Active Directory (Entra ID), and Azure Storage extends functionality and optimizes workload management. By leveraging Azure services alongside AKS, companies can: Ensure enhanced security for their containerized applications. Build robust monitoring and logging solutions to monitor the state of applications at all times. Set up automated pipelines for deployment and scaling. Seamlessly exchange data and status information across various Azure services. Key Azure Services to Integrate with Your AKS Platform Azure Active Directory (AAD) for Authentication and Security Azure Active Directory (AAD) provides comprehensive identity and access management that can be directly integrated with AKS. This ensures that only authorized users and services can access your Kubernetes clusters. With Azure RBAC (Role-Based Access Control), you can define granular access permissions for different users and teams, increasing the security of your environment. AAD Pod Managed Identities enable your AKS applications to securely access Azure resources like Azure Key Vault or Azure Storage without the need to manually manage sensitive credentials. Azure DevOps for CI/CD Pipelines Azure DevOps is one of the best solutions for automating CI/CD

The Quest Begins (The "Why") I still remember the first time our API started choking under a sudden traffic spike. It was a Friday afternoon, the kind where you’re just about to log off, and the monitoring dashboard lit up like a Christmas tree. Requests were piling up, latency shot through the roof, and our users began seeing those dreaded “429 Too Many Requests” errors. We had a naive rate limiter in place—a simple fixed‑window counter that reset every minute. It worked fine when traffic was steady, but as soon as a burst hit, the counter would either let too many through (because we hadn’t hit the limit yet) or block everything for the whole minute (because we’d already exhausted the quota). It felt like trying to hold back a tsunami with a sandbag. Honestly, I was frustrated. I knew there had to be a smarter way to smooth out those bursts without penalizing honest users or over‑protecting the system. That’s when I dove into the world of rate‑limiting algorithms, and the token bucket caught my eye like a shiny loot drop in a dungeon. The Revelation (The Insight) The token bucket is deceptively simple, yet it solves the exact pain points we were experiencing. Imagine a bucket that holds a fixed number of tokens. Tokens drip into the bucket at a steady rate (say, 10 tokens per second). Each incoming request consumes a token. If the bucket is empty, the request is denied or delayed; if there’s a token, the request proceeds and the token is removed. Why does this beat the fixed‑window counter? Burst tolerance – The bucket can store up to its capacity, allowing a short burst of requests up to that limit without waiting for the next window. Smooth throttling – Because tokens are added continuously, the limiter adapts to the actual request rate rather than resetting abruptly at arbitrary intervals. Memory‑light – We only need to track two numbers: the current token count and the last time we refilled the bucket. No arrays of timestamps per key. Here’s a quick ASCII sket

A scheduler can be perfectly healthy and still run the wrong job at the wrong time. The failure is usually not the cron expression. It is the boundary between wall-clock time, monotonic elapsed time, leases, retries, and a process that may pause or restart. A reliable agent scheduler needs an explicit clock contract. Without one, a clock correction can make a job run twice, never run, or run after its authorization window has expired. The three clocks an agent should not conflate Use wall-clock time for human meaning and durable records: scheduled_at: when the user asked for the run not_before: the earliest acceptable dispatch time expires_at: the latest acceptable dispatch time Use a monotonic clock for elapsed-time decisions inside one process: lease renewal deadlines backoff timers watchdog intervals drain deadlines Use a database or provider sequence for ordering across processes: scheduler ownership fencing tokens attempt numbers reconciliation order A monotonic timestamp cannot be compared across hosts, and a wall-clock timestamp cannot safely measure a five-minute lease if NTP steps the clock backward. Store both kinds of evidence instead of pretending one timestamp answers every question. A small scheduling contract Here is a deliberately boring record shape: action: send_digest run_id: 01J... scheduled_at: 2026-08-19T08:00:00Z not_before: 2026-08-19T08:00:00Z expires_at: 2026-08-19T08:05:00Z lease_owner: worker-7 lease_token: 1842 attempt: 1 state: READY The important part is not the field names. It is the decision rule: The scheduler claims the run with a durable lease and fencing token. It checks wall-clock eligibility against not_before and expires_at. The worker checks that its lease token is still current before starting. The effect layer checks the token again before a side effect. If the outcome is ambiguous, record UNKNOWN and reconcile by the provider's idempotency key instead of blindly retrying. That last step matters after restarts. A clean rest

I've written before about SMESH, a coordination protocol modelled on mycorrhizal networks — the fungal web that lets trees in a forest warn each other about drought and disease with nothing in charge of the network. Signals diffuse, decay on their own, and get reinforced when independently confirmed. Consensus emerges instead of being orchestrated. That was the idea. This post is about the part where I found out whether it worked. The transport that had never run SMESH has had a QUIC transport in it for a while. Roughly 500 lines: a quinn endpoint that is simultaneously server and client, self-signed certs, length-prefixed bincode frames over unidirectional streams, an accept loop that spawns per-connection and per-stream tasks, connection pooling. Every test passed. The workspace was green. I could point at smesh-runtime/src/transport.rs and say "yes, it does peer-to-peer." Then I grepped for who actually constructed it: $ grep -rn "QuicTransport" --include = '*.rs' . smesh-runtime/src/transport.rs:177:pub struct QuicTransport { smesh-runtime/src/transport.rs:192:impl QuicTransport { smesh-runtime/src/lib.rs:16:pub use transport:: { QuicTransport, ... } ; Its own definition, and a re-export. Nothing else in the workspace had ever instantiated it. No binary opened a socket. SmeshRuntime imported TransportConfig , stored it in a struct field, and never looked at it again. I had a networking layer with tests, docs, and zero executions. Three bugs in the first twenty minutes I wrote an integration test that starts two runtimes, has one dial the other, and asserts a signal crosses. Here is what fell out before it went green. 1. It panicked on the first call. Could not automatically determine the process-level CryptoProvider from Rustls crate features. rustls 0.23 refuses to pick a crypto backend when more than one is compiled in, and quinn pulls in both through its own feature set. Every call to QuicTransport::new would have panicked for anyone, ever. Nobody noticed bec

"Nearby" sounds like a simple feature. Calculate the distance between the user and every location. Sort by distance. Done. In practice, that's not enough. A useful local discovery engine has to understand more than geography. That's one of the problems we're tackling with LeeX. The basic version A traditional nearby query might look like: User location ↓ Calculate distance ↓ Sort ascending ↓ Return results If Restaurant A is 500 meters away and Restaurant B is 2 kilometers away, Restaurant A wins. But what if Restaurant A is permanently closed? What if Restaurant B is much more relevant to the user's category? What if Restaurant B is currently featured? What if thousands of people have recently interacted with Restaurant B? Distance alone doesn't capture usefulness. Our discovery model We're thinking about discovery as a combination of signals: Discovery Score = Distance + Relevance + Activity + Popularity + Featured status + Availability + User context The exact weighting can evolve. The important part is that proximity is one signal, not the entire algorithm. Distance still matters We don't want to ignore geography. For local discovery, distance is extremely important. A user looking for a restaurant probably cares whether it is: 500 m 1 km 2 km 5 km 10 km That's why LeeX can expose radius-based discovery. But distance should normally be combined with other information. Category context Suppose someone opens LeeX and selects: Restaurants The discovery engine should not treat every listing equally. The system already knows the user's current intent. That gives us a stronger query: Nearby + Restaurant + Open + Relevant rather than: Nearby + Everything Featured listings LeeX also has a promotion layer. Featured listings can receive additional visibility across relevant discovery surfaces. But promotional ranking needs to be handled carefully. A featured listing shouldn't necessarily make every other result useless. Instead, we can think of featured placement as an ad

In this article, we'll explore the mechanisms to solve the coordination problem. 8. Introducing Leases To address the problem of permanent ownership, distributed systems typically replace it with temporary ownership. This concept is known as a lease . Instead of granting indefinite control over a resource, the coordination service assigns ownership for a limited period of time. Rather than stating, “You own this resource until you explicitly release it,” the system instead says, “You own this resource for the next 30 seconds.” This changes the interaction model significantly. Acquire Lease | v Execute Work | v Renew Lease | v Continue Processing As long as the application remains healthy, it periodically renews the lease to maintain ownership. If the application crashes or becomes unresponsive, it can no longer renew the lease. Once the lease duration expires, ownership is automatically revoked. At that point, another application becomes eligible to acquire the lease and continue the work. Leases solve a critical problem in distributed systems: they prevent abandoned locks from blocking progress indefinitely . The system can recover automatically without manual intervention. However, while leases improve availability, they also introduce a new class of subtle and more complex problems. Leases Depend on Time To understand the next challenge, assume the lease duration is thirty seconds. Application A successfully acquires the lease. Lease Granted Duration = 30 seconds After twenty seconds, the JVM begins a long Full Garbage Collection cycle. This pause lasts forty seconds, significantly longer than the lease duration. The timeline now becomes problematic. Lease Granted | | Processing | | GC Pause (40 sec) | | Lease Expires While Application A is paused, the lease expires. During this time, another application requests access to the same resource. The coordination service observes that the previous lease has expired and therefore grants ownership to Application B. Appl

The textbook version of the transactional outbox is tight. You save the domain entity and an outbox row in one local transaction. A background scheduler picks up PENDING rows and publishes them to Kafka. You never publish inside the request thread — no dual-write, no atomicity breach. The pattern closes the consistency gap. Then you load-test it. I ran 1,000 authenticated requests through my event-driven platform in 70 seconds. The gateway returned 201 for every one of them. The outbox absorbed every row. The consumer drained everything. By every visible metric the system looked healthy. Underneath that health, I found three production-grade problems the textbook never mentioned. What a correct implementation looks like Before the problems, the shape of the solution. The outbox publisher runs on a @Scheduled virtual-thread worker: @Scheduled ( fixedDelay = 5000 ) @Transactional public void publishPendingEvents () { List < OutboxEvent > batch = outboxRepository . findTop20ByStatusOrderByCreatedAtAsc ( OutboxStatus . PENDING ); for ( OutboxEvent event : batch ) { event . setStatus ( OutboxStatus . PROCESSING ); outboxRepository . save ( event ); try { kafkaTemplate . send ( event . getTopic (), event . getPayload ()). get (); event . setStatus ( OutboxStatus . PUBLISHED ); } catch ( Exception e ) { event . incrementRetryCount (); if ( event . getRetryCount () >= MAX_RETRIES ) { event . setStatus ( OutboxStatus . FAILED ); } else { event . setStatus ( OutboxStatus . PENDING ); } } outboxRepository . save ( event ); } } This is correct. The PROCESSING state prevents another scheduler instance from claiming the same row. The retry cap prevents infinite cycling. The PENDING fallback on transient errors gives the event another chance. The dual-write problem is genuinely closed. Here is what that correctness does not cover. Gap 1: Your throughput ceiling is a config line fixedDelay = 5000 means the scheduler runs every 5 seconds. findTop20 means it picks up 20 rows per cycl

A stored fact can be wrong in a quiet way. The answer still reads clean. A preference from an old exchange gets reused, the message goes out with confidence, and later nobody can tell why that detail was allowed back into the result. That is the failure I built around. When a system returns remembered material, the caller needs the text plus the reason it passed the reuse check. A log line found after the action is weak evidence. The object that leaves the memory service has to carry the admission record with it. 1. Keep the outside surface small This is the pattern I used in Holographic, Law-Bound Memory (HLM), a stand-alone memory brain outside application code. The README describes public Application Programming Interface (API) routes under /api/brain/* , with internal /api/v1/* services behind that layer. The outside shape is intentionally thin: register an agent, write a fact, build a capsule. The Python Software Development Kit (SDK) in sdks/python/hlm_sdk/client.py shows the boundary without exposing table names or policy code: import httpx class HLMClient : def __init__ ( self , base_url : str , token : str | None = None ): self . base_url = base_url . rstrip ( " / " ) self . _client = httpx . AsyncClient ( headers = { " Authorization " : f " Bearer { token } " } if token else None ) async def register_agent ( self , name : str ): r = await self . _client . post ( f " { self . base_url } /api/brain/agents/register " , json = { " name " : name }) r . raise_for_status return r . json async def write_fact ( self , text : str , tags : list [ str ] | None = None , selectors : list [ str ] | None = None ): r = await self . _client . post ( f " { self . base_url } /api/brain/memory/facts " , json = { " text " : text , " tags " : tags or [], " selectors " : selectors or []}) r . raise_for_status return r . json async def build_capsule ( self , query : str , budget_tokens : int = 2048 ): r = await self . _client . post ( f " { self . base_url } /api/brain/context/cap

Seed three events into an audit database, then reach past the application and change one row by hand: $ sqlite3 audit.db "UPDATE audit_log SET principal='attacker' WHERE id=2" $ trelix audit verify --db audit.db Audit chain TAMPERED — first divergent entry id: 2 $ echo $? 1 Delete the newest row instead and it still catches it, naming id 3, even though the surviving rows form a perfectly valid chain. Point it at something SQLite cannot open and it exits 2 rather than 0, because "I could not check" and "I checked and it is clean" must never collapse into the same green build. None of that existed six releases ago. trelix audit verify is one command out of six feature areas that landed in trelix v3.0.0, and it is the one that most changes what the project is for. What the major bump actually is The span from v2.11.0 to v3.1.1 is six releases — v2.11.1, v2.12.0, v3.0.0, v3.0.1, v3.1.0 and v3.1.1, the last of them dated 2026-08-15 — 68 commits, 137 files changed, +19,829/-1,211 lines. v2.11.0 closed out the Jira and Linear connector work, which has its own story. Everything after it is a different kind of release. v3.0.0 carries six new feature areas: Anthropic extended thinking, a model-aware context budget, a VS Code extension that acts instead of merely displaying, a hash-chained append-only audit trail, OIDC SSO, and query-conditioned context compression. Alongside them, an opt-in FTS5 declaration boost for keyword ranking. It is a major bump because of scope, not breakage. Every one of those six is additive and off by default: TRELIX_AUDIT_ENABLED=false , TRELIX_OIDC_ENABLED=false , TRELIX_LLM_THINKING_ENABLED=false , TRELIX_RETRIEVAL_COMPRESSION=false , declaration_boost_enabled False, and context_token_budget still the exact 12_000 integer it was in v2.12.0. A default v3.0.0 install assembles context byte-identically to a default v2.12.0 install, and there is a test that proves it rather than a release note that asserts it. An audit trail you can hand to somebody

Rate limiting sounds like a solved problem until you actually implement one and watch it fail in a way your load test didn't predict: legitimate bursts getting rejected, or a limiter that lets through 2x its stated limit at window boundaries. The failure modes are specific enough that it's worth working through the two dominant algorithms — token bucket and sliding window — with actual code, not just the diagrams. The problem with fixed windows The naive approach almost everyone reaches for first is a fixed window counter: pick a window size (say, 60 seconds), count requests in that window, reset the counter when the window rolls over. import time class FixedWindowLimiter : def __init__ ( self , limit : int , window_seconds : int ): self . limit = limit self . window_seconds = window_seconds self . count = 0 self . window_start = time . time () def allow ( self ) -> bool : now = time . time () if now - self . window_start >= self . window_seconds : self . window_start = now self . count = 0 if self . count < self . limit : self . count += 1 return True return False This is simple and cheap, and it's also broken in a specific, exploitable way. Say the limit is 100 requests/minute. A client can send 100 requests in the last second of window N, then another 100 in the first second of window N+1. That's 200 requests in roughly two seconds, well within the letter of "100/minute" as the code enforces it, but nowhere near the spirit of it. This is the classic boundary-burst problem, and it's the reason fixed windows get replaced once traffic is adversarial or bursty enough to find the seam. Sliding window: smoothing the boundary A sliding window log fixes this by tracking actual timestamps instead of a single counter, and counting how many fall within the trailing window at the moment of the request: from collections import deque import time class SlidingWindowLogLimiter : def __init__ ( self , limit : int , window_seconds : float ): self . limit = limit self . window_seco

TL;DR — I've been building the Social Resource Floor: an open blueprint for coordinating one person's access to basic survival resources — food, housing, energy, healthcare, and more — across many independent providers, so that reaching those resources is grounded in being human rather than in financial access. The first blueprint version is now complete: language-neutral schemas, prose specifications, a reference implementation, and a first adapter. This post is about the engineering choices behind it, and the reasons for each — how it stays a contract rather than a product, how it keeps personal data out of the coordination layer, why it binds to existing standards instead of inventing new ones, and how I check that the contracts are implementation-independent rather than just claiming they are. The problem the Floor is trying to help with Today, for most people, survival routes through financial access. To reach food, housing, energy, or healthcare you generally need money, and to hold or move money you need banking, employment, or purchasing power. Financial access has become the gate standing in front of the resources a person needs to stay alive. The goal of the Social Resource Floor is narrow and specific: to help make it so that financial status is not the condition that determines whether a person can reach the basic resources required to survive. It does not try to abolish money, banks, or markets — money stays a first-class resource and delivery method. It aims at one thing: a floor beneath which no person should fall, defined locally, reachable regardless of financial circumstances. That's the mission. Everything technical below exists to make that mission buildable by the institutions — governments, municipalities, NGOs, cooperatives, community providers — that would actually run it, without asking any of them to give up their own systems or hand over their data. Where the Floor sits The delivery systems for social protection already exist and are stron

Paperwork-heavy workflows rarely fail because a database cannot store another PDF. They fail because the system loses the relationship between the document, the real-world object, the decision it supports, and the stage of work it represents. Permits provide a useful example. A complete project record is not one uploaded form. It is an evidence chain that changes over time. A recent Local Service Ledger guide to Pasco County septic-repair records organizes the file into eight stages: property, existing system, site, pump-out, water and sewer, application, permit, and closeout. The guide's most important software lesson is that a receipt or contractor proposal alone does not establish the complete chain from reported problem to final recorded status. That distinction generalizes well beyond permits. 1. Give every workflow a stable subject Every document should attach to a stable entity: a property, customer, asset, case, project, or account. Do not rely on a filename or free-form address as the only identifier. Normalize enough data to prevent obvious duplication, preserve the source value, and retain a stable internal ID. For a property workflow, several records may contain slightly different owner names or address formatting. The system should help a reviewer determine whether they refer to the same site without silently overwriting those differences. 2. Separate observations, proposals, and decisions These are different kinds of facts: an owner reports a symptom; a contractor proposes a scope; an authority authorizes specific work; an inspector records a result; a final status closes the file. Collapsing them into one “project description” field destroys provenance. Model the actor, date, source, and status of each statement. The interface can display the current operational summary while preserving the earlier language that explains how the record evolved. 3. Make state transitions explicit A reliable workflow should not infer completion because a document exists

While mass-producing web tools with an AI, I've changed how I lock the design in three stages. The previous post I wrote about that got this comment: "I'd like to see the JSON approach and the design-system approach side by side." Taken at face value, I should just put the two side by side. But first, let me add a short preface. I don't want to frame this as "the JSON approach versus the design-system approach." When I called the JSON approach a "failure" in that post, I didn't mean the method is inferior; I meant it didn't suit my particular set of tools. A page made with the JSON approach does look thin. But where that thinness comes from is easily misread. Whether the design drifts and whether it looks rich are decided separately. What stops the drift is locking the design; whether it looks rich is how much you build out. What locking with JSON removes is drift in the items you specified in the schema. Whether the screen becomes rich, on the other hand, is determined by how much you've built out the machinery that turns that JSON into a screen. So it isn't that locking with JSON is what made it look like a spreadsheet. In the previous post, too, I wrote that fattening the schema and the renderer does increase the expression itself. But that came with a caveat: past a point, it heads toward rebuilding HTML and CSS by hand. What I really want to check is one step past that. If the template sets the ceiling on expression, then building out the JSON side's template as much as the current one should produce the same screen. So what does that build-out demand? I actually built it and measured. I'll share the result, along with the JSON-approach and design-system-approach screens placed side by side under matched test conditions. I'll admit up front: at the time, I chose the design system without running this comparison. So this is me building the road I didn't take, after the fact, and measuring what that cost consists of. Same order, same one-shot So that the comparis

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry . You Know What's Worse Than Not Being Able to Log In? Being told everything worked right up until you try to actually use your account. Yes, that was a real bug. And, somehow, I ended up being pulled into another authentication mystery. At this point, I’m starting to think authentication bugs have a personal grudge against me. 😅 In my previous Smash Story , I wrote about a bug where users simply couldn't log in. This time, the problem was sneakier because most of the flow looked completely healthy. The user was approved, the background task ran, the email and SMS arrived, and Cognito had a user. Then the user actually tried to use their account. And everything fell apart. It Started With Two User Pools The authentication setup was fairly large and had evolved over time, so there wasn't one shiny User Pool doing everything. We had an older Cognito User Pool supporting existing authentication flows, including mobile-based signup, while a newer User Pool handled a newer flow where users received an email containing their PIN. Both pools were intentional because they supported different parts of the authentication journey. That wasn't the problem. The interesting part was that the application database had its own representation of a user, while Cognito had another. On top of that, some of the work connecting those two systems happened asynchronously. As long as everyone agreed about who the user was, nobody cared. The moment they disagreed, authentication became very interested. The Tiny Timing Window The problem appeared in the partner and dependant journey. A member could create a partner or dependant during signup or later from the member details area. A relevant non-member user would then approve the account, which scheduled an asynchronous task called SendingEmailsAfterApprovalBot in a TaskList database table. That task ran every 15 minutes, and once it executed, the partner or depend

Imagine this: It's salary day. It's 2 PM. Millions of people across India suddenly open their UPI apps and start paying rent, sending money to family, paying credit-card bills, and shopping online. Now here's the system-design interview question: If millions of people make payments at almost exactly the same time, is every request hitting one central server? What prevents the entire payment system from freezing? At first glance, it sounds like a scaling problem. It isn't just a scaling problem. It's a combination of: horizontal scaling concurrency distributed systems database consistency retries idempotency backpressure failure isolation downstream bottlenecks And that's what makes payment systems such an interesting system-design problem. First: Don't Imagine One Giant UPI Server A common mental model looks like this: Millions of users | v +-------------+ | UPI Server | +-------------+ | v Bank If that were literally true, we'd have a pretty serious problem. One machine cannot safely process the country's entire payment traffic. Instead, think about a distributed system: Users | v +---------------+ | API / Gateway | +---------------+ / | \ / | \ v v v [S1] [S2] [S3] | | | +------+------+ | Payment Services | +--------+--------+ | | Bank A Bank B The exact implementation of a real payment network is much more complicated than this diagram, but this is the right system-design mental model . The important idea is: The system is distributed across many machines and participating institutions. Step 1: The First Problem — Traffic Spikes Let's take a concrete example. You want to pay your landlord: ₹25,000 At the same moment, millions of other people are doing something similar. Suddenly: Normal traffic: 100K requests/sec Salary day: ████████████████████████ 1M+ requests/sec The first question is: How do we handle the additional traffic? Naive Solution: One Powerful Server We could buy a massive machine. 1M requests/sec | v +---------------+ | HUGE SERVER | | 256 CPU core