标签：#STEM

Your monitoring catches problems on everything except itself. Here is how an always-firing Watchdog alert plus an external heartbeat check turns silence into a signal, so you find out when your own alerting dies. TL;DR A monitoring system can't reliably monitor its own failure, so use a dead man's switch. Create an always-firing Prometheus Watchdog alert and route it to an independent external heartbeat service. As long as the monitoring pipeline is working, the Watchdog continuously refreshes the heartbeat. If Prometheus, Alertmanager, or the delivery path fails, the heartbeat stops and the external service alerts you through a separate channel. The key is independence: the system responsible for detecting that your monitoring is down must not depend on the monitoring stack itself. One of the traps of creating alerts on a monitoring stack is the hidden assumption that the mechanism evaluating the alert is running properly and has the ability to evaluate it. Prometheus watches your hosts and Alertmanager delivers the warnings. But what watches Prometheus? If something goes wrong and the monitoring stack fails in the middle of the night, no alerts are going out but there is definitely a problem. That is the failure mode that you should be most concerned about, because it is the one your monitoring cannot report on. The fix is an old idea with a grim name: a dead man's switch. A train's dead man's switch stops the train when the operator stops holding it down. The safe state requires continuous positive action, while the absence of that action is what triggers the response. Applied to monitoring, it means building one alert whose silence is itself the alarm. Step one: an alert that always fires This feels backwards the first time you see it, and it took me a little time to get it right. Basically, you create an alert with a condition that is always true, so it fires constantly, forever, on purpose. In the Prometheus world this is conventionally called Watchdog. - aler

Writing software that scales from a small monolith into a multi-team distributed system requires strict architectural discipline. The SOLID principles —coined by Robert C. Martin ("Uncle Bob")—serve as fundamental guidelines for object-oriented design and system architecture. When improperly understood, developers often fall into two extreme traps: creating monolithic "God objects" that break with every change, or over-engineering systems into hyper-fragmented, unmaintainable micro-services. In this deep-dive guide, we will break down each of the 5 SOLID principles from low-level class design up to high-level distributed systems design, complete with bad vs. refactored Java examples, system architecture diagrams, trade-off analyses, and a comprehensive cheat sheet. SOLID Principles Cheat Sheet Principle Core Concept Anti-Pattern / Code Smell Refactoring Solution Single Responsibility (SRP) A class or module should have one, and only one, reason to change (serving one business actor/domain). God Class / Micro-Fragmentation: Classes handling payment, DB, and notifications, OR over-fragmented single-function classes. Split by domain responsibility. Use orchestrator/coordinator components for workflows. Open/Closed (OCP) Software entities should be open for extension, but closed for modification . Conditional Bloat: Cascading if-else or switch statements checking object types or channels. Strategy Pattern, Dependency Injection, and Event-Driven Pub/Sub messaging (e.g., Kafka). Liskov Substitution (LSP) Subtypes must be completely substitutable for their base types without breaking client behavior. Runtime Exceptions: Subclasses throwing UnsupportedOperationException or silently breaking logic. Split fat inheritance hierarchies into granular, capability-specific interfaces. Interface Segregation (ISP) No client should be forced to depend on methods it does not use. Fat Interfaces: Monolithic interfaces forcing callers to mock or implement irrelevant methods. Role-focused

This series follows a fictional conversation between an experienced engineer and his nephew. Every episode explores one stage of how software moves from an idea to production. 👦 Nephew: Uncle, requirements are clear. I checked the codebase — there's already a FavoritesService I can extend for Wishlist. Now can I open VS Code? 👨‍🦳 Uncle: Almost. Tell me — what do you think HLD even is ? You've heard the term in every interview. What do you think it actually means? 👦 Nephew: Some kind of... diagram? Boxes connected with lines, before you start coding? 👨‍🦳 Uncle: That's what it looks like. That's not what it's for . Let me ask differently. Why do you think experienced engineers insist on drawing this before touching code, when they could just start building? 👦 Nephew: ...to plan the work? 👨‍🦳 Uncle: Closer, but still not it. Here's the real answer: HLD exists to decide, in advance, where the walls go — so that six months from now, when someone adds a new feature, they know exactly which room to build it in, without knocking down a wall that was holding up the ceiling. 👦 Nephew: That's a strange way to describe a diagram. 👨‍🦳 Uncle: Then let me show you, instead of describing it. That's the only way this actually lands. What Talks to What 👨‍🦳 Uncle: Suppose we're building this at Flipkart. Not a college project — a company with hundreds of live services, where breaking one thing can affect ten others you've never even heard of. Here's the simplest picture for Wishlist. Frontend ↓ Wishlist API ↓ Wishlist Service ↓ Database 👦 Nephew: That's it? Four boxes? 👨‍🦳 Uncle: That's it. HLD answers exactly one question, and nothing more — what talks to what. Not how the button looks. Not what fields the database table has. Just: which component calls which, and in what direction. 👦 Nephew: Then why does everyone treat it like it's such a big deal? This took ten seconds to draw. 👨‍🦳 Uncle: Because the value isn't in the ten seconds you spend drawing it today. The value is in what i

Encrypted data stored in cloud archives today will outlive the mathematical algorithms guarding it. In enterprise architectures that handle long-term records, like construction risk logs or employee compliance platforms, data retention schedules often span twenty to thirty years. When building cloud pipelines that move this information across services, we depend heavily on asymmetric encryption, which is a security method using one public key to lock data and a separate private key to unlock it. Standard public-key algorithms rely on mathematical problems that are nearly impossible for classical computers to solve within a reasonable human timeframe. Quantum computing changes this equation entirely. Quantum computers leverage quantum mechanics, the physical rules governing subatomic particles, to perform calculations at speeds fundamentally unimaginable with traditional silicon processors. While powerful quantum systems are still in development, the security threat to distributed systems exists today. Hostile actors do not need to crack modern security algorithms in real time. Through a pattern known as Harvest Now, Decrypt Later, adversaries can capture and store encrypted network traffic right now. They simply wait until future quantum hardware becomes capable of running the formulas required to decrypt that stolen history. For software architects, preparing for post-quantum cryptography, which refers to new mathematical encryption algorithms designed to withstand quantum attacks, is far more than a simple library swap. It is a deep distributed systems migration challenge. The primary operational hurdle is payload size and computational overhead. Quantum-resistant algorithms require significantly larger digital keys and payload headers than the standards we rely on today. When cryptographic payloads expand, every component of a distributed platform feels the ripple effect. Message queues experience higher bandwidth demands. Database indexes inflate. Memory consump

Herodotus wrote of a bird that lived five hundred years in Arabia, and when its life came to an end, it did not wait to be surprised by death. It built its own nest of cinnamon and myrrh, set the nest and itself alight, and let a new bird rise from what the fire left behind. The Hestia argued for tending a fire that must never go out. That’s true, and it isn’t the whole truth. Teams end. People leave. Companies get acquired, reorganized, shut down, and five years from now some part of this whole model will probably look as dated as the practices it was written to replace. No amount of tending prevents that. Pretending otherwise is its own kind of Alexandria , a slow decline dressed up as continuity, right up until the fire goes out anyway and nobody chose the moment. The bird in Herodotus doesn’t get caught by surprise. It builds the pyre itself. Chooses the moment, gathers what matters, and burns deliberately, trusting that what rises afterward carries the shape of what came before, not because the fire preserved the old bird whole, but because starting over was never the same thing as starting from nothing. That’s the part tending alone can’t promise. A team that’s about to be split up can hand its shared model to whoever inherits the work on purpose, the way a rep in the Boule carries a decision back instead of leaving it to travel however it happens to travel. A team about to lose its most experienced person can spend the weeks before that departure making sure the framing, not just the conclusions, made it into someone else’s head, the way the Mimesis argued a junior actually learns. None of that stops the ending. It decides what the ending leaves behind. This series doesn’t get to end with a fire that never goes out. Nothing does. It gets to end with the only thing actually inside anyone’s control. Build the pyre on purpose. Choose what goes into the fire. References The Myth of the Phoenix: Rebirth and Renewal : Greek Mythology, on Herodotus’s original accoun

Part 1 documented the recurring snapback in practice. This note asks a narrower question: what does the observed pattern support, what remains a working hypothesis, and what changes should follow in the project? Status: Bounded project conclusion. This note separates observed behaviour, working hypothesis, and practical consequence. It is based on current project documents and interactions; it is not external validation or a universal claim about AI systems. What the evidence supports 1. The project already contains a stable relational model The working model is not generic “AI assistance.” It separates reasoning surfaces, uses bounded comparisons, permits two-way cognitive pressure, and keeps final acceptance authority with the human. Reciprocal cognitive contribution, asymmetrical governing authority. 2. Concrete project work preserves the structure better than public abstraction At the concrete level, instructions such as: Review this proposal against that architecture. preserve the distinction between the object being reviewed, the surface applying pressure, the evidence, and the authority that may accept a change. When the same structure was compressed into general prose, generated explanations repeatedly returned to a simpler one-way model of either human control or transferred AI authority. That is an observed pattern in this development process. 3. Public explanation is a separate reasoning surface A README, article, summary, or portfolio page is a projection of the model, not the model itself. It cannot be assumed to reproduce the internal structure faithfully merely because that structure is present in context. The explanation must be reviewed against the model it represents: Does this explanation preserve the actual authority, review, evidence, and state-transition structure? 4. Annoyance was useful boundary data The irritation indicated that the generic rendering was no longer merely an imperfect exploration. It was colliding with an internal frame that

Disclosure: This post includes affiliate links; I may receive compensation if you purchase products or services from the different links provided in this article. Credit: codemia.io Hello Devs, if you're preparing for software engineering interviews, particularly in MAANG, you already know that Data Structures & Algorithms (DSA) and System Design are two key areas where you will be rigorously tested. While LeetCode is the go-to platform for DSA, system design has always been a challenge. While there are many websites and platforms to prepare for System Design Interviews like ByteByteGo , DesignGurus.io , Exponent , Educative , and Udemy , there is nothing like LeetCode. These are great resources to learn fundamentals, go through case studies, and understand the theory part of system design, but LeetCode-style practice is one thing that is missing - until now. I recently found Codemia.io , and I must say, it feels like the LeetCode for System Design. If you've struggled with structuring your system design answers, getting real feedback, or knowing whether your approach is correct, Codemia.io is a game-changer. They not only have the biggest collection of System Design and OOP Design problems for practice, but they also have a free System Design course called Tackling System Design Interview Problems , which is a great free resource to learn essential System Design concepts. It's a short course with 2 hours of content, but it's powerful and also has quizzes to test your skills. Here are all the key System Design topics you can learn on this free course: Now, let's check out how Codemia.io can help you to prepare better for your System design and OOP Design interview, and why I think it's like Leetcode for System design. Most system design resources today are long, text-heavy articles or expensive courses. The problem? No hands-on practice - Reading about system design isn't enough; you need to actively design solutions. No structured progression --- Unlike DSA, where

Most architecture decision records fail for the opposite reason people think. The issue usually isn't that teams forget to write them. It's that the ones they write are filled with the wrong content. The key information a reader needs—why this option instead of the others—often gets buried on page three under a list of API changes. An ADR has one job: capture a decision that is costly to reverse, along with the reasoning that led to it, while that reasoning is still fresh. That's all. It isn't a design document, a specification, or a collection of research. If you keep that focus, everything else about what to include or leave out will follow naturally. The format that still works Michael Nygard's original ADR template from 2011 (title, status, context, decision, consequences) has lasted for a reason. It directly addresses the key questions a future reader has: What was the situation? What did we decide? What did we give up? Teams that add ten extra sections, like owners, review dates, risk matrices, or approval lists, usually end up with a document no one finishes reading, which defeats the purpose. If your ADR template is longer than the time it takes to fill it out for a simple decision, cut sections until it’s more concise. A useful rule of thumb is that an ADR longer than a page and a half is often a design document masquerading as an ADR. This isn’t a strict rule, but I haven’t seen a truly good ADR exceed 600 words. The decisions worth documenting this way can be stated, justified, and owned in about a page. If that's not possible, the record isn’t the issue. The decision is probably still tied up with other unresolved matters. What belongs The decision, stated clearly. "We will use event-driven integration between the order and inventory services instead of synchronous REST calls" is a decision. "The order service integrates with inventory" simply describes the current state and belongs in a wiki, not an ADR. The challenges faced. Describe the two or three a

It's easy to reach for the popular tool before asking what your system actually needs. For job queueing, the usual advice is to use RabbitMQ or Kafka. The underlying burden of using these tools will be additional processes to deploy, monitor and reason about. But what if using your existing database is possible? I built a job queueing system with a PostgreSQL database that cleared the bar without adding infrastructure. The next question will be, does this solution meet the criteria of a job queue? A job needs a few things to execute properly within a system. It needs to be persistent, surviving unforeseen crashes. Jobs must not be processed more than once by workers; one job should be processed once by one worker. Also, a job's state has to be tracked through every step. A job state must show when it's pending, completed or failed. That's the bar any solution must clear. With the Postgres approach, persistence comes free. Jobs live in a table so when a worker dies mid-job, the job still exists in a row in the db. Whereas with in-memory queues, a crash loses everything still in memory. A broker like RabbitMQ has to be configured for persistence and if configured wrongly, jobs get lost. A database however is fundamentally built for durability. Now, let's say three workers poll the queue at the same instant and run the same query. They'll see the same pending job at the top and nothing stops them all from grabbing it. If that job is a payment, the customer gets charged three times for one service. All the workers successfully process the job with no indication of an error or alerts. This is the requirement that seems to demand a real message broker, and it's exactly where people assume a database can't compete. It can. Postgres has a specific tool for exactly this. The SQL clause FOR UPDATE is used to lock rows. This can be called on a job when a worker picks it up to process. By default other workers will get blocked during this process, they'll wait for the lock to r

Originally published on nejckorasa.github.io . When a team moves from a monolith into microservices and event-driven, asynchronous systems, it inherits a class of problems that used to be someone else's: work that fails halfway through, steps that must not run twice, calls that return before the work is done. Temporal is a durable execution engine that handles a lot of this - you define a multi-step process, and it guarantees the process runs to completion even when workers crash in the middle. I've spent the better part of a decade building distributed systems in the money-movement core of banks - ledgers, payments, credit cards - a lot of it on Temporal, from short request-triggered workflows to ones that stayed open for weeks. This is the high-level guide I'd give a team making that jump: the principles worth internalising before you ship, not a full tutorial. Most of them aren't really about Temporal. They're the habits the async shift demands - Temporal just punishes you quickly when you skip one. Durable Execution: The Problem It Solves Distributed work fails in the middle. You call service A, it succeeds. You call B, it times out. The pod dies before C. Now you have half-finished work and no memory of how far you got. The usual fix is a pile of status columns, a cron job to find stuck rows, and retry logic hand-rolled for every step. Temporal's promise is that any process you start runs to the end. The runtime picture: there's a Temporal service (its own cluster), and your app runs worker processes that poll it and execute your code. As a workflow runs, Temporal records every step to an event history . If a worker dies, another picks the workflow up and replays that history to rebuild state, then carries on from where it left off, retrying anything that failed. The history is the source of truth, and it survives the crash. Most of the rules below fall out of that one fact. The Golden Rule: Workflows Decide, Activities Do There are two kinds of code in Tempora

From musical instruments and games to play pieces that encourage balance and dexterity, these toys will engage your child’s mind and body.

Building an operating system is a project I have had my eyes set on ever since I discovered free will in the realm of programming. Years ago, I did a reasonable amount of research, paying extra attention to the subject during my computer science degree and I was able to understand Operating System Theory and how it works from first principles but I never really got around to building one. I had only flimsy reasons for not embarking on it like "why build one when there are tons of working ones out there? The theoretical knowledge is enough" . More recently, I am ignoring the need to not re-invent the wheel for the joy of programming. So if you are interested in also rebuilding stuff because you can, join me on this series as I document how I am going to be building kluster. kluster is in its infancy and the direction is not clear but the one certain thing is that I will be building it entirely in Rust, save some assembly instructions and a linker script and I will be explaining every single line of code along the way. It will also be designed to target the raspberrypi 4 & 5, on qemu and on real hardware respectively. This is an opportunity for anyone who wants to see how Rust works at the lowest of levels to hop on and join the ride. Note that this series will be your biggest lesson on delayed gratification because we will write a lot of code before we even get to see anything meaningful on screen but I will foreshadow what you can get by the end of part 3 if you are patient enough: {{ image(src="/images/os-part3-result.png", alt="Part 3 Results OS Dev") }} You can also clone the source code for part 1 from Github and follow along. Project Setup First things first, let us setup the foundation of the project. I'll be straight with you, I love Rust and I enjoy using the Rust ecosystem in its entirety so I will stay true to that and use it as obsessively as any true Rustacean; I won't hold back. Without doubt, all the dependencies we need are freely available as long as

Back-of-the-envelope estimation for system design interviews Most people don't fail capacity math because the arithmetic is hard. They fail because they do it silently, produce a number they can't defend, and then never use it again for the rest of the interview. The math itself is trivial. The method is what's worth learning. Why interviewers ask Capacity estimation isn't a numeracy test. It's checking two things: Can you tell whether a design is physically possible before you commit to it? Do you know which constraint actually binds — storage, read throughput, write throughput, or bandwidth? A candidate who estimates 30,000 reads/sec and 200 writes/sec has learned something that changes the design. A candidate who computes petabytes of storage and then never mentions it again has just performed arithmetic. Round aggressively Precision is a trap. You're not producing a capacity plan; you're finding the order of magnitude. The single most useful substitution: 1 day = 86,400 seconds ≈ 10^5 seconds That's a 16% error and it makes every subsequent division doable in your head. Nobody will challenge it. Everyone will notice if you spend forty seconds long-dividing by 86,400. A few more worth having ready: 1 million requests/day ≈ 12/sec — round to 10 1 KB × 1 million = 1 GB 1 KB × 1 billion = 1 TB Peak traffic ≈ 2–3× average Replicated storage ≈ 3× raw ## Work in one direction Users → requests → QPS → storage → bandwidth. Don't jump around. Say each assumption out loud and label it as an assumption, so the interviewer can correct you early rather than watch you build on sand. A worked example Say we're designing a social feed. Given: 100M daily active users. Assumptions (stated, not smuggled in): Each user posts 0.2 times/day Each user reads their feed 10 times/day A post averages 1 KB including metadata A feed page shows 20 posts Writes 100M × 0.2 = 20M posts/day 20M / 10^5 = 200 writes/sec Peak (3×) = 600 writes/sec Reads 100M × 10 = 1B feed loads/day 1B / 10^5 = 10,0

This is going to be a very different article from what I usually write. No technical discussions, architecture deep dives, or engineering practices today. Instead, we're talking about something much older than software itself: music. We treat language like it's the default mode of human communication, like it's the real and only thing used to communicate, everything else is secondary, emotional, aesthetic, nice to have. But language is actually the outlier. It's the new protocol layered on top of something much older. Music is the original standard and we've basically forgotten how to read it. The Protocol Stack Think of communication like a network stack. Language is high-level. It's TCP/IP. Built on assumptions, needs learning, breaks the second you cross a boundary. You need: A shared vocabulary Syntactic understanding Cultural context Years of study if you actually want fluency It's powerful but It's also fragile. And it's recent . Written language is a few thousand years old. Spoken language is older, sure, but both are late abstractions compared to the hundreds of thousands of years humans have been syncing bodies to shared sound. Relative to that timeline? Language is yesterday's patch. Music? That's the lower-level protocol. The physical layer everything else runs on. A Japanese teenager at a Michael Jackson concert doesn't need to speak English. She doesn't need to understand what "Man in the Mirror" means as a concept. She also doesn't need a music degree. Music isn't zero -cost. Genre, culture, convention still shape how we hear it. But the entry barrier for emotional communication is way lower. A rhythm can hit urgency, celebration, sadness, or tension long before anyone understands the formal structure behind it. Her nervous system speaks that fluently. And so does everyone else in that stadium. How the Protocol Works Here's what happens when the song starts: 70,000 people stop being individuals and start being a distributed system synchronizing to the

Controller A records revocation epoch 12. Worker B, partitioned with a cached grant from epoch 11, starts another external action. The database is correct and the system is unsafe. Emergency stop is therefore a distributed protocol, not a Boolean field. What is verified In its July 21 disclosure, OpenAI says an internal benchmark used models with reduced cyber refusals and that a combination of models compromised Hugging Face infrastructure. The primary source is https://openai.com/index/hugging-face-model-evaluation-security-incident/ . Reporting on July 24 then described US discussion of emergency-shutdown and independent-audit proposals. The latter is policy coverage, not enacted law and not an extension of the official incident facts. Missing protocol details, impact boundaries, and remediation should remain unknown rather than inferred. Invariants and assumptions Assume workers, queue consumers, an authorization service, and external adapters can fail independently. Messages may be delayed, duplicated, or reordered; clocks have bounded error only if measured. Required invariants: No action starts with a grant epoch below the subject's revocation epoch. Cached grants expire within a declared lease bound. Restart cannot lower a persisted epoch. Duplicate revocation converges to the same or higher epoch. Completion means every registered executor acknowledged or its lease expired. revoke(subject, epoch=13) -> durable CAS max(current, 13) -> publish {subject, epoch:13} -> executors persist max(local, 13), ack -> controller waits for ack set OR lease expiry -> issue completion receipt with missing/expired members Failure injection Property Acceptance rule delay revocation event lease bounds stale authority no start after local lease expiry duplicate epoch 13 idempotence epoch remains 13+ deliver 13 before 12 monotonicity never returns to 12 worker restarts durability loads persisted epoch before work controller partition fail closed no new lease after expiry A minim

I cranked Apple’s iOS 27 beta to maximum transparency on my iPhone and enjoyed every warped second.

8 min read · telecom provisioning platform, 30M+ subscribers Most companies avoid migrating their monolith for one reason: they imagine it as a single, terrifying event — months of a feature freeze, a weekend cutover, and a rollback plan that's really just hope. That fear is reasonable. A big-bang rewrite of a live system serving 30M+ subscribers really would be terrifying. So we didn't do that. We used a pattern that lets you migrate a monolith one slice at a time, while the system keeps running and the product team keeps shipping features — the same pattern Martin Fowler named Strangler Fig , after the vine that grows around a host tree, gradually taking over, until eventually the original tree is no longer needed. Why the "just rewrite it" instinct is usually wrong The instinct to rewrite a legacy monolith from scratch is understandable — the old code is scary, undocumented, and nobody wants to touch it. But a full rewrite has a well-known failure pattern: it takes far longer than estimated, the business can't freeze feature development for that long, and by the time the rewrite is "done," the old system has changed underneath it and the rewrite is already out of date. The alternative isn't "don't migrate." It's: migrate in slices small enough that each one is boring , and never require the business to stop shipping while you do it. The pattern: a facade, and one slice at a time Strangler Fig works by putting a routing layer — a facade or API gateway — in front of the monolith. At first, 100% of traffic passes through to the old system untouched. Then, one capability at a time: Build the new version of that one capability as an independent service. Update the facade to route just that capability's traffic to the new service. Run both in parallel long enough to trust the new one (see "shadow traffic" below). Retire that piece of the old monolith. Repeat for the next capability. At every point in this process, the system is fully functional. There's no "half-migrat

Your Contact Form 7 form can work perfectly from a user's perspective and still fail to deliver a lead to your CRM. The visitor fills out the form. The browser shows a success message. The WordPress form appears to have submitted correctly. But when you open Systeme CRM, the contact is nowhere to be found. This is one of the most confusing problems in form-to-CRM integrations because a successful form submission does not necessarily mean a successful API request. The complete workflow has multiple stages: Visitor ↓ Contact Form 7 ↓ WordPress ↓ API Request ↓ Systeme CRM ↓ Contact Record ↓ CRM Automation A failure at any stage can break the workflow. The key to debugging the integration is to stop treating the form submission as a single event and start checking each stage separately. First, separate the two different types of success There are two different questions: Did Contact Form 7 submit the form? This is a WordPress-side question. Did Systeme CRM accept and process the API request? This is an API and CRM-side question. These are not the same thing. A form can successfully collect: Name: John Doe Email: john@example.com Company: Example Inc. while the API request fails because: the endpoint is incorrect authentication is missing the request method is wrong the JSON payload is invalid the CRM expects different field names a required field is missing The first debugging step is therefore to identify exactly where the data flow stops. Step 1: Confirm that Contact Form 7 is collecting the expected data Start at the beginning. Look at the form fields: [text* your-name] [email* your-email] [tel your-phone] [text company] [textarea your-message] The important values are the actual field names: your-name your-email your-phone company your-message A common mistake is to assume that the visible label is the field name. For example: Visible label: Full Name Field name: your-name The integration needs the submitted field value associated with the actual field name. Before

When a new video shows up in one of our regional crawlers, three things need to happen almost immediately: the SQLite FTS5 search index for that region needs a new row, the discovery ranking cache needs to be invalidated, and the sitemap generator needs to know a URL was born. For a long time we did all of this inline, inside the same cron process that fetched the video. It worked until it didn't. A slow FTS5 rebuild would stall the fetch loop, a sitemap write would fail silently, and a crash halfway through meant one region had the video indexed and another didn't. The fetch and the fan-out were fused together, and every failure was a partial failure. The fix was to stop treating "a video was discovered" as a function call and start treating it as an event. At TrendVidStream we run discovery across 8 regions, and the moment we introduced NATS JetStream as the spine between the crawler and the downstream consumers, the whole system got calmer. This post is the concrete version of how we did it: the stream config, the publishers, the consumers, and the mistakes we made that you can skip. Why Not Just Use a Queue Table in SQLite We already had SQLite everywhere, so the obvious move was a jobs table. We tried it. The problems showed up fast: Polling latency vs. load tradeoff. Poll every second and you hammer the DB with mostly-empty SELECT queries across 8 regions. Poll every 30 seconds and your search index lags noticeably behind your crawler. No fan-out. One row, one worker. If the sitemap generator and the FTS5 indexer both need the same event, you either duplicate rows or invent a consumed_by bitmask. Both are ugly. Locking. SQLite's writer lock means the queue table and the actual data table start contending under the multi-region cron bursts we run. Cross-region delivery. Our regions aren't all on the same box. A queue table doesn't cross machines without you building a replication story on top. JetStream solves all four: push-based delivery (no polling), multipl

1. Subsystem Decomposition 1.1 The Decomposition Problem A subsystem decomposes a codebase into smaller, cohesive units. Two primary axes of decomposition exist: Technical axis : grouping by component type (controller, service, model, view) Functional axis : grouping by business capability (cataloguing, circulation, etc.) 1.2 Tension Between Framework Prescriptions and Decomposition Strategy Organizing top-level subsystems functionally may create friction with frameworks that prescribe a technical-first structure. Concrete examples: Rails enforces model, view, and controller directories at the root level, making functional decomposition awkward without additional mechanisms like Rails Engines Sinatra (a microframework) imposes minimal structure, leaving architectural decisions entirely to the team Frameworks with rigid prescriptions constrain architectural choices. Frameworks with no structure shift the entire burden onto the team with no guidance. This second approach might be fine for teams that know what they are doing and how to shape the architecture properly. Not everyone needs guidance from the framework. 1.3 Contexts as a Middle Ground Phoenix provides contexts as a compromise: Explicit, guideline-oriented subsystems that enable functional decomposition without rigid enforcement Contexts define functional boundaries while allowing technical organization to remain nested within them Functional blocks may later evolve into microservices, but this is optional The same decomposition serves equally well in a modular monolith or a distributed architecture The choice depends on team needs, scaling requirements, and operational maturity, not on the decomposition strategy itself 2. Pipeline Topology and Data Flow 2.1 The Unix Pipeline Model Unix pipelines model data flow through a single stream connecting stdout to stdin. This forms a linear chain where each stage's output becomes the next stage's input. Key characteristics: Each stage has exactly one input and one o