标签：#systems

Originally published on nejckorasa.github.io . When a team moves from a monolith into microservices and event-driven, asynchronous systems, it inherits a class of problems that used to be someone else's: work that fails halfway through, steps that must not run twice, calls that return before the work is done. Temporal is a durable execution engine that handles a lot of this - you define a multi-step process, and it guarantees the process runs to completion even when workers crash in the middle. I've spent the better part of a decade building distributed systems in the money-movement core of banks - ledgers, payments, credit cards - a lot of it on Temporal, from short request-triggered workflows to ones that stayed open for weeks. This is the high-level guide I'd give a team making that jump: the principles worth internalising before you ship, not a full tutorial. Most of them aren't really about Temporal. They're the habits the async shift demands - Temporal just punishes you quickly when you skip one. Durable Execution: The Problem It Solves Distributed work fails in the middle. You call service A, it succeeds. You call B, it times out. The pod dies before C. Now you have half-finished work and no memory of how far you got. The usual fix is a pile of status columns, a cron job to find stuck rows, and retry logic hand-rolled for every step. Temporal's promise is that any process you start runs to the end. The runtime picture: there's a Temporal service (its own cluster), and your app runs worker processes that poll it and execute your code. As a workflow runs, Temporal records every step to an event history . If a worker dies, another picks the workflow up and replays that history to rebuild state, then carries on from where it left off, retrying anything that failed. The history is the source of truth, and it survives the crash. Most of the rules below fall out of that one fact. The Golden Rule: Workflows Decide, Activities Do There are two kinds of code in Tempora

Building an operating system is a project I have had my eyes set on ever since I discovered free will in the realm of programming. Years ago, I did a reasonable amount of research, paying extra attention to the subject during my computer science degree and I was able to understand Operating System Theory and how it works from first principles but I never really got around to building one. I had only flimsy reasons for not embarking on it like "why build one when there are tons of working ones out there? The theoretical knowledge is enough" . More recently, I am ignoring the need to not re-invent the wheel for the joy of programming. So if you are interested in also rebuilding stuff because you can, join me on this series as I document how I am going to be building kluster. kluster is in its infancy and the direction is not clear but the one certain thing is that I will be building it entirely in Rust, save some assembly instructions and a linker script and I will be explaining every single line of code along the way. It will also be designed to target the raspberrypi 4 & 5, on qemu and on real hardware respectively. This is an opportunity for anyone who wants to see how Rust works at the lowest of levels to hop on and join the ride. Note that this series will be your biggest lesson on delayed gratification because we will write a lot of code before we even get to see anything meaningful on screen but I will foreshadow what you can get by the end of part 3 if you are patient enough: {{ image(src="/images/os-part3-result.png", alt="Part 3 Results OS Dev") }} You can also clone the source code for part 1 from Github and follow along. Project Setup First things first, let us setup the foundation of the project. I'll be straight with you, I love Rust and I enjoy using the Rust ecosystem in its entirety so I will stay true to that and use it as obsessively as any true Rustacean; I won't hold back. Without doubt, all the dependencies we need are freely available as long as

This is going to be a very different article from what I usually write. No technical discussions, architecture deep dives, or engineering practices today. Instead, we're talking about something much older than software itself: music. We treat language like it's the default mode of human communication, like it's the real and only thing used to communicate, everything else is secondary, emotional, aesthetic, nice to have. But language is actually the outlier. It's the new protocol layered on top of something much older. Music is the original standard and we've basically forgotten how to read it. The Protocol Stack Think of communication like a network stack. Language is high-level. It's TCP/IP. Built on assumptions, needs learning, breaks the second you cross a boundary. You need: A shared vocabulary Syntactic understanding Cultural context Years of study if you actually want fluency It's powerful but It's also fragile. And it's recent . Written language is a few thousand years old. Spoken language is older, sure, but both are late abstractions compared to the hundreds of thousands of years humans have been syncing bodies to shared sound. Relative to that timeline? Language is yesterday's patch. Music? That's the lower-level protocol. The physical layer everything else runs on. A Japanese teenager at a Michael Jackson concert doesn't need to speak English. She doesn't need to understand what "Man in the Mirror" means as a concept. She also doesn't need a music degree. Music isn't zero -cost. Genre, culture, convention still shape how we hear it. But the entry barrier for emotional communication is way lower. A rhythm can hit urgency, celebration, sadness, or tension long before anyone understands the formal structure behind it. Her nervous system speaks that fluently. And so does everyone else in that stadium. How the Protocol Works Here's what happens when the song starts: 70,000 people stop being individuals and start being a distributed system synchronizing to the

Controller A records revocation epoch 12. Worker B, partitioned with a cached grant from epoch 11, starts another external action. The database is correct and the system is unsafe. Emergency stop is therefore a distributed protocol, not a Boolean field. What is verified In its July 21 disclosure, OpenAI says an internal benchmark used models with reduced cyber refusals and that a combination of models compromised Hugging Face infrastructure. The primary source is https://openai.com/index/hugging-face-model-evaluation-security-incident/ . Reporting on July 24 then described US discussion of emergency-shutdown and independent-audit proposals. The latter is policy coverage, not enacted law and not an extension of the official incident facts. Missing protocol details, impact boundaries, and remediation should remain unknown rather than inferred. Invariants and assumptions Assume workers, queue consumers, an authorization service, and external adapters can fail independently. Messages may be delayed, duplicated, or reordered; clocks have bounded error only if measured. Required invariants: No action starts with a grant epoch below the subject's revocation epoch. Cached grants expire within a declared lease bound. Restart cannot lower a persisted epoch. Duplicate revocation converges to the same or higher epoch. Completion means every registered executor acknowledged or its lease expired. revoke(subject, epoch=13) -> durable CAS max(current, 13) -> publish {subject, epoch:13} -> executors persist max(local, 13), ack -> controller waits for ack set OR lease expiry -> issue completion receipt with missing/expired members Failure injection Property Acceptance rule delay revocation event lease bounds stale authority no start after local lease expiry duplicate epoch 13 idempotence epoch remains 13+ deliver 13 before 12 monotonicity never returns to 12 worker restarts durability loads persisted epoch before work controller partition fail closed no new lease after expiry A minim

I cranked Apple’s iOS 27 beta to maximum transparency on my iPhone and enjoyed every warped second.

When a new video shows up in one of our regional crawlers, three things need to happen almost immediately: the SQLite FTS5 search index for that region needs a new row, the discovery ranking cache needs to be invalidated, and the sitemap generator needs to know a URL was born. For a long time we did all of this inline, inside the same cron process that fetched the video. It worked until it didn't. A slow FTS5 rebuild would stall the fetch loop, a sitemap write would fail silently, and a crash halfway through meant one region had the video indexed and another didn't. The fetch and the fan-out were fused together, and every failure was a partial failure. The fix was to stop treating "a video was discovered" as a function call and start treating it as an event. At TrendVidStream we run discovery across 8 regions, and the moment we introduced NATS JetStream as the spine between the crawler and the downstream consumers, the whole system got calmer. This post is the concrete version of how we did it: the stream config, the publishers, the consumers, and the mistakes we made that you can skip. Why Not Just Use a Queue Table in SQLite We already had SQLite everywhere, so the obvious move was a jobs table. We tried it. The problems showed up fast: Polling latency vs. load tradeoff. Poll every second and you hammer the DB with mostly-empty SELECT queries across 8 regions. Poll every 30 seconds and your search index lags noticeably behind your crawler. No fan-out. One row, one worker. If the sitemap generator and the FTS5 indexer both need the same event, you either duplicate rows or invent a consumed_by bitmask. Both are ugly. Locking. SQLite's writer lock means the queue table and the actual data table start contending under the multi-region cron bursts we run. Cross-region delivery. Our regions aren't all on the same box. A queue table doesn't cross machines without you building a replication story on top. JetStream solves all four: push-based delivery (no polling), multipl

The pricing formulas in Motor, the estimating engine I built for a water feature shop, did not come from the manual. I pulled 32 of them out of the JavaScript behind Aquascape's contractor calculator, the tool contractors actually use to bid jobs. The manual was sitting right there, official and free. Ignoring it was the best design decision in the whole system. A vendor never ships a sloppy calculator Why trust the calculator over the manual? Because of what happens when each one is wrong. If the manual sizes a pump wrong, a reader shrugs and moves on. If the calculator sizes a pump wrong, a contractor bids a job at that number, wins it, and loses money on the install. Then the phone rings. So calculators get fixed and manuals drift. Give it ten years and the two quietly disagree, and everyone in the trade knows which one to trust without anyone saying so. A vendor will ship a sloppy PDF. They will never ship a sloppy calculator. Documentation is what a domain says about itself. The artifacts money flows through are what it actually believes. Once you see that split, you cannot stop seeing it. The other half was in old invoices Formulas only get you to cost. What a shop charges on top of cost is a belief about its market, and no vendor document holds that number. So I pulled 132 historical quotes out of the shop's CRM. Real quotes, sent to real customers, most of them paid. I calibrated Motor's markup against those, then checked its output against what the shop had actually charged. The result: Calibrated against 132 real quotes, Motor's estimates landed within 5 percent of what the shop actually charged, with no pricing rule taken from documentation. I could have just asked the owner what his markup was. But what an owner says and what his invoices show are rarely the same number, and the invoices are the ones customers paid. When the two disagree, believe the invoices. The same bug in a different industry I build and run systems in several industries, and the sur

Before a young Athenian took his full place in the city, he spent two years in the ephebeia. Training happened in the gymnasion, organized by tribe, the same tribes that would later send him to represent them in the Boule itself. Nobody handed him full standing first and hoped the judgment would follow. This should have been post seven. It’s showing up as sixteen because the gap only became visible once the room was real enough to test against. The Agora described what a Spec Session does. It never asked whether everyone walking into that room shares an accurate picture of what the agent can actually do. Most rooms don’t. Someone watched a demo and thinks the agent can do anything. Someone else got burned by a bad output three weeks ago and doesn’t trust it with anything real. Nobody’s intuition has been tested against the same tasks, and the spec that comes out of that room ends up too ambitious or too conservative depending on whose untested belief happened to speak first. That gap has a name in Athens. The gymnasion existed because nobody was handed a place in the city first and expected to develop judgment on the job. The ephebeia ran two years, training built around a specific fact. Physical readiness and civic judgment weren’t taught in separate places. They happened in the same space, under the same supervisors, organized by the same tribal groupings that would later structure how the city actually governed itself. The same word, gymnasion, ended up naming both the training ground for eighteen-year-olds and the buildings where Plato and Aristotle did their most serious thinking. Academy and Lyceum were gymnasia first. Nobody separated the trial from the reflection. The trial was how the reflection got earned. That’s the part worth taking seriously. Testing a tool and understanding a tool were never two different activities. The testing is how the understanding gets built. A team that reads documentation about what an agent can do has a description. A team tha

In this article, we're going to explore Event Schema evolution with Event versioning 10. Event Schemas Will Eventually Change No event schema stays the same forever. As businesses grow, regulations shift, products gain new features, and processes become more complex, the data shared between services must evolve as well. This evolution is not optional—it is a natural consequence of a system adapting to changing requirements. Many teams initially assume they can simply update an event whenever needed. This assumption may hold when there is only one producer and one consumer, but real-world systems rarely remain that simple. Over time, multiple consumers emerge, each with its own responsibilities and release cycles. A typical system often looks like this: OrderConfirmed | +------------------+-------------------+ | | | v v v Inventory Billing Notification | v Analytics | v Customer Insights Each consumer evolves independently. Some services may deploy updates weekly, while others might release changes quarterly. In some cases, consumers may even belong to external teams with entirely different priorities and timelines. Because of this, producers cannot assume that all consumers will upgrade simultaneously. Schema evolution, therefore, is not just about modifying data structures. It is fundamentally about maintaining compatibility across independently evolving systems. Compatibility Is More Important Than Version Numbers When discussing schema evolution, teams often focus immediately on versioning. While versioning is useful, compatibility is far more critical. Without compatibility, versioning alone cannot prevent system breakage. Consider the following event: { "orderId" : "ORD-1001" , "customerId" : "CUS-501" , "totalAmount" : 249.99 } Now imagine a new requirement introduces currency. One approach might replace the existing field entirely: { "orderId" : "ORD-1001" , "customerId" : "CUS-501" , "amount" : { "value" : 249.99 , "currency" : "USD" } } Although the data mo

I want to start with a moment most of us have lived through. It's 3 a.m. A dashboard is red. You're eight terminals deep in grep , trying to work out which service actually fell over and why. And the whole time there's this nagging feeling that you're doing archaeology on a system you wrote last month. Here's what got under my skin about it. The answer was never actually lost. Back in the source code it said, in plain terms, that the payment service talks to Postgres through a connection pool. That this retry backs off three times. That this particular dependency is external and you must never, ever try to "just restart it." That was all right there at build time. Then we packaged everything up, deployed, threw that structure in the bin, and asked a sleep-deprived human to reconstruct it from log lines. That gap bugged me enough that I spent a while building something around it. This post is about that. Autoscaling is good at the wrong problem We've gotten genuinely good at reacting to resource pressure. Traffic climbs, a box gets slow, CPU pins, and the autoscaler adds capacity or sheds load. No complaints there, it's kept things running for years. The problem is it has no idea what your app is for . It can't tell a service that's slow because it's healthy and hammered from a service that's fast because it's quietly writing garbage to the database. It never had a model of the application in the first place. So a whole category of failures just sails right past it. A connection pool getting drained by something downstream. A poison message kicking off a retry storm. A schema change that breaks one code path and leaves the other one looking perfectly fine. Infrastructure that only thinks in CPU and memory is blind to all of that. The "throw an LLM at it" era The going answer right now is to bolt a large language model onto your observability stack. Fire hose all the logs, traces, and metrics at a big central model and ask it what happened. I get why. I also think it'

Metron was the Greek word for a measure: the standard you judge a thing against. It gives us metric, and metronome. Pick the wrong metron and everything you count against it comes out wrong, no matter how carefully you count. Ask most teams how they know AI adoption is working, and the answer is some version of: we’re shipping more. More PRs, more tickets closed, more velocity on the board. None of those numbers were ever measuring what people thought they were measuring. They were proxies, and bad ones, for whether the team was creating value. AI didn’t break that. It just made the proxies lie louder. An agent can produce PRs faster than any team can review them. It can close tickets all day. None of that tells you whether the thing built was worth building, or whether it fixed the actual problem, or whether anyone downstream is better off. Burning tokens isn’t the same thing as creating value. It just looks the same on a dashboard built to reward the first thing. The same DORA numbers I pointed at earlier are blunt about it: individual output climbs while delivery throughput and stability drop. Same bottleneck, counted at the wrong end . This is Theory of Constraints in one sentence: speeding up a step that wasn’t the bottleneck doesn’t speed up the system. It just piles more work in front of whatever the real bottleneck is. Implementation used to be the constraint. Now it isn’t. Review is. Coordination is. Making sure everyone agrees on what “done” even means is. Agents made the fast part faster and left the slow part exactly where it was, except now it’s buried under more output arriving to be reviewed by fewer people who understand any of it. Measuring individual speed after that shift is measuring the wrong clock. The team can be faster and worse off in the same quarter. The instinct, when this gets noticed, is to add another skill to the agent. Automate the review step too. Automate the coordination. Whatever’s slow, throw a capability at it. That doesn’t fix

Xona aims to deploy 258 satellites into low-Earth orbit as a GPS alternative.

"We're using Kafka with exactly-once semantics, so we don't have to worry about duplicates." I've heard this in architecture reviews, design docs, and postmortem explanations. It represents a misunderstanding of what Kafka's exactly-once guarantee actually covers, and the gap between the promise and the reality has caused real production incidents. What Kafka's Exactly-Once Actually Covers Kafka's exactly-once semantics (EOS), introduced in 0.11.0, operates at two levels: Producer idempotence ( enable.idempotence=true ): The producer assigns each message a sequence number. The broker deduplicates messages with the same producer ID and sequence number. This prevents duplicates caused by producer retries — the message lands in the Kafka partition exactly once, regardless of retry count. Transactions ( transactional.id ): Allows a producer to write to multiple partitions atomically. Either all writes commit or none do. Combined with isolation.level=read_committed on consumers, readers only see committed transactions. Together, these give you exactly-once message delivery within the Kafka cluster. What Exactly-Once Does Not Cover Here's the boundary that engineers miss: Kafka's exactly-once guarantee is scoped to the Kafka cluster. The moment your consumer does anything outside Kafka — writes to a database, calls a REST API, publishes to a cloud queue — you're outside the transaction boundary. Consider a typical consumer: consumer . poll ( records ); for ( record : records ) { database . save ( process ( record )); // External write — outside Kafka transaction } consumer . commitSync (); If the application crashes after database.save() but before commitSync() , Kafka re-delivers the message. The consumer reprocesses it. The database now has two writes for the same event. Enabling producer idempotence on the consumer's Kafka writes does not fix this. The Patterns That Actually Give You End-to-End Safety Idempotent Consumers Design consumer processing logic to be idempote

The Future of Rust: Why This Memory-Safe Language is Dominating Systems Programming in 2026 In its early years, Rust was often viewed as a "rising star"—a promising language with significant potential but a steep learning curve. As we navigate through 2026, that narrative has fundamentally shifted. Rust has transitioned from a niche interest to a cornerstone of modern, memory-safe infrastructure. The language is no longer just proving its worth; it is defining the gold standards for systems programming, cloud-native backend services, and kernel development. This deep dive explores the key pillars driving Rust's evolution, its massive ecosystem growth, and the roadmap for the years ahead. Seamless Ownership: The New Era of Rust Ergonomics One of the most significant shifts in recent Rust development has been the relentless focus on language ergonomics. Historically, developers occasionally felt that custom smart pointers were "second-class citizens" compared to built-in references. The "Beyond the & " initiative has successfully bridged this gap. Through advancements in Smart Pointer Parity , developers can now use custom pointers—such as Rc , Arc , or specialized interop pointers—with the same intuitive syntax and capabilities as standard references. Furthermore, the introduction of sophisticated field projection mechanisms and &own references allows for unprecedented precision in managing ownership and borrowing. This makes complex data structures much easier to implement and reason about, drastically reducing the friction typically associated with high-level abstractions. The Async Revolution: Making Asynchronous Code Natural Rust has achieved a milestone often referred to as "Async Parity," aiming to make asynchronous programming feel as natural and seamless as synchronous code. Several key developments have fueled this revolution: Async-in-Traits: The stabilization of async fn in traits has removed the need for external crates like async-trait , greatly simplify

Changing the model for a running AI task is not a settings update. It is a distributed operation: read current task -> prepare credentials/config -> request restart -> receive result -> persist active model If two switches overlap, completion order can differ from request order. The system needs a rule for which intent wins. The concrete case At commit c58bcd4 , MonkeyCode records model-switch attempts with from/to model IDs, request ID, load-session flag, success, message, session ID, and timestamps in TaskModelSwitch . The reviewed task use case creates a switch record, asks taskflow to restart with the target model configuration, and completes the switch record and task model based on the response. The accompanying tests cover success and failure paths. From this source review, I could not establish an explicit compare-and-swap generation or a per-task serialization contract around overlapping requests. That does not prove an exploitable race: serialization may exist elsewhere in the deployment or taskflow boundary. It means concurrency semantics deserve an explicit test and contract. Why last completion is unstable Assume request A selects model A, then request B selects model B: time -> A: request ---- restart ---------------- complete B: request -- restart -- complete If each successful completion writes its model, B applies first and late A overwrites it. Reverse network timing and the result changes. The companion simulator makes that order dependence visible: export function naiveCompletionOrder ( completions ) { let model = " initial " ; for ( const completion of completions ) { if ( completion . success ) model = completion . model ; } return model ; } [A, B] ends on B. [B, A] ends on A. The caller's latest intent is not part of the rule. Add a monotonic generation Assign a generation while accepting each request: A -> generation 41 B -> generation 42 Completion may update active state only when its generation equals the task's current requested generatio

DoorDash details the architecture behind Ask DoorDash, its AI-powered conversational shopping assistant, combining LLMs, specialized AI agents, MCP-based tooling, and an intelligence layer with persistent consumer memory and live backend data. Early results show up to 24% higher checkout conversion, 17% larger baskets, and improved intent accuracy using memory-backed sessions. By Leela Kumili

Episode 1 established the mindset: failure is normal, not a sign of bad engineering. Episode 2 gets specific — you can't detect or handle a failure you can't even name. Saturday, Round 2 👦 Nephew: Uncle, last time you convinced me failure is basically guaranteed. Fine, I accept it. So what actually fails ? 👨‍🦳 Uncle: You tell me. Start listing things that could go wrong in your app right now. 👦 Nephew: Uh... the server could crash. The database could go down. My code could have a bug. 👨‍🦳 Uncle: Keep going. 👦 Nephew: The network? Someone could deploy the wrong thing? Payment gateway dies mid-checkout? 👨‍🦳 Uncle: You just named six of the seven categories without trying. You already know this. You've just never sorted it. 1. Hardware Failure 2. Software Failure 3. Network Failure 4. Database Failure 5. Third-Party Failure 6. Human Error 7. Resource Exhaustion 👦 Nephew: Then why do we need the list at all, if I already know it instinctively? 👨‍🦳 Uncle: Because "instinctively" isn't fast enough at 2 AM. Let's trace each one properly. Part 1 — Hardware Failure 👦 Nephew: This one's obvious anyway — I deploy to AWS. The cloud hides hardware failure from me. 👨‍🦳 Uncle: Does it? 👦 Nephew: ...doesn't it? That's the whole point of paying for EC2 instead of buying a server. 👨‍🦳 Uncle: Let's trace it. Your app sits on an EC2 instance. What's underneath the instance? 👦 Nephew: Virtual machine stuff, I guess? 👨‍🦳 Uncle: And underneath that ? 👦 Nephew: ...an actual physical machine somewhere. In a data center. 👨‍🦳 Uncle: There it is. Your app | "Virtual" server (EC2/Droplet) | ACTUAL physical hardware somewhere in a data center | Still capable of failing — just less visible to you 👦 Nephew: So it's not hidden. It's just one layer further away than I thought. 👨‍🦳 Uncle: Exactly. AWS absorbs a lot of it — that's part of what you're paying for — but disks still fail, instances still get abruptly terminated, whole availability zones still go down. That's Hardware Failure . Hardware Fa

When I wrote about shifting the engineering process left — spec sessions, autonomous agents, humans reviewing output rather than writing code — a question kept coming up. Where does the Product Owner fit in all of this? It’s the right question. And I think the answer is more interesting than “the PO disappears.” Let’s start with acceptance criteria. We invented them to bridge a gap. The team needed to know when something was done. The PO needed confidence that what got built matched the intent. Acceptance criteria were the contract between the two. But if the Spec Session is where intent gets defined — by the whole team, together, before the agent runs — that gap closes. What the team agreed on in the room is the definition of done. The spec is the acceptance criteria. You don’t need a separate validation step because the planning and the agreement happened at the same time. The tighter the loop, the less ceremony you need around it. There’s a caveat though. The spec is a necessary contract. It’s not a sufficient one. Simon Martinelli’s work on the AI Unified Process validates the spec-driven approach technically. But his model is about the artifact — requirements at the center, AI generating everything else from them. How the team actually builds shared understanding before the spec exists isn’t something it addresses. That’s not a criticism. It’s just a different question. A spec written after a real Spec Session — where the team worked through edge cases together, disagreed, got to resolution — is different from a spec written by one person and signed off asynchronously. Same artifact. Different quality of shared understanding. That distinction matters when the agent hits an edge case the spec didn’t anticipate. So what’s actually left for a dedicated PO? Two things. And they’re very different. The first is product thinking — challenging intent, representing user needs, asking why before the agent runs with something. That’s valuable. But it doesn’t require a ded

An astrolabe doesn’t map every star. It gives you a way to find your position relative to the ones that hold still. That’s the instrument I reach for when someone asks which AI tool they should be using. The honest answer is that the tools will be different in six months. The layers won’t. I spent a week trying to make sense of a handful of names that kept showing up in the same conversations. Tessl . Goose . Archestra . Kestra . Modelplane . RAG , MCP , half a dozen others orbiting nearby. Each one has its own pitch, its own funding round, its own reason it’s the thing you should adopt next. Taken together they read like noise. Taken apart, they sit on different floors of the same building. The agent loop again, the one I keep coming back to. Once you place each tool on a floor, the noise turns into a map. Tessl sits left of the loop , at the intent layer. Turn a spec into something an agent runs against directly. This is the one tool on the list that pushes back instead of going along with it. A well-formed spec is not the same thing as a team that agrees on what the spec means. The Agora produces the second thing as a byproduct of producing the first. Tessl produces the first and assumes the second follows. It doesn’t, automatically. That’s the whole argument. RAG and MCP are plumbing. Protocol, not position. They carry context into the loop and don’t take a side in any argument about who should be in the room when the spec gets written. They’re also the one floor with an actual standard. MCP, A2A , ACP , all under Linux Foundation governance now, joint working groups, cross-protocol commitments. Passing data between systems is a solved problem with decades of precedent behind it, so it standardized almost on contact. Nothing else on this floor plan has that. Governance, orchestration, the harness, the spec layer: every vendor is still building its own version and calling it the obvious one. The standard showed up first at the floor that mattered least to this ar

Here are five architectural lessons we've learned from designing software for modern retailers.* Designing ERP Software for Retail: Five Lessons Every Software Engineer Should Know When people hear the word ERP , they often think of accounting software, dashboards, or inventory management. As software engineers, we see something different. We see distributed systems. Complex business workflows. Real-time data synchronization. Concurrent transactions. Event-driven architecture. And perhaps the biggest challenge of all—representing how real businesses actually operate. At RetailWings , we've learned that building an ERP for retail isn't simply a software engineering challenge. It's a business engineering challenge. Here are five lessons every engineer should understand before designing an ERP platform for modern retail. 1. Retail Doesn't Run in Modules—It Runs as One Business One of the biggest architectural mistakes in business software is treating departments as isolated applications. Many systems separate: Sales Inventory Finance Procurement HR But retailers don't experience their businesses that way. One sale immediately affects inventory. Inventory influences procurement. Procurement impacts finance. Finance drives reporting. Everything is connected. A well-designed ERP should reflect these relationships rather than forcing departments into disconnected silos. 2. Inventory Is More Than a Database Table To many engineers, inventory may appear to be a simple CRUD problem. Create. Read. Update. Delete. Retail quickly proves otherwise. Inventory changes through: Sales Returns Transfers Damages Procurement Stock adjustments Warehouse movements Manual reconciliations Every movement has financial implications. Every movement must be traceable. Designing inventory requires thinking in terms of events, not just records. 3. Real-Time Data Changes Everything Retail managers don't want yesterday's reports. They want answers now. How much stock is left? Which branch is sellin