标签：#architecture

Building a Thriving Package Marketplace: The Complete MarketHub Guide Introduction If you're building a platform where developers can discover, share, and monetize packages, you're tackling one of the most complex problems in the software ecosystem. From managing publisher reputations to handling analytics at scale, marketplace dynamics require careful orchestration across multiple user roles. Enter MarketHub — a comprehensive three-app marketplace system designed to handle exactly this challenge. Whether you're creating a plugin ecosystem, SaaS integrations hub, or package distribution platform, MarketHub provides a battle-tested architecture for managing the complete marketplace lifecycle. The Problem: Why Marketplaces Are Hard Building a marketplace isn't just about creating a catalog. You need to solve several interconnected problems simultaneously: Discovery : How do users find quality packages in a sea of options? Trust : How do you build confidence in unfamiliar publishers? Quality Control : How do you maintain standards without stifling innovation? Incentives : How do you motivate publishers to create excellent packages? Scale : How do you manage analytics, reputation, and community as the ecosystem grows? Most teams try to bolt these features onto a basic catalog — resulting in fragmented systems where reputation tracking doesn't align with analytics, and community features feel disconnected from the review process. MarketHub Architecture: A Three-App Approach MarketHub solves this by separating concerns into three distinct applications, each optimized for its audience: 1. Public Discovery App — The Storefront This is where users find packages. The discovery app features: Intelligent Search & Filtering : Search across package names, descriptions, and tags with category-based filtering Featured Packages : Curated collections to highlight quality and trending packages Smart Ranking Algorithm : Packages rank based on quality signals — not just download counts

There are only 3 LLM API protocols, but unlimited providers running the same protocol. Separate protocol from identity — protocol is code, provider is data — and complexity drops from N×M to N+M. 300 lines of TypeScript. Zero dependencies. The problem isn't "it doesn't work." It's "it won't tell you it broke." In May, I built a Claude Code skill called unblind . I use DeepSeek as my daily driver, but it can't see images. So unblind forwards images to Mimo and OpenAI's vision APIs. The MVP had two providers. A few dozen lines of if-else. It worked. Then I noticed something more unsettling: an expired API key — no warning. A network hiccup — no retry. A missing permission — silently skipped. This tool didn't fail. It quietly stopped working without telling you. I added Phase 0 self-healing, circuit breakers, persistent caching, and a security sandbox. Now unblind wouldn't fail silently. But then I noticed something else. The circuit breaker doesn't care if you're calling a vision API or a translation API. The cache doesn't care if the response is an image description or OCR text. The error normalization doesn't care whether the other end is Mimo or OpenAI. A universal provider infrastructure, trapped inside a vision skill. First attempt: follow the ecosystem, hit the ceiling The largest similar project in the ecosystem is vision-support, with 19 providers. The pattern is standard—base class + subclasses, GoF Template Method. I followed it for v2.0. class BaseProvider { async analyzeImage ({ image , prompt , options }) { const { url , body , headers } = this . _buildRequest ( image , prompt , options ); const res = await apiRequest ( url , { body , headers }); return { content : await this . _parseResponse ( res ), model : this . _model }; } } class MimoProvider extends BaseProvider { ... } // 54 lines class OpenAIProvider extends BaseProvider { ... } // 45 lines class GeminiProvider extends BaseProvider { ... } // ~50 lines One subclass per provider. I expanded unblin

At Twio we picked pg-boss for our job queue, ran into trouble when we went serverless, looked at Pub/Sub, and ended up on Google Cloud Tasks. This is what each queue got right, what it got wrong for our workload, and the rule we landed on for choosing between them. The workload Twio is an AI SaaS for loan brokers. The piece that needs a job queue is email processing: download an email, parse the body and attachments, OCR, classify with an LLM, write structured data, and index for RAG. One email with five attachments easily becomes 30+ background jobs. A batch upload becomes hundreds. Why pg-boss worked — until it didn't Our database was Postgres on Neon, so pg-boss was the obvious starting point. No extra infrastructure, and one feature we genuinely loved: transactional enqueue . Because jobs live in the same database as business data, you can create a job in the same transaction as the row that triggered it. No dual-write problem, no "DB succeeded but the queue API failed" inconsistency. It also gave us retries, delayed jobs, dead-letter queues, dedup keys, and full SQL visibility into stuck or failed jobs. For a Postgres-first app on always-on infra, it's an excellent tool. Then we moved heavy processing to Cloud Run, and the cracks showed up. pg-boss polls. Neon suspends. They want opposite things. pg-boss runs a query roughly every 1–2 seconds to look for the next job, plus maintenance queries. Neon autosuspends compute when nothing touches the database. If the queue is polling every second, Neon's idle timer never expires — you pay for always-on compute even when the queue is empty. Worse, when Neon did manage to suspend, the next poll had to wake it. That wake-up takes hundreds of ms to a few seconds, and queries that triggered it would fail with Connection terminated , ECONNRESET , or timeouts. Pooled connections made it worse: the pool kept sockets that the server had already closed during suspend, and the next polling cycle picked one up and broke. This isn

TL;DR Day 1 of AI Native DevCon was a practical reality check for AI-native software...

Introduction In contemporary research, the volume of biological data deposited in public repositories is growing exponentially. The Gene Expression Omnibus (GEO), NCBI Gene, PubMed, and UniProt accumulate thousands of new records daily, including sequences, expression profiles, scientific articles, and functional annotations. On the one hand, this scenario represents a unique opportunity for biomedical research. On the other hand, the diversity of data formats, access protocols, and metadata models creates a significant barrier: each source requires a specific collector, distinct rate-limiting strategies, and its own validation logic. Above all, the lack of standardization in data storage compromises the reproducibility of scientific studies. The need for integrated tools capable of unifying data extraction, curation, and persistence has been widely discussed. In practice, ad hoc solutions such as isolated scripts for individual repositories generate redundant work and make maintenance difficult. First and foremost, it is necessary to establish an architecture that treats data collection as a service rather than a collection of scattered artifacts. This work presents Project 1 of the Integrated Bioinformatics Platform: a containerized Biomedical Data Collector coupled with a Data Lake. Its objective is to provide a REST API capable of triggering asynchronous data collections from the four aforementioned sources, storing immutable raw data in MinIO, and persisting metadata in PostgreSQL, all while ensuring traceability and resilience. Development The system architecture is divided into three main layers. The first is the API and orchestration layer , implemented using FastAPI. Its five endpoints — POST /collections , GET /collections , GET /collections/{id} , GET /collections/{id}/download/{dataset_id} , and GET /health — expose a clean interface for initiating and monitoring collection processes. The second layer is the collector engine , composed of abstract classe

Shopify Staff Engineer Guilherme Carreiro discusses building and scaling highly customizable platforms. Using Shopify’s Liquid theme system as a case study, he explains how to balance extreme design flexibility with low-latency performance under massive traffic. He shares insights on implementing secure domain-specific languages, native code extensions, and resilient developer tooling. By Guilherme Carreiro

Michael Stiefel spoke to Sonya Natanzon, about the intersection of technical and social aspects of software architecture. Understanding the business and how a company operates is more important than the specific technologies used. Effective requirements analysis requires focusing on problems to be solved that describe good and bad outcomes, rather than statements of need or solution statements. By Sonya Natanzon

I run Sola, a travel app for people who move differently from the traveller the industry was built for. While building it, I kept hitting the same wall. The data I wanted to query did not exist. Not because nobody collected it, but because the schema underneath the whole industry never had a field for it. So on 27 May 2026 I sat down and audited Booking.com. The homepage form, the currency selector, a Bangkok search results page. I wrote down what it accepts and what it refuses. Then I looked at the new AI travel agents shipping on top of it. Here is what I found, and why it matters to anyone building in this space right now. The form is the spec Booking.com's homepage search bar accepts exactly four inputs: A destination, as a single text field A check-in date and check-out date, as one range An occupancy counter, defaulting to "2 adults · 0 children · 1 room" A search button That is the spec. An online travel agency (OTA) is a CRUD app over this spec, and Expedia, Agoda, and Hotels.com run the same four fields. Airbnb lets you skip the dates. The destination stays a single field everywhere. Think about what a spec encodes. The default occupancy is a couple. Not a solo traveller, not a parent with one child, not three generations, not seven people eating from one host's kitchen. The form cannot accept a circuit ("Bangkok, then Hanoi, then Jakarta" forces three separate searches). It cannot accept an open date ("October, not sure which week"). It has no field for the part of a trip where you sleep at family but spend money in restaurants. When you fill that form, you have not searched. You have submitted to a schema. Most of the world's travellers fail the schema before they fail the search. The data receipts I am a builder, so I went for counts, not adjectives. Everything below rendered on the platform on 27 May 2026. Currencies: 52 offered, about 180 in circulation. Eight currencies sit featured at the top of the dropdown. On the day I ran it the order was EUR, US

You've seen the developer. Maybe you are the developer. They discover DRY — ✨ Don't Repeat Yourself...

Stepping up to give my first-ever presentation at UbuCon 2026 was a massive milestone, and honestly, it was pretty intimidating. The stakes felt high, especially with the live demo. It was a race against the clock to get everything running, and it only finally came together exactly ten minutes before I went on stage. Talk about a close call. While I am proud of what I delivered, I originally wanted to pack even more into the session. I had planned to showcase a simulated mission, Gazebo visualizations and RViz path simulations. While time caught up with me for the presentation, these features are still actively in the works over at the aeronix project. My goal is to have the entire end-to-end setup completed and ready by the end of the year. I connected with some incredible engineers and industry peers and I am looking forward to building on those conversations for future professional collaborations. This experience proved that the best way to grow is to just put yourself out there. Moving forward, I plan to keep speaking on topics that challenge me. It is the ultimate way to deepen my own technical understanding share what I have learned with the community and grow professionally.

A year ago, I wrote about our first year with micro-frontends . It was a frank retrospective: the win was framework autonomy and independent deploys, the cost was 20 GB of RAM during local dev, 15–20 minute deploy cycles, and a monorepo that became its own coordination tax. The closing line was the honest one — if I could choose again, I would think carefully about using Microfrontend or not. This article is the follow-up the previous one didn't write: what we actually built around Single-SPA to make it work. Not the architecture diagram. The workarounds. The dead code that's not really dead. The regexes we wish we didn't need. The .NET app no Single-SPA tutorial mentions. The browser-side singletons we use as message buses. The dev experience we documented and the parts we didn't. If you're evaluating Single-SPA in 2026, this is the article I wish I'd had a year ago. Where we left off The platform is a Single-SPA monorepo with seven in-repo apps and five shared packages. On top of that sit roughly ten standalone repositories for newer features, loaded at runtime via SystemJS import maps. Shared dependencies — React 18.2, ReactDOM, axios, react-bootstrap, single-spa 5.9.5 — are served from CDN and externalized in every webpack config. The architecture is textbook. The reality, a year in, is that the textbook stops where the workarounds start. The shell isn't where you think it is If you read our frontend repo you'd conclude the shell is whatever directory has single-spa in its package.json . That's where Single-SPA gets configured. That's where start() is called. There's even a webpack entry that, until recently, was producing an index.ejs HTML template. That HTML template is dead code. The real shell — the HTML the browser actually receives, in every environment — is rendered by an ASP.NET MVC application, in a Razor view. The frontend repo ships JavaScript modules. The .NET host ships the page that loads them. That's the unlock. Once you see this, every other work

The Theorem That Changed How We Think About Databases In 2000, Eric Brewer stood at a conference and proposed a conjecture that would reshape distributed systems forever: "You can only guarantee two of these three properties at the same time: Consistency, Availability, and Partition Tolerance." Two years later, Seth Gilbert and Nancy Lynch proved it mathematically. It became known as the CAP Theorem — and every distributed system architect since has had to wrestle with it. It sounds abstract. But once you understand it, you'll never look at a database choice the same way again. You'll understand why Amazon DynamoDB and Google Spanner make opposite architectural choices. You'll know why your bank uses PostgreSQL while Twitter uses Cassandra. Let's break it down from first principles. The Three Properties C — Consistency Every read receives the most recent write, or an error. There's only one version of the truth — all nodes agree. Not the same consistency as ACID . CAP consistency (linearizability) means every read reflects the latest write across all nodes. ACID consistency means transactions don't violate database constraints. Different concepts, same confusing word. A — Availability Every request receives a non-error response — though it might not be the most recent data. The system is always up and answering. Note: "Available" in CAP doesn't mean "fast." It means "responds without error." A system that always returns a (possibly stale) answer is Available. P — Partition Tolerance The system continues operating even when network messages between nodes are lost or delayed. A partition is when part of your distributed system can't communicate with another part. The Unavoidable Truth: P Is Not Optional Here's the insight that makes CAP actually useful: In any real distributed system, partitions will happen. Networks fail. Cables get cut. Data centers lose connectivity. AWS regions go down. Since you must tolerate partitions (or have a single-server system, which does

"AI slop" is not a model problem. It's an engineering problem you decided not to solve. The slop is the bland, off-voice, half-hallucinated, occasionally-just-an-error-message text that your LLM emits maybe 5% of the time — and that 5% is the part users screenshot. The instinct is to fix it in the prompt: add three more sentences of "be concise, be accurate, match my tone." That treats a stochastic system as if it were deterministic. It isn't. You cannot prompt your way to a guarantee. What actually works is treating the model like any other unreliable upstream dependency: wrap it in a harness that validates, rejects, and retries before anything reaches a user. The model proposes; the harness disposes. Here's how to build one. Slop is a systems problem, not a prompt problem Every production LLM feature I've shipped converged on the same shape: the model is one stage in a pipeline, not the pipeline itself. You don't trust raw generation any more than you'd trust raw user input. You parse it, you validate it against constraints you can express in code, and you reject anything that fails — automatically, before a human ever sees it. The key insight is that most slop is detectable . Empty output, a leaked stack trace, the wrong language, a 900-word answer when you asked for 200, a banned phrase like "in today's fast-paced world" — these are all checkable with deterministic code. You don't need a judge model to catch them (though a judge model has its place at the end). You need a gate that runs on every generation, costs microseconds, and never gets tired. Think of it as five layers, each rejecting a different class of failure. Layer 1: Structured output, not freeform text The single biggest reduction in slop comes from refusing to accept prose where you can demand structure. If you ask for a JSON object with named fields and a schema, the failure modes collapse from "infinite" to "a handful you can enumerate." Use the provider's native structured-output / tool-calling

A Stanford and Tsinghua paper ran a controlled experiment earlier this year. Same model. Same task. Different harness architecture. The result: a 6x performance gap driven entirely by the system built around the model. Not the model itself. This is not a prompt engineering insight. It is a systems architecture insight, and it changes where developers should invest their time when building agentic systems. The 6x Gap Meta-Harness tested Claude Opus 4.6 across two harness configurations on TerminalBench-2. The only variable was the scaffold: the code that manages tool calls, context windows, error recovery, and state persistence. One version scored at baseline. The other, with structured tool orchestration and context management, scored 18.4 points higher. Same inference cost. Same model. Different architecture. This pattern replicates across multiple independent studies: LangChain DeepAgents (2026): Same GPT-5.2-Codex model. Harness-only changes moved it from Top 30 to Top 5. That is a 13.7-point gain. Can Bölük (Hashline, 2026): Same model, same task. Changed the edit tool format. Performance went from 6.7% to 68.3%. That is a 10x improvement with 61% fewer tokens. Vercel's d0 agent : A production agent had 16 tools. Removing 14 of them (leaving only bash) took success rate from 80% to 100%. The bottleneck was not capability. It was decision surface. Why This Matters Practically The cheapest Haiku call with an optimised harness (37.6% on TerminalBench-2) outperformed the most expensive Opus call with a default harness (58.0%). That is at 1/50th the inference cost. Most teams are optimising at the wrong layer. They swap models, tune prompts, add retrieval. The structural leverage is in how the system manages tool calls, handles state, and recovers from failure. What Changes The practical takeaway for anyone building with AI agents: Audit your tool surface. Every tool your agent can call is a decision it must make. Vercel found 16→1 tool reduction improved everything.

c_std: A Leak-Free, Cross-Platform Standard Library for Modern C Bringing the comfort of the C++ STL and Python's standard library to C17 — without leaving C A technical white paper. Executive summary C is still the substrate of the computing world — kernels, databases, language runtimes, embedded firmware, and the inner loops of nearly everything else. Yet the moment you step away from the kernel and try to write ordinary application code in C, you feel the gap: no growable vector, no hash map, no JSON parser, no string type that doesn't invite a buffer overflow. You either pull in a grab-bag of mismatched third-party libraries, each with its own conventions and failure modes, or you re-implement the same dynamic array for the hundredth time. c_std is an attempt to close that gap deliberately and coherently. It is a single, consistent library — written in pure C17 — that reimplements a large slice of the C++ Standard Library (containers, algorithms, smart pointers) alongside many Python-style conveniences ( json , regex , random , statistics , csv , config , even turtle graphics). It targets Windows and Linux from one source tree, compiles cleanly under -Wall -Wextra , and — this is the part I care about most — is verified leak-free under Valgrind , module by module, example by example. This paper explains the design philosophy, the architecture, and the engineering discipline that makes a library like this trustworthy enough to build on. 1. The problem: C's missing middle Every C programmer knows the two extremes. At the bottom, the language itself: pointers, malloc , memcpy , raw arrays. At the top, whatever the platform hands you — <windows.h> or POSIX, OpenSSL, a JSON library someone wrapped a decade ago. The middle — the layer the C++ STL and Python's batteries-included standard library occupy — is missing. That missing middle has a real cost. It shows up as: Re-invention. Teams write their own vector, their own string builder, their own linked list, each subt

Event sourcing gets sold on immutability. You don't update, you don't delete, you only append, so the history is permanent. It mostly isn't. The events are immutable because your code agrees not to touch them, not because anything actually stops it. Underneath they're still rows in Postgres, and rows have a DBA with write access. A migration that "cleans up" old data. A 2 a.m. query run against the wrong connection. A backup restored with slightly different bytes in it. Change one of those rows and a replay won't blink. The aggregate rebuilds, the projections rebuild, everything looks fine. Usually the first person to notice is a customer whose balance is off, and by then the trail is cold. Chain each event into the next The trick is small. Give every row two extra columns: a hash of its contents, and the hash of the row before it. #1 AccountOpened prev=00000… hash=70be4f… │ ▼ #2 AmountDeposited prev=70be4f… hash=796018… │ ▼ #3 AmountWithdrawn prev=796018… hash=6a0260… The hash is SHA-256(previousHash || json(payload)) . Nothing exotic. The point is that each hash depends on the one before it. Edit a payload and its hash stops matching. Rewrite that hash to cover for the edit, and now the next row's pointer is wrong. You can't fix one without breaking the next. About forty lines of it Appending an event hashes it together with the previous one: public HashChainedEntry Append ( object payload ) { var previousHash = _entries . Count == 0 ? GenesisHash : _entries [^ 1 ]. Hash ; var hash = ComputeHash ( previousHash , payload ); var entry = new HashChainedEntry ( _entries . Count + 1 , payload , previousHash , hash ); _entries . Add ( entry ); return entry ; } internal static byte [] ComputeHash ( byte [] previousHash , object payload ) { var payloadJson = JsonSerializer . SerializeToUtf8Bytes ( payload , payload . GetType ()); var combined = new byte [ previousHash . Length + payloadJson . Length ]; Buffer . BlockCopy ( previousHash , 0 , combined , 0 , previousHash .

> TL;DR — I run four different domains off one Next.js codebase: a marketing site at pagestrike.com , an authenticated app at app.pagestrike.com, a public publishing domain at pagestrike.app, and customer-owned domains. The trick isn't deploying four apps — it's a single proxy.ts that reads the host and rewrites/redirects/passes-through per-request. This post walks through why I chose this shape, the parts I got wrong, and the cookie-domain trick that makes it all stick. Stack: Next.js 16 App Router , Supabase , Vercel , one proxy.ts file (~370 lines). This is the second post in my build-in-public series on PageStrike . Last week I wrote about the 6-CTA architecture — modeling conversion intent as a discriminated union so one launch could be a checkout, a COD form, or a calendar booking. This post is about a different primitive: modeling host as routing context so one codebase can serve four very different audiences. Why four domains, not one Most SaaS apps live at one domain — say myapp.com with /dashboard under it. That works until you grow into edge cases that don't fit: Marketing pages get spammed by your own dashboard headers. Your marketing nav says "Sign in / Pricing / Blog". Your dashboard nav says "Launches / Contacts / Settings". You either A/B them with conditional logic everywhere or you live with the noise. Public user-generated pages share your domain reputation. When a customer publishes a landing page at myapp.com/p/[slug] , every spammy LP from a free-tier user drags down myapp.com 's sender reputation, search trust, and ad-account standing. Google and Meta penalize the host, not the path. Custom domains don't route cleanly. A customer who buys acmewidgets.com and points it at your app expects their LP at acmewidgets.com/ — not myapp.com/p/acme-widgets . You need a rewrite that's transparent to the visitor, doesn't 404 on _next/static/* , and survives RSC prefetches. I split PageStrike — a free AI landing page builder — across four hosts to solve al

Look at your modern Agentic AI stack. An agent wants to execute a tool, trigger a deployment, access a database, or call an external API. Because nobody fully trusts a probabilistic black box, many teams now use a second probabilistic black box to validate the first one. Think about what is actually happening. You are running hundreds of billions of parameters, consuming tokens, burning GPU resources, and adding hundreds or thousands of milliseconds of latency just to answer a simple operational question: PASS HOLD RED Or in plain English: Continue Verify Stop For many production systems, that's the only decision that matters. Yet we often spend orders of magnitude more compute determining whether an action should execute than executing the action itself. That feels dangerously close to architectural bankruptcy. The Illusion of Prompt-Based Safety We've all done it. You create a prompt: "You are a security validator. If the action appears unsafe, return RED." Then reality arrives. Prompt injections appear. Edge cases appear. Different model versions behave differently. The same input occasionally produces different outputs. And your cloud bill keeps growing. At some point, a difficult architectural question emerges: Can a probabilistic system reliably govern another probabilistic system? Many teams assume the answer is yes. I'm not convinced. The Problem Isn't Intelligence This is where I think the industry may be looking at the problem incorrectly. The challenge is not intelligence. The challenge is governance. LLMs are exceptional at: Reasoning Summarization Code generation Natural language interaction But governance is a different problem. Governance is not asking: "What is the best answer?" Governance is asking: "Should this action be allowed to proceed?" Those are fundamentally different questions. A Different Architecture While exploring this problem, we ended up building a separate deterministic governance layer internally. Instead of generating text, it perf

In November 2023 we ran our first global Hytale servers on Google Kubernetes Engine using Veltrix 3.2 as our configuration orchestrator. The Treasure Hunt Engine—a service that fans spawn to claim event loot—started crashing every time search volume exceeded 12 k RPM. Grafana showed a steady climb of 503 errors on /hunt/claim until the autoscaler maxed out at 32 G1 CPU cores and still couldnt keep up. Operators kept filing tickets that boiled down to one sentence: We click the map, nothing happens. We never saw the actual error because the ingress controller was swallowing it and returning a generic Too many requests. What we tried first (and why it failed) Our first move was to crank up the nginx-ingress-controller replicas from 3 to 12 and switch the load-balancer tier from GKE Standard to Premium. The 503 rate dropped to 8 k RPM, but now the p99 latency on claims spiked from 80 ms to 420 ms. The culprit was a recursive call in the hunt service: every claim required a round trip to the player-profile service to validate tier eligibility, and that service was on a shared Postgres 15.4 cluster with 3 k TPS of unrelated traffic. The error stack in Jaeger was literally tracing_id=7f3a1c8… server=profile-db pool_timeout . We tried adding connection pooling with PgBouncer, but the hunt service was using raw libpq and refused to reuse connections—no matter how many times we told it. The Architecture Decision We ripped the validation out of the synchronous path and made the hunt engine publish an event called HuntTierCheckRequired to a dedicated Kafka topic player-events-tier . The hunt service would respond to the client with a 202 Accepted immediately, then the loot-claim worker would listen to that topic and, if the tier passed, publish HuntLootReady . The worker ran in the same pod but on a separate goroutine with a 60-second TTL so we didnt leak memory if the tier service hung. We moved the player-profile service to an SSD-backed CloudSQL instance and gave it 32 GB R

Google Cloud's automated systems suspended Railway's production account without notice, triggering an eight-hour platform-wide outage affecting 3 million users. The cascade took down workloads across all providers including AWS and bare metal because Railway's control plane was hosted on GCP. Railway is demoting GCP to backup-only status. By Steef-Jan Wiggers