标签：#STEM

If you’ve used S3, MinIO, or any cloud storage API, it’s easy to assume object storage is just a “cloud folder system.” That assumption is wrong — and it leads to confusion when you start working with distributed systems. Object storage is not a file system. It’s closer to a distributed key-value system with strong durability guarantees and a very specific access model . Once you understand that shift, a lot of cloud infrastructure starts to make more sense. The mental model most people start with When people first see object storage, they imagine something like this: /photos/cats.png /photos/dogs.png A hierarchical file system: folders subfolders files inside directories This is how traditional systems like ext4 or NTFS work. But object storage doesn’t actually work this way. The actual model: key → object Object storage is much simpler at its core: key → value Example: key : photos/cats.png value : <binary data> There are no real folders. “folders” are just string prefixes used for organization. That’s it. Why this design exists This model isn’t accidental. It solves real distributed system problems. Traditional file systems struggle when you try to: scale across many machines replicate data reliably handle partial failures coordinate metadata changes at scale Object storage avoids many of these problems by simplifying the model. Instead of supporting complex file operations, it focuses on: store object retrieve object delete object list objects by prefix Nothing more. The most important design choice: immutability In most object storage systems: Objects are not modified in place. If you “update” a file, what actually happens is: upload a new object replace the key pointer old object becomes orphaned (eventually cleaned up) This is a huge shift from file systems. Why this matters Immutability makes distributed systems easier because: no concurrent write conflicts on the same object replication becomes simpler caching becomes safer failure recovery is easier to rea

Every engineering team hits this conversation at some point. Someone proposes GraphQL. Someone else says REST is fine. A third person mentions gRPC and half the room goes quiet. The debate usually ends with the most senior person in the room picking what they're most familiar with. That's not a strategy — that's habit. Here's an objective breakdown of all three, when each one wins, and how to actually make the decision for your specific use case. The Core Mental Model Before comparing them, understand what each one is optimizing for: REST optimizes for simplicity and broad compatibility GraphQL optimizes for flexibility and precise data fetching gRPC optimizes for performance and strongly-typed contracts None of them is universally better. Each one is a tradeoff. The right answer depends entirely on who is consuming your API and what they need from it. REST — The Default That Still Wins Most of the Time REST (Representational State Transfer) is not a protocol. It's an architectural style built on HTTP — verbs, URLs, and status codes most developers already understand. Where REST genuinely wins: Public APIs. If external developers are consuming your API, REST is the only reasonable default. The tooling, documentation patterns, and developer familiarity are unmatched. Stripe, Twilio, GitHub — all REST. Simple CRUD services. If your resource model is straightforward, REST maps cleanly to it. No overhead, no learning curve, no ceremony. Browser-native requests. REST over HTTP works directly in the browser without any special client. Fetch it, done. Where REST struggles: Over-fetching and under-fetching. A single REST endpoint returns a fixed shape. Mobile clients that need 3 fields get 40. Separate data needs often require multiple round trips. Versioning overhead. As covered in our previous post — every breaking change forces a versioning decision. This compounds quickly on complex APIs. GraphQL — Powerful, But You Need to Earn It GraphQL is a query language for your A

Last year I was on a team that pushed 40 million events per day through Kafka. We had consumer lag alerts, rebalancing incidents, and a whole runbook for when the broker got behind. I understood how to operate Kafka. But I did not understand how Kafka works. So I built a tiny one. No dependencies. No Zookeeper. No JVM. Just Python and the core ideas. Here is what I learned. The Three Things Kafka Actually Does People say "Kafka is a message queue." That is not quite right. Kafka is a distributed commit log . It has three jobs: Accept writes from producers and append them to a log Let consumers read from any offset in that log Remember where each consumer group is up to That third one is the thing that makes Kafka different from a traditional queue. A queue forgets a message once it is consumed. Kafka remembers. You can replay. You can have 10 different consumer groups reading the same topic at different speeds. The code to implement this is smaller than you think. brokelite: A Message Broker in 120 Lines import threading import time from collections import defaultdict from typing import Dict , List , Tuple class Partition : """ Append-only log for one partition of a topic. """ def __init__ ( self ): self . _log : List [ Tuple [ int , bytes ]] = [] # (offset, message) self . _lock = threading . Lock () self . _next_offset = 0 def append ( self , message : bytes ) -> int : with self . _lock : offset = self . _next_offset self . _log . append (( offset , message )) self . _next_offset += 1 return offset def read_from ( self , offset : int , max_count : int = 100 ) -> List [ Tuple [ int , bytes ]]: with self . _lock : return [ ( off , msg ) for off , msg in self . _log if off >= offset ][: max_count ] def __len__ ( self ): return self . _next_offset class Topic : """ A topic is just N partitions. """ def __init__ ( self , name : str , num_partitions : int = 3 ): self . name = name self . partitions = [ Partition () for _ in range ( num_partitions )] def route ( self , k

Most AI demos work perfectly on a laptop. But production AI systems can become fragile when everything is handled inside one synchronous API call. A user sends a request. The API extracts text. The API chunks the content. The API generates embeddings. The API stores data. The API waits for everything to finish. This may look simple in a demo, but it quickly becomes a problem in real systems. The problem with one giant API call In many AI applications, the API is expected to do too much. For example, in a document processing or RAG pipeline, one request may trigger multiple heavy steps: text extraction chunking embedding generation indexing summarization database updates If all of this happens inside one synchronous request, the API becomes slow and fragile. If one downstream step fails, the complete request may fail. If traffic increases suddenly, the API may become overloaded. This is why event-driven architecture becomes useful for AI workloads. A better approach: API + Kafka + workers Instead of making the API do everything, we can split the workflow into smaller services. The API accepts the request and publishes an event. Background workers consume events and continue the processing asynchronously. A simple flow looks like this: User Request ↓ FastAPI ↓ Kafka / Redpanda Topic ↓ Python Worker ↓ Next Processing Stage In my practical demo, I am using: FastAPI Redpanda Python workers Docker Compose Kafka-compatible messaging Why Redpanda? Redpanda is Kafka-compatible, which makes it useful for local demos and event-driven architecture experiments. It allows us to work with Kafka-style topics, producers, and consumers while keeping the setup simple for development. What this architecture gives us This approach helps with: decoupling services handling bursty workloads moving long-running tasks to background workers improving scalability isolating failures building production-style AI pipelines This pattern is especially useful for AI systems involving: document proce

Martin Kleppmann, an associate professor at Cambridge and author of Designing Data-Intensive Applications, discusses the evolution of data systems over the last decade, mainly the shift from monolithic databases to modular building blocks. Kleppmann underlines the importance of moving from cloud-centric data storage systems to decentralised data storage similar to Bluesky’s AT protocol. By Martin Kleppmann

Quorum is an incident command plane built on Amazon Aurora DSQL. The failover story lives in another post. This one is about a narrower question that turned out to be the foundation: when several responders write to the same incident at the same moment, across regions, during the worst minutes of an outage, how do you guarantee the record never forks into two conflicting truths. The answer is two design choices that are really one choice seen from two angles: event sourcing, and DSQL's optimistic concurrency control. The data model is append-only Quorum is event-sourced across four tables. Every state change is an immutable event appended to a log, not an in-place update. The current state of an incident is a fold over its events. There is no UPDATE incidents SET status = ... ; there is an acknowledged event, a note event, a resolved event, and the status you render is computed from them. The event's UUID is its primary key and its idempotency key at the same time. A retried write carrying the same UUID cannot double-apply: the insert collides on the primary key and becomes a no-op. That property sounds minor until you remember what kind of system this is. A tool designed to survive network failure retries writes constantly, and "the responder tapped resolve twice because the first response was slow" must not produce two resolutions. Append-only also suits the domain directly. For an incident system the audit trail is the product, not a side effect. "Who acknowledged this, at what time, and what did the timeline look like at 02:14" is a first-class question for the post-incident review and a compliance requirement in regulated environments. Event sourcing gives you that for free. It also gives DSQL a write pattern it likes, which matters more than you would expect. The stack, briefly TypeScript end to end. Kysely as a typed query builder rather than an ORM, because I wanted type safety without surrendering control of the SQL: on a distributed database the exact shap

Every time you tap "Book Ride," a system makes dozens of decisions in under two seconds: Which driver? What route? What's the real ETA? This article breaks down exactly how the dispatch algorithm works — from the greedy approach that fails at scale, to the bipartite graphs, batched matching, and surge pricing mechanics that power Uber, Lyft, Grab, and Gojek today. Why a Greedy Dispatch Algorithm Fails (Closest Driver Problem) The first instinct when designing a matching system is to pair every customer with their nearest driver. However, this Greedy approach causes massive losses at a system-wide scale: Example: 3 riders (R1, R2, R3) and 3 drivers (D1, D2, D3) Greedy Matching (closest driver): R1 ← D1 (ETA 2 mins) ✓ R2 ← D3 (ETA 8 mins) ← D2 was "taken" by R1, even though D2 is closer to R2 R3 ← D2 (ETA 10 mins) ← Terrible outcome Total ETA: 2 + 8 + 10 = 20 minutes Optimal Matching (global optimal): R1 ← D2 (ETA 3 mins) R2 ← D1 (ETA 3 mins) R3 ← D3 (ETA 4 mins) Total ETA: 3 + 3 + 4 = 10 minutes ← 50% better! Uber refers to this problem as Global Optimization — finding an assignment strategy that minimizes the total ETA of the entire system , rather than optimizing just for individual pairs. Bipartite Graph Matching: The Mathematical Foundation (Lyft) Before diving into the systems, it helps to understand the mathematical model that all ride-hailing matching engines share at their core. Lyft formalizes dispatch as a bipartite graph matching problem : Bipartite Graph: Set A (Riders): { R1, R2, R3, R4 } Set B (Drivers): { D1, D2, D3, D4, D5 } Edges: every possible Rider ↔ Driver pair Edge Weight: cost of that match (e.g., ETA, driver rating, distance) Goal: Find a set of edges (a "matching") where: - No rider is matched to more than one driver - No driver is matched to more than one rider - The total cost of all selected edges is minimized This is known as the Minimum Weight Bipartite Matching problem. The classical algorithm for solving it is the Hungarian Algorithm (

Network Protocols Network protocols define how computers communicate over a network. Whether you're opening Instagram, sending a WhatsApp message, watching Netflix, or transferring money through a banking app, some protocol is working behind the scenes to make communication possible. Client-Server Model What is it? The Client-Server model is a communication architecture where: Client requests a service or data. Server processes the request and returns a response. Most modern applications follow this architecture. How Does It Work? Client ---------- Request ----------> Server Client <--------- Response ---------- Server The client always initiates communication, and the server listens for incoming requests. Real World Example Instagram When you open Instagram: Mobile app sends a request. Instagram servers process the request. Feed data is fetched from databases. Posts are returned to your phone. Instagram App | V Instagram Server | V Database Advantages Centralized control Easier security management Easy maintenance Easier data consistency Disadvantages Server can become a bottleneck Single point of failure if not replicated Interview One-Liner Client-Server architecture is a centralized model where clients request resources and servers provide them. Peer-to-Peer (P2P) Model What is it? In a Peer-to-Peer network, every machine can act as both: Client Server There is no central server controlling communication. How Does It Work? Peer A <------> Peer B ^ ^ | | V V Peer C <------> Peer D Each peer can directly share resources with others. Real World Example BitTorrent Instead of downloading a file from one server: User | +--> Peer 1 | +--> Peer 2 | +--> Peer 3 Different parts of the file are downloaded from multiple peers simultaneously. Blockchain Bitcoin and Ethereum networks operate using Peer-to-Peer communication. Advantages Highly scalable No central server cost Better fault tolerance Disadvantages Harder to manage Security challenges Data consistency issues Inter

I Built a Consistent Hashing Ring in Pure Python and Finally Understood How Cassandra Distributes Data I've been using Cassandra and Redis Cluster for years. I knew consistent hashing was "how they work." But I never truly got it until I built one myself from scratch, in pure Python, with zero dependencies. This post is about what I learned doing that. The Problem Consistent Hashing Solves Imagine you have 3 servers and 1 million keys. The naive approach: server = hash(key) % 3 . It works great until you add or remove a server. Change 3 to 4, and almost every key remaps to a different server. In a caching layer, that means near 100% cache miss. In a database, it means massive data movement. That's the problem consistent hashing solves. When you add or remove a node, only a fraction of keys move. Specifically, 1/n of the keys, where n is the number of nodes. Building the Ring The core idea: place both nodes and keys on a circular number line from 0 to 2^32 (or any large integer). To find which node owns a key, walk clockwise until you hit a node. Here's the minimal version: import hashlib import bisect class ConsistentHashRing : def __init__ ( self , replicas = 150 ): self . replicas = replicas self . ring = {} # hash -> node name self . sorted_keys = [] # sorted hash positions def _hash ( self , key : str ) -> int : return int ( hashlib . md5 ( key . encode ()). hexdigest (), 16 ) def add_node ( self , node : str ): for i in range ( self . replicas ): virtual_key = f " { node } :vnode: { i } " h = self . _hash ( virtual_key ) self . ring [ h ] = node bisect . insort ( self . sorted_keys , h ) def remove_node ( self , node : str ): for i in range ( self . replicas ): virtual_key = f " { node } :vnode: { i } " h = self . _hash ( virtual_key ) del self . ring [ h ] idx = bisect . bisect_left ( self . sorted_keys , h ) self . sorted_keys . pop ( idx ) def get_node ( self , key : str ) -> str : if not self . ring : raise ValueError ( " Ring is empty " ) h = self . _hash

If you're building a modern data stack that requires either high-throughput transaction processing or large-scale analytical workloads, you've likely come across both Vertica and VoltDB (now rebranded as Volt Active Data). While both are distributed relational database management systems (RDBMS), they are architected for completely opposite use cases — choosing the wrong one can lead to 10x higher costs, missed latency SLAs, and poor application performance. In this guide, we break down every key difference between OpenText Vertica and Volt Active Data, with practical examples, real-world use cases, and best practices to help you make the right choice for your team. Table of Contents What is OpenText Vertica? What is Volt Active Data (Formerly VoltDB)? Core Differences Between Vertica and VoltDB Real-World Use Cases: When to Pick Which Best Practices & Common Mistakes Conclusion & Key Takeaways References What is OpenText Vertica? OpenText Vertica (formerly Micro Focus Vertica) is a columnar relational DBMS built exclusively for analytical (OLAP) workloads, first launched in 2005. As of 2026, the latest stable version is 26.1, with native lakehouse and Apache Iceberg export support for modern data ecosystems. Core Vertica Architecture Vertica's design is optimized for fast queries across massive datasets: Columnar storage : Data is stored by column instead of row, enabling significantly higher compression ratios and faster aggregation queries that only access a small subset of columns Massively Parallel Processing (MPP) : Query execution and data are distributed across hundreds of nodes for parallel processing Dual deployment modes : Enterprise Mode : Shared-nothing architecture with data stored locally on nodes for maximum performance Eon Mode : Compute and storage separated, using shared object storage (S3, GCS, ADLS) to scale compute independently of storage for cloud workloads Projections : Physical, sorted copies of data optimized for common query patterns (ins

As my backend engineering internship wraps up, I’ve been reflecting on the tasks that pushed me the hardest. Building minimum viable products is one thing, but making them resilient, scalable, and fault-tolerant is an entirely different beast. Here are two of the most memorable tasks from my time here—one solo dive into system scaling, and one team effort tackling asynchronous voice processing. Task 1: Scaling the Insighta Labs+ Query Engine (Individual) What it was Insighta Labs+ is a demographic intelligence platform where analysts and engineers run structured queries on user profiles via a CLI and a Web Portal (backed by GitHub OAuth and RBAC). My task was to take a functional MVP and evolve it into a robust query engine capable of handling tens of millions of records and hundreds of concurrent queries per minute. The problem it was solving The initial architecture worked flawlessly for a few thousand records, but under scale, it started showing cracks. Latency : Without indexing, every filter query triggered a full-table scan. Redundancy : Identical queries from different users wasted CPU and DB cycles. Write-Pressure : Users needed to bulk-upload CSVs containing up to 500,000 rows. Processing these synchronously locked the database, bringing read operations to a halt. How I approached it Instead of blindly throwing more server power at the problem, I focused on doing less work. Targeted Indexing : I added indexes only to frequently filtered columns. Caching & Normalization : I introduced Redis for TTL-based caching. To maximize cache hits, I built a query normalization layer. Whether a user queried "young males" or "men under 30" , the parser normalized the filter object into a canonical form before hashing the cache key. Connection Pooling : I set up PgBouncer to manage database connections and prevent exhaustion under high concurrency. Chunked Ingestion : For the massive CSV uploads, I implemented chunked streaming. Rows were validated individually; valid row

なぜ“特別な決済システム”の話は危険なのか？ 近年、SNSやメッセージアプリを通じて、「特別なPOS端末」や「秘密の決済システム」に関する話を目にすることがあります。 「通常の銀行システムを経由しない」 「オフラインでも大金を受け取れる」 「特別なカードと専用POSがあれば送金できる」 こうした説明は一見すると高度な金融技術のように聞こえます。 しかし、実際の決済システムを理解すると、多くの主張が現実的ではないことが分かります。 まず、POS端末とは何か？ POS（Point of Sale）端末は、店舗でクレジットカードやデビットカードによる支払いを処理するための装置です。 一般的な決済は以下のような流れで行われます。 顧客 ↓ POS端末 ↓ 加盟店契約銀行 ↓ カードブランド ↓ カード発行銀行 ↓ 承認または拒否 重要なのは、最終的な資金の確認を行うのはカード発行銀行であるという点です。 POS端末そのものが資金を生み出すことはありません。 「オフライン決済だから大丈夫」は本当か？ 一部の詐欺では、 「この端末はオフラインで動作する」 という説明が行われます。 確かに、現実の決済システムにはオフライン処理が存在します。 しかし、それは通信障害時の一時的な仕組みであり、最終的には銀行側との照合が行われます。 つまり、 オフライン処理 ≠ 資金の創造 です。 銀行が承認していない資金は、後の精算時に拒否される可能性があります。 なぜ人は信じてしまうのか？ 理由は単純です。 専門用語が多いからです。 例えば、 決済ネットワーク 国際ブランド オフライン認証 ISO規格 特殊プロトコル こうした言葉が並ぶと、本物らしく見えます。 しかし、本当に重要なのは技術用語ではありません。 重要なのは、 「お金はどこから来るのか？」 という一点です。 詐欺を見抜くための3つの質問 1. お金の出所はどこか？ 利益や送金の原資を説明できない場合は要注意です。 2. 誰が監督しているのか？ 銀行、決済事業者、規制当局など、責任主体が明確か確認しましょう。 3. 第三者による検証は可能か？ 説明が内部関係者の証言だけに依存している場合は危険です。 テクノロジーと金融リテラシー 新しい技術は私たちの生活を便利にします。 しかし、技術的な言葉が使われているからといって、その仕組みが正しいとは限りません。 本当に優れた金融サービスほど、 透明性が高い 説明が分かりやすい リスクが明示されている という特徴があります。 逆に、 「秘密」 「特別」 「限定」 「誰にも教えないでほしい」 といった言葉が頻繁に出てくる場合は、一度立ち止まって考えるべきです。 まとめ 金融詐欺の多くは、技術ではなく心理を利用します。 人々はお金を失うから騙されるのではありません。 「理解したつもりになる」から騙されるのです。 だからこそ、最も重要な防御策は、 「そのお金はどこから来るのか？」 というシンプルな質問を忘れないことです。 金融の世界に魔法はありません。 あるのは、透明な仕組みと説明可能な資金の流れだけです。

Removing expf() from a fire detector: one header, 1.95x faster, zero accuracy loss A smoke detector is not a demo project. When it fires, someone either evacuates in time or doesn't. The firmware running on that microcontroller has one job, and it needs to do it without hesitation, without bloat, and without dependencies that can fail in unexpected ways. Last May 28th I published a bare-metal fire detection system built with Hasaki 刃先 — a neural network trainer that exports standalone C headers with no runtime, no Python, no TensorFlow. The model is a 12-8-4-1 MLP trained on 28,596 sensor readings. It fits in 3.8 kB of Flash and achieves 99.93% accuracy on held-out data, with a single missed fire event out of 3,599. But there was something in that header that bothered me. static inline float sigmoid ( float x ) { return 1 . 0 f / ( 1 . 0 f + expf ( - x )); } expf() . Right there in a life-safety application. On a microcontroller that may not have a hardware FPU. The problem with expf() on bare metal On processors with a hardware FPU — like the ESP32-C3 — expf() is fast. But the moment you deploy to an ATmega328P, an ATtiny85, or any Cortex-M0 target, that call becomes software floating-point. The CPU has to simulate the operation in firmware, cycle by cycle. It works. But it carries hidden cost: unpredictable latency, dependency on math.h , and a transcendental function sitting in the critical path of every single inference. For a smoke detector running at 1 Hz this might seem irrelevant. But inference latency compounds with sensor reads, normalization, and communication overhead. And more importantly — if you're deploying to a truly constrained target, expf() might be the difference between fitting in Flash or not. The fix: one header from kigu-quant kigu-quant(comming soon) is a new tool in the Rosito Bench ecosystem. It generates ready-to-include C headers for evaluating mathematical functions on microcontrollers — no FPU, no libm, no dependencies. One command: k

Lyft details a new pickup experience to improve reliability in gated communities, where 25–30% of rides face routing and access challenges. The system uses mapping signals, boundary detection, and routing improvements to reduce cancellations and coordination overhead between riders and drivers, highlighting how real-world constraints drive evolution in geospatial systems. By Leela Kumili

Cross-border purchasing platforms commonly use marketing tactics such as coupons, registration points, order rebates, and spend-based discounts to acquire new users and boost engagement. However, public promotions are prime targets for “wool hunters” (fraudsters) who exploit batch account registration, fake orders, malicious order spamming, and combined discount abuse to drain platform benefits, causing direct financial losses. Most purchasing systems lack dedicated event risk controls, making them highly vulnerable to batch exploitation as soon as a promotion goes live. This not only leads to financial losses but also crowds out genuine user benefits and distorts campaign effectiveness. This article details Taocarts’ comprehensive anti-fraud risk control framework, which uses multi‑dimensional behavior detection, rule‑based blocking, and account risk assessment to accurately distinguish real users from malicious exploiters, ensuring fair and controllable marketing activities. First, we identify common malicious exploitation scenarios and system vulnerabilities in cross‑border platforms: Batch account registration – Using new‑user exclusive coupons and registration points to harvest benefits repeatedly. Fake order placement and cancellation – Repeatedly claiming limited‑time discounts or rebates by placing and then canceling orders. Multiple accounts from the same device/IP – Spamming orders to consume activity quotas. Illegal discount stacking – Violating platform rules by combining multiple coupons or point deductions. Activity volume manipulation – Generating fake orders to earn activity rewards or points, creating false engagement data. Traditional systems have no risk rules and cannot detect batch operations or abnormal behavior, leading to wasted marketing spend and campaigns that actually lose money. Taocarts builds a fine‑grained anti‑exploit rule engine based on four dimensions: user behavior, device information, network characteristics, and order data, ena

Most developers never have to design a network protocol from scratch. You use HTTP, gRPC, WebSockets, or something else that already exists and has been debugged by thousands of people over many years. That is the right call for most situations. I did not take that path when building Vaylix, a key-value database engine. I designed a custom binary protocol called VTP2, and the process taught me things about networking that I would not have picked up any other way. This is not an argument that you should also build a custom protocol. For most things, you should not. This is an honest account of what I ran into. Why not HTTP The first question anyone reasonably asks is: why not just use HTTP? HTTP is everywhere. The tooling is excellent. Every language has a client. Debugging with curl is trivial. If I had used HTTP, I would have had working client libraries in a dozen languages before writing a single line of server code. The problem is that HTTP is stateless by design. Every request is independent. Every request carries headers. Every response carries headers. The model assumes that each round trip is a fresh conversation with no memory of what came before. A database session is the opposite of that. A client connects, authenticates, and then issues many commands over the same connection. The authentication should happen once. The session should carry state. Pipelining requests without waiting for each response to return should be natural, not something you fight the protocol to achieve. HTTP/2 closes some of this gap. But using HTTP/2 correctly for a stateful session model involves working against the grain of what HTTP was designed for. I would have been spending a lot of time on infrastructure that exists to make HTTP behave less like HTTP. The other issue is overhead. HTTP headers are verbose. For small key-value operations, the headers can easily exceed the payload. That felt wrong for something designed to be a tight operational data store. So I went with TCP d

I built a distributed compute grid where your idle laptop runs ML jobs — the orchestrator behind it The pitch: a single FastAPI hub takes compute jobs from ML researchers, and a fleet of home PCs and gaming rigs (RTX 4090s, M2 MacBooks, anything with a GPU and a Python interpreter) polls in, picks up work, and ships results back. A 20% platform fee funds the hub. An interactive dashboard shows the mesh in real time. I have been living inside this codebase for a few weeks. This post is about the part that actually determines whether the thing works or does not — the orchestrator . No frontend, no marketing — just the brain. Live dashboard: man44.zo.space/compute-pool Repo: github.com/AmSach/ComputePool-Grid The problem with "dumb" schedulers The first version of ComputeOrchestrator had a one-line bug that took down a 12-node stress test. Two jobs hit the hub at the same millisecond. Both saw the same node as idle . Both wrote busy to the same row. One node ended up double-allocated, the other starved, and the test logs looked like a hostage negotiation. The fix had to be three things at once: A scoring function that picks the right node, not just the first idle one. An async lock so concurrent submissions cannot race on a single node. A heartbeat monitor that reclaims nodes that ghosted. Here is what it looks like now. The scoring algorithm def _calculate_score ( self , capacity : Dict [ str , Any ], requirements : Dict [ str , Any ]) -> float : """ Heuristic for node-task matching. """ score = 0.0 if capacity . get ( " gpu_vram " , 0 ) >= requirements . get ( " min_vram " , 0 ): score += 10.0 if capacity . get ( " cpu_cores " , 0 ) >= requirements . get ( " min_cores " , 0 ): score += 5.0 return score The weights are deliberately lopsided. A node that satisfies a job's VRAM requirement gets a 2x bonus over a node that just barely has enough cores. The intuition: GPU work is the long pole. If you cannot fit the model in VRAM, nothing else matters, no matter how many

Every backend project I've worked on eventually hits the same wall. You start clean — one service, simple routes, everything works. Then slowly the requirements creep in. "We need rate limiting." "Can we add auth middleware?" "What happens when the user service goes down — does it take everything else with it?" You either bolt these things onto every service individually, copy-paste the same middleware across projects, or pay for a managed gateway like Kong or AWS API Gateway and hope it does what you need. I wanted to actually understand how these things work under the hood. So I'm building one — and this is what I've learned so far. What is Ferrox? Ferrox is a self-hosted, programmable API gateway written entirely in Rust. It sits in front of your backend services and handles everything a production system needs in one place: Dynamic routing — point any path prefix to any upstream service Authentication — JWT and API key validation on protected routes Rate limiting — Redis-backed per-IP and per-API-key limiting Circuit breaking — stops hammering a dead upstream service Response caching — Redis-backed TTL cache per route Real-time observability — WebSocket dashboard with live request stats Prometheus metrics — plug straight into Grafana The idea is simple. Instead of this: Client → Service A (has its own auth, rate limiting, logging) Client → Service B (has its own auth, rate limiting, logging) Client → Service C (has its own auth, rate limiting, logging) You get this: Client | v FERROX (auth, rate limiting, circuit breaking, logging — once) | +--------+--------+ | | | Svc A Svc B Svc C (clean) (clean) (clean) Your services stay clean. Ferrox handles the cross-cutting concerns. Why Rust? Honest answer — I already knew Rust from my backend work. But for a gateway specifically, it felt like the obvious choice. A gateway sits on the critical path of every single request. Every millisecond of latency it adds is latency your users feel. You need predictable performance

Earlier parts examined the performance characteristics of sequential and random access under single-threaded execution, and noted in passing the destructive effect of random access on the TLB. This part devotes full attention to the TLB: what it is, why a TLB miss is more severe than a cache miss, why a page table walk constitutes one of the longest dependency chains a CPU can encounter, how huge pages fundamentally alter TLB reach, and where memory-level parallelism falters in the face of TLB misses. Page Boundaries: Where the Prefetcher Halts Part III, in its discussion of prefetchers, noted a hard constraint: a prefetcher must not cross page boundaries on its own authority. The operating system manages virtual memory in units of pages (typically 4 KB, i.e., 64 cache lines). When a program reaches the end of one page and is about to step into the next, the prefetcher cannot proceed. The reason is that the next page may not reside in physical memory (it may have been swapped out to disk), or it may be an entirely invalid virtual address — if the prefetcher were to speculatively initiate an access to the next page, it would trigger a page fault: the OS would have to suspend the process and swap the page in from disk; in the case of an invalid address, the OS would terminate the process outright. From a security standpoint, the prefetcher neither can nor is permitted to autonomously cross page boundaries without TLB approval. Hence a performance brake appears every 4 KB — even when traversing an array sequentially, after every 64 cache line accesses the prefetch pipeline must pause and await confirmation of an address translation. This is not to say that modern CPU prefetchers are completely unable to cross pages. Intel's Next Page Prefetcher and AMD's equivalent mechanism can consult the TLB when approaching a page boundary — if the address mapping for the next page is already registered in the TLB, the prefetcher receives clearance to continue prefetching across th

When you expose inventory or checkout endpoints to public-facing front-ends or third-party webhooks, safeguarding those APIs from brute-force scripts, scraping bots, and inventory hoarding algorithms becomes a critical requirement. Without defensive rate limiting, a single coordinated script can easily overwhelm your database connections. The Problem with Simple Counter Resets A common mistake when setting up basic API protection is using a rigid "Fixed Window" counter (e.g., allowing 100 requests per minute, resetting exactly at the turn of the clock). This creates a massive flaw where a developer can flood your server with 100 requests at 11:59:59 and another 100 requests at 12:00:01, effectively doubling your acceptable burst traffic and causing severe performance dips. To handle uneven burst traffic safely without crashing your database, the standard approach is implementing a token bucket algorithm. The Token Bucket Pattern The token bucket algorithm maintains a centralized bucket that holds a maximum capacity of tokens. Tokens are added back to the bucket at a constant, predictable rate over time. Each incoming API request consumes exactly one token. If the bucket is completely empty, the request is instantly rejected with a 429 Too Many Requests status code, protecting your core server threads. javascript // Quick Redis-based token bucket rate limiter concept async function isRateLimited(userId) { const key = `rate:${userId}`; const now = Date.now(); // Use a Redis multi-exec transaction to atomically check and update tokens const [tokens, lastRefill] = await redis.hmget(key, 'tokens', 'lastRefill'); // Calculate token replenishment based on time elapsed... // Return true if tokens <= 0, otherwise decrement tokens and update timestamp }