标签：#systems

Introduction This is the second part of sharing my journey to create a microservices-based backend system, in the previous part, I introduced the system and its services. In this part I am going to mention the main things that happened with me from then until now. Transitioning to online DBaaS platforms The first main thing was to transition my databases from my local device to accessible over the internet. I used Neon and MongoDB Atlas . Although performance-wise it is better to deploy the database on the same server that the whole system so all services can access the databases without network overhead , This solution is better from a point, that is avoiding consuming the free VM instance that I got to host my main services, as with DBaaS platforms I will not require storage for databases or additional overhead to handle the DBMS on a free small resources-constrained VM. Observability Observability is one of the main concerns in any system, it gives the ability for the system to expose what happens inside it without any need to guess and manually trace the code to determine where the error may happened. I utilized OpenTelemetry , as I found it is most used and accepted tool to log, trace and collect metrics about the system and the traffic. Also, I loved that it is not related to specific framework, that is .NET by the way, I loved the idea that it is standalone tool. But logging and tracing and metrics collection will not be beneficial if we do not see it actually and can achieve monitoring from those telemetry data, so I used Grafana Cloud to export the telemetry data to some place that is accessible online and a tool that can generate dashboards and visualizing for the metrics ( Prometheus ), and UI that can show all the telemetry data easily. Deployment Until this point, we have a system that is fully functional and have consistent docker configuration for its services via Docker Compose, and it database is already there and available. Now this is the point at

The Problem: Feature Velocity Was Creating Structural Debt The system originally started as a simple feature delivery backend: A Django API powering agricultural insights Celery workers handling asynchronous processing Independent endpoints for each new capability A growing set of Earth Observation computations (NDVI, NDWI, etc.) At first, it worked. But as more features were added, a pattern emerged: Each feature introduced its own pipeline logic Observability was inconsistent across services API contracts drifted between frontend and backend Debugging required tracing multiple disconnected systems We weren’t scaling functionality. We were scaling fragmentation. The Turning Point: Features vs Platforms The key realization was simple: Features solve user problems. Platforms solve system problems. We were repeatedly rebuilding: Authentication flows Data ingestion logic Processing pipelines API validation layers Monitoring hooks Each feature was solving its own version of these concerns. That is where platform engineering became necessary. The Shift: Introducing a Platform Layer We introduced a platform layer between feature delivery and infrastructure. Instead of building isolated pipelines, we standardized: 1. Unified API Surface All Earth Observation workflows (NDVI, NDWI, and future indices) were normalized into a consistent API contract. Shared request/response structure Versioned endpoints Schema validation through serializers Central routing logic This eliminated endpoint fragmentation. 2. Standardized Processing Pipeline Celery tasks were refactored into a reusable pipeline pattern: Ingestion Validation Computation Storage Publishing Instead of feature-specific workers, we moved toward composable tasks. This allowed new indices or processing logic to plug into the same execution flow. 3. Observability as a First-Class Layer One of the biggest failures in the original system was visibility. We introduced: Structured logging across all services Traceable job IDs

I bombed a system design interview last year — not because I didn't know the architecture, but because I spent the first 5 minutes fighting Excalidraw. So I built SystemDesignBoard — a free, keyboard-first whiteboard specifically for system design interviews. What it does You open it, press a key, and start drawing. No account, no onboarding, no drag-from-a-sidebar friction. R → place a Service node C → place a Database/Cache/Queue A → connect two nodes N → open the scratchpad for scale math The features I'm most proud of Animated connectors that show communication type Instead of just drawing arrows, connectors visually encode how services talk: ⇄ sync — paired dashes (request + ACK) ≋ stream — near-solid fast line with glow (continuous pipeline) This matters in interviews — your interviewer can glance at your diagram and immediately understand the communication pattern. Cloud provider badges Tag any node as AWS (EC2, Lambda, RDS, S3), GCP (GKE, Cloud Run, Firestore), or Azure. Each subtype has its own icon. Trade-off logging Right-click any node → Log Trade-offs → attach your CAP theorem stance, consistency level, and scaling strategy directly to the component. Diagram-as-Code Type: [Mobile App] -> [API Gateway] [API Gateway] -> [Auth Service] [Auth Service] -> [Users DB] [Feed Service] -> [Posts DB x3] [Feed Service] -> [Redis Cache] Hit Apply — it auto-lays out the whole architecture in seconds. Export to animated GIF Export your diagram as a GIF that shows live traffic flow animations. Great for sharing after an interview or in a design doc. Tech stack React + TypeScript + Vite @xyflow/react (ReactFlow v12) for the canvas Zustand + Immer for state with full undo/redo html-to-image + gifshot for PNG/GIF export It's free and open No signup required. Works entirely in the browser. Free during beta. 👉 systemdesignboard.com Would love feedback — especially from anyone who's done system design interviews recently. What's missing? What's annoying? Drop a comment below

The Counterintuitive Truth: Feature Flags Are Not Config Files Most engineers first encounter feature flags as a simple abstraction: a key-value lookup that returns true or false. That mental model works fine for a single service handling a few hundred requests per minute. It becomes actively dangerous at scale. A mature feature flag system isn't a config file with an API wrapper — it's a distributed control plane . The distinction matters architecturally. A control plane manages the real-time behavior of a running system across many nodes simultaneously, with its own consistency guarantees, failure semantics, and propagation latency. That's a fundamentally different design problem than reading a YAML file on startup. One constraint drives every downstream decision: user traffic must never block on a remote flag service call. If evaluation requires a synchronous RPC, you've coupled your request path to the availability and latency of an external system. Netflix's Archaius library enforces this by evaluating flags entirely in-process against a locally-cached configuration snapshot. A network round-trip per evaluation injects 10–50ms of tail latency at p99 — catastrophic when you're competing on streaming start times measured in hundreds of milliseconds. Google, Meta, and Netflix collectively evaluate flags against millions of requests per second with sub-millisecond overhead. That figure is only achievable through local evaluation backed by an async synchronization layer, not RPC. The other failure mode engineers underestimate is flag sprawl . Systems accumulate flags the way codebases accumulate dead functions — gradually, then all at once. I've seen services carrying thousands of flags where fewer than 10% were actively managed. The operational weight alone becomes a liability: which flags are safe to remove? Which ones are kill switches for production behavior that no one documented? Knight Capital's $440M loss in 45 minutes in 2012 remains the canonical cautionar

Failures are obvious. Retries are sneaky. When something fails, everyone notices. An alert goes off. A request errors out. Someone starts investigating. Retries are different. They look harmless. Most of the time, they save the system. But sometimes, retries create bigger problems than the original failure. Imagine an API call times out. No problem. The system retries. But what if the first request actually succeeded and only the response was lost? Now the retry creates: duplicate orders repeated emails inconsistent records workflows running twice The failure happened once. The retry multiplied it. Another thing I've seen: One slow dependency causes requests to pile up. Retries start firing. Those retries create even more traffic. Which slows things down further. Which triggers even more retries. Suddenly, the system is spending more effort retrying than doing useful work. Retries also hide problems. A temporary issue gets retried five times and eventually succeeds. Everything looks normal. Meanwhile: latency increases queues grow users experience delays Nothing technically failed. But the system is getting less healthy. What changed for me is that I stopped treating retries as free. Every retry has a cost. It consumes resources. It increases load. And if actions aren't designed carefully, retries can repeat side effects that should only happen once. Now when I build something, I don't ask: "What happens if this fails?" I ask: "What happens if this runs again?" Because in production, things almost always run again. And if the answer is "bad things happen," the retry mechanism isn't helping. It's making things worse. Failures are part of every system. Retries are too. The difference is that failures usually happen once. Retries can turn one problem into hundreds if you don't design for them. This is something we think about constantly at BrainPack when operating long-running workflows across multiple systems. AI and automation layers make retries even more common, wh

pgrac is an open attempt to rebuild Oracle RAC's core machinery (shared-everything storage, multiple active nodes all writing one database, a cluster-wide change number) on top of PostgreSQL 16. Build log #1 laid out the four problems that fight back. This one is about the problem that turns a node failure into silent data corruption, and the first, deliberately modest, layer pgrac ships against it. The failure mode In a shared-nothing cluster an evicted node is mostly harmless: it owns its own disks, so the cluster routes around it. In a shared-everything cluster the same event is dangerous, because every node writes the same storage. Picture the classic split: node 2 misses heartbeats, the cluster declares it dead and remasters its work elsewhere, but node 2 is not actually dead. It is frozen on a long GC pause, or its interconnect NIC flaked, and it is about to wake up and finish the write it started. Now two nodes believe they own the same blocks, and shared storage will accept both writes. That is not a crash. It is corruption you find three days later. Oracle RAC's answer is I/O fencing: before remastering a dead node's resources, you make certain it can no longer touch the storage, with STONITH, SCSI-3 persistent reservations, or a hardware watchdog. The node is fenced at a layer below its own software, because the whole point is that you cannot trust the dead node's software to behave. That hardware layer is real work, and it is not what pgrac built first. What it built first is the layer above it: an in-process cooperative write-fence, now default-ON. The rest of this is precise about what that does and does not buy you, because "we have fencing" is the kind of claim that is worth less than nothing if it is overstated. A fence needs an authority everyone can agree on You cannot fence on local opinion, because the whole problem is that the dead node disagrees about being dead. Authority has to live on durable, shared, quorum-backed storage. pgrac writes a sm

When organizations scale, a simple monolithic architecture often becomes difficult to maintain, deploy, and scale. This is where microservices come into the picture. However, moving to microservices introduces new challenges: How do we migrate from a monolith safely? How do we handle transactions across multiple services? How do we scale read-heavy applications efficiently? Three popular patterns solve these problems: Strangler Pattern – Monolith to Microservices Migration Saga Pattern – Distributed Transaction Management CQRS (Command Query Responsibility Segregation) – Read/Write Scalability 1. Strangler Pattern Why Do We Need It? Most companies cannot shut down a production monolith and rewrite everything from scratch. A complete rewrite is risky because: Development takes a long time. Existing customers are affected. Bugs can impact business operations. Rollback becomes difficult. The Strangler Pattern allows teams to migrate gradually with minimal risk. What Is It? The Strangler Pattern is a migration strategy where new microservices slowly replace parts of a monolithic application until the monolith is no longer needed. The name comes from the strangler fig tree, which gradually grows around another tree and eventually replaces it. How Does It Work? [Insert diagram here showing Client → API Gateway → Monolith + Microservices] Step 1 All requests go to the monolith. Client | v Monolith Step 2 Introduce an API Gateway (or Controller). Client | v API Gateway | v Monolith Step 3 Extract one module into a microservice. Client | v API Gateway |------> Order Service | v Monolith Step 4 Gradually move more modules. Client | v API Gateway |------> Order Service |------> Payment Service |------> Inventory Service | v Monolith Step 5 Eventually remove the monolith completely. Example Consider an e-commerce application. Initially, everything exists inside one application: Monolith ├── Orders ├── Payments ├── Inventory └── Users Over time: Orders become Order Service Payme

Every non-trivial business operation touches more than one system. An e-commerce order reserves inventory, charges a payment method, and schedules a shipment — three services, three databases. A bank transfer debits one account and credits another across two ledgers that may not even be in the same data center. A cloud VM provisioning workflow reserves a network port, allocates storage, starts the hypervisor, registers billing, and sends a notification — five services, five independent state stores. The question is: what happens when step four fails after steps one through three have already succeeded? In a monolith backed by a single database, the answer is simple: roll back the transaction. The database engine guarantees atomicity; either everything commits or nothing does. But when your workflow spans multiple services, each owning its own storage, there is no transaction boundary that wraps them all. There is no rollback button. Step one through three have already made durable changes to systems that do not know about each other, and step four's failure has left the system in an inconsistent state. This is not a pathological edge case. It is the default condition in any distributed architecture. And it gets worse: the failure might not be a hard error. The network might time out. The billing service might return a 503. You do not know whether step four applied its effect or not — you only know you did not receive a success response. Now what? This is the problem sagas were designed for. Client Inventory Svc Payment Svc Shipping Svc │ │ │ │ 1 │──reserve(item)──►│ │ │ │◄──── 200 OK ─────│ │ │ │ [reserved ✓] │ │ │ │ │ │ 2 │──────────── charge(card, $99) ────►│ │ │◄───────────────── 200 OK ──────────│ │ │ │ [charged ✓] │ │ │ │ │ 3 │─────────────────────── schedule(order) ─────────────►│ │◄─────────────────────────── 503 ──────────────────── │ │ │ │ [no record ✗] │ │ │ │ ╔══════════════════════════════════════════════════════╗ ║ ⚠ Inconsistent state ║ ║ Inventory: it

Rate Limiting and Circuit Breakers in Distributed AI Systems Distributed AI systems are inherently complex, handling massive volumes of requests, variable latency from model inference, and dependencies on external services like GPU clusters, databases, or third-party APIs. Without proper safeguards, a single misbehaving component or a sudden traffic surge can cascade into system-wide failure. Two fundamental patterns— rate limiting and circuit breakers —provide essential protection. This post explores their roles, implementation strategies, and practical Python examples tailored for AI workloads. Why Distributed AI Systems Need These Patterns Consider a typical AI pipeline: a user sends a prompt, which hits a load balancer, then an API gateway, then an inference service (e.g., a large language model), which may call a vector database or a fine-tuning API. Each component has capacity limits: GPU inference servers can handle limited concurrent requests. External APIs (e.g., OpenAI, HuggingFace) impose rate limits. Database connections are finite. Without rate limiting, a single abusive client can exhaust resources. Without circuit breakers, a failing downstream service can cause cascading timeouts and resource exhaustion across the entire system. Rate Limiting: Controlling Request Flow Rate limiting restricts how many requests a client, user, or service can make in a given time window. It prevents resource starvation and ensures fair access. Common Algorithms Algorithm Pros Cons Token Bucket Smooth burst handling, easy to implement Memory per bucket Leaky Bucket Constant outflow rate, simple Less flexible for bursts Fixed Window Simple, low overhead Boundary spikes (reset issues) Sliding Window Smoother than fixed, accurate Slightly more complex For AI systems, token bucket is often preferred because it allows short bursts (e.g., a user sending a batch of prompts) while maintaining a long-term average. Python Implementation: Token Bucket Rate Limiter import time impor

If you’ve used S3, MinIO, or any cloud storage API, it’s easy to assume object storage is just a “cloud folder system.” That assumption is wrong — and it leads to confusion when you start working with distributed systems. Object storage is not a file system. It’s closer to a distributed key-value system with strong durability guarantees and a very specific access model . Once you understand that shift, a lot of cloud infrastructure starts to make more sense. The mental model most people start with When people first see object storage, they imagine something like this: /photos/cats.png /photos/dogs.png A hierarchical file system: folders subfolders files inside directories This is how traditional systems like ext4 or NTFS work. But object storage doesn’t actually work this way. The actual model: key → object Object storage is much simpler at its core: key → value Example: key : photos/cats.png value : <binary data> There are no real folders. “folders” are just string prefixes used for organization. That’s it. Why this design exists This model isn’t accidental. It solves real distributed system problems. Traditional file systems struggle when you try to: scale across many machines replicate data reliably handle partial failures coordinate metadata changes at scale Object storage avoids many of these problems by simplifying the model. Instead of supporting complex file operations, it focuses on: store object retrieve object delete object list objects by prefix Nothing more. The most important design choice: immutability In most object storage systems: Objects are not modified in place. If you “update” a file, what actually happens is: upload a new object replace the key pointer old object becomes orphaned (eventually cleaned up) This is a huge shift from file systems. Why this matters Immutability makes distributed systems easier because: no concurrent write conflicts on the same object replication becomes simpler caching becomes safer failure recovery is easier to rea

Last year I was on a team that pushed 40 million events per day through Kafka. We had consumer lag alerts, rebalancing incidents, and a whole runbook for when the broker got behind. I understood how to operate Kafka. But I did not understand how Kafka works. So I built a tiny one. No dependencies. No Zookeeper. No JVM. Just Python and the core ideas. Here is what I learned. The Three Things Kafka Actually Does People say "Kafka is a message queue." That is not quite right. Kafka is a distributed commit log . It has three jobs: Accept writes from producers and append them to a log Let consumers read from any offset in that log Remember where each consumer group is up to That third one is the thing that makes Kafka different from a traditional queue. A queue forgets a message once it is consumed. Kafka remembers. You can replay. You can have 10 different consumer groups reading the same topic at different speeds. The code to implement this is smaller than you think. brokelite: A Message Broker in 120 Lines import threading import time from collections import defaultdict from typing import Dict , List , Tuple class Partition : """ Append-only log for one partition of a topic. """ def __init__ ( self ): self . _log : List [ Tuple [ int , bytes ]] = [] # (offset, message) self . _lock = threading . Lock () self . _next_offset = 0 def append ( self , message : bytes ) -> int : with self . _lock : offset = self . _next_offset self . _log . append (( offset , message )) self . _next_offset += 1 return offset def read_from ( self , offset : int , max_count : int = 100 ) -> List [ Tuple [ int , bytes ]]: with self . _lock : return [ ( off , msg ) for off , msg in self . _log if off >= offset ][: max_count ] def __len__ ( self ): return self . _next_offset class Topic : """ A topic is just N partitions. """ def __init__ ( self , name : str , num_partitions : int = 3 ): self . name = name self . partitions = [ Partition () for _ in range ( num_partitions )] def route ( self , k

Martin Kleppmann, an associate professor at Cambridge and author of Designing Data-Intensive Applications, discusses the evolution of data systems over the last decade, mainly the shift from monolithic databases to modular building blocks. Kleppmann underlines the importance of moving from cloud-centric data storage systems to decentralised data storage similar to Bluesky’s AT protocol. By Martin Kleppmann

Quorum is an incident command plane built on Amazon Aurora DSQL. The failover story lives in another post. This one is about a narrower question that turned out to be the foundation: when several responders write to the same incident at the same moment, across regions, during the worst minutes of an outage, how do you guarantee the record never forks into two conflicting truths. The answer is two design choices that are really one choice seen from two angles: event sourcing, and DSQL's optimistic concurrency control. The data model is append-only Quorum is event-sourced across four tables. Every state change is an immutable event appended to a log, not an in-place update. The current state of an incident is a fold over its events. There is no UPDATE incidents SET status = ... ; there is an acknowledged event, a note event, a resolved event, and the status you render is computed from them. The event's UUID is its primary key and its idempotency key at the same time. A retried write carrying the same UUID cannot double-apply: the insert collides on the primary key and becomes a no-op. That property sounds minor until you remember what kind of system this is. A tool designed to survive network failure retries writes constantly, and "the responder tapped resolve twice because the first response was slow" must not produce two resolutions. Append-only also suits the domain directly. For an incident system the audit trail is the product, not a side effect. "Who acknowledged this, at what time, and what did the timeline look like at 02:14" is a first-class question for the post-incident review and a compliance requirement in regulated environments. Event sourcing gives you that for free. It also gives DSQL a write pattern it likes, which matters more than you would expect. The stack, briefly TypeScript end to end. Kysely as a typed query builder rather than an ORM, because I wanted type safety without surrendering control of the SQL: on a distributed database the exact shap

I Built a Consistent Hashing Ring in Pure Python and Finally Understood How Cassandra Distributes Data I've been using Cassandra and Redis Cluster for years. I knew consistent hashing was "how they work." But I never truly got it until I built one myself from scratch, in pure Python, with zero dependencies. This post is about what I learned doing that. The Problem Consistent Hashing Solves Imagine you have 3 servers and 1 million keys. The naive approach: server = hash(key) % 3 . It works great until you add or remove a server. Change 3 to 4, and almost every key remaps to a different server. In a caching layer, that means near 100% cache miss. In a database, it means massive data movement. That's the problem consistent hashing solves. When you add or remove a node, only a fraction of keys move. Specifically, 1/n of the keys, where n is the number of nodes. Building the Ring The core idea: place both nodes and keys on a circular number line from 0 to 2^32 (or any large integer). To find which node owns a key, walk clockwise until you hit a node. Here's the minimal version: import hashlib import bisect class ConsistentHashRing : def __init__ ( self , replicas = 150 ): self . replicas = replicas self . ring = {} # hash -> node name self . sorted_keys = [] # sorted hash positions def _hash ( self , key : str ) -> int : return int ( hashlib . md5 ( key . encode ()). hexdigest (), 16 ) def add_node ( self , node : str ): for i in range ( self . replicas ): virtual_key = f " { node } :vnode: { i } " h = self . _hash ( virtual_key ) self . ring [ h ] = node bisect . insort ( self . sorted_keys , h ) def remove_node ( self , node : str ): for i in range ( self . replicas ): virtual_key = f " { node } :vnode: { i } " h = self . _hash ( virtual_key ) del self . ring [ h ] idx = bisect . bisect_left ( self . sorted_keys , h ) self . sorted_keys . pop ( idx ) def get_node ( self , key : str ) -> str : if not self . ring : raise ValueError ( " Ring is empty " ) h = self . _hash

If you're building a modern data stack that requires either high-throughput transaction processing or large-scale analytical workloads, you've likely come across both Vertica and VoltDB (now rebranded as Volt Active Data). While both are distributed relational database management systems (RDBMS), they are architected for completely opposite use cases — choosing the wrong one can lead to 10x higher costs, missed latency SLAs, and poor application performance. In this guide, we break down every key difference between OpenText Vertica and Volt Active Data, with practical examples, real-world use cases, and best practices to help you make the right choice for your team. Table of Contents What is OpenText Vertica? What is Volt Active Data (Formerly VoltDB)? Core Differences Between Vertica and VoltDB Real-World Use Cases: When to Pick Which Best Practices & Common Mistakes Conclusion & Key Takeaways References What is OpenText Vertica? OpenText Vertica (formerly Micro Focus Vertica) is a columnar relational DBMS built exclusively for analytical (OLAP) workloads, first launched in 2005. As of 2026, the latest stable version is 26.1, with native lakehouse and Apache Iceberg export support for modern data ecosystems. Core Vertica Architecture Vertica's design is optimized for fast queries across massive datasets: Columnar storage : Data is stored by column instead of row, enabling significantly higher compression ratios and faster aggregation queries that only access a small subset of columns Massively Parallel Processing (MPP) : Query execution and data are distributed across hundreds of nodes for parallel processing Dual deployment modes : Enterprise Mode : Shared-nothing architecture with data stored locally on nodes for maximum performance Eon Mode : Compute and storage separated, using shared object storage (S3, GCS, ADLS) to scale compute independently of storage for cloud workloads Projections : Physical, sorted copies of data optimized for common query patterns (ins

なぜ“特別な決済システム”の話は危険なのか？ 近年、SNSやメッセージアプリを通じて、「特別なPOS端末」や「秘密の決済システム」に関する話を目にすることがあります。 「通常の銀行システムを経由しない」 「オフラインでも大金を受け取れる」 「特別なカードと専用POSがあれば送金できる」 こうした説明は一見すると高度な金融技術のように聞こえます。 しかし、実際の決済システムを理解すると、多くの主張が現実的ではないことが分かります。 まず、POS端末とは何か？ POS（Point of Sale）端末は、店舗でクレジットカードやデビットカードによる支払いを処理するための装置です。 一般的な決済は以下のような流れで行われます。 顧客 ↓ POS端末 ↓ 加盟店契約銀行 ↓ カードブランド ↓ カード発行銀行 ↓ 承認または拒否 重要なのは、最終的な資金の確認を行うのはカード発行銀行であるという点です。 POS端末そのものが資金を生み出すことはありません。 「オフライン決済だから大丈夫」は本当か？ 一部の詐欺では、 「この端末はオフラインで動作する」 という説明が行われます。 確かに、現実の決済システムにはオフライン処理が存在します。 しかし、それは通信障害時の一時的な仕組みであり、最終的には銀行側との照合が行われます。 つまり、 オフライン処理 ≠ 資金の創造 です。 銀行が承認していない資金は、後の精算時に拒否される可能性があります。 なぜ人は信じてしまうのか？ 理由は単純です。 専門用語が多いからです。 例えば、 決済ネットワーク 国際ブランド オフライン認証 ISO規格 特殊プロトコル こうした言葉が並ぶと、本物らしく見えます。 しかし、本当に重要なのは技術用語ではありません。 重要なのは、 「お金はどこから来るのか？」 という一点です。 詐欺を見抜くための3つの質問 1. お金の出所はどこか？ 利益や送金の原資を説明できない場合は要注意です。 2. 誰が監督しているのか？ 銀行、決済事業者、規制当局など、責任主体が明確か確認しましょう。 3. 第三者による検証は可能か？ 説明が内部関係者の証言だけに依存している場合は危険です。 テクノロジーと金融リテラシー 新しい技術は私たちの生活を便利にします。 しかし、技術的な言葉が使われているからといって、その仕組みが正しいとは限りません。 本当に優れた金融サービスほど、 透明性が高い 説明が分かりやすい リスクが明示されている という特徴があります。 逆に、 「秘密」 「特別」 「限定」 「誰にも教えないでほしい」 といった言葉が頻繁に出てくる場合は、一度立ち止まって考えるべきです。 まとめ 金融詐欺の多くは、技術ではなく心理を利用します。 人々はお金を失うから騙されるのではありません。 「理解したつもりになる」から騙されるのです。 だからこそ、最も重要な防御策は、 「そのお金はどこから来るのか？」 というシンプルな質問を忘れないことです。 金融の世界に魔法はありません。 あるのは、透明な仕組みと説明可能な資金の流れだけです。

Removing expf() from a fire detector: one header, 1.95x faster, zero accuracy loss A smoke detector is not a demo project. When it fires, someone either evacuates in time or doesn't. The firmware running on that microcontroller has one job, and it needs to do it without hesitation, without bloat, and without dependencies that can fail in unexpected ways. Last May 28th I published a bare-metal fire detection system built with Hasaki 刃先 — a neural network trainer that exports standalone C headers with no runtime, no Python, no TensorFlow. The model is a 12-8-4-1 MLP trained on 28,596 sensor readings. It fits in 3.8 kB of Flash and achieves 99.93% accuracy on held-out data, with a single missed fire event out of 3,599. But there was something in that header that bothered me. static inline float sigmoid ( float x ) { return 1 . 0 f / ( 1 . 0 f + expf ( - x )); } expf() . Right there in a life-safety application. On a microcontroller that may not have a hardware FPU. The problem with expf() on bare metal On processors with a hardware FPU — like the ESP32-C3 — expf() is fast. But the moment you deploy to an ATmega328P, an ATtiny85, or any Cortex-M0 target, that call becomes software floating-point. The CPU has to simulate the operation in firmware, cycle by cycle. It works. But it carries hidden cost: unpredictable latency, dependency on math.h , and a transcendental function sitting in the critical path of every single inference. For a smoke detector running at 1 Hz this might seem irrelevant. But inference latency compounds with sensor reads, normalization, and communication overhead. And more importantly — if you're deploying to a truly constrained target, expf() might be the difference between fitting in Flash or not. The fix: one header from kigu-quant kigu-quant(comming soon) is a new tool in the Rosito Bench ecosystem. It generates ready-to-include C headers for evaluating mathematical functions on microcontrollers — no FPU, no libm, no dependencies. One command: k

Lyft details a new pickup experience to improve reliability in gated communities, where 25–30% of rides face routing and access challenges. The system uses mapping signals, boundary detection, and routing improvements to reduce cancellations and coordination overhead between riders and drivers, highlighting how real-world constraints drive evolution in geospatial systems. By Leela Kumili

Most developers never have to design a network protocol from scratch. You use HTTP, gRPC, WebSockets, or something else that already exists and has been debugged by thousands of people over many years. That is the right call for most situations. I did not take that path when building Vaylix, a key-value database engine. I designed a custom binary protocol called VTP2, and the process taught me things about networking that I would not have picked up any other way. This is not an argument that you should also build a custom protocol. For most things, you should not. This is an honest account of what I ran into. Why not HTTP The first question anyone reasonably asks is: why not just use HTTP? HTTP is everywhere. The tooling is excellent. Every language has a client. Debugging with curl is trivial. If I had used HTTP, I would have had working client libraries in a dozen languages before writing a single line of server code. The problem is that HTTP is stateless by design. Every request is independent. Every request carries headers. Every response carries headers. The model assumes that each round trip is a fresh conversation with no memory of what came before. A database session is the opposite of that. A client connects, authenticates, and then issues many commands over the same connection. The authentication should happen once. The session should carry state. Pipelining requests without waiting for each response to return should be natural, not something you fight the protocol to achieve. HTTP/2 closes some of this gap. But using HTTP/2 correctly for a stateful session model involves working against the grain of what HTTP was designed for. I would have been spending a lot of time on infrastructure that exists to make HTTP behave less like HTTP. The other issue is overhead. HTTP headers are verbose. For small key-value operations, the headers can easily exceed the payload. That felt wrong for something designed to be a tight operational data store. So I went with TCP d

I built a distributed compute grid where your idle laptop runs ML jobs — the orchestrator behind it The pitch: a single FastAPI hub takes compute jobs from ML researchers, and a fleet of home PCs and gaming rigs (RTX 4090s, M2 MacBooks, anything with a GPU and a Python interpreter) polls in, picks up work, and ships results back. A 20% platform fee funds the hub. An interactive dashboard shows the mesh in real time. I have been living inside this codebase for a few weeks. This post is about the part that actually determines whether the thing works or does not — the orchestrator . No frontend, no marketing — just the brain. Live dashboard: man44.zo.space/compute-pool Repo: github.com/AmSach/ComputePool-Grid The problem with "dumb" schedulers The first version of ComputeOrchestrator had a one-line bug that took down a 12-node stress test. Two jobs hit the hub at the same millisecond. Both saw the same node as idle . Both wrote busy to the same row. One node ended up double-allocated, the other starved, and the test logs looked like a hostage negotiation. The fix had to be three things at once: A scoring function that picks the right node, not just the first idle one. An async lock so concurrent submissions cannot race on a single node. A heartbeat monitor that reclaims nodes that ghosted. Here is what it looks like now. The scoring algorithm def _calculate_score ( self , capacity : Dict [ str , Any ], requirements : Dict [ str , Any ]) -> float : """ Heuristic for node-task matching. """ score = 0.0 if capacity . get ( " gpu_vram " , 0 ) >= requirements . get ( " min_vram " , 0 ): score += 10.0 if capacity . get ( " cpu_cores " , 0 ) >= requirements . get ( " min_cores " , 0 ): score += 5.0 return score The weights are deliberately lopsided. A node that satisfies a job's VRAM requirement gets a 2x bonus over a node that just barely has enough cores. The intuition: GPU work is the long pole. If you cannot fit the model in VRAM, nothing else matters, no matter how many