标签：#net

Imagine opening a website from India while its servers are located in the United States. You request an image. Your request travels thousands of kilometers to the server, the server processes it, and the response travels all the way back to you. It works. But what happens when millions of users around the world do the same thing? This is where a CDN (Content Delivery Network) comes in. A CDN helps websites deliver content from servers that are geographically closer to users, reducing latency, improving performance, and taking load away from the main server. In this article, we'll understand how CDNs work, why they're important, and how they're used in large-scale systems. What Is a CDN? A Content Delivery Network is a globally distributed network of servers that stores and delivers frequently requested content closer to users. Without a CDN, requests might look like this: User ↓ Main Server ↓ Content With a CDN, a distributed layer is added between users and the origin server: ┌── CDN Edge Server ── User (India) │ Origin Server ────┼── CDN Edge Server ── User (Europe) │ └── CDN Edge Server ── User (USA) The main server is called the origin server . The distributed servers are commonly called edge servers or Points of Presence (PoPs) . Why Do We Need a CDN? Without a CDN, users from different parts of the world may have to communicate with the same origin server. For example: User in India ───────┐ User in Germany ─────┤ User in USA ─────────┼──→ Origin Server User in Japan ───────┘ As traffic grows, this creates several problems: Higher latency More traffic reaching the origin Increased server load Slower image and video delivery Poor performance for users far away from the server A CDN solves this by distributing frequently requested content geographically. How Does a CDN Work? Suppose your website contains an image: /images/product.jpg A user in India requests it. Instead of immediately contacting your origin server, the request goes through the CDN: User ↓ CDN ↓

Microsoft has released .NET 11 Preview 7 with a substantial set of .NET MAUI updates, including cross-platform passkey authentication, a new incremental XAML Hot Reload implementation, Shell route templates, and additional AOT-safe bindings. The release also continues MAUI’s migration from legacy renderers to handlers and improves development workflows on Android and Apple platforms. By Edin Kapić

A bad cutout rarely announces itself as a bad cutout. The car lands on a new backdrop, the paint looks clean, then a thin piece is gone. An antenna. A tire lip. The dark seam under a rocker panel. The complaint that comes back is never technical. The vehicle looks wrong. I wanted the last correction stage to fix fuzzy edges without handing it the whole car to rewrite. That sounds like a small distinction. It stops being small the first time a model improves one boundary and quietly damages another. So the rule is physical. Edit the uncertain strip. Leave the settled area alone. This is Part 2. Part 1, "Negative Space Is a Label", was about supervision: what the pixels beside an object teach a model, and why a shadow touching a tire has to be labeled as evidence against foreground. This one moves from training to runtime. A mask already exists. Where is a learned stage allowed to act? 1. The contract lives in the band CarSegNet is the research implementation here. Its pipeline module splits the route by media type, and the docstring says the design more clearly than any diagram I could draw after the fact. Stills run SAM 3 text concept, then NSJ alpha, then composite. A detector box prompt and a depth prior are optional inputs. Video runs SAM 3.1 multiplex propagation, per-frame NSJ with temporal handling, a depth-parallax plate, composite, encode. The list matters less than the handoff. SAM gives a semantic prior. NSJ receives a trimap band. The compositor receives a matte only after the prior and the refiner have each done bounded work. flowchart TD image[Vehicle Image] segment[Concept Mask] trimap[Trimap Band] refiner[NSJ Alpha Refiner] depth[Depth Prior] composite[Showroom Composite] frozen[Prior Frozen Outside Band] image --> segment segment --> trimap trimap --> refiner image --> depth depth --> refiner refiner --> composite segment -.-> frozen frozen --> composite The diagram is a contract. It is not a model zoo. The refiner edits the uncertain strip. The sema

xUnit 4.0.0 makes full test-case parallelization an explicit option. That is useful, but xUnit 4 ParallelMode.All changes a quiet assumption in many suites: tests in the same class, including separate rows of one theory, may now overlap. A static fake, shared fixture, temporary file, or database record that was safe under collection-level parallelism can become a race. I treat this as an isolation change, not a speed switch. Before enabling it across a suite, I want a deterministic failure that proves the risk and a deterministic check for each guardrail. What xUnit 4 ParallelMode.All changes The xUnit.net v3 4.0.0 release notes describe full test-case parallelization as a new feature. The default is still ParallelMode.Collections , so upgrading does not silently enable the broader mode. I have to opt in at the assembly level: using Xunit.Sdk ; using Xunit.v3 ; [ assembly : Parallelization ( Mode = ParallelMode . All , MaxThreads = 2 , Algorithm = ParallelAlgorithm . Conservative )] With Collections , tests within a collection are serialized. With All , every test case is eligible to run beside every other test case. That includes two cases from the same class and two pre-enumerated rows from the same theory. The official parallel test execution guide documents the modes, algorithms, and available opt-out scopes. I set MaxThreads = 2 in the sample so the scheduling condition is easy to inspect. It is a demonstration setting, not a recommendation for CI. The right value depends on available CPU, memory, and the external systems touched by the tests. Before changing the mode, I scan for mutable static fields, IClassFixture and ICollectionFixture implementations, fixed file names, environment-variable changes, test servers bound to fixed ports, and records addressed by shared IDs. I also check theory data sources for objects that rows can mutate. That inventory tells me whether the resource should become concurrency-safe, receive a unique per-test identity, or stay beh

Created a VM with hardware from information in one of the many online tutorials on this topic, that is a few years old, and recommends 8GB HDD space on the VM Started the VM with the OPNsense installer DVD ISO mounted on virtual dvdrom drive Went through installer steps to the end where the installer says No space left on device Luckily... I found a forum post where someone mentions the swap partition alone consumes 8GB now, and the VM HDD needs to be 20-30GB Searched ProxMox docs and find the option to expand the size of disk, and enlarge the virtual HDD to 30GB Restarted, but HDD boot "bit" is already set from the "swap partition only" failed install, so the ISO installer won't "try again" and offers no option to wipe the HDD and start over. Decided to drop and re-create the HDD, so I "detached" it, but didn't notice that it hangs around until you "remove" it also, in a separate step. Added a new "blank" HDD... 32GB this time Did another full boot from ISO -> install process... which seemed to work this time (found enough disk space for swap AND install) The reboot at the end boots from installer ISO again. None of the tutorials I found mention that the ISO image must be unmounted before the reboot. Stopped the VM and unmounted the ISO from virtual dvdrom drive Restarted the VM but now the boot process / BIOS won't do anything but PXE network boot. 13 Noticed the initial "unused" HDD and used "remove" to finish getting rid of it. Restarted the VM and went through install process again (not sure why boot bit on the virtual HDD wasn't blocking it this time). Unmounted the ISO from the virtual dvdrom drive again, and rebooted again, but the BIOS is still skipping any attempt to boot from the HDD and tries to do PXE network boot again. Assumed that no boot from HDD might be because the attached HDD is the 2nd one added and has ID=1 (not 0). This might be adjacent to the actual problem (more on that later), but probably wasn't the actual cause. Detached AND "removed" A

When every unauthorized request becomes the same dashboard line, diagnosis turns into guessing. ASP.NET Core 10 authentication metrics give me a better split: did the handler have nothing to authenticate, reject supplied credentials, or accept them? That distinction matters because a client deployment that drops credentials needs a different response from a surge of malformed or expired credentials. ASP.NET Core 10 added built-in authentication and authorization instruments to System.Diagnostics.Metrics . I can collect them without rewriting each handler, and I can lock their behavior into an offline test before wiring up a production exporter. Why one 401 hides two different problems A protected endpoint normally challenges an unauthenticated caller. The final status is 401 whether the caller sent nothing or the handler rejected what it received. The authentication duration histogram exposes the missing context through aspnetcore.authentication.result : Result What the handler reported A common interpretation none No authentication result No applicable credentials were available failure Authentication failed Supplied credentials were rejected or processing failed success A principal was created Authentication completed successfully _OTHER Another framework result Preserve it as an explicit catch-all none is a handler result, not a universal synonym for “missing Authorization header.” A policy scheme or custom handler can make a different choice. I verify the behavior of the schemes I actually deploy instead of building an alert from the label alone. Likewise, success means the handler produced an authentication ticket. Authorization can still deny that principal, so it does not promise a 2xx response. The separate aspnetcore.authentication.challenges counter answers another question: how often was a scheme challenged? Both a none result and a failure result can be followed by a challenge, so challenge count cannot replace the result split. A challenge is an authent

🌐 Network Devices Explained The Foundation Every Cloud & DevOps Engineer Needs Series: Networking Fundamentals for Cloud & DevOps — Part 1 of 6 Before VPCs, subnets, route tables, and security groups make sense, you need to understand what's happening beneath them. This series builds that foundation — starting with the devices that make networks work. Why Networking Before Cloud? I hit a wall during my AWS VPC sessions. Route tables, subnets, gateways, NACLs — the concepts existed in isolation. I could follow steps in the console, but I couldn't reason about why traffic was or wasn't flowing. The fix wasn't more AWS documentation. It was going back to networking fundamentals. Once I understood what a router actually does — how it makes forwarding decisions, what a routing table really is — the AWS route table stopped being a mysterious config screen and became something I could think through. That's what this series is. Six posts covering the networking concepts that directly underpin Cloud and DevOps work. No exam prep framing, no CCNA depth. Just what you actually need. 1. What is a Host? A host is any device that participates in network communication by sending or receiving traffic. That's broader than most people assume. Examples: your laptop, your phone, an EC2 instance, a web server, a virtual machine. The word "host" doesn't imply a server — your laptop is a host just as much as a data center machine is. 2. Client vs Server — Roles, Not Hardware A client is a host that initiates a request. A server is a host that responds. The critical point: a server is not a special type of computer . It's just a computer running software that listens and responds. Your Browser (Client) │ │ HTTP Request ▼ Web Server (Server) │ │ HTTP Response ▼ Your Browser (Client) The same machine can be a client in one communication and a server in another. Your EC2 running a web app is a server to users hitting it — and a client when it queries RDS. 3. IP Address — The Network Identity

ASP.NET Core Output Caching: How to Make Web APIs Faster in .NET When an API receives the same request repeatedly, performing the same database query and rebuilding the same response every time can waste valuable resources. For example, imagine this endpoint: GET /api/products If thousands of users request the same product catalog, your application might repeatedly: HTTP Request ↓ Controller ↓ Database Query ↓ Business Logic ↓ JSON Response For data that doesn't change frequently, this can create unnecessary database load. ASP.NET Core provides Output Caching to help solve this problem. Instead of executing the complete request pipeline every time, the application can temporarily store the generated response and reuse it for subsequent requests. In this tutorial, we'll look at how Output Caching works, how to configure it, how to invalidate cached responses, and when you should avoid using it. What Is Output Caching? Output caching stores the generated response from an endpoint. For example: First request ↓ GET /api/products ↓ Execute controller ↓ Query database ↓ Generate response ↓ Store response in cache Later: Second request ↓ GET /api/products ↓ Cached response ↓ Return immediately The database doesn't need to be queried again while the cached response is valid. Output Caching vs Response Caching These two concepts are often confused. Response Caching Response caching mainly relies on HTTP caching semantics and headers. Output Caching Output caching is controlled by ASP.NET Core and allows your application to decide which responses should be cached and for how long. Output caching provides more control over server-side response caching. 1. Add Output Caching Start by registering the output-cache services. var builder = WebApplication.CreateBuilder(args); builder.Services.AddControllers(); builder.Services.AddOutputCache(); var app = builder.Build(); app.UseOutputCache(); app.MapControllers(); app.Run(); The important pieces are: AddOutputCache() ↓ Configure cac

The Problem In one of our Azure DevOps pipelines, nuget restore suddenly started failing with an error stating, in essence, that the requested package could not be found in the referenced version. The task referencing the package hadn't changed — yet the restore stage kept failing. At first glance, this looks like an issue with the package source, some caching effect, or a broken .nuspec/lockfile. It wasn't. The Root Cause The actual culprit was the version of the NuGetToolInstaller@1 task itself. The pipeline had NuGet pinned to version 6.12.2. The Fix Bump the versionSpec in the NuGetToolInstaller@1 task from 6.12.2 to 7.9.0: - task : NuGetToolInstaller@1 displayName : ' Use NuGet 7.9.0' inputs : versionSpec : 7.9.0 checkLatest : false That's it. After the update, nuget restore ran through cleanly again.

The Quest Begins (The “Why”) Honestly, I used to stare at my CI/CD yaml files like they were ancient runes. Every push felt like a gamble: “Will the build pass this time?” I’d spend Friday nights hunting down a missing node_modules cache in Jenkins, only to realize the agent had run out of disk space because I’d forgotten to add a cleanup step. The pain was real, and the feedback loop was slower than a dial‑up modem. I kept asking myself: Why does this feel like wrestling a dragon every time I want to ship a feature? The answer was simple—I hadn’t yet found a pipeline that just worked out of the box. I wanted something that gave me confidence, not anxiety. So I embarked on a quest to compare the three big contenders: GitHub Actions, GitLab CI, and good ol’ Jenkins. Spoiler: the treasure wasn’t in the tool itself, but in how you shape the pipeline around your team’s flow. The Revelation (The Insight) The big “aha!” moment came when I stopped treating CI/CD as a one‑size‑fits‑all script and started seeing it as a contract between my code and my environment. The contract says: Every commit gets a clean slate. Dependencies are restored, not guessed. Tests run in parallel, not sequentially. Artifacts are published only if the gate passes. When I wrote that contract down, the yaml stopped looking like magic incantations and started looking like a checklist. The tools differ in syntax, but the underlying principles are the same. Here’s the secret: cache wisely, fail fast, and keep the pipeline short enough to give you feedback before you’ve even finished your coffee. Wielding the Power (Code & Examples) Below are three pipelines—one for each platform—that embody the contract above. I’ll first show a “struggle” version (the common pitfalls) and then the victorious version. 1. GitHub Actions – The Struggle name : CI on : [ push , pull_request ] jobs : build : runs-on : ubuntu-latest steps : - uses : actions/checkout@v3 - name : Install deps run : npm install # <-- no cache,

A two-factor flag in the user store looks like a reassuring authorization check. It tells us the account has a second factor configured. For a sensitive operation, however, that is only half the question. The other half is about the session in front of us: did this cookie actually complete a second-factor challenge? Those facts can change independently. Treating them as interchangeable can silently promote an old password-only session after the account enables MFA. Two questions that look like one Account capability answers questions such as: Is a factor enrolled now? Could the account complete an MFA challenge? Has that capability since been disabled? Session assurance answers different questions: Which authentication steps produced this session? Did the framework issue this cookie after an MFA challenge? Is the evidence trusted, or merely a user-supplied claim? An enrolled account can still have a password-only session. A previously verified session can also outlive a later change to the account’s factor state. One signal cannot safely stand in for both. The transition that exposes the gap Snapshot tests often miss this because the final state looks correct. The account has MFA enabled, the user is authenticated, and a policy succeeds. Now test the transition instead: Sign in with a password and receive a normal application cookie. Enable MFA for the account without replacing that cookie. Use the original cookie against a sensitive operation. If authorization checks only the current enrolment flag, step three may succeed. Nothing about the original authentication ceremony changed, but the session has effectively been upgraded by a later database write. That is the important boundary: changing account capability must not rewrite the history of an already-issued session. Use two independent signals A generalized policy can be expressed like this: if (! session . IsAuthenticated || ! session . HasTrustedMfaEvidence ) return Deny ; if (! await accountStore . IsMfaStil

Your AI Slop Bores Me is brilliant in its simplicity. There are two tabs: human and LARP as an AI. On one side you enter a request. On the other, you submit an answer. But the important thing is that there's a human on both sides of the equation. Prompts can request a response as […]

The difference between a good infrastructure troubleshooter and someone who restarts services and hopes is a mental model. When "HTTPS times out" lands in your inbox, you don't guess — you know exactly which layer to interrogate first, and in what order. The network is a stack, so treat it like one Every request rides through the same layers, top to bottom: Application → TLS → Port → DNS → Gateway → Route → Interface That's the dependency order — TLS can't work if the port is closed, the port is meaningless if DNS resolved to the wrong host, and none of it matters if your interface has no IP. So you verify in the inverse order, from the ground up: Interface → IP → Route → Gateway → DNS → Port → TLS → Application Start at the bottom because a broken lower layer produces confusing symptoms higher up. Confirm each layer is healthy before you climb. The moment a layer fails, you've found your problem — everything above it is a red herring. Walk it: "HTTPS to api.example.com times out" 1. Interface — do we have a link and an address? ip addr show Look for your primary interface (say eth0 ) in state UP with an inet line like 192.168.1.20/24 . No inet ? DHCP failed or the link is down — stop here, nothing above will work. If the address is present and sane, climb. 2. Route — is there a path to the destination? ip route get 93.184.216.34 This shows the exact route the kernel would pick, including the source IP and gateway ( via 192.168.1.1 dev eth0 src 192.168.1.20 ). If you get "Network is unreachable" or no default route, you've found it. This is also the signature behind the classic curl error "No route to host." 3. Gateway — can we reach the first hop? ping -c3 192.168.1.1 ip neigh show ping tests reachability; ip neigh shows the ARP table. A gateway entry in state REACHABLE with a MAC address means L2 is fine. FAILED or INCOMPLETE means the gateway isn't answering ARP — a VLAN, cabling, or firewall problem. Note that many hosts drop ICMP, so treat a failed ping as a hi

The Quest Begins (The "Why") I remember the first time I tried to take a weekend side‑project from my laptop to something that felt “real”. I had a cute Express API that talked to Postman, a PostgreSQL container spun up with docker-compose up , and a React front‑end that lived in its own dev server. Everything worked beautifully … until I hit Ctrl+C on my laptop and the whole thing vanished. I needed a way to say, “Hey, keep this running even if I close my laptop, and if something crashes, bring it back up automatically.” I started poking at Docker Swarm, then Nomad, but the docs felt like reading ancient runes. That’s when a coworker slid over a Slack message: “Just try a Kind cluster. It’s K8s locally, and you’ll see why everyone talks about it.” Spoiler: it felt like discovering the secret level in a classic arcade game. Suddenly I could describe what I wanted my system to look like, and the cluster would make it happen — no more babysitting containers. The Revelation (The Insight) Kubernetes isn’t a mystical black box; it’s a declarative orchestrator . You tell it the desired state of your application (how many replicas, which image, what ports to expose) and it works relentlessly to match reality to that state. If a pod dies, Kubernetes spins up a new one. If you ask for three replicas and only two are running, it creates the missing pod. If you update the image tag, it rolls out the change pod‑by‑pod, keeping traffic flowing. Think of it like the save‑game system in a RPG: you define the story you want to experience, and the engine handles the gritty details of loading, saving, and recovering from crashes. The core objects you’ll meet early on are: Pod – the smallest deployable unit (one or more tightly coupled containers). Deployment – manages a set of identical pods, handles updates and rollbacks. Service – a stable network endpoint that load‑balances traffic to a set of pods. Ingress (optional) – exposes HTTP/HTTPS routes from outside the cluster to service

I built and open-sourced PacketVoyage —an Agent Skill & MCP server that turns boring traceroute outputs into fascinating stories about physics, geography, and undersea cables. europeanplaice / packetvoyage MCP server & Agent Skill for educational network traceroute analysis, fiber-optic physics verification, and packet voyage storytelling 🚢 PacketVoyage Model Context Protocol (MCP) Server & Agent Skill for educational network traceroute analysis, fiber-optic physics verification, and packet voyage storytelling. Zero external commercial APIs, zero bundled copyright data — pure physical laws and detective insight. 🏛️ Architecture: The Two Pillars PacketVoyage is built around two complementary layers designed specifically for AI-native workflows: ┌────────────────────────────────────────────────────────┐ │ AI Agent (LLM) │ └──────────────┬──────────────────────────┬──────────────┘ │ │ ▼ ▼ ┌──────────────────────────────┐ ┌──────────────────────────────┐ │ 🧠 Agent Skill │ │ 🛠️ MCP Server │ │ (Knowledge / Playbook) │ │ (Capabilities / Execution)│ ├──────────────────────────────┤ ├──────────────────────────────┤ │ • Speed of Light in Fiber │ │ • analyze_voyage_text │ │ (~0.67c, ~10ms / 1,000km) │ │ • voyage_investigate │ │ • Control vs Data Plane math │ │ • run_protocol_experiment │ │ • Disproving GeoIP illusions │ │ • research_host │ │ • Decision Flow & Heuristics │ │ • list_known_iata_airports │ └──────────────────────────────┘ └──────────────────────────────┘ 🛠️ MCP Server (Capabilities & … View on GitHub Ever wondered what’s actually happening behind a trace like this? 1 gateway (192.168.1.1) 0.8 ms 2 * * * 3 ae-1.tokyo-hnd.bb.net (203.0.113.1) 2.1 ms 4 xe-0-0.sjc-core.bb.net (198.51.100.25) 88.5 ms 5 one.one.one.one (1.1.1.1) 88.7 ms Behind these lines lies real-world physics: • The * * * at Hop 2 isn't packet loss: Normal traffic runs at line rate in hardware ASICs (Data Plane), while diagnostic ICMP responses are rate limited by router CPUs (Control Plane). • The +

Cloud computing is a fundamental pillar of modern network architecture, shifting infrastructure management from physical data centers to flexible, virtualized environments. This guide breaks down core cloud concepts, architecture models, service types, and operational characteristics aligned with CompTIA Network+ objectives. Virtualization and Network FoundationsNetwork Functions Virtualization (NFV)NFV replaces dedicated, proprietary hardware appliances (such as firewalls, load balancers, and routers) with virtual appliances running on standard servers. This decouples network functions from physical hardware, allowing for rapid deployment, easier scaling, and reduced capital expenditure.Virtual Private Cloud (VPC)A Virtual Private Cloud (VPC) provides an isolated, private cloud environment dedicated to a single customer within a shared public cloud infrastructure.Resource Separation: Uses subnets, VLANs, and tunneling to isolate compute, storage, and networking resources.Control: Customers have full administrative control over their network configuration, IP address ranges, and routing tables.Security: Regulated via Network Security Groups (NSGs) and Access Control Lists (ACLs) to govern traffic entering and leaving subnets.Cloud Gateways & Connection MethodsCloud gateways serve as translation points or secure entryways between on-premises networks and cloud environments. Organizations connect to cloud resources using several methods:Site-to-Site VPNs: Encrypted tunnels over the public internet connecting an on-premises office or data center to a VPC.Dedicated Interconnects (e.g., AWS Direct Connect, Azure ExpressRoute): High-speed, private, dedicated circuits that bypass the public internet for enhanced security, lower latency, and predictable performance. Cloud Deployment ModelsCloud architecture defines where infrastructure is hosted and who manages the underlying hardware.ModelCharacteristicsBest Suited ForPublic CloudOwned and operated by a third-party provide

This is Level 5 of our Kubernetes networking series. So far, we've built up a solid foundation: LEVEL 1 — Pod networking LEVEL 2 — Pod-to-Pod communication LEVEL 3 — Service (a stable internal endpoint) LEVEL 4 — DNS (service name → Service IP) But we still have a glaring gap: how does a real user on the internet actually reach your Kubernetes application? That's exactly what this article covers — Ingress, Ingress Controllers, and the newer Gateway API. Table of Contents The Problem: The Internet Can't Reach a ClusterIP The Basic Solution: Ingress and Gateway API What Is Ingress? A Routing Example Ingress Is Not the Actual Proxy A Simple Analogy: Traffic Police A Basic Ingress YAML Example Breaking Down the Key Fields Host-Based Routing Path-Based Routing Why Not Just Use a LoadBalancer Service for Everything? The Complete Traffic Flow Where Does DNS Fit In? The Ingress Controller A Typical Architecture Ingress vs Service Ingress vs LoadBalancer Service HTTPS and TLS Termination Why Terminate TLS at the Edge? Referencing a TLS Certificate Routing Multiple Domains The Gateway API GatewayClass, Gateway, and HTTPRoute Ingress vs Gateway API Important Distinctions: Ingress Is Not CNI or Service Troubleshooting Ingress Layer by Layer Common Ingress Mistakes The Complete Kubernetes Networking Picture (Levels 1–5) The Mental Model to Memorize Level 5 Checkpoint What's Next: NetworkPolicy The Problem: The Internet Can't Reach a ClusterIP Suppose you want users to reach your application at myapp.example.com . Inside your cluster, you have: Service : frontend ClusterIP : 10.96.20.10 frontend Service ├── Pod 1 ├── Pod 2 └── Pod 3 A user on the internet can't simply visit http://10.96.20.10 — that's a private Kubernetes Service IP, invisible outside the cluster. We need something sitting at the edge of the cluster to bridge that gap. The Basic Solution: Ingress and Gateway API Historically, Kubernetes solved this with Ingress . More recently, Kubernetes introduced a more expres

How we turned always-on Kafka sinks into on-demand workers that shrug off nightly bombardments — by scaling on the right signal, tuning per-pod drain rate, and keeping autoscaling from sabotaging itself. Every number in this post is measured from a local lab you can run yourself — the full code is on GitHub , and the Appendix has the commands. The problem We run a fleet of Kafka sinks — consumer services that read change events from Kafka, apply business logic, and write the result into a service-local database as a query-friendly materialized view. It keeps reads fast and independent from upstream systems, and it's a great pattern. But the workload has an awkward shape. Most sinks are idle most of the day, then buried in minutes. Traffic isn't steady: changes arrive in bursts, usually from nightly imports or CDC jobs. The rest of the day the topic is quiet. topic activity over 24h msgs ▲ │ ██ nightly import / CDC burst │ ██ │______________██______________ flat, idle ~22h/day └───────────────────────────────▶ time That shape creates two problems at once : Idle waste. When the topic is quiet, each sink still runs — it polls Kafka, holds connections, emits metrics, and occupies CPU and memory. Multiply one "small" sink across dozens of them and several regions, and you're paying around the clock for work that happens for a couple of hours a night. Spike lag. When the burst lands, a backlog builds fast. If consumers can't drain it quickly enough, consumer lag — the gap between what's been produced and what's been processed — climbs, and downstream reads start serving stale data. We want two things that sound contradictory: cost almost nothing when idle , and absorb the spike fast when it hits. Why the obvious autoscaler doesn't help The reflex is a Kubernetes Horizontal Pod Autoscaler (HPA) on CPU or memory. For sinks, that's the wrong signal. Sink work is I/O-bound : the consumer spends its time waiting on Kafka polls and database writes, not burning CPU. So when a ba

Originally published at prepstack.co.in Part 4 of 4 — 14 Years of Enterprise ASP.NET (finale). Where the system actually runs: choosing Azure architecture by cost and scaling profile, making the system observable, and treating AI as a real architectural component — not a demo. Running example: Mattrx — .NET 9 / ASP.NET Core, 110k MAU, Azure SQL, ~3,200 req/sec peak. Lesson 10 — Azure: match the platform to the workload Pick the compute by your scaling and operational profile, then right-size — don't default to the biggest box or the trendiest platform. Most enterprise .NET runs perfectly on Azure App Service; you reach for Container Apps or AKS when you have a specific reason, not because Kubernetes is on your résumé. The decision framework: App Service for standard web/API (default), Container Apps when you want containers + scale-to-zero without running a cluster, AKS only when you genuinely need its control plane and have the ops capacity. A 5-person team has no business running Kubernetes. Over-provisioning is the most common and most invisible cloud waste — it never pages anyone, so nobody fixes it. Right-sizing the web tier (P2v3×6 always-on → P1v3×2 + autoscale), moving to managed Redis, and tuning the SQL tier saved roughly $2,000/month total — with better peak headroom, because autoscale handles the month-end burst the fixed fleet was over-sized for. Lesson 11 — Observability is essential For years I "had logging" and was still blind in production. The shift from logging to observability — answering new questions about a running system without shipping new code — is the difference between a 4-minute incident and a 4-hour one. You can't fix what you can't see, and you can't see what you didn't instrument. Three pillars, tied by a correlation ID: logs (what happened), metrics (how much/how often), traces (where the time went). // structured fields + a correlation scope so every line in the request is linkable using ( logger . BeginScope ( new Dictionary < str

Most Spring Boot services end up with a scheduled job in them somewhere. A nightly reconciliation, a report, an export to some partner system. It starts small, and it goes in the main app because that's where the domain code already is. One artifact, one deployment, one pipeline. That's a real advantage and it's why most teams do it. This post is about when that stops being a good trade, how to split the job out, and when you shouldn't. The memory problem Look at how much memory each workload uses over a day. The API is fairly flat. Warm heap, connection pool, some caches. It moves with traffic but it doesn't swing much. The batch job uses close to nothing for 23 hours, jumps while it runs, then drops back to nothing. When both live in the same JVM, the pod has to be sized for the peak. So every replica of your API holds batch-sized memory all day, for a job that runs once. With three replicas you're reserving that headroom three times over so one job can use it once, at 2am. Memory limits are not like CPU limits CPU is compressible. Go over your CPU limit and the kernel throttles you. The app gets slower and keeps running. Memory doesn't work that way. There's no "run with less" mode. If the container goes over its memory limit, the kernel kills the process. What you get is a container that exited with code 137 (that's 128 + 9, where 9 is SIGKILL). What you don't get is anything useful in the logs. No OutOfMemoryError , no stack trace, no heap dump unless you configured one and it had time to write, no shutdown hook. The JVM was running fine, asked for another page of memory, and got killed for it. So a batch job sharing a pod with your API is a way for a nightly job to take down the pods serving traffic. If the job's working set grows (bigger dataset, a table that keeps growing, one unusually heavy day) the thing that dies is the API. There's a quieter version of the same problem. Even when the job stays under the limit, it allocates heavily and triggers longer GC