标签：#DevOps

Your CircleCI pipeline is green. Every job passes. And yet your app is running against a dependency version that hasn't shipped in a month — nobody committed it, nobody bumped it, it just quietly showed up in production. If you've chased a bug like this, the culprit is almost never your code. It's your cache key. This is a five-minute read and a fifteen-minute fix. Quick Win Friday, deployed to your .circleci/config.yml . The failure mode CircleCI's dependency caching works on a simple contract: you compute a key from something that changes when your dependencies change (usually a lockfile checksum), and you save/restore a cache tied to that key. The contract breaks in three specific, extremely common ways: You checksum the wrong file. {{ checksum "package.json" }} looks reasonable until someone bumps a transitive dependency via package-lock.json without touching package.json . The checksum doesn't move. CircleCI happily hands back last week's node_modules . restore_keys does prefix matching, and people think it does exact matching. CircleCI tries your primary key first, then falls through restore_keys in order, and the first one is a prefix match against existing cache entries — not "give me the newest exact match." If your restore_keys list is too coarse (e.g. just v1-deps- ), you can restore a cache built from a completely different branch, with a completely different lockfile, and the job won't fail. It'll just quietly install nothing (cache hit, npm ci sees the modules are "there") or run against the wrong versions. There's no version escape hatch. When you inevitably need to force everyone's cache to invalidate — a corrupted cache entry, a package manager migration, a lockfile format change — there's no cheap way to do it, because the key format was never designed with a manual buster in mind. Each of these fails silently. No red X. No error in the logs. Just a build that ran with stale state, and a bug report three days later that nobody can reproduce locally

A tool claiming "up to 15x faster than cdk deploy" showed up in my feed a while back. Drop-in replacement, it said: keep your CDK app exactly as it is, just swap cdk deploy for cdkd deploy . I've learned to be skeptical of "Nx faster" claims. So I actually deployed something real to AWS with both tools and timed it. Short version: it really is that fast. What cdkd actually is cdkd deploys an existing AWS CDK app without going through CloudFormation. It calls the AWS SDK directly instead. It's built by go-to-k (Kenta Goto), an AWS DevTools Hero and CDK top contributor who also maintains cls3 (a fast S3 bucket emptier) and delstack (for cleaning up stuck CloudFormation/CDK stacks) — tools that quietly fix the annoying parts of working with AWS. cdkd feels like the biggest one yet, and I mean that as a compliment grounded in actually using it, not a throwaway one. The mechanism is straightforward. cdkd runs the exact same CDK synth step as the CDK CLI, producing the same CloudFormation template. What changes is everything after that: instead of handing the template to CloudFormation, cdkd's own engine reads the resource dependency graph ( Ref , Fn::GetAtt ), builds a DAG, and fires AWS SDK / Cloud Control API calls directly, in parallel, as soon as each resource's dependencies are satisfied. Worth saying up front: cdkd calls itself not production-ready, dev/test only. This isn't a "replace CloudFormation in prod" pitch. I actually ran both, on real AWS cdkd's own README backs up the 15x number with a VPC + Lambda + SQS + CloudFront benchmark. So I wrote that same stack as a CDK app and deployed it twice — DeployRaceCfn via cdk deploy , DeployRaceCdkd via cdkd deploy — to the same AWS account, same region (ap-northeast-1). The stack: VPC (2 AZ + NAT Gateway) with a Lambda inside it, fronted by a Function URL CloudFront, origin set to that Function URL SQS + EventSourceMapping + a consumer Lambda First attempt failed. The account had hit its VPC limit (five, the default)

A green required check is not a merge decision. It is a signal that one job graph finished without a red X. If a pre-push hook was skipped, or a snapshot fixture was regenerated without a reason, you can still ship a lie. This article walks through a merge packet: a small JSON artifact your CI publishes next to the check. The packet records hook results, fixture drift, and required-job status. A model may write the eight-line brief. It does not get a vote. Why green still lies CI dashboards collapse many facts into one glyph. You see green. You click merge. You miss three common failures. First, someone pushed with --no-verify and skipped the hook that keeps fixture hashes honest. Second, a test helper rewrote golden files because a serializer added a field. Third, a retry job went green on the second attempt and nobody recorded that the first attempt failed. You do not need a platform rewrite to catch this. You need a contract the merge button cannot ignore. Cheap code generation makes the second failure more common. When it is easy to regenerate tests, it is easy to regenerate the fixtures those tests pin. The pin becomes a moving target. Treat unexplained fixture diffs as merge blockers, the same way you treat a failed unit job. What the merge packet contains Keep the packet boring. One file. One schema. Commit it as a CI artifact, not as a comment that can be edited after the fact. { "commit" : "REPLACE_WITH_SHA" , "generated_at" : "2026-09-03T00:00:00Z" , "hooks" : { "pre_push_fixture_guard" : "passed" }, "fixtures" : { "manifest_path" : "tests/fixtures.sha256" , "changed_paths" : [], "unexplained_paths" : [] }, "required_jobs" : [ { "name" : "unit" , "conclusion" : "success" }, { "name" : "contract" , "conclusion" : "success" } ], "merge_ready" : false , "brief" : null } merge_ready is computed by a script you own. Not by a prompt. The brief is optional prose for humans who will not open the JSON. Step 1: Pin fixtures with a manifest Pick a directory you alrea

Moving a Next.js app off Vercel and onto a plain Ubuntu VPS usually starts with a painful realization: either your serverless functions are timing out on background jobs, or your client just handed you a strict "you must host this on our infrastructure" requirement. Deploying the app itself is easy. What trips people up (and what cost me hours of debugging and locking myself out of my own server) is everything around the app. Here are the 12 things that actually break when you leave the serverless ecosystem, in the order you'll hit them. 1. Next.js needs a process manager, not just npm start Running npm start in a terminal dies the moment you disconnect. You need something that keeps the process alive, restarts it on crash, and survives a reboot. PM2 is the simplest option for a single-server Node deploy. npm install -g pm2 // ecosystem.config.js module . exports = { apps : [{ name : " my-app " , script : " node_modules/.bin/next " , args : " start " , cwd : " /var/www/my-app " , instances : 1 , exec_mode : " fork " , autorestart : true , max_memory_restart : " 512M " , env : { NODE_ENV : " production " , PORT : 3000 }, }], }; cd /var/www/my-app && pm2 start ecosystem.config.js pm2 save pm2 startup systemd -u YOUR_USER --hp /home/YOUR_USER That last line is the one people forget - without it, PM2's process list doesn't survive a server reboot. 2. Nginx needs to proxy to the port, not serve the files Next.js is not a static site (unless you've explicitly exported it as one). Nginx's job is to forward requests to the Node process, not serve files from disk: upstream nextjs_upstream { server 127.0.0.1 : 3000 ; keepalive 64 ; } server { listen 80 ; server_name example.com www.example.com ; location / { proxy_pass http://nextjs_upstream ; proxy_set_header Host $host ; proxy_set_header X-Real-IP $remote_addr ; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for ; proxy_set_header X-Forwarded-Proto $scheme ; # WebSocket support - required for HMR and any realtime f

There is a category of defect that cannot appear in your acceptance criteria. Not because nobody thought of it, but because the shape of a requirement has no room for it. A requirement describes a state and a rule. A customer can apply a valid promo code at checkout. State: the code is valid. Rule: it is accepted. Both are evaluated at a single instant, because a sentence has one tense. Real systems do not have one instant. They have two, and sometimes a lot more. The gap between checking and using Take that promo code. The system validates it when the customer types it into the basket. The system commits it when the customer pays. Between those two events sits an unbounded amount of time — thirty seconds if they have their card handy, three days if they leave the tab open on a laptop lid. If the code expires in that gap, what happens? The requirement cannot tell you. It never contemplated a gap, because it was written as one sentence about one moment. And a test written by hand almost certainly cannot tell you either, because a person writing a test naturally writes it the way they would perform it: enter code, assert accepted, pay, assert charged. Three lines, one instant, no gap. This is time-of-check to time-of-use. Most developers first meet it as a security problem — access() then open() , and a symlink swapped in between. The same shape appears at business timescale, and there it is far more common and far less discussed: Stock is reserved at basket, decremented at dispatch. Someone else buys the last one. A permission is checked when the page loads, enforced when the action fires. The role changed. A price is quoted at quote time, charged at renewal. The tariff moved. A rate limit is checked at admission, consumed at execution. The window rolled over. A feature flag is read at session start, branched on at submit. Someone flipped it. A token is validated at the gateway, used by a downstream call. It expired in flight. Every one of those is a real defect clas

Plenty of test data tools are built as a UI first and an API second, if there's an API at all. You open a dashboard, configure some fields, click "generate," and download a file. That works fine for a one-off demo. It falls apart the moment test data generation needs to be part of your actual engineering workflow — running in CI, seeding a database on every branch, or producing ten thousand records instead of ten. That's the case for a test data generation API over a click-driven dashboard: the primary interface is a request you can make from code, and everything else — a UI, a CLI — is built on top of that same API. Automation and CI integration A UI is something a person operates. CI doesn't have a person sitting at it. If test data generation only exists behind a login screen and a click, it can't run as a step in your pipeline — someone has to generate the data ahead of time, commit it, and hope it doesn't drift from what the tests actually need. An API-first tool is just another HTTP call your pipeline makes: fetch fresh, schema-conformant data as part of the build, every run, with no manual step in between. Scriptability — no clicking required Generating test data through a UI means clicking through the same sequence of dropdowns and fields every time you need a new batch. That's tedious for one dataset and untenable for the dozens of shapes a real test suite needs — different entity types, different edge cases, different volumes. An API call is a script. Write it once, parametrize it, and reuse it for every collection you need, without a human repeating the same clicks. Wiring a test data generation API into pipelines and seed scripts Seed scripts are code that runs at a specific point in a workflow — before a test suite, on container startup, in a migration. They need a function call or an HTTP request they can invoke programmatically, not a browser tab. With a test data generation API, "seed the dev database with realistic orders" is a line in a setup scrip

An agent patch can pass every unit test and still write outside the workspace, call an undeclared tool, or read an env key the task never named. Gate the blast radius first. Score the prose later. This article is a method, not a field report. It proposes a fail-closed envelope around filesystem roots, tool names, environment keys, and network hosts. Side-effect violations never freeze. Only a dual-runner disagreement on a non-envelope property may freeze, and only with a hashed evidence bundle. The conclusion in one rule Treat an agent patch as a capability change. If the run touches anything outside a declared envelope, the gate fails closed. Flakes in ranking, wording, or latency do not override that rule. Cheap generation does not make side effects cheap to reverse. A green suite that never watched /tmp , os.environ , or outbound sockets is not a verification result. It is a missing observer. What this gate is not It is not a golden-file of model text. It is not a mutation score. It is not a full-suite rerun after every hunk. It answers four questions only: Did the run write or delete outside allowed roots? Did it invoke a tool name that is not on the allowlist? Did it read an environment key that is not on the allowlist? Did it open a network host that is not on the allowlist? If any answer is yes, fail. Do not freeze. Do not retry for luck. Artifact: a locked envelope and an observer log Pin the envelope as a fixture. Hash it. Refuse to run if the hash drifts without a review note. { "envelope_id" : "agent-patch-envelope-v3" , "allowed_roots" : [ "/work/repo" , "/tmp/agent-scratch" ], "allowed_tools" : [ "read_file" , "apply_patch" , "run_tests" ], "allowed_env" : [ "CI" , "RUN_ID" , "ENVELOPE_HASH" ], "allowed_hosts" : [], "network" : "deny" } sha256sum envelope.json > envelope.json.sha256 # CI must compare this digest before the agent process starts. Label the next block as a proposed harness, not a production sandbox. User-space tracing will miss kernel-leve

I sat in on Margaret-Anne Storey's DORA community session last week, and she put a name on the thing I'd been circling since April. It isn't technical debt. Her ACM Queue piece splits software health into three debts. Technical debt is the familiar one: implementation choices that make tomorrow's change harder. Intent debt is the missing rationale, the goals and constraints that say what a system is even for, which now has to be legible to agents and not just to people. Cognitive debt is the one that stopped me. It's the erosion of shared understanding, the state where nobody on the team can confidently explain how the system works or predict what a change will break. Read that again if you review pull requests for a living. I closed a thirteen-post retrospective last month admitting I couldn't answer one question: how many AI-generated pull requests a week can a review process absorb before it stops working as a control? I still don't have that number. What I have now is a name for what you accumulate while you don't have it, and a way to find yours. Approval velocity measures motion Every metric most teams watch gets better as review collapses. Merge rate climbs. Time-to-approve drops. The throughput chart looks terrific right up until the incident review, because a reviewer who has quietly become a rubber stamp is indistinguishable from a fast reviewer in every dashboard you own today. Cognitive debt doesn't announce itself as a red number. It shows up as green ones, arriving faster. I know this failure mode from the inside. Two months of green CI on conformance checks that had never once passed , on my own project. A human audit caught it. No metric I was watching came close. What you need to measure is detection. Almost nobody does. Mutation testing, pointed at the reviewers We solved this once already, for test suites. Mutation testing injects known bugs into code and checks whether the tests catch them. A suite that passes everything might be thorough or migh

A recorded expected value is a leak. An agent that can read assert f(x) == y can patch f until that line is green and leave every unlisted input broken. A metamorphic relation does not publish y . It only publishes a constraint the output must keep under a known transform. That is the gate worth automating. Fixtures still matter, but only as seeds. Flaky tests still need a freeze, but the freeze must not cover the relation itself. This article is a proposed layout, not a production case study. No runtime metrics are claimed. The commands and modules below are labeled so they can be copied into a scratch repo and executed against your own function under test. Why snapshots fail as a merge gate Golden files encode one transcript. An agent patch is a search over many transcripts. If the search can see the answer key, the cheapest passing program is a lookup table for the keys in tree. That program is green. It is also wrong on the next customer file. Property-style checks reduce that leak because they do not ship the answer. They still need a seed corpus, a replay runner that the patch cannot edit, and a quarantine file that expires. Mix those three and you get a gate that fails closed when the agent rewrites tests, when a fixture drifts, or when a flake is used to hide a broken invariant. Three relation classes worth encoding first Start with relations you can state in one line. If you cannot state the line, you do not have a gate. You have a recorder. Idempotence. f(f(x)) == f(x) for normalizers, formatters, and canonicalizers. Round-trip. parse(serialize(x)) equals x on the fields you actually guarantee, not on whitespace you do not. Oracle-free comparison. f(t(x)) relates to t(f(x)) for a transform t you control: shuffle independent rows, rename equivalent keys, NFC vs NFD unicode, scale a quantity and its unit together. These are not universal laws. They are hypotheses about your function. Write them down as code. Keep the seed inputs boring. The relation, not the

An agent patch is only as trustworthy as the checks it cannot rewrite. If properties, fixtures, and flake policy live in the same tree as src/ , the diff can weaken the proof. Move the oracle out of the writable tree and run it as a control loop with hysteresis, not as a skip list. Co-located tests fail this requirement in a predictable way. The agent adds an assertion that matches the new code. A fixture grows a default that hides a broken parser. A flaky case becomes skip . The suite stays green. Production still drifts. This article proposes a sidecar oracle: human-owned properties, sealed fixtures, and a two-threshold flake freeze. The design is a workflow, not a production case study. Treat the code as a proposed runner you can execute locally, not as a claim about a live fleet. What the loop decides The loop answers three questions on every candidate patch: Do independent properties still hold on generated inputs? Did the patch mutate a sealed fixture or depend on an unsealed one? Is a failing test a regression, or does it belong in a measured freeze? A skip list answers none of those. It only records that someone got tired of a red job. Layout: oracle beside the repo, not inside the diff Keep the application repo writable for the agent. Keep the oracle in a second directory that the agent cannot include in its patch. app/ # agent may write src/, not oracle paths src/ pyproject.toml oracle/ # human-owned; hashed before every gate properties/ test_invariants.py fixtures/ manifest.json http_empty_body.json flake_ledger.json path_deny.txt run_gate.py path_deny.txt is the first control, not the last. If the patch touches oracle files, tests the agent authored, or lockfiles it did not need, the gate fails before pytest starts. # oracle/path_deny.txt oracle/ **/test_*.py **/*_test.py **/conftest.py **/__snapshots__/ The deny list is deliberately blunt. Agent-authored tests can still exist as scratch. They do not count as evidence. Step 1 — Hash the oracle before the

Ever Wondered How Netflix Never Seems to Go Down? Think about this for a second. Netflix has over 260 million subscribers worldwide. People are watching shows in Tokyo, London, Lagos, and New York — all at the same time. And yet, when was the last time Netflix crashed on you? Now think about your favourite food delivery app. You open it, order food, track your driver in real time, and get a notification the moment your burger arrives. All of that happens in seconds. Behind all of this is a way of working called DevOps. And by the end of this article, you'll understand exactly what it is — no jargon, no complicated diagrams, just plain English. The Old Way (And Why It Was a Nightmare) To understand DevOps, we first need to understand the problem it solved. Imagine a software company in the early 2000s. They had two completely separate teams: The Developers — the people who wrote the code and built new features The Operations team — the people who managed the servers and kept everything running These two teams barely talked to each other. Developers would spend months building new features, then hand over a massive pile of code to the operations team and say "here you go, make it work." The operations team would panic. They hadn't been involved in building it, had no idea what it did, and now they had to deploy it to millions of users without breaking anything. The result? Deployments took weeks. Bugs slipped through. Systems crashed. Customers complained. And the two teams blamed each other. Sound stressful? It was. So What is DevOps? DevOps is simply the practice of bringing developers and operations teams together to build, test, and release software faster and more reliably. The name itself is a combination of Dev (Development) and Ops (Operations). Instead of two teams working in silos, they work as one team with shared goals, shared tools, and shared responsibility. Think of it like a restaurant kitchen. In a badly run kitchen, the chefs cook the food and just s

The outage that teaches you deployment A service passes every test locally. It fails in staging because the API calls localhost:5432 for Postgres — but Postgres is in another container, reachable only as db:5432 . This is not a Docker problem. It is a boundary problem: your code assumed an environment it does not own. Engineers who have shipped on AWS Lambda already avoid a class of these mistakes. They never SSH into a function to hot-fix. They inject config at deploy time. They treat each invocation as disposable. Containers reward the same discipline with different vocabulary. This article maps what transfers, what breaks, and what I require before any Python backend goes to production in a container. What serverless already taught you Immutable deployments Lambda versions are replaced, not patched. Container images work the same way: build a new image, roll out, roll back by tag. If your incident runbook includes "edit files inside the running box," you have a design problem. Configuration at runtime Secrets belong in Secrets Manager or injected env vars — not in source control, not in the image layer cache. Docker does not change the rule; it changes where you mount the values. Single responsibility per unit One Lambda, one job. One container, one main process. Compose and Kubernetes add orchestration; they do not remove the rule. Cold start awareness Slim packages on Lambda map to slim base images ( python:3.12-slim , multi-stage builds). Startup time affects autoscaling and health-check windows the same way cold starts affect user-facing latency. If you understand why a Lambda deployment package should stay small, you understand why a 2 GB container image is a liability. Where the mental model breaks 1. Network identity Inside Compose or Kubernetes, localhost is the container itself. Services discover each other by DNS name ( api , db , redis ). This is the most common first-production failure I see in teams moving from bare metal or single-host deploys. 2. P

1. Provisionar não é configurar No artigo anterior desta série, vimos o OpenTofu como uma alternativa (ou substituto direto) ao Terraform para a tarefa de provisionar infraestrutura — criar VMs, redes, bancos de dados gerenciados, buckets. Mas provisionar um servidor é só o primeiro passo: depois que a VM existe, alguém precisa instalar pacotes, configurar usuários, aplicar hardening, subir a aplicação e manter tudo isso consistente ao longo do tempo. É nesse espaço que o Ansible entra — e é comum ver os dois trabalhando juntos no mesmo pipeline, não como concorrentes. 2. Onde o Terraform para e o Ansible começa A distinção mais útil na prática é: Terraform (e OpenTofu) são ferramentas de provisionamento : elas conversam com APIs de nuvem para criar, atualizar ou destruir recursos. O modelo mental é declarativo e orientado a estado desejado do recurso : "quero uma VM com esse tipo de instância, nessa rede, com esse disco". Ansible é uma ferramenta de gerenciamento de configuração : ela conecta em máquinas já existentes (via SSH, sem precisar de agente instalado) e executa tarefas para deixá-las em um estado desejado: "quero o Nginx instalado, essa versão, esse arquivo de configuração, esse serviço rodando". Não é incomum ver os dois no mesmo pipeline: o Terraform cria a VM e expõe o IP como output; o Ansible usa esse IP para conectar e configurar o que está dentro dela. Um cuida do "hardware" (ainda que virtual), o outro do "software". 3. Conceitos fundamentais do Ansible Antes de ver exemplos reais, vale fixar o vocabulário: Inventory: a lista de máquinas que o Ansible gerencia, agrupadas logicamente (por exemplo, webservers , databases ). Pode ser um arquivo estático (INI ou YAML) ou gerado dinamicamente (ex.: a partir de tags de uma conta AWS). Playbook: um arquivo YAML que descreve, em ordem, quais tarefas ( tasks ) devem ser executadas em quais grupos de máquinas do inventory. Module: a unidade de trabalho executada por uma tarefa — existem módulos prontos para

If your AI application suddenly becomes 300–500ms slower after adding an AI gateway, the first...

Security controls in AI systems often end up duplicated across applications. One team adds prompt validation, secrets detection, PII filtering, authentication, logging, and rate limits to an LLM application. A second team builds similar controls around another provider. Once agents enter the mix, the organization has several implementations of policies that should be shared. An AI gateway provides a central place to enforce those controls. Traditional infrastructure already centralizes TLS termination, authentication, rate limiting, traffic routing, and observability instead of rebuilding them in every microservice. The same pattern applies to AI traffic. What is an AI gateway? Without a gateway: +--> OpenAI Application -------+--> Anthropic +--> Gemini With a gateway: Applications | v +-------------------------+ | AI Gateway | | | | Authentication | | Rate limits | | Guardrails | | Routing | | Logging / observability | +------------+------------+ | +------+------+ | | | v v v OpenAI Anthropic Gemini Products in this space include Bifrost , Kong AI Gateway , LiteLLM , and Cloudflare AI Gateway . Their feature sets differ, but each can route model traffic through a common control layer. Security policy can run at that layer before traffic reaches a provider. Why application-level guardrails become painful To prevent users from accidentally sending credentials to an LLM, a simple implementation might look like this: def ask_llm ( prompt ): if contains_secret ( prompt ): raise SecurityError ( " Potential secret detected " ) return llm . chat ( prompt ) For one application, this is reasonable. Across 30 services, enforcement starts to drift. Some services call OpenAI directly, others use Anthropic, and several teams maintain their own wrappers. One application omits the secret check, another uses an old version, and a third checks prompts but no other AI interactions. Policy enforcement now depends on convention. Moving enforcement to the gateway changes the model: Requ

developer context switching security DevSecOps flow state developer velocity security alerts batch security patching ROI cost of context switching developer productivity security security alert fatigue developer cognitive load ad-hoc security patching interrupting developer flow engineering vp productivity metrics DevSecOps velocity context switching recovery time 23 minute recovery context switch batching security alerts SLA-backed fix campaigns security SLA for developers minimizing context switching feature delivery vs security developer experience DevSecOps The Real Cost of Context Switching What Security Alerts Actually Do to Developer Flow Back to blog What interruptions actually cost Is it worse for developers specifically? The research says probably yes The alert volume isn't imaginary — but be careful which numbers you cite The fix: batch the routine work, protect the calendar The important exception: not everything can wait for the batch A more honest way to estimate the ROI The takeaway Sources The Real Cost of Context Switching: What Security Alerts Actually Do to Developer Flow Companies keep investing in better frameworks, tighter deployment gates, and broader platform suites — and feature delivery keeps getting slower anyway. For engineering leaders trying to explain that paradox to the board, the usual suspects (headcount, tooling, talent) rarely hold up. The more useful place to look is something less visible: how often developers get pulled out of what they're doing, and what it costs them to get back in. As "shift-left" security practices spread, developers absorb a steady stream of vulnerability alerts, automated pull-request comments, and one-off Jira tickets throughout the day. The goal — a more secure codebase — is the right one. The delivery mechanism is often the problem. Scattering fixes across random moments in the workday erodes productivity without necessarily making the codebase safer any faster. The alternative a growing number of engi

There was a deployment pipeline at one job that everyone called "Tomasz's script." It did roughly nine critical things in a precise order, it had saved us thousands of hours over the years, and exactly one human on the planet understood how it worked. When Tomasz was around, this was invisible. When Tomasz went on holiday and the script failed at eleven at night, it stopped being a convenience and became the single scariest object in the company. We stood around a terminal reading code none of us had written, afraid to touch it and unable to leave it alone. This is the quiet paradox of automation. The whole point is to remove human effort, and it succeeds so completely that the humans forget how the thing works, or never learn in the first place. A manual process, for all its tedium, keeps knowledge distributed across everyone who performs it. A perfect automation concentrates that knowledge into whoever wrote it and then lets everyone else safely forget. The more indispensable the script becomes, the more dangerous its single point of understanding grows. What makes it worse is that these scripts accrete. They start simple and legible, then someone adds a special case for a weird environment, then a workaround for a vendor bug, then a hack to handle the one customer who is different. Each addition makes sense in the moment and makes the whole slightly more opaque. By the time it is truly load-bearing, it has become a small undocumented system that only its author can reason about, and its author is a busy person who is one job offer away from taking all of it with them. I have stopped treating a working automation as finished. Working is only half the requirement. The other half is that at least one other person can read it, understand what it does, and safely change it. That means the script explains its intent, not just its steps. It means the tribal knowledge lives somewhere other than one skull. It means occasionally, deliberately, having someone who did not wr

WebLLM: The Rise of AI That Runs Directly in Your Browser For the last few years, the dominant architecture for generative AI has been straightforward: Your application → Cloud API → Large Language Model → Response Every time you interact with an AI application, your prompt or data is typically sent to a remote inference service. But a different architecture is emerging: Your browser → Local AI model → Your device's GPU This is where WebLLM becomes interesting. WebLLM is an open-source, high-performance inference engine that allows large language models to run directly inside a web browser using WebGPU . The inference can happen on the user's device rather than on an application server. That seemingly simple change has significant implications for privacy, cost, offline AI, AI agents, enterprise applications, and cybersecurity . What exactly is WebLLM? WebLLM is not another large language model like Llama, Qwen, Gemma, or Mistral. Instead, think of WebLLM as an AI runtime for the browser . It provides the infrastructure required to load compatible open-source models and perform inference using the user's hardware. The basic architecture looks like this: Traditional AI User ↓ Web Application ↓ Backend Server ↓ LLM API / GPU Infrastructure ↓ Response With WebLLM: Web Application ↓ WebLLM ↓ WebGPU ↓ User's GPU / Device ↓ Local LLM inference WebLLM uses WebGPU for hardware acceleration and provides an OpenAI-compatible API, making it possible to integrate local models into JavaScript/TypeScript applications using familiar patterns. Why does this matter? The most important word is: Local Instead of sending every request to a remote AI service, an application can perform inference locally in the browser. That creates several potential advantages. 1. Privacy Consider an employee using an AI-powered security assessment tool. They may upload: Architecture diagrams Security policies Source code Vulnerability reports Compliance evidence Internal documents Configuration files W

These articles come from lessons learned while building Eterna Clarity and the operating system I use to run it. Some of the most misleading moments in building software happen when the page looks finished. The button is there. The layout is polished. The flow works in a test account. The code has been merged. It is very easy to look at that and think the product has moved forward. Then production reminds you that a product is larger than its frontend. I learned this repeatedly while building Eterna Clarity. A customer-facing change could depend on application code, a database function, authentication, storage rules, an email template, environment configuration and the way a demo account was isolated from real customer data. If one of those pieces stayed behind, the screenshot could be correct while the product was not. That changed the way I think about releases. A release is not “the code shipped.” A release is the smallest complete set of owned systems that have to advance together for the accepted behavior to become true in production. The browser can hide a lot of unfinished work Frontend work is unusually visible. That makes it easy to use as a proxy for progress. Back-end state is less visible. So are permissions, production configuration, storage policy, transactional email, tenant boundaries and data migrations. They tend to reveal themselves only when something goes wrong. That asymmetry can create a strange kind of false confidence. A team can spend hours polishing the thing a customer sees while the systems underneath it still describe an older product. In Eterna, the correction was to stop treating the repository as the whole release. Source code still matters. It is simply one owner among several. If a new customer flow requires a database change, the production database has to advance. If it requires a new authentication behavior, the production auth configuration has to advance. If it depends on storage permissions, those permissions have to exist in

The AWS API has become infrastructure's common language, and a whole ecosystem has grown up around running it somewhere other than AWS. Some tools mock it for testing. Others implement it for real. Knowing which is which saves you from deploying a dev tool to production or wiring a production platform into your CI pipeline. Two categories The tools split into emulators and real cloud platforms. Emulators intercept AWS API calls and return plausible responses without provisioning real infrastructure, where state is usually ephemeral, VMs never boot, and the goal is behavioural approximation fast enough for a developer's inner loop. Real cloud platforms provision actual infrastructure where EC2 calls boot real virtual machines and block storage carries real persistence guarantees. Emulators Moto Moto ( github.com/getmoto/moto , Apache 2.0, 8,400+ stars) has been around since 2013, making it the oldest option here. It works differently from the rest because rather than running a local server, it patches boto3 calls in-process through a test decorator. A function wrapped in @mock_aws intercepts all AWS SDK calls and returns mock responses without any network traffic. This makes it fast and easy to drop into Python test suites, but it only works for Python. Teams using the AWS CLI, Terraform, or Go SDKs need a server-based option. LocalStack LocalStack ( github.com/localstack/localstack , 64,000+ stars) is the dominant name in local AWS development. It runs as a Docker container exposing the AWS API on localhost:4566 and covers over 120 services. In March 2026, LocalStack archived its Community Edition repository and moved core services behind a paid plan. A free tier remains for non-commercial use and open source projects, but the Base plan covering Cloud Pods persistent state costs $39 per month and the Ultimate plan runs $89 per month. Teams that depended on CE for commercial CI pipelines are now evaluating alternatives. Floci Floci ( floci.dev , github.com/floci-io/f