标签：#cicd

1. Retomando: da aplicação funcionando ao container pronto para produção Esta série cobriu, até aqui, o suficiente para desenvolver com Docker no dia a dia: conceitos fundamentais, comandos essenciais, Dockerfiles eficientes, rede, volumes e Compose para orquestrar múltiplos serviços. Este último artigo fecha a lacuna entre "funciona no meu Compose local" e "pronto para rodar em produção": imagens menores via multi-stage builds, segurança básica e não negociável, e como tudo isso se integra a um pipeline de CI/CD. 2. O problema que multi-stage builds resolve Compilar ou empacotar uma aplicação frequentemente exige ferramentas que a aplicação não precisa em tempo de execução : compiladores, headers de desenvolvimento, o próprio código-fonte antes de ser transpilado/buildado. Um Dockerfile ingênuo carrega tudo isso para a imagem final: # Ruim: ferramentas de build viajam junto para produção FROM node:20 WORKDIR /app COPY . . RUN npm install && npm run build CMD ["node", "dist/server.js"] Essa imagem inclui o npm , todo o node_modules (incluindo dependências de desenvolvimento), o código-fonte original e as ferramentas de build — frequentemente centenas de MBs de peso morto que nunca são usados depois que npm run build termina, e que ainda aumentam a superfície de ataque da imagem (mais binários, mais coisa que pode ter vulnerabilidade). Multi-stage builds resolvem isso permitindo múltiplos blocos FROM no mesmo Dockerfile, onde estágios posteriores copiam seletivamente apenas o que precisam dos anteriores — o restante do estágio de build simplesmente não existe na imagem final: # Estágio 1: build, com todas as ferramentas necessárias FROM node:20 AS build WORKDIR /app COPY package*.json . RUN npm ci COPY . . RUN npm run build # Estágio 2: produção, só com o resultado do build FROM node:20-slim WORKDIR /app COPY --from=build /app/dist ./dist COPY --from=build /app/node_modules ./node_modules COPY package*.json . CMD ["node", "dist/server.js"] A imagem final não contém o

1. Retomando: do docker run repetido a um arquivo único No artigo anterior, subir uma API e um Postgres conectados exigiu dois comandos docker run longos, com flags de rede, volume e variáveis de ambiente para lembrar (e digitar) toda vez. Em um projeto real, com mais serviços — cache, fila, worker em background — isso rapidamente vira inviável de manter na cabeça ou em um script solto. O Docker Compose resolve isso descrevendo toda a aplicação multi-container em um único arquivo declarativo, versionado junto com o código. 2. O arquivo compose.yaml Compose lê um arquivo YAML (por convenção compose.yaml , ou o nome legado docker-compose.yml , ainda amplamente usado) descrevendo serviços (cada um vira um ou mais containers), redes e volumes: # compose.yaml services : api : build : . ports : - " 8000:8000" environment : DATABASE_URL : postgresql://postgres:segredo@banco:5432/postgres depends_on : - banco banco : image : postgres:16 environment : POSTGRES_PASSWORD : segredo volumes : - pg-dados:/var/lib/postgresql/data volumes : pg-dados : Isso substitui inteiramente os dois docker run do artigo anterior. Uma diferença importante já aparece aqui: por padrão, Compose cria uma rede própria para o projeto e conecta todos os serviços a ela automaticamente — não é preciso um docker network create manual, nem declarar --network em cada serviço. Cada serviço já é acessível pelos demais pelo nome declarado em services: (aqui, banco resolve para o container do Postgres), exatamente como as redes definidas pelo usuário do artigo anterior. 3. Comandos essenciais do Compose docker compose up -d # sobe todos os serviços em segundo plano docker compose ps # lista os containers do projeto e seu status docker compose logs -f api # segue os logs de um serviço específico docker compose logs -f # segue os logs de todos os serviços, intercalados docker compose exec api bash # abre um shell dentro do container de um serviço docker compose stop # para os containers sem removê-los docker comp

A month ago I shipped aicraft-code-review , an MCP server that reviews code locally. This week I added a CLI mode — because not everyone wants to wire up an MCP client just to check a diff. Now the same reviewer runs three ways: MCP tools — review_code / review_diff / review_file inside Claude Code, Cursor, Cline CLI — mcp-code-review review-file path/to/file.py CI — pipe git diff into it and branch on the exit code The CLI pip install aicraft-code-review # a single file (config auto-discovered from the file's directory upward) mcp-code-review review-file src/api.py # the current diff git diff | mcp-code-review review-diff # a snippet mcp-code-review review-code "import os; os.system('ls')" Exit codes are CI-friendly: Code Meaning 0 clean, or only info-level findings 1 high / medium issues found 2 critical issues found What it catches out of the box Security (OWASP patterns), performance (N+1, unbounded growth), quality (bare excepts, TODOs, missing type hints), style (naming, line length). Real output: ### 🟠 High (2) | Line | Issue | Category | Fix | | 4 | Command injection risk | security | subprocess.run with args list | | 9 | N+1 query in loop | performance | batch query / eager loading | ### 🟢 Info (2) — missing return type annotations Verdict: Conditional Pass — address high/medium issues Making it match YOUR rules The config file is the part I'd actually show a teammate: custom_rules : - name : no-console-log pattern : ' console\.log\(' severity : high category : quality issue : Console logging left in production code fix : Use a structured logger instead disabled_checks : - todo_comment severity_overrides : hardcoded_secret : critical .mcp-code-review.yaml is auto-discovered from the reviewed file's directory upward MCP_CODE_REVIEW_CONFIG points a whole team at one shared profile valid severities: critical / high / medium / info regex patterns work best in single quotes (double quotes will error on escapes like \. ) One caveat if you're also shipping Python

GitHub Actions is GitHub's built-in CI/CD system — it spins up a fresh virtual machine, runs whatever commands you tell it to, and tears the machine down when it's done. It supports Linux, Windows, and macOS runners. The macOS runners are the interesting part here, because they're actual macOS machines with Xcode's command-line build tools available, which means they can build and sign iOS apps — not just run tests. The headline rule: public repos are free On a public repository, standard GitHub-hosted runner minutes — including macOS — don't cost anything, on any plan, including the free plan. It's a genuine free tier, not a trial or a limited allowance that runs out. One nuance: this covers standard runners. GitHub also offers "larger runners" (more CPU/RAM) — those are billed regardless of repo visibility. A default macOS build for signing and archiving a typical app doesn't need one, so this rarely matters in practice. What changes on a private repo If your repository is private, you get a monthly allowance of free minutes instead of unlimited free usage: Plan Included minutes / month Free 2,000 Pro 3,000 Team 3,000 The number that actually matters for iOS builds is how fast macOS runners burn through that allowance. macOS minutes cost roughly 10x GitHub applies a multiplier against your included minutes: Linux runners run at the baseline rate, macOS runners run at roughly 10x that rate. A 6-minute macOS build eats through the same allowance as roughly 60 minutes of Linux CI. Applied to the table above, a Free-plan private repo effectively gets around 200 macOS-runner-minutes worth of free build time per month before you're billed per minute past it (Pro/Team works out to roughly 300). The practical upshot: if you're fine building in the open, a public repo gets you unlimited macOS build minutes at zero cost, indefinitely. If you'd rather keep the code private, everything about the pipeline still works the same way — you're just drawing from a metered allowance

Gate your CI on a dollar ceiling, not a percentage — the number your finance team actually asks for Most cost gates for agent/LLM workflows check a delta : did this PR make the run more expensive than the last one, by more than X%? That's a good regression alarm. But it answers a developer's question ("did I make it worse?"), not a budget owner's question ("are we going to blow the monthly number?"). Those are genuinely different gates, and a team that only has the percentage one keeps getting surprised. A workflow can pass every percentage check — each PR adds a harmless-looking 3% — and still cross the line where the absolute monthly spend stops being okay. Percentages compound quietly; dollars are what shows up on the invoice. So the second gate I want on any agent workflow is an absolute ceiling : "a single run of this job must not cost more than $N," full stop, regardless of whether it went up or down since yesterday. Three things make that gate actually usable rather than theater: 1. The ceiling is priced, not token-counted. "Under 2M tokens" is meaningless to the person who signs off on spend, because a token of Opus output and a token of cached Haiku input differ by ~100× in price. The gate has to multiply each token bucket (input, output, cache-write at ~1.25×, cache-read at ~0.1×) by that model's real per-token price and sum to an actual dollar figure. If your gate reports tokens and makes a human convert, nobody converts, and the ceiling drifts. 2. The ceiling is per-run and per-workflow, not global. A nightly full-repo audit and a per-PR lint agent have wildly different legitimate costs; one global number is either too loose for the small job or too tight for the big one. You want to set max-usd on the specific workflow, so each job carries the ceiling that matches what it's for . 3. It shows the headroom, not just pass/fail. "$0.43 of a $0.50 ceiling — 86%" on every run is the line that lets you move the limit before it starts failing builds, instead of

The Quest Begins (The "Why") I remember the first time I tried to take a weekend side‑project from my laptop to something that felt “real”. I had a cute Express API that talked to Postman, a PostgreSQL container spun up with docker-compose up , and a React front‑end that lived in its own dev server. Everything worked beautifully … until I hit Ctrl+C on my laptop and the whole thing vanished. I needed a way to say, “Hey, keep this running even if I close my laptop, and if something crashes, bring it back up automatically.” I started poking at Docker Swarm, then Nomad, but the docs felt like reading ancient runes. That’s when a coworker slid over a Slack message: “Just try a Kind cluster. It’s K8s locally, and you’ll see why everyone talks about it.” Spoiler: it felt like discovering the secret level in a classic arcade game. Suddenly I could describe what I wanted my system to look like, and the cluster would make it happen — no more babysitting containers. The Revelation (The Insight) Kubernetes isn’t a mystical black box; it’s a declarative orchestrator . You tell it the desired state of your application (how many replicas, which image, what ports to expose) and it works relentlessly to match reality to that state. If a pod dies, Kubernetes spins up a new one. If you ask for three replicas and only two are running, it creates the missing pod. If you update the image tag, it rolls out the change pod‑by‑pod, keeping traffic flowing. Think of it like the save‑game system in a RPG: you define the story you want to experience, and the engine handles the gritty details of loading, saving, and recovering from crashes. The core objects you’ll meet early on are: Pod – the smallest deployable unit (one or more tightly coupled containers). Deployment – manages a set of identical pods, handles updates and rollbacks. Service – a stable network endpoint that load‑balances traffic to a set of pods. Ingress (optional) – exposes HTTP/HTTPS routes from outside the cluster to service

1. Retomando: de imagens bem construídas a containers que conversam entre si Os artigos anteriores desta série cobriram como criar imagens eficientes e rodar containers isolados. Mas uma aplicação real raramente é um único container: normalmente há uma API, um banco de dados, um cache, talvez uma fila de mensagens — cada um em seu próprio container, precisando se comunicar. E containers, por padrão, são efêmeros: qualquer dado escrito dentro deles some quando são removidos. Este artigo cobre as duas peças que resolvem isso: redes (comunicação entre containers) e volumes (persistência de dados). 2. O problema do isolamento de rede por padrão Cada container recebe seu próprio namespace de rede, isolado dos demais e do host. Isso é uma característica de segurança, não um bug — mas significa que dois containers rodados de forma independente não conseguem se encontrar automaticamente: docker run -d --name api minha-api docker run -d --name banco postgres De dentro do container api , tentar acessar banco por esse nome simplesmente falha — cada container, isolado, só enxerga localhost como a si mesmo. A solução do Docker para isso é criar uma rede e conectar ambos os containers a ela. 3. Redes definidas pelo usuário (User-Defined Networks) docker network create minha-rede docker run -d --name banco --network minha-rede postgres docker run -d --name api --network minha-rede minha-api A partir daqui, dentro do container api , o hostname banco resolve automaticamente para o IP do container banco — o Docker roda um DNS interno para qualquer rede definida pelo usuário, resolvendo containers pelo nome (ou pelo alias definido com --network-alias , se houver mais de um). Isso é o motivo pelo qual strings de conexão em aplicações containerizadas costumam usar o nome do serviço em vez de um IP fixo: DATABASE_URL = postgresql :// usuario : senha @ banco : 5432 / meudb Comandos úteis para inspecionar redes: docker network ls # lista todas as redes docker network inspect minha-rede # d

A merge request changes one README line. The pipeline still calls a model. It costs tokens. It adds latency. It tells you almost nothing. Sound familiar? If you maintain a small CI setup, this failure keeps showing up. The instinct is to put model-based review everywhere. Then the free tier dies in a week. The fix isn't another monitor. It's a small decision gate that decides whether a diff deserves a model call at all. The operator-supplied availability claims for MonkeyCode include free model access and a free server option. I treat those claims as a starting point, not a quota guarantee. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Why every diff shouldn't hit the model Free model access is not infinite. Even if it feels free, there are hidden ceilings. Free tiers often cap requests, tokens, or time-based windows. Model output variance on trivial diffs adds noise, not signal. CI latency grows. A two-second call across a hundred merge requests is real time. The highest-value model review is rare, not constant. If you call a model on every change, you pay the full cost while getting almost none of the benefit. The gate is supposed to fix that. A three-tier escalation ladder I use a small decision table. It doesn't need to be perfect. It needs to be boring and predictable. Tier Trigger Action Model call? 0 Up to 50 added+removed lines, only docs or config suffixes, no sensitive paths Run lint and skip the model No 1 Code or test files touched, 51–400 lines, no lockfile, no migration, no sensitive path Send one bounded prompt to the free model Yes, once 2 Over 400 lines, new lockfile, migration, auth or secret paths Require human review first. Use a model only to summarize, not to decide Optional The exact numbers are arbitrary. They matter less than the fact that tier 0 never reaches the model. The code Here is a plain Python gate. It reads simple diff stats and changed paths. from pathlib import Path DOC_OR_CONFIG = { ' .md ' , '

The retry trap A free model CI job fails on a timeout. You click retry. The whole pipeline starts over: checkout, build, dependencies, model call. That is the trap. Why re-run the world for one timeout? Retrying the pipeline does not isolate the flaky step. It makes a small problem expensive. I wanted a workflow that replays just the model call, not the whole pipeline. So I made every free model call leave behind a tiny reproducible record. A record has two halves: the input envelope and the output hash. If the job fails, I can replay the input against the same model and compare the output hash. No full pipeline re-run. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I use MonkeyCode's free model access for the model step and its free server option as a small replay store. I do not assume exact quotas, model names, or availability windows here. The pattern works with any free HTTP model endpoint and any tiny key-value store or CI artifact. Why a hash and not the full prompt Full prompt logs are useful until they are not. A free model job may receive a snippet of a merge request, an error message, or an environment variable. Store the raw text in CI logs and you can accidentally leak source or secrets. Store a hash and the replay input in a locked artifact, and the risk drops. A hash also gives me one cheap comparison target. I do not need to reason about the entire response to see that an endpoint changed. I only need byte-level equality. The record shape For every model call, I save the fields below. request_id: a hash derived from model, prompt hash, and a timestamp. prompt_hash: the hash of the normalized prompt. response_hash: the hash of the raw response. status: the HTTP status of the original call. bytes: the length of the response. The exact hash algorithm matters less than using the same one on both sides. I use SHA-256 because it is available everywhere. GitLab CI wiring I run two jobs. The first job calls the model and post

1. Retomando: do Dockerfile mínimo a um Dockerfile de verdade Na segunda parte desta série, um Dockerfile de poucas linhas já foi suficiente para empacotar uma aplicação Python. Isso funciona, mas um Dockerfile escrito sem pensar em camadas e cache de build gera imagens maiores do que precisam ser e builds que demoram muito mais do que deveriam a cada mudança pequena no código. Este artigo aprofunda como o Docker constrói uma imagem por dentro, e como escrever um Dockerfile que tira proveito disso. 2. Como funcionam as camadas (layers) Cada instrução de um Dockerfile ( FROM , RUN , COPY , ADD ) que modifica o sistema de arquivos gera uma camada — um diff read-only armazenado separadamente e empilhado sobre as anteriores. A imagem final é simplesmente a soma de todas essas camadas, e o container em execução adiciona uma camada gravável no topo (union filesystem). Container (camada gravável) ────────────────────────── Camada 4: COPY . . Camada 3: RUN pip install -r requirements.txt Camada 2: COPY requirements.txt . Camada 1: FROM python:3.12-slim Duas consequências práticas importantes: Camadas são reaproveitadas entre imagens. Se duas imagens diferentes compartilham as mesmas primeiras instruções (por exemplo, a mesma FROM e o mesmo RUN apt-get install ), o Docker armazena essa camada uma única vez em disco, mesmo que várias imagens a usem. Camadas são cacheadas entre builds. Ao rodar docker build de novo, o Docker verifica cada instrução, na ordem: se a instrução e seus arquivos de entrada não mudaram desde o último build, ele reaproveita a camada já construída em vez de refazer o trabalho. Isso é a base de todo o próximo tópico. 3. Cache de build: ordenar o Dockerfile por frequência de mudança O cache de build é invalidado a partir do primeiro ponto de mudança : se a instrução N mudou (ou um arquivo que ela copia mudou), toda camada a partir de N é reconstruída — mesmo que as instruções seguintes sejam idênticas ao build anterior. Isso significa que a ordem das ins

1. Retomando: de imagens a containers em execução Na primeira parte desta série vimos o que é o Docker, o problema que ele resolve e os três conceitos fundamentais — imagens, containers e registries. Agora que a base teórica está posta, o foco deste artigo é prático: os comandos que efetivamente viram hábito no uso diário — run , exec , logs , ps , build — aplicados a containers reais, não só ao hello-world . 2. docker run além do básico O artigo anterior já usou docker run para subir um Nginx. Vale conhecer as flags que aparecem o tempo todo: # Modo interativo, útil para explorar uma imagem manualmente docker run -it ubuntu bash # Variáveis de ambiente docker run -e POSTGRES_PASSWORD = segredo -d postgres # Montar um diretório do host dentro do container (volume bind mount) docker run -v $( pwd ) /dados:/dados -d minha-imagem # Remover o container automaticamente quando ele parar docker run --rm -it python:3.12 python3 # Limitar recursos docker run --memory = 512m --cpus = 1 minha-imagem -it combina -i (interativo, mantém STDIN aberto) com -t (aloca um pseudo-terminal) — é o par de flags para "entrar" em um container e usar um shell como se fosse uma máquina normal. --rm evita acumular containers parados no disco depois de testes rápidos e descartáveis — sem ela, cada docker run deixa um container parado para trás até ser removido manualmente. -e define variáveis de ambiente; imagens oficiais como a do Postgres costumam documentar quais variáveis elas esperam (usuário, senha, nome do banco inicial). 3. Inspecionando o que está rodando O comando mais usado para ter uma visão geral do que o Docker está gerenciando na máquina: docker ps # containers em execução docker ps -a # todos, incluindo parados docker ps -q # só os ids (útil em scripts) Para investigar um container específico mais a fundo: docker inspect meu-container # todos os metadados em JSON: rede, volumes, config docker top meu-container # processos rodando dentro do container docker stats # uso de CPU/mem

You installed GitLab Community Edition, pushed a .gitlab-ci.yml , and watched the pipeline sit at pending until it went grey. No error, no failed job, nothing in the logs worth reading. Nothing is broken. Your instance has no runners, and it never had any. Why a fresh instance has none GitLab is two things that people assume are one thing. There is the application: repositories, issues, merge requests, the CI/CD system that reads your .gitlab-ci.yml and builds a pipeline out of it. And there is GitLab Runner: a separate program, on a separate machine, that actually executes jobs. The Omnibus package installs the first. It does not install the second, and it does not come with any machines to run it on. The confusion comes from GitLab.com, where shared runners are switched on by default and most people's first experience of CI is that it simply works. That shared fleet is hardware GitLab owns and operates as part of their hosted service. It is not part of the software you downloaded, so it does not come across when you run your own instance. So on a self-managed install, GitLab will happily accept your pipeline definition, parse it, create the jobs, and queue them. Then it waits for a runner to ask for work. If no runner ever asks, the jobs wait indefinitely. Confirming it in thirty seconds Go to Admin Area → CI/CD → Runners on your instance. If the list is empty, that is your answer. For a single project, Settings → CI/CD → Runners shows the same thing scoped narrower. A fresh instance shows nothing in either place. From the command line on the GitLab server: sudo gitlab-rails runner "puts Ci::Runner.count" If that prints 0 , no runner has ever been registered against this instance. One thing worth ruling out at the same time: a job can also sit pending when runners do exist but none of them match the job's tags. If your runner list is not empty, check whether your jobs specify tags: that no runner carries. That is a different problem with a different fix, and it is

1. O Problema que o Docker Resolve "Na minha máquina funciona." Poucas frases resumem tão bem um problema que atormentou (e ainda atormenta) times de desenvolvimento: um código que roda perfeitamente no notebook do desenvolvedor, mas quebra no servidor de produção — porque a versão do Python é outra, uma biblioteca do sistema está faltando, uma variável de ambiente não foi configurada, ou o sistema operacional simplesmente se comporta de forma diferente. O Docker resolve exatamente isso: ele empacota uma aplicação junto com tudo que ela precisa para rodar — código, dependências, bibliotecas do sistema, variáveis de ambiente, configuração — em uma unidade isolada e portátil chamada container . Essa unidade roda da mesma forma em qualquer lugar que tenha o Docker instalado: no notebook do desenvolvedor, no servidor de CI, ou em produção. Esta é a primeira parte de uma série que vai do zero ao avançado em Docker: hoje o foco é entender o problema que ele resolve, os conceitos fundamentais e como eles se encaixam. 2. Containers vs Máquinas Virtuais A comparação mais comum ao explicar Docker é com máquinas virtuais (VMs), porque ambos resolvem um problema parecido — isolar e empacotar aplicações — mas de formas muito diferentes. Uma máquina virtual virtualiza o hardware inteiro: cada VM roda seu próprio sistema operacional completo (kernel incluso), gerenciado por um hypervisor. Isso garante isolamento forte, mas tem um custo alto: cada VM consome centenas de MBs a alguns GBs de disco e memória só para o SO, e leva de dezenas de segundos a minutos para inicializar. Um container , por outro lado, virtualiza no nível do sistema operacional: todos os containers em uma máquina compartilham o mesmo kernel do host, mas cada um enxerga seu próprio sistema de arquivos, processos e rede isolados — usando recursos do kernel Linux como namespaces (isolamento de visão) e cgroups (limites de CPU/memória). O resultado é que containers são muito mais leves: alguns MBs a poucas centenas

The Slack thread started with a screenshot of a green GitHub Actions run. By the third reply someone had pasted a Lighthouse JSON artefact, a link to a Chrome release note, and a question nobody wanted to own: "Which client repository still pins Lighthouse 10?" That is the week Lighthouse CI stopped being a merge gate and became a second job. The pipeline still passed and the portfolio still needed evidence, but the difference was who paid in hours: the developer shipping a feature, or the one person who inherited every lighthouserc file across client repositories. When does Lighthouse CI maintenance outgrow a CI/CD pipeline? Lighthouse CI earns its place early. You wire assertions on a preview URL, block a CLS regression, and the team trusts the red build. The cost is front-loaded configuration, not ongoing calendar time. The shift happens when success creates obligations a CI/CD pipeline was never designed to carry: Every new client repository needs a copied workflow, pinned Chrome, and preview URL rules that match their host. Assertions need tuning after flaky LCP on cold runners, so thresholds loosen until they barely catch real regressions. Account managers ask for client-ready reports, and the only export is a CI/CD artefact someone must turn into slides. Production URLs outside the two preview paths regress while the job stays green. At that point you are not "running Lighthouse CI in a pipeline." You are operating a small internal product: version pins, runner hygiene, assertion policy, and reporting glue. For a single product team that can be fine. For an agency portfolio it competes with billable delivery. How do you know Lighthouse CI in CI/CD became an unpaid side role? We treat these as signals to shrink CI/CD scope or add a managed monitoring layer, not as moral failure. Teams hit them around five to fifteen client sites, sometimes sooner when preview hosts differ wildly. Flaky Lighthouse CI runs on GitHub Actions Engineers merge after the third "Re-ru

The owner had already asked for the alert emails to stop. A fix shipped. Then another email landed. Then another. "ong it just ssent me abother email," he said, voice-dictated, unedited. Fifteen minutes later: "go another one." The system was reporting an outage that did not exist. The Transport That Only Ever Failed A 14-PR merge train had just moved every cron producer's alerting off shared email and onto Buzz, a Nostr-relay team chat. One producer per PR, each with its own liveness contract and a bead receipt. It shipped cleanly. But the library backing those producers carried a default that had only one job: fail. AF_BUZZ_CMD = " ${ AF_BUZZ_CMD :- af_default_buzz_post } " af_default_buzz_post returned 1 with "no Buzz transport injected". Every caller that sourced the library (which is every cron producer) exhausted its Buzz retries and fell through to the email floor. The system reported a false Buzz outage while the relay was healthy. It did this 2 to 5 times per hour. Evidence arrived in the logs: 581 dedup markers, a steady stream of "[INTENT ALERT FLOOR: Buzz unreachable]" emails, and sweep.log showing buzz=ok only for the handful of callers invoked through the CLI entrypoint rather than by sourcing the library. That asymmetry was the bug. The CLI had a one-line fixup swapping in the real transport, annotated in a comment as "the library path is unchanged". The library path did not, and the cron producers all take the library path. The fix promoted the real transport to the default for both seams. af_buzz_transport already discovers the installed buzz-notify.sh and already fails closed when it is genuinely missing. The dead CLI fixup was deleted. Fail-closed behavior survives, but now it is conditional on genuine absence rather than on every caller remembering to opt in. Why not migrate callers one at a time? Because the per-caller route leaves the next new producer to rediscover this the same way. Flipping the default fixes the class, not the instance. The

Week 3. "The deploy is done. Everything's green. Now what am I actually supposed to be looking at?" Previously Runner ↓ Cache ↓ Artifact ↓ Deployment Today ↓ Monitoring Junior Engineer: The canary rolled out fine yesterday. 100% traffic, all healthy. I closed my laptop. Was that wrong? Senior Engineer: Not wrong, exactly. But let me ask you something first. Your service is running on a server somewhere. Right now, this second — is it healthy? Junior Engineer: I mean... I assume so? Nobody's messaged me. Senior Engineer: "Nobody's messaged me" isn't an answer. It's the absence of one. That's the entire problem monitoring exists to solve. The Thing Nobody Says Out Loud Senior Engineer: Here's an uncomfortable fact about production systems: you cannot see them. Not directly. You're not standing next to the server, watching electricity move through it. Everything you know about whether it's healthy is a claim — something a piece of software told you, that you're choosing to trust. Junior Engineer: That sounds obvious when you say it, but I don't think I've ever actually thought about it that way. Senior Engineer: Most engineers don't, until the gap between "the system told me it's fine" and "the system is actually fine" bites them. Monitoring is the discipline of shrinking that gap — of making sure what you're told is close to what's actually true, and told to you fast enough to matter. 📒 Senior Engineer's Notebook You don't monitor a system because you don't trust it. You monitor it because you can't see it. Trust isn't the issue — visibility is. The Car Dashboard Analogy Junior Engineer: Can you make this concrete? Senior Engineer: Think about driving a car. You can't see the engine. You can't see the oil level, the coolant temperature, how much fuel is actually left in the tank, mid-drive. All of that is invisible to you, sealed inside metal, while you're doing 100 km/h. So the car gives you a dashboard. Speed, fuel, engine temperature, warning lights. You're not wat

Originally published at wostal.eu . TL;DR : I handed a CI re-platform — GitHub Actions → Argo Workflows, GitHub → Codeberg — to an AI agent. The rewrite looked correct and passed lint, but it didn't account for one thing: the runtime environment had changed completely. The old runner was an external VM; the new one is a pod inside the cluster, where Tailscale IPs aren't routable. Nobody ran the pipeline end-to-end, so every difference stayed as a landmine — six of them. The worst part wasn't the AI's mistake. It was that I spent hours blaming the network when the real culprit was a single missing newline in an SSH key. This is the companion to a different incident. While I was migrating my homelab k3s control plane from SQLite to etcd , the pipeline that ran that migration turned out to be broken in ways that taught me more than the migration itself. In this post I'll cover: Where the mess came from — an AI-assisted CI re-platform that nobody validated end-to-end The phantom CNI bug — hours of chasing MTU, Tailscale, firewall, and conntrack The real culprit — a single missing newline in an SSH key Why early failures hide later ones — six latent bugs, surfaced one at a time The lesson about delegating migrations — to a human or an AI Where the mess came from The entire CI had just been migrated from GitHub Actions to Argo Workflows , and from GitHub to Codeberg — and that migration was handed to an AI agent 🙂. The re-platforming rewrote the logic roughly 1:1, but it didn't catch that the runtime environment had changed fundamentally . The old runner (GitHub Actions / ARC) behaved one way; the new one is a pod in the cluster — from which Tailscale IPs aren't routable, but private IPs are. Nobody (human or AI) ran the pipeline end-to-end, so all of those differences stayed as landmines. That's why there were six latent bugs and the hang I'm about to describe — not random faults, but traces of a GHA→Argo translation that was never validated in the new environment. The p

After setting up Jenkins and creating my first Declarative Pipeline, the next step was preparing my machine to build Docker images. Since my pipeline will eventually clone code from GitHub, build a Docker image, and push it to a container registry, Jenkins needs access to Docker. Without Docker installed, the docker build stage would fail because Jenkins wouldn't be able to execute Docker commands. Installing Docker On my Ubuntu machine, I installed Docker using: sudo apt update sudo apt install docker.io -y Once the installation completed, I verified it using: docker --version This confirmed that Docker was successfully installed and ready to use. Verifying Docker Installation Installing Docker is only half the job. The next step is to check whether your current user has permission to use Docker. Run: docker ps Expected Output If everything is configured correctly, you should see something similar to: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES Even if no containers are running, getting an empty table like the one above means Docker is working correctly. What If You Get a Permission Error? If you see an error like: permission denied while trying to connect to the Docker daemon socket it means your current user doesn't have permission to access the Docker daemon. First, check which user you're currently logged in as: whoami Example output: nishant Now add your user to the docker group: sudo usermod -aG docker $USER What does this command do? usermod modifies a user account. -aG means append the user to a supplementary group without removing existing groups. docker is the group that has permission to communicate with the Docker daemon. $USER automatically refers to your currently logged-in username. Apply the Changes The group membership won't take effect immediately. You have two options: Option 1 (Recommended): Log out of your Ubuntu session and log back in. This refreshes your user groups and is the most reliable method. Option 2: Simply restart your ter

The Situation Our team's CI/CD pipeline on Azure DevOps was taking 15 minutes to complete on every push to develop. You'd merge a PR, grab a coffee, come back — and it was still running. A 15-minute feedback loop breaks flow state — by the time the pipeline finishes, you've already switched context twice and forgotten what you were checking. I spent an afternoon digging into the Azure DevOps logs. Here's what I found. The Numbers (Before) Artifact content (uncompressed): 1,218 MB (1.2 GB) Artifact downloaded (compressed): 614 MB Download time: 3-4 min Pipeline breakdown: Build stage: ~5 min (Docker build + artifact) Download artifact: ~3 min (614 MB over the wire) Configure App Service: 2m54s (5 Azure API calls) Deploy (AzureWebApp@1): ~1 min Validate: 2m07s (sleep 30 + 3×30s probes) ───────────────────────────────── Total: ~15 min Root Cause #1: Ignoring output: 'standalone' next.config.js had this: const nextConfig = { output : ' standalone ' , // ← was there the whole time ... }; output: 'standalone' tells Next.js to produce .next/standalone/ — a self-contained directory with only what's needed at runtime. Trimmed node_modules . Auto-generated server.js . No source files. No dev dependencies. But the pipeline was ignoring it: # Old pipeline — copies everything from Docker docker cp deployImage:/app/node_modules . # 600 MB 😱 docker cp deployImage:/app/src . docker cp deployImage:/app/.next . docker cp deployImage:/app/server.js . # ... more files /bin/zip -r deploy.zip .env .next public node_modules package.json \ next.config.js jsconfig.json postcss.config.mjs decs.d.ts src server.js # Then published the ENTIRE working directory as the artifact - task : PublishPipelineArtifact@0 inputs : targetPath : ' $(System.DefaultWorkingDirectory)' # 1.2 GB of loose files + zip Azure DevOps compressed this to 614 MB for transfer. The deploy stage downloaded 614 MB to use a 24 MB zip buried inside it. The fix: # New pipeline — standalone only docker cp deployImage:/app/.next/