标签：#Docker

Docker VMM (virtual machine monitor) is Docker's new, first-party virtualization layer for Docker Desktop, replacing third-party virtualization components with an engine that Docker can directly control and optimize specifically for container workloads. The public beta launched with Docker Desktop 4.86 for Mac and Windows. By Sergio De Simone

post 7: A practical guide to understanding Docker Compose—what it is, how it works, and the misconceptions that catch most beginners. You've mastered single containers. Now it's time to build a real application. A frontend. A backend. A database. A Redis cache. Suddenly you're juggling multiple docker run commands. Ports. Networks. Volumes. Environment variables. Chaos. Then someone says: "Just use Docker Compose." It works beautifully. But here's the twist most people never realize… Why Docker Compose Exists Imagine starting an application like this: Frontend Backend PostgreSQL Redis Running each container manually quickly becomes repetitive and error-prone. Docker Compose lets you describe your entire application in a single YAML file and start everything with one command. Instead of remembering dozens of commands, you define your infrastructure once. What Docker Compose Actually Is Docker Compose is not a container orchestrator . Docker Compose is a tool that reads your Compose YAML file and uses the Docker Engine to create and manage the resources defined in it.” Modern Docker uses Compose V2 , which runs as: docker compose instead of the older: docker-compose Compose runs only when you execute a command. It creates the required Docker resources, starts the containers, and then exits. This makes it ideal for development, testing, and single-host deployments , but it doesn't provide orchestration features like automatic scheduling, self-healing, or multi-node management. A Simple docker-compose.yml services : web : build : . ports : - " 8080:80" environment : - DB_HOST=db depends_on : - db db : image : postgres:15 volumes : - postgres_data:/var/lib/postgresql/data redis : image : redis:alpine volumes : postgres_data : YAML Quick Reference Key Purpose services Defines containers (web, db, redis) build Builds an image from a Dockerfile image Uses an existing image from a registry ports Maps host ports to container ports environment Sets environment variables depend

The Quest Begins (The “Why”) Honestly, I used to stare at my CI/CD yaml files like they were ancient runes. Every push felt like a gamble: “Will the build pass this time?” I’d spend Friday nights hunting down a missing node_modules cache in Jenkins, only to realize the agent had run out of disk space because I’d forgotten to add a cleanup step. The pain was real, and the feedback loop was slower than a dial‑up modem. I kept asking myself: Why does this feel like wrestling a dragon every time I want to ship a feature? The answer was simple—I hadn’t yet found a pipeline that just worked out of the box. I wanted something that gave me confidence, not anxiety. So I embarked on a quest to compare the three big contenders: GitHub Actions, GitLab CI, and good ol’ Jenkins. Spoiler: the treasure wasn’t in the tool itself, but in how you shape the pipeline around your team’s flow. The Revelation (The Insight) The big “aha!” moment came when I stopped treating CI/CD as a one‑size‑fits‑all script and started seeing it as a contract between my code and my environment. The contract says: Every commit gets a clean slate. Dependencies are restored, not guessed. Tests run in parallel, not sequentially. Artifacts are published only if the gate passes. When I wrote that contract down, the yaml stopped looking like magic incantations and started looking like a checklist. The tools differ in syntax, but the underlying principles are the same. Here’s the secret: cache wisely, fail fast, and keep the pipeline short enough to give you feedback before you’ve even finished your coffee. Wielding the Power (Code & Examples) Below are three pipelines—one for each platform—that embody the contract above. I’ll first show a “struggle” version (the common pitfalls) and then the victorious version. 1. GitHub Actions – The Struggle name : CI on : [ push , pull_request ] jobs : build : runs-on : ubuntu-latest steps : - uses : actions/checkout@v3 - name : Install deps run : npm install # <-- no cache,

1. Retomando: da aplicação funcionando ao container pronto para produção Esta série cobriu, até aqui, o suficiente para desenvolver com Docker no dia a dia: conceitos fundamentais, comandos essenciais, Dockerfiles eficientes, rede, volumes e Compose para orquestrar múltiplos serviços. Este último artigo fecha a lacuna entre "funciona no meu Compose local" e "pronto para rodar em produção": imagens menores via multi-stage builds, segurança básica e não negociável, e como tudo isso se integra a um pipeline de CI/CD. 2. O problema que multi-stage builds resolve Compilar ou empacotar uma aplicação frequentemente exige ferramentas que a aplicação não precisa em tempo de execução : compiladores, headers de desenvolvimento, o próprio código-fonte antes de ser transpilado/buildado. Um Dockerfile ingênuo carrega tudo isso para a imagem final: # Ruim: ferramentas de build viajam junto para produção FROM node:20 WORKDIR /app COPY . . RUN npm install && npm run build CMD ["node", "dist/server.js"] Essa imagem inclui o npm , todo o node_modules (incluindo dependências de desenvolvimento), o código-fonte original e as ferramentas de build — frequentemente centenas de MBs de peso morto que nunca são usados depois que npm run build termina, e que ainda aumentam a superfície de ataque da imagem (mais binários, mais coisa que pode ter vulnerabilidade). Multi-stage builds resolvem isso permitindo múltiplos blocos FROM no mesmo Dockerfile, onde estágios posteriores copiam seletivamente apenas o que precisam dos anteriores — o restante do estágio de build simplesmente não existe na imagem final: # Estágio 1: build, com todas as ferramentas necessárias FROM node:20 AS build WORKDIR /app COPY package*.json . RUN npm ci COPY . . RUN npm run build # Estágio 2: produção, só com o resultado do build FROM node:20-slim WORKDIR /app COPY --from=build /app/dist ./dist COPY --from=build /app/node_modules ./node_modules COPY package*.json . CMD ["node", "dist/server.js"] A imagem final não contém o

1. Retomando: do docker run repetido a um arquivo único No artigo anterior, subir uma API e um Postgres conectados exigiu dois comandos docker run longos, com flags de rede, volume e variáveis de ambiente para lembrar (e digitar) toda vez. Em um projeto real, com mais serviços — cache, fila, worker em background — isso rapidamente vira inviável de manter na cabeça ou em um script solto. O Docker Compose resolve isso descrevendo toda a aplicação multi-container em um único arquivo declarativo, versionado junto com o código. 2. O arquivo compose.yaml Compose lê um arquivo YAML (por convenção compose.yaml , ou o nome legado docker-compose.yml , ainda amplamente usado) descrevendo serviços (cada um vira um ou mais containers), redes e volumes: # compose.yaml services : api : build : . ports : - " 8000:8000" environment : DATABASE_URL : postgresql://postgres:segredo@banco:5432/postgres depends_on : - banco banco : image : postgres:16 environment : POSTGRES_PASSWORD : segredo volumes : - pg-dados:/var/lib/postgresql/data volumes : pg-dados : Isso substitui inteiramente os dois docker run do artigo anterior. Uma diferença importante já aparece aqui: por padrão, Compose cria uma rede própria para o projeto e conecta todos os serviços a ela automaticamente — não é preciso um docker network create manual, nem declarar --network em cada serviço. Cada serviço já é acessível pelos demais pelo nome declarado em services: (aqui, banco resolve para o container do Postgres), exatamente como as redes definidas pelo usuário do artigo anterior. 3. Comandos essenciais do Compose docker compose up -d # sobe todos os serviços em segundo plano docker compose ps # lista os containers do projeto e seu status docker compose logs -f api # segue os logs de um serviço específico docker compose logs -f # segue os logs de todos os serviços, intercalados docker compose exec api bash # abre um shell dentro do container de um serviço docker compose stop # para os containers sem removê-los docker comp

Cómo solucionar docker run con Exited (1) en Raspberry Pi ¿Por qué ocurre este error? El código de salida 1 indica que el proceso principal del contenedor terminó con un error genérico. En Raspberry Pi, los casos más comunes son: Arquitectura incompatible : La imagen fue construida para amd64 (x86_64), pero Raspberry Pi usa arm32v7 o arm64v8 . Falta de binarios compatibles : El ENTRYPOINT o CMD del contenedor intenta ejecutar un binario compilado para otra arquitectura. Problemas de permisos o dependencias faltantes en el entorno embebido (especialmente en Raspberry Pi OS Lite sin GUI). Uso incorrecto de --net=host : En algunas versiones de Docker en Raspberry Pi, el flag --net=host puede causar fallos si el sistema no lo soporta correctamente. 🔍 Nota crítica : En tu comando original docker run --net = host -d -t myimage , hay un error de sintaxis: --net = host tiene espacios alrededor del = . Docker lo interpreta como un nombre de red literal " = host" , lo que probablemente falla. Pasos para solucionarlo Paso 1: Corrige la sintaxis del comando # ❌ Incorrecto (con espacios en `--net`) docker run --net = host -d -t myimage # ✅ Correcto (sin espacios) docker run --net host -d -t myimage ⚠️ Importante : En Docker CLI, los flags con valores no deben tener espacios entre el = . Usa --net=host o --net host , pero nunca --net = host . Paso 2: Verifica la arquitectura de la imagen Ejecuta en tu Raspberry Pi: docker inspect myimage --format '{{.Architecture}}' Si el resultado es amd64 , la imagen no es compatible con Raspberry Pi . Solución: Reconstruir la imagen para ARM Si tienes el Dockerfile , usa multi-arch build: # Al inicio del Dockerfile (antes de FROM) # syntax=docker/dockerfile:1 FROM --platform=$BUILDPLATFORM golang:1.21-alpine AS builder ... O construye explícitamente para ARM: # En tu máquina de desarrollo (x86_64) docker buildx create --use docker buildx build --platform linux/arm/v7 -t myimage:armv7 . --push # o para Pi 4 (64-bit): docker buildx build --platf

The Quest Begins (The "Why") I remember the first time I tried to take a weekend side‑project from my laptop to something that felt “real”. I had a cute Express API that talked to Postman, a PostgreSQL container spun up with docker-compose up , and a React front‑end that lived in its own dev server. Everything worked beautifully … until I hit Ctrl+C on my laptop and the whole thing vanished. I needed a way to say, “Hey, keep this running even if I close my laptop, and if something crashes, bring it back up automatically.” I started poking at Docker Swarm, then Nomad, but the docs felt like reading ancient runes. That’s when a coworker slid over a Slack message: “Just try a Kind cluster. It’s K8s locally, and you’ll see why everyone talks about it.” Spoiler: it felt like discovering the secret level in a classic arcade game. Suddenly I could describe what I wanted my system to look like, and the cluster would make it happen — no more babysitting containers. The Revelation (The Insight) Kubernetes isn’t a mystical black box; it’s a declarative orchestrator . You tell it the desired state of your application (how many replicas, which image, what ports to expose) and it works relentlessly to match reality to that state. If a pod dies, Kubernetes spins up a new one. If you ask for three replicas and only two are running, it creates the missing pod. If you update the image tag, it rolls out the change pod‑by‑pod, keeping traffic flowing. Think of it like the save‑game system in a RPG: you define the story you want to experience, and the engine handles the gritty details of loading, saving, and recovering from crashes. The core objects you’ll meet early on are: Pod – the smallest deployable unit (one or more tightly coupled containers). Deployment – manages a set of identical pods, handles updates and rollbacks. Service – a stable network endpoint that load‑balances traffic to a set of pods. Ingress (optional) – exposes HTTP/HTTPS routes from outside the cluster to service

1. Retomando: de imagens bem construídas a containers que conversam entre si Os artigos anteriores desta série cobriram como criar imagens eficientes e rodar containers isolados. Mas uma aplicação real raramente é um único container: normalmente há uma API, um banco de dados, um cache, talvez uma fila de mensagens — cada um em seu próprio container, precisando se comunicar. E containers, por padrão, são efêmeros: qualquer dado escrito dentro deles some quando são removidos. Este artigo cobre as duas peças que resolvem isso: redes (comunicação entre containers) e volumes (persistência de dados). 2. O problema do isolamento de rede por padrão Cada container recebe seu próprio namespace de rede, isolado dos demais e do host. Isso é uma característica de segurança, não um bug — mas significa que dois containers rodados de forma independente não conseguem se encontrar automaticamente: docker run -d --name api minha-api docker run -d --name banco postgres De dentro do container api , tentar acessar banco por esse nome simplesmente falha — cada container, isolado, só enxerga localhost como a si mesmo. A solução do Docker para isso é criar uma rede e conectar ambos os containers a ela. 3. Redes definidas pelo usuário (User-Defined Networks) docker network create minha-rede docker run -d --name banco --network minha-rede postgres docker run -d --name api --network minha-rede minha-api A partir daqui, dentro do container api , o hostname banco resolve automaticamente para o IP do container banco — o Docker roda um DNS interno para qualquer rede definida pelo usuário, resolvendo containers pelo nome (ou pelo alias definido com --network-alias , se houver mais de um). Isso é o motivo pelo qual strings de conexão em aplicações containerizadas costumam usar o nome do serviço em vez de um IP fixo: DATABASE_URL = postgresql :// usuario : senha @ banco : 5432 / meudb Comandos úteis para inspecionar redes: docker network ls # lista todas as redes docker network inspect minha-rede # d

Learn how containers communicate with each other and how to keep data alive even after containers are removed. Modern applications rarely run as a single container. A typical application might include a web application, a database, a cache layer, and background workers. For these services to work together, containers need a reliable way to communicate and share data. In this article, we'll learn: How Docker networking works How containers discover each other Docker network drivers Persistent storage with Docker volumes Essential networking and volume commands A real-world multi-container example By the end, we'll understand two of the most important concepts in Docker: networking and data persistence . Why Docker Networking Matters Every container runs inside its own isolated network namespace. This isolation improves security and prevents conflicts, but it also creates an important challenge: If containers are isolated, how does a web application connect to a database? Imagine a web application running inside one container and MongoDB running inside another. Without networking, they cannot communicate. Docker solves this problem using Docker Networks . A Docker network allows containers to communicate with each other while remaining isolated from unrelated containers. Web App Container | v Docker Network | v Database Container Without a shared network, containers cannot easily find or communicate with each other. Docker Network Drivers Docker supports several network drivers, but most developers primarily use three. Bridge Network A bridge network creates a private virtual network on the Docker host. Containers connected to the same bridge network can communicate with each other securely. Create a custom bridge network: docker network create my-app-network Benefits of bridge networks: Container-to-container communication Isolation from other applications Built-in DNS resolution Easy management For most Docker projects, a user-defined bridge network is the recommend

1. Retomando: do Dockerfile mínimo a um Dockerfile de verdade Na segunda parte desta série, um Dockerfile de poucas linhas já foi suficiente para empacotar uma aplicação Python. Isso funciona, mas um Dockerfile escrito sem pensar em camadas e cache de build gera imagens maiores do que precisam ser e builds que demoram muito mais do que deveriam a cada mudança pequena no código. Este artigo aprofunda como o Docker constrói uma imagem por dentro, e como escrever um Dockerfile que tira proveito disso. 2. Como funcionam as camadas (layers) Cada instrução de um Dockerfile ( FROM , RUN , COPY , ADD ) que modifica o sistema de arquivos gera uma camada — um diff read-only armazenado separadamente e empilhado sobre as anteriores. A imagem final é simplesmente a soma de todas essas camadas, e o container em execução adiciona uma camada gravável no topo (union filesystem). Container (camada gravável) ────────────────────────── Camada 4: COPY . . Camada 3: RUN pip install -r requirements.txt Camada 2: COPY requirements.txt . Camada 1: FROM python:3.12-slim Duas consequências práticas importantes: Camadas são reaproveitadas entre imagens. Se duas imagens diferentes compartilham as mesmas primeiras instruções (por exemplo, a mesma FROM e o mesmo RUN apt-get install ), o Docker armazena essa camada uma única vez em disco, mesmo que várias imagens a usem. Camadas são cacheadas entre builds. Ao rodar docker build de novo, o Docker verifica cada instrução, na ordem: se a instrução e seus arquivos de entrada não mudaram desde o último build, ele reaproveita a camada já construída em vez de refazer o trabalho. Isso é a base de todo o próximo tópico. 3. Cache de build: ordenar o Dockerfile por frequência de mudança O cache de build é invalidado a partir do primeiro ponto de mudança : se a instrução N mudou (ou um arquivo que ela copia mudou), toda camada a partir de N é reconstruída — mesmo que as instruções seguintes sejam idênticas ao build anterior. Isso significa que a ordem das ins

How to rescue abandoned open-source projects, modernize build systems, and generate multi-architecture Docker images (x86_64, ARM64) in a single afternoon with Antigravity.

1. Retomando: de imagens a containers em execução Na primeira parte desta série vimos o que é o Docker, o problema que ele resolve e os três conceitos fundamentais — imagens, containers e registries. Agora que a base teórica está posta, o foco deste artigo é prático: os comandos que efetivamente viram hábito no uso diário — run , exec , logs , ps , build — aplicados a containers reais, não só ao hello-world . 2. docker run além do básico O artigo anterior já usou docker run para subir um Nginx. Vale conhecer as flags que aparecem o tempo todo: # Modo interativo, útil para explorar uma imagem manualmente docker run -it ubuntu bash # Variáveis de ambiente docker run -e POSTGRES_PASSWORD = segredo -d postgres # Montar um diretório do host dentro do container (volume bind mount) docker run -v $( pwd ) /dados:/dados -d minha-imagem # Remover o container automaticamente quando ele parar docker run --rm -it python:3.12 python3 # Limitar recursos docker run --memory = 512m --cpus = 1 minha-imagem -it combina -i (interativo, mantém STDIN aberto) com -t (aloca um pseudo-terminal) — é o par de flags para "entrar" em um container e usar um shell como se fosse uma máquina normal. --rm evita acumular containers parados no disco depois de testes rápidos e descartáveis — sem ela, cada docker run deixa um container parado para trás até ser removido manualmente. -e define variáveis de ambiente; imagens oficiais como a do Postgres costumam documentar quais variáveis elas esperam (usuário, senha, nome do banco inicial). 3. Inspecionando o que está rodando O comando mais usado para ter uma visão geral do que o Docker está gerenciando na máquina: docker ps # containers em execução docker ps -a # todos, incluindo parados docker ps -q # só os ids (útil em scripts) Para investigar um container específico mais a fundo: docker inspect meu-container # todos os metadados em JSON: rede, volumes, config docker top meu-container # processos rodando dentro do container docker stats # uso de CPU/mem

Docker and Kubernetes are two of the most consequential infrastructure technologies of the last decade. They changed how software is built, packaged, and deployed. They are also two technologies that most engineers use before they understand, which creates gaps in knowledge that show up at the worst times: a production outage, a security incident, a performance problem you cannot diagnose. This guide builds understanding from the ground up. Every concept is introduced with the problem it solves. You will understand why containers exist before you understand what they are. You will understand why Kubernetes exists before you understand how it works. By the end, you will know not just how to run these technologies but how to reason about them. Table of Contents The Problem Containers Solve - Why Docker Exists Docker Internals - What a Container Actually Is Images - Building Portable Application Packages Dockerfile - Writing Reproducible Builds Docker Networking - Container Communication Docker Volumes - Managing State Docker Compose - Multi-Container Applications The Problem Kubernetes Solves - Why Orchestration Exists Kubernetes Architecture - The Control Plane and Data Plane Core Kubernetes Objects - Pods, Deployments, Services, ConfigMaps, Secrets Namespaces and RBAC - Multi-Tenancy and Access Control Storage in Kubernetes - Persistent Volumes Ingress - Routing External Traffic Helm - Package Management for Kubernetes Service Mesh - Istio and Advanced Traffic Management AWS Container Services - ECS and EKS Real Architecture Patterns The Problem Containers Solve - Why Docker Exists The Classic Failure Mode A developer builds an application on their MacBook. It works. They hand it to the QA team. It does not work. They hand it to the operations team to deploy to production. It works differently than in QA. "It works on my machine" is not a joke. It is a description of a real, chronic infrastructure problem. The application depends on: A specific version of Python, No

1. O Problema que o Docker Resolve "Na minha máquina funciona." Poucas frases resumem tão bem um problema que atormentou (e ainda atormenta) times de desenvolvimento: um código que roda perfeitamente no notebook do desenvolvedor, mas quebra no servidor de produção — porque a versão do Python é outra, uma biblioteca do sistema está faltando, uma variável de ambiente não foi configurada, ou o sistema operacional simplesmente se comporta de forma diferente. O Docker resolve exatamente isso: ele empacota uma aplicação junto com tudo que ela precisa para rodar — código, dependências, bibliotecas do sistema, variáveis de ambiente, configuração — em uma unidade isolada e portátil chamada container . Essa unidade roda da mesma forma em qualquer lugar que tenha o Docker instalado: no notebook do desenvolvedor, no servidor de CI, ou em produção. Esta é a primeira parte de uma série que vai do zero ao avançado em Docker: hoje o foco é entender o problema que ele resolve, os conceitos fundamentais e como eles se encaixam. 2. Containers vs Máquinas Virtuais A comparação mais comum ao explicar Docker é com máquinas virtuais (VMs), porque ambos resolvem um problema parecido — isolar e empacotar aplicações — mas de formas muito diferentes. Uma máquina virtual virtualiza o hardware inteiro: cada VM roda seu próprio sistema operacional completo (kernel incluso), gerenciado por um hypervisor. Isso garante isolamento forte, mas tem um custo alto: cada VM consome centenas de MBs a alguns GBs de disco e memória só para o SO, e leva de dezenas de segundos a minutos para inicializar. Um container , por outro lado, virtualiza no nível do sistema operacional: todos os containers em uma máquina compartilham o mesmo kernel do host, mas cada um enxerga seu próprio sistema de arquivos, processos e rede isolados — usando recursos do kernel Linux como namespaces (isolamento de visão) e cgroups (limites de CPU/memória). O resultado é que containers são muito mais leves: alguns MBs a poucas centenas

If you love the simplicity of Docker Swarm (native Compose files, lightweight single binary) but miss the advanced capabilities of Kubernetes (targeted label placement, SRE-grade observability, built-in DNS service discovery, and zero-trust ingress), meet Gubernator (gbnt) . We are excited to release Gubernator v2.13.0 , introducing three massive feature suites natively integrated into a single binary and a modern Material Design 3 Flutter Web Dashboard: Google SRE Multi-Burn-Rate SLO Engine & Interactive Suite CoreDNS 4-Tab Management Suite & Interactive Dig Playground Caddy Ingress & Zero-Trust Reverse Proxy Suite Fun Fact: The entirety of Gubernator's codebase, multi-node deployment pipelines, and SRE features were designed, built, and pair-programmed using **Google Antigravity (AGY) , Google DeepMind's agentic AI coding assistant! Let's dive into what's new and how you can level up your self-hosted or production container clusters! 1. Google SRE Multi-Burn-Rate SLO Engine & Web Suite Defining Service Level Objectives (SLOs) and tracking Error Budgets is the gold standard of Site Reliability Engineering. Until now, implementing SLOs meant running heavy Kubernetes CRDs (via tools like Sloth or Pyrra) or using costly SaaS platforms. Gubernator v2.13.0 brings Google SRE Workbook (Chapter 5) compliant multi-burn-rate alerting straight to simple docker-compose.yml services: version : " 3.8" services : payment-api : image : hashicorp/http-echo:latest labels : gbnt.slo.enable : " true" gbnt.slo.target : " 99.9" gbnt.slo.window : " 30d" gbnt.slo.template : " caddy-http" gbnt.slo.journey : " Checkout Flow" What makes Gubernator's SLO Suite unique? Google Multi-Burn-Rate Alerting : Automatically generates standard 4-window Prometheus recording and alert rules ( Critical Page 1h/6h & Warning Ticket 3d/14d ). Dynamic "No-Code" Management : Click "+ Configure / Add SLO" in the Web UI or call POST /v1/slo/edit to create, edit, or disable SLOs on the fly without editing Compose

I thought this would be a simple task. I already had a Python Kafka producer running. Kafka was up in Docker. The topic existed, and I could send a message into it successfully. The next step sounded straightforward: Python Producer ↓ Kafka ↓ Spark Structured Streaming All I wanted Spark to do was read a JSON message from a Kafka topic. Instead, I ran into one error after another. At first, it looked like one problem: Spark cannot read Kafka. It was not one problem. It turned into a chain of failures across several different layers: Python / PySpark ↓ Spark runtime ↓ Kafka connector ↓ Hadoop / Windows ↓ Docker ↓ Kafka networking ↓ Ivy dependency resolution The useful part of this experience was not any single fix. It was learning how to separate the layers and stop treating every error as a problem in my Python code. This is the full debugging path. What I Was Building This was part of an financial data engineering project. The batch side of the project already looked roughly like this: Financial Data Source ↓ Python ingestion ↓ AWS S3 ↓ Snowflake ↓ dbt ↓ Financial anomaly models I wanted to add a streaming extension for newly arriving financial events. For the first version, I kept it intentionally simple: Python Kafka Producer ↓ Kafka topic: financial_events ↓ Spark Structured Streaming The producer sent a simulated financial event: { "company_id" : "COMPANY_001" , "company_name" : "Sample Company" , "report_type" : "quarterly_report" , "reporting_date" : "2026-08-08" , "event_id" : "FIN-20260808-001" , "source" : "simulated_financial_event" } Kafka accepted the message successfully. I could even read it with Kafka's console consumer. So Kafka itself was working. Then Spark entered the picture. Failure #1: PySpark Worked, but spark-submit Didn't I installed PySpark: pip install pyspark Then I installed Java 17 and verified it: java -version After reopening my terminal, Java was available. I tested Spark directly through Python: python -c "from pyspark.sql import S

I put an AI into a Google Meet call. It transcribed Japanese, generated a reply, and spoke it out loud. Total new spend: $0 . Then I asked it the one question I actually needed answered, and it said: "I think there's still room for discussion. How about we set up a session to align our understanding?" That is exactly what a person says when they don't know. TL;DR: I had a latency problem and an "is this model smart enough" problem. Neither was real. Same model, same question, 5.80s → 5.68s — 2,545 characters of context turned a deflection into a claim you could argue with. The stack, and what it replaced I wanted an AI participant in a real meeting. Not a note-taker — something that answers when someone demands specifics. The obvious stack bills you three times: a hosted meeting-bot API, a speech-to-text vendor, and a text-to-speech vendor. I replaced all three. Layer Obvious choice What I used Why Meeting bot Recall.ai, $0.50/hour Attendee (OSS, self-hosted) no per-hour billing Speech-to-text Deepgram / AssemblyAI Google Meet's own captions the meeting already generates them Text-to-speech Google Cloud TTS raw audio POST (below) no GCP project at all Reasoning + voice LLM + TTS, two hops Gemini Live (speech-to-speech) one model, one hop Attendee is 699 stars, last pushed 2026-08-07. Google Meet exposes no bot API, so it drives a full Chrome instance — which is why setup hurt before anything else did. The setup tax, compressed Two problems were routine. The image pins FROM --platform=linux/amd64 ubuntu:22.04 , and my machine is Apple Silicon, so colima with Rosetta: colima start --vm-type = vz --vz-rosetta --cpu 6 --memory 12 --disk 60 docker run --rm --platform = linux/amd64 alpine:3.20 uname -m # x86_64, 5.6s cold Then the build died at step 35 of 42 with the --chmod option requires BuildKit — colima's docker CLI ships without the buildx plugin. brew install docker-buildx , point ~/.docker/config.json at /opt/homebrew/lib/docker/cli-plugins via cliPluginsExtraDirs

Docker for Beginners: Images, Containers, Ports, and Volumes Explained If you've ever followed a programming tutorial and seen something like: docker run ... you've probably wondered: What exactly is Docker doing? I had the same question when I started learning Docker. At first, I thought Docker was simply a way to "run applications in containers." But there is much more to it. Once I understood four concepts — images, containers, ports, and volumes — Docker became much easier to understand. So let's break it down from the beginning. What Is Docker? Docker is a platform for building, packaging, and running applications in isolated environments called containers . The basic idea is simple: Package an application together with the things it needs to run, and make that package portable. For example, imagine you build a Python application. Your application might depend on: Python 3.12 FastAPI Uvicorn Several Python packages Environment variables Certain system libraries On your computer, everything works. Then someone else downloads your project. They install a different Python version. A package is missing. Something behaves differently. Now you have: "It works on my machine." Docker helps reduce this problem by allowing you to define the environment your application should run in. The Four Concepts You Need to Understand Before learning Docker commands, understand these four things: Docker Image ↓ Docker Container ↓ Ports ↓ Volumes Let's look at each one. 1. What Is a Docker Image? A Docker image is a packaged, read-only template used to create containers. Think of it like a blueprint. For example: Docker Image │ ├── Ubuntu ├── Python ├── Application code ├── Dependencies └── Configuration An image contains the instructions and filesystem needed to create a container. You can download images from container registries such as Docker Hub. For example: docker pull nginx This downloads the Nginx image. You can see your downloaded images with: docker images You might see s

If you want AI agents running on your own machine, with your own models, and no data leaving your computer, this is the article :). This is part three of the series. In part one we set up PostgreSQL, and in part two we covered the LLM concepts (models, parameter, quantization, context, capabilities, VRAM). Today we put them to work: n8n for the workflows and Ollama for the models. One prerequisite: Docker. If you do not have it yet, install Docker Desktop for Mac following the official guide [ Docker docs ]. Quick setup: n8n The fastest path is n8n's official Self-hosted AI Starter Kit, a Docker Compose template that ships n8n, Ollama, Qdrant (a vector store) and PostgreSQL preconfigured to talk to each other [ n8n docs ]. git clone https://github.com/n8n-io/self-hosted-ai-starter-kit.git cd self-hosted-ai-starter-kit cp .env.example .env # file where your passwords are stored The .env file is hidden by default. In Finder, press Command + Shift + Period to show hidden files, or just edit it from the terminal. Update the credentials, for example: POSTGRES_USER = admin POSTGRES_PASSWORD = root POSTGRES_DB = n8n Also replace the N8N_ENCRYPTION_KEY and N8N_USER_MANAGEMENT_JWT_SECRET values with your own random strings. Now one Mac-specific detail. Docker on Apple Silicon cannot use the Mac's GPU, so the kit's README recommends running Ollama natively on your Mac for speed and letting the containers connect to it [ starter kit README ]. That is what we'll do. Set this in your .env : OLLAMA_HOST = host.docker.internal:11434 Then start everything: docker compose up Open http://localhost:5678 to create your n8n account (once), and http://localhost:5678/home/workflows is where your workflows and agents live. If you only want n8n without the rest of the kit, this single command works too [ n8n docs ]: docker run -it --rm --name n8n -p 5678:5678 -v n8n_data:/home/node/.n8n docker.n8n.io/n8nio/n8n Quick setup: Ollama On the Mac side (from part two, condensed): brew install olla

There is a line in almost every Python web tutorial that nobody explains: uvicorn main:app --host 0.0.0.0 --port 8000 I copied it for weeks without thinking about it. Then I deployed the same application three times — to a local VM, to a production server, and into a container — and the correct value was different every time. Twice it was 0.0.0.0 . Once, in the place that mattered most, it was not. That gap is worth writing about, because the setting itself is trivial and the reasoning behind it is not. What the Flag Actually Controls A server process doesn't "open a port." It creates a socket and binds it to an address. The bind address answers one question: which network interfaces should this socket accept connections from? A machine has more than one interface: lo (loopback) — reachable only from inside the machine ( 127.0.0.1 ). Packets addressed there never reach a physical network card; the kernel loops them straight back. 0.0.0.0 — a wildcard meaning every interface this machine has , including ones added later. So the flag isn't about security or convenience. It's about reachability — and reachability depends entirely on what sits in front of the process. Case 1: The Local VM — 0.0.0.0 I was running the service inside a Multipass VM and wanted to hit it from the browser on my laptop. The laptop is outside the VM, so binding to loopback would have made the service invisible to it. curl inside the VM would work; the browser outside would get connection refused. Decision: wildcard bind. Nothing sits in front of the process, and nothing needs protecting. Case 2: Production — 127.0.0.1 Here I copied the same line at first, and it was wrong. The production box has a public IP. Binding to 0.0.0.0 there means the application is directly exposed to the internet: no TLS, no rate limiting, no authentication. Within hours of provisioning that server, its SSH logs showed hundreds of automated login attempts against usernames like admin and oracle . The same scanners try