标签：#DevOps

AI agents are getting very good at writing final reports. The problem is not only that they make mistakes. The problem is that sometimes they make mistakes with excellent presentation. Proofline is a 5-skill Markdown pack that catches fake-ready output before it turns into a release, handoff, public post, or "yeah, looks done". What Proofline does It is not trying to be another giant agent. It works as a review route after the agent produces a result: Reference Gap Ready Gate Reality QA Lean Pass Repair Report Compiler Each step asks an annoying but useful question: what is missing from the references, what was not checked, where did the agent pretend everything was fine, and what actually needs to be fixed? Who it is for Builders working with Codex-style agent chats, AI coding workflows, Markdown handoffs, and any process where "done" needs to mean more than a confident paragraph. Release: https://github.com/aisflows/proofline/releases/tag/v0.2.0-rc5

Texto com base no livro "Manual de DevOps" WIP significa "Work In Progress". É uma métrica essencial que representa a quantidade de trabalho iniciado, mas ainda não concluído. Na prática, ela ajuda a entender quantos tickets, tarefas, histórias ou demandas estão sendo executados simultaneamente pelo time. WIP Alto (Ruim) Time com 5 pessoas 20 histórias abertas Todos pegam várias tarefas ao mesmo tempo Dezenas de branches simultâneas Dezenas de PRs simultâneos WIP Baixo (Bom) Time com 5 pessoas Apenas 5 histórias abertas Cada pessoa trabalhando em uma tarefa por vez O time termina as tarefas antes de começar outras Menor Lead Time Essa métrica é essencial para uma boa estratégia de DevOps, além de ser um baita indicador para a saúde do projeto ou da companhia. Quanto maior o WIP: Maior troca de contexto Mais conflitos de merge Mais difícil rastrear e validar as entregas Maior "latência" no tempo de aprovação dos PRs Se seu time está começando muitas frentes e terminando poucas demandas, você está com um WIP alto, e isso afeta diretamente a qualidade das entregas e a qualidade de vida das pessoas. Sei que WIP aparece bastante nos princípios Lean, porém ainda não li o suficiente sobre o tema para me aprofundar nele.

Most developers run git pull dozens of times a week without thinking about it. And most of the time, it works. Then one day you open a PR and the reviewer says "can you clean up the merge commits?" You look at your branch and see three "Merge branch 'main' into feature/login" commits scattered through history. The feature itself is 5 commits. The log is a mess. That mess comes from one decision: using git pull instead of git pull --rebase . Here's what's actually happening, and why the rebase variant produces cleaner history for teams. The setup: diverged history You're working on feature/login . You commit two changes locally ( X , Y ). Meanwhile, your teammate pushes two commits to main ( C , D ). Your branch and main have now diverged . Neither is a strict superset of the other. Git needs to reconcile them when you pull. Shared history: A → B Your local: A → B → X → Y (you added X, Y) Remote main: A → B → C → D (teammate added C, D) Git has two strategies for this reconciliation. Strategy 1: git pull (merge) A plain git pull creates a merge commit that joins your local history with the remote. Your commits and the remote's commits both appear in the log, connected by a merge node. The git log reads: M Merge branch 'main' into feature/login D fix: timeout on slow connections Y feat: client-side validation C chore: upgrade eslint X feat: login form B (shared) A (shared) This is honest history — it records exactly what happened: parallel development that was joined at a specific point. But it's also noisy history — the merge commit has no meaningful changes, and the log interleaves commits that weren't conceptually related. Strategy 2: git pull --rebase With --rebase , Git takes a different approach. It: Temporarily sets aside your local commits ( X , Y ) Fast-forwards your branch to the tip of the remote ( D ) Replays your commits on top, one by one, creating new commits ( X' , Y' ) The git log reads: Y' feat: client-side validation X' feat: login form D fix: timeo

IaC tools built for single-team deployments fail structurally at 200 accounts because the failure modes are architectural, not configurational. Why 200 Accounts Is Where IaC Tools Break IaC tools built for single-team deployments fail structurally at 200 accounts because the failure modes are architectural, not configurational. Scale Threshold State Management Provider Auth Overhead Execution Time Impact ~10 accounts One backend bucket, one workspace; quirks are routable Not a meaningful bottleneck Sequential plan/apply is manageable ~50 accounts Sequential execution still viable; blast radius contained Per-account latency exists but tolerable Below threshold where parallelism is required 200+ accounts 200 separate plan operations per shared-module refactor 3-sec per-account auth × 200 = 10 min added per plan cycle Sequential Terraform applies measured at 4.1 hours end-to-end A 10-account environment forgives sloppy state management. One backend bucket, one workspace convention, one pipeline. Engineers learn the tool's quirks and route around them. At 200 accounts, those same quirks compound. Why the threshold is 200 State lock contention, cross-account provider authentication chains, and module resolution latency stack on top of each other. The result is not slower deploys. It is non-deterministic deploys, which is operationally worse. The specific threshold matters. Below roughly 50 accounts, most teams run plan and apply sequentially without parallelism because the blast radius of a runaway apply is contained. Above 200 accounts, sequential execution becomes untenable. We measured a 200-account org where sequential Terraform applies across all accounts took 4.1 hours end-to-end. That latency made emergency remediation impossible inside a standard incident window. Three compounding failure modes State file proliferation. Each account carries its own state file, and each state file is a consistency boundary. At 200 accounts, a single refactor touching a shared modu

Time-series data is one of the most common types of data generated by modern applications. Every log entry, API request, metric, transaction, sensor reading, or user interaction is recorded with a timestamp, making time the primary dimension for analysis. As organizations collect billions of these records, efficiently storing and querying them becomes increasingly challenging. This is where ClickHouse® excels. Although ClickHouse is not a dedicated time-series database, its columnar storage architecture, vectorized query execution, high compression ratios, and massively parallel processing make it an excellent choice for time-series analytics at scale. It is capable of ingesting large volumes of data while delivering analytical queries in milliseconds. The article begins by explaining the fundamentals of time-series data and highlighting common real-world use cases such as application monitoring, IoT sensor data, financial market analysis, server metrics, user activity tracking, and business analytics. These workloads typically involve continuous data ingestion, time-based filtering, aggregations, and trend analysis. One of ClickHouse's biggest strengths is its optimization for analytical workloads. Since data is stored column-wise rather than row-wise, only the required columns are read during query execution. Combined with compression and vectorized processing, this significantly reduces I/O and improves query performance over massive datasets. The article also demonstrates how to create an optimized table for time-series workloads using the MergeTree engine. Proper partitioning by month and ordering data by dimensions and timestamps help ClickHouse prune unnecessary partitions and efficiently locate relevant data during queries. Several practical SQL examples are covered, including: Filtering records within a specific time range Aggregating metrics by hour, day, week, or month Calculating averages, sums, minimums, and maximums Grouping events over time Working wi

You run a Telegram bot through OpenClaw on your own Linux server. One day it goes quiet. The bot can't send or receive. But the server itself is fine — SSH works, apt works, other sites load. The reason: your server can't reach api.telegram.org . Some networks block or throttle it, so every call times out while everything else is fine. The clean fix: route only OpenClaw's Telegram traffic through Cloudflare WARP (1.1.1.1) . Everything else on the box stays direct and fast — including your SSH login. Here is the full setup, step by step. First, confirm it's a Telegram-only problem curl --max-time 8 https://api.telegram.org/ # hangs / times out curl --max-time 8 https://www.google.com/ # works instantly If Telegram times out but other sites are quick, this guide is for you. How it works We chain three small tools: OpenClaw ──▶ iptables ──▶ redsocks ──▶ WARP (SOCKS5) ──▶ Cloudflare ──▶ api.telegram.org WARP gives us a local proxy that exits through Cloudflare's network (which can reach Telegram). redsocks turns normal connections into proxy connections (so the app needs no proxy support — OpenClaw has none). iptables picks only OpenClaw's Telegram traffic and sends it to redsocks. The trick is in that last step. We match by the app's user and Telegram's IP ranges, so nothing else is touched. Step 1: Install WARP in proxy mode # Add Cloudflare's package repo curl -fsSL https://pkg.cloudflareclient.com/pubkey.gpg \ | gpg --yes --dearmor -o /usr/share/keyrings/cloudflare-warp-archive-keyring.gpg echo "deb [signed-by=/usr/share/keyrings/cloudflare-warp-archive-keyring.gpg] \ https://pkg.cloudflareclient.com/ $( lsb_release -cs ) main" \ > /etc/apt/sources.list.d/cloudflare-client.list apt-get update && apt-get install -y cloudflare-warp # Sign up (free) and switch to proxy mode warp-cli --accept-tos registration new warp-cli --accept-tos mode proxy # opens a SOCKS5 proxy on 127.0.0.1:40000 warp-cli --accept-tos connect Now check that WARP can reach Telegram: curl --socks5-

"An AI agent just built a production landing page, with GDPR audit logs and encryption baked in. I wasn't even at my desk." That is not a lucky one-shot. It is a repeatable workflow. Piotr Karwatka recorded a full tutorial showing how to go from idea to a production-ready app on Open Mercato - the AI-Engineering Foundation Framework for CRM/ERP - with no babysitting and no ping-pong prompting. This is the technical version: what the loop actually looks like, why it doesn't fall apart, and which patterns you can lift into your own stack. The problem with conversational coding The default AI coding loop is single-threaded and human-bound: prompt -> generate -> you spot a bug -> correct -> re-prompt -> repeat It holds for snippets. It collapses the moment the task touches real architecture - multi-tenancy, RBAC, event flow, encryption, audit logging. Corrections pile up in the context window, the agent loses the thread, and you are back to typing. You are the bottleneck, sitting in the inner loop. The workflow in the tutorial moves you to the outer loop : you review a finished, tested PR instead of every keystroke. goal -> agent: branch + implement + test + open PR -> you: review PR The reason this is even possible on Open Mercato is that the hard architectural decisions are already encoded as conventions, specs and agent-readable skills ( AGENTS.md , task routing, spec skills). The agent is not inventing how RBAC or GDPR logging should work - it reads the foundation and follows it. 1. Fire-and-forget: the autonomous PR loop The execution agent owns the full unit of work: 1. git checkout -b feat/lead-capture-landing 2. implement against framework conventions 3. run the test suite (Playwright integration tests included) 4. open a structured PR: what changed, why, how it was verified You are no longer correcting tokens. The deliverable is a reviewable artifact. In the tutorial the output is concrete: a live site capturing leads straight into the Open Mercato CRM, with GD

The deployment should have taken a few minutes. The application was running, DNS was configured correctly, and the domain was already pointing to the server's public IP. Caddy was configured as a reverse proxy and was listening on ports 80 and 443. Every item on my deployment checklist appeared healthy. Yet every Let's Encrypt validation attempt kept failing. The error looked simple enough: authorization failed timeout during connect likely firewall problem At first, I believed it. I checked DNS resolution, verified firewall rules, confirmed that Caddy was listening on the expected ports, and made sure the application itself was reachable. Every check came back clean. That was the first clue that the problem might not be where the logs were pointing. The Obvious Things The first assumption was DNS. I verified that the domain resolved to the correct public IP. dig +short my-domain.com Everything looked correct. Next came the firewall. sudo ufw status Ports 80 and 443 were open. There were no unexpected deny rules, and nothing suggested inbound traffic was being blocked. Then I checked whether Caddy was actually listening. sudo ss -tulpn | grep -E ':80|:443' Again, everything looked normal. The application itself was healthy too. curl http://localhost:3001 returned a valid response. At this point I had checked most of the things engineers typically check when certificate validation fails. DNS looked good, the firewall looked good, the reverse proxy was healthy, and the application was running. Yet the validation errors continued. The Part That Sent Me In The Wrong Direction The error messages kept mentioning connectivity problems and possible firewall issues. That wording influenced my thinking more than it should have. I spent time investigating firewall rules, reverse proxy configuration, TLS settings, and domain configuration. Every new hypothesis felt reasonable, but none of them explained why local tests consistently succeeded while external validation continued

Introduction Dans le cadre de mon apprentissage des pratiques DevOps modernes, j’ai conçu et implémenté un pipeline CI/CD (Continuous Integration / Continuous Deployment) capable de déployer automatiquement une application web frontend sur deux environnements de production distincts : Vercel et GitHub Pages . Cette mission constitue une application concrète des concepts fondamentaux du DevOps, notamment l’automatisation des processus, la réduction des interventions manuelles et la mise en place d’une chaîne de livraison logicielle fiable et reproductible. Tableau comparatif des plateformes Dans ce projet, le déploiement sur les deux plateformes n’est pas un doublon mais une démarche pédagogique et technique délibérée permettant de tester la flexibilité de l'orchestrateur. Voici comment elles se comparent : Critère GitHub Pages Vercel Hébergement Statique uniquement Statique + SSR + Serverless Domaine gratuit username.github.io projet.vercel.app CI/CD intégré Via GitHub Actions Natif + GitHub Actions Performance Bonne Excellente (Edge Network) Previews PR Non Oui (automatique) Gratuit Oui (illimité) Oui (avec limites) Cas d’usage Portfolios, docs Apps React/Next.js, SaaS ⚠️ Le problème du double déploiement : Laisser Vercel en mode automatique génère un conflit critique avec GitHub Actions. Pour éviter que deux builds s'exécutent en parallèle, j'ai désactivé le déploiement natif de Vercel en ajoutant un fichier vercel.json à la racine contenant "git": { "deploymentEnabled": false } . Le pipeline complet — deploy.yml Voici le code source du fichier de configuration de l'orchestrateur GitHub Actions ( .github/workflows/deploy.yml ). Ce script gère séquentiellement l'installation, le build et la publication vers nos deux cibles : name : CI/CD -- Deploy to Vercel & GitHub Pages # Declenchement : uniquement sur push vers main on : push : branches : - main jobs : build-and-deploy : runs-on : ubuntu-latest permissions : contents : write steps : # Etape 1 : Recuperer le code

When was the last time you deleted an email account on purpose? For most teams the answer is never, and that tells you something. We treat mailboxes the way we treated servers in 2008: hand-built, carefully named, kept alive indefinitely because recreating one is painful. They're pets. Meanwhile every other piece of our infrastructure — compute, queues, databases — became cattle: numbered, provisioned by code, destroyed without sentiment when the job ends. Email is finally catching up. With Nylas Agent Accounts (in beta), a mailbox is created with one call and destroyed with another, and that symmetry is the whole point. The full lifecycle in two commands Provisioning, from the CLI: nylas agent account create signup-agent@agents.yourdomain.com Or via POST /v3/connect/custom with "provider": "nylas" — no OAuth, no refresh token, just an address on a domain you've registered. Teardown is equally unceremonious: nylas agent account delete signup-agent@agents.yourdomain.com --yes (The API equivalent is a DELETE on the grant.) The signup automation recipe treats this as a loop: provision a fresh inbox, point a third-party signup form at it, catch the verification email through a message.created webhook, follow the confirmation link, delete the grant. No human inbox involved at any step, and nothing left behind. The middle of that loop is about twenty lines of webhook handler, and the recipe's version filters hard before acting — right grant, right sender, right URL shape: const { grant_id , id : messageId , from } = event . data . object ; if ( grant_id !== AGENT_GRANT_ID ) return ; const sender = from [ 0 ]?. email ?? "" ; if ( ! sender . endsWith ( " @saas-you-care-about.example.com " )) return ; const match = /https: \/\/ saas-you-care-about \. example \. com \/ confirm \? token= [^ " \s < ] +/ . exec ( message . body , ); if ( match ) await fetch ( match [ 0 ]); Nylas fires message.created within a second or two of mail arriving, so the whole signup round-trip typical

Last quarter our OpenAI bill went from $620 to $2,480 in 23 days. No new features shipped. No traffic spike. Zero error alerts. Deployment logs were clean. Just a number climbing in silence while five engineers stared at dashboards that gave us totals and nothing else. This is what we found. And why "cost monitoring" is completely the wrong mental model. The dashboard that answers the wrong question First thing I did was open the OpenAI usage dashboard. It showed me a total. A graph going up. A model breakdown. I knew we spent $2,480. I still had no idea which feature spent it, which service triggered it, or which user was responsible. The dashboard was answering "how much" while we were desperately asking "what caused it." Those are completely different questions. Almost every cost tool on the market only answers the first one. That distinction matters more than most engineering teams realise until they are staring at a bill like ours. Three features, zero visibility We had three features hitting GPT-4o: A document summariser, triggered manually by users An inline suggestion engine, triggered on keystrokes A batch report generator, triggered on export Any one of them could be the problem. Or all three. Or one specific tenant hammering one endpoint in a loop nobody noticed. Without attribution at the feature, service, and user level, we were just guessing. So I did what most engineers do: optimised the feature that felt most expensive. Added caching to the one that ran most often. Two weeks later the bill was still climbing. Guessing at cost problems without attribution data is exactly like debugging a performance issue without a profiler. You move things around and hope. 48 hours of real data A teammate dropped CostReveal in our Slack. I set it up that evening. The Node.js SDK wraps your existing provider calls. You instrument each one with a feature name, service context, and user or tenant ID. That is the entire integration for the base case: import { CostReveal

Last year my side project was running on AWS. A t3.small EC2 instance, an RDS PostgreSQL db.t3.micro, an S3 bucket, and a CloudFront distribution. Total bill: $47/month for an app with 200 daily users. Then someone on Reddit told me to look at Hetzner. I now run the same stack on a single CAX21 (4 vCPU ARM, 8GB RAM, 80GB SSD) for €5.49/month. Here's exactly what happened. The Migration What I was running on AWS: Node.js API (Express) PostgreSQL database Redis for sessions Nginx reverse proxy Static files on S3 + CloudFront What I moved to Hetzner: Same Node.js API PostgreSQL installed directly on the server Redis installed directly on the server Nginx + Certbot for SSL Static files served by Nginx Total migration time: one Saturday afternoon. The hardest part was setting up automated backups (solved with a cron job + Hetzner's snapshot API). What Broke Nothing critical, but: No managed database failover. On RDS, if the database crashes, AWS restarts it automatically. On Hetzner, if PostgreSQL crashes at 3 AM, I'm the one fixing it. In 8 months, this has happened zero times. But it could. No CDN by default. My static assets now serve from a single Hetzner datacenter in Germany. For my EU-heavy userbase, this is actually faster than CloudFront. For US users, it's about 50ms slower. I added Cloudflare (free tier) in front and the problem disappeared. Deployment changed. No more eb deploy or push-to-deploy. I wrote a 12-line bash script that SSHs in, pulls from git, runs migrations, and restarts PM2. Takes 8 seconds. Honestly prefer it — I know exactly what's happening. The Cost Comparison at Every Scale This is what surprised me most. The gap isn't just at my small scale — it gets wider as you grow: SpecAWSDigitalOceanVultrHetzner2 vCPU, 4GB$30/mo$24/mo$24/mo€4.50/mo4 vCPU, 8GB$61/mo$48/mo$48/mo€8.50/mo8 vCPU, 16GB$122/mo$96/mo$96/mo€16/mo Hetzner is roughly 5-7x cheaper than AWS at every tier. DigitalOcean and Vultr sit in the middle. 👉 Calculate your exact costs When

TL;DR Provider invoices aggregate by model and billing period. They cannot tell you which team, product, or agent caused a cost spike. Request-level AI cost attribution links every API call to structured owner metadata (team, product, environment, trace ID) so investigations take minutes, not days. Three approaches exist: provider dashboard, gateway log enrichment, and application trace attribution. They differ sharply in setup cost and query granularity. Gateway log enrichment is the highest-leverage first step for most teams. It requires no changes to application code and covers all traffic behind the gateway. Real example: a platform team at a 60-person AI company discovered that 31% of their $18k/month spend came from a misconfigured retry loop in a background job, identified in under 20 minutes once request-level logs were searchable. Why Your Invoice Is Lying to You Your OpenAI invoice for last month shows $22,400. Your Anthropic invoice shows $6,800. Total: $29,200. Your CFO wants to know which business unit owns each line. You forward the invoices to your finance partner, who forwards them to three engineering managers, who reply with estimates that sum to $24,000 and do not match any real allocation. This is the standard state of LLM spend governance at companies between $5k and $50k per month in AI API costs. The invoices arrive, the spend is real, and attribution is a spreadsheet exercise done with guesses. The problem is structural. Provider billing aggregates by model and by billing period. It has no concept of your internal ownership model, your product boundaries, your tenant hierarchy, or your agent topology. A single gpt-4o line in your invoice might represent spend from a customer-facing chat feature, an internal summarization service, a nightly batch job, and three developers running experiments against production endpoints. You get one number. You have four or more owners. Request-level AI cost attribution is the practice of enriching every API c

Coinbase has published a detailed postmortem of its May 7, 2026, outage, revealing how a localized cooling failure inside an AWS data center escalated into a multi-hour disruption that halted nearly all trading activity across the cryptocurrency exchange By Craig Risi

Every day, startups rent expensive GPUs to power AI applications. The problem is that most of those GPUs spend a surprising amount of time doing nothing. Imagine renting an apartment and only using one room while paying for the entire building. That's effectively what many AI teams do with GPU infrastructure. The Hidden Cost of GPU Rentals When you rent a GPU, you're usually paying for uptime. Whether your application is processing requests or sitting idle at 3 AM, the bill keeps running. For many early-stage products: Traffic is inconsistent Usage spikes are unpredictable Most requests arrive in short bursts As a result, GPU utilization can be far lower than expected. The Utilization Problem A startup might rent a GPU for an entire month. But how much of that compute is actually being used? During development: Developers test occasionally Demos happen a few times a day Customer requests arrive sporadically The GPU remains available 24/7, but actual inference workloads often occupy only a small fraction of that time. Yet the infrastructure bill reflects full-time usage. Why This Matters For startups, infrastructure costs directly affect runway. Every dollar spent on idle compute is a dollar that cannot be spent on: Product development Customer acquisition Hiring Experiments Reducing wasted infrastructure spend can significantly improve efficiency. A Different Model Instead of paying for GPU uptime, what if developers only paid when inference actually occurred? For example: Pay per token generated Pay per image generated Pay per second of video generated This approach aligns cost with actual usage rather than reserved capacity. The Future of AI Infrastructure As AI adoption grows, efficiency becomes increasingly important. The next generation of AI infrastructure may look less like traditional server rentals and more like utilities: Use what you need. Pay for what you use. Nothing more. What has your experience been with GPU utilization and AI infrastructure costs? I

Stack Overflow has announced Stack Overflow for Agents, a beta API-first knowledge exchange aimed at AI coding agents rather than human developers. The service is presented as a way to close what the company calls the Ephemeral Intelligence Gap, where agents repeatedly rediscover the same fixes and patterns in isolation instead of sharing them through a common memory. By Matt Saunders

The problem IAM wildcards and public S3 buckets keep slipping through Terraform code review. Tools like Checkov and tfsec exist but they live in CI, require config files, and developers ignore the output because it's not where they're working. What I built TerraWatch is a GitHub App that scans every pull request that touches .tf files automatically. If it finds a security issue it blocks the merge and posts the exact code fix as a PR comment. The developer sees something like this in their PR: ⚠️ PUBLIC_S3_BUCKET - main.tf (Line 6) Severity: HIGH Risk: S3 bucket allows public read access. Fix: acl = "public-read" acl = "private" block_public_acls = true restrict_public_buckets = true They copy the fix, push, and the merge unblocks automatically. How it's different No YAML, no CI config - installs in 2 minutes via GitHub App Fixes are hardcoded diffs, not AI generated Nothing auto-applied - you review every fix No Checkov dependency - own lightweight rules engine Only reads changed .tf files in the PR, never your full codebase 29 rules covering S3 public access, IAM wildcards, open ports (SSH/RDP/MySQL/Postgres), unencrypted EBS/RDS, public databases, hardcoded secrets, EKS public endpoints, CloudTrail disabled, IMDSv1, and more. Try it Free during beta - terrawatch.dev Also launching on Product Hunt today if you want to show some support!

Agentic QA Pipelines: Why Your Test Scripts Are Already Obsolete You wrote the test. You maintained the test. The app changed. You rewrote the test. If that loop sounds familiar, you're not alone — and in 2026, you're also not competitive. Agentic QA pipelines are replacing script-based test automation not because AI is smarter than your QA engineers, but because describing goals is faster than maintaining instructions. Here's what's actually changing, why it matters, and how forward-thinking teams are shipping without the script debt. The Script Maintenance Tax Is Killing Velocity Traditional test automation follows a simple premise: write explicit instructions, run them, check results. It worked when applications changed slowly and test environments were stable. In 2026, neither is true. AI-generated code ships faster. Features change in days. UI components regenerate. And every change breaks a percentage of your carefully maintained test scripts — creating a maintenance tax that grows proportionally with your automation coverage. Quash's 2026 State of QA Automation Report found that teams spending more than 30% of QA bandwidth on script maintenance are shipping 2.4x slower than teams that have automated that maintenance layer away. The irony: the more test coverage you write, the more you're paying the tax. What Agentic QA Actually Means (Without the Buzzwords) An agentic QA system doesn't follow a script. It follows a goal. Instead of: Click the login button Enter " testuser@example.com " in the email field Enter "password123" in the password field Assert redirect to /dashboard An agentic QA agent receives: Goal: Verify that a registered user can successfully authenticate and access their dashboard. Context: Auth flow supports email/password and OAuth. Dashboard loads user-specific data. The agent then: Explores the auth flow autonomously Generates test scenarios, including edge cases it infers from the UI Executes tests, reads failures, and adapts to UI changes

Most teams I talk to have "evals." I ask them where the evals run. The answer is almost always the same: a notebook, a dashboard, a spreadsheet someone updates after a bad week. That is not an eval suite. That is a museum. Here is the opinion I will defend for the rest of this post: if your agent's quality checks cannot block a merge, they are decorative. The entire value of an eval is that it stops a regression before it reaches a user. A score you read on Monday about a deploy you shipped Friday is a postmortem, not a gate. We gate code with unit tests. We gate APIs with contract tests. We gate infra with terraform plan . Then we take the single most non-deterministic component in the stack — an LLM agent that can silently change behavior when a vendor ships a new checkpoint — and we let it through on vibes. That asymmetry is the actual bug. Why "run it locally and eyeball it" rots The failure isn't that engineers are lazy. It's that manual eval runs degrade under exactly the conditions where you need them most: Prompt edits look harmless. You reword one line of a system prompt to fix a tone complaint and tank tool-selection accuracy on three unrelated tasks. Nobody re-ran the full set because it's a 40-minute slog. The model moves under you. You pinned gpt-4o , but gpt-4o is not a constant — providers roll checkpoints. Your prompt is identical and your behavior shifted anyway. Dependencies leak. A retrieval index gets re-embedded, a tool's API changes a field name, and the agent starts confidently citing stale data. None of that shows up in a code diff. Every one of these passes code review. Every one of these is caught by a regression suite that runs on the PR. The fix is boring and it works: treat agent behavior like any other thing you'd protect with CI. The two halves you actually need A CI gate for agents needs two things, and people consistently build only one. The first is a scorer : something that takes the agent's output for a fixed set of inputs and ret

Here's a thing that catches almost everyone the first week they run a model locally. You paste a 600-line file into your shiny new local assistant, ask it to find the bug, and it confidently rewrites a function that isn't even in the part it read. No error. No warning. It just... silently dropped most of your file on the floor before the model ever saw it. That's not the model being dumb. That's Ollama doing exactly what it was told. By default it gives every model a context window of 2048 tokens and quietly truncates anything past that. It's one of a handful of small surprises that separate "I installed Ollama" from "I actually understand what's running on my machine." Let's go through the ones that matter: how the thing works under the hood, what hardware you really need, the gotchas, and the honest answer to "should I even bother instead of just calling an API?" What Ollama actually is Ollama gets described as "Docker for LLMs," and that's a decent first approximation. You pull a model, you run it, there's a registry. But it hides what's doing the heavy lifting. Underneath, Ollama is a friendly wrapper around llama.cpp , the C/C++ inference engine that made running these models on consumer hardware practical in the first place. When you type ollama run , you're really booting a llama.cpp runtime with a sane default config and a tidy HTTP server bolted on. The models it runs are in a format called GGUF (GPT-Generated Unified Format). A GGUF file isn't just weights. It's a self-contained package that bundles the tensors, the tokenizer config, the architecture details, and hyperparameters like the trained context length, all in one file. That's why ollama pull llama3.1 gives you something that just works: everything the runtime needs to reconstruct the model is in the box. Ollama itself is young. The project shipped its first release in early July 2023 , and it rode the wave of open-weight models (Llama 2 landed that same month) that suddenly made "run a real LLM on