标签：#devops

Target: pnpm/pnpm Issue: pnpm/pnpm#12240 PR: pnpm/pnpm#12301 Public branch: https://github.com/scarab-systems/pnpm/tree/fix/deps-status-no-manifest Latest pushed commit: cb68ac1af0dcffbe4fb607a10b0df2046d2490ba This field test targeted a pnpm command-routing failure where pnpm self-upgrade could fail outside a project directory with: ERR_PNPM_NO_PKG_MANIFEST The issue looked simple at the surface: a global/self command should not require a project manifest just because the current working directory is not inside a package. But the repair boundary was more specific than “ignore missing manifest.” The problem was in the dependency-status verification path. When dependency status was unavailable because there was no project manifest, the command could fall through into the auto-install path. That made a self-upgrade/global-style command behave as if it needed a local project manifest. Failure shape The failing behavior was: pnpm self-upgrade run outside a project directory dependency status cannot be established from a project manifest the command path falls into install/manifest expectations result: ERR_PNPM_NO_PKG_MANIFEST That is the wrong ownership boundary. A self-upgrade command should not inherit project-manifest preconditions when there is no local project context. Boundary The boundary here is: global/self command execution versus project dependency-status verification Dependency-status verification can be useful when a command is operating inside a project. But when there is no project manifest and the command is not recursive/all-projects, “dependency status unavailable” should not automatically mean “try to auto-install project dependencies.” There are two different cases: Dependency status is unavailable because there is no project manifest. Dependency status is unexpectedly unavailable even though a root project manifest exists. Those cases should not behave the same. The repair preserves that distinction. What changed The patch updates: exec/commands/src

The biggest mistake teams make when comparing testing tools is treating the feature list like the decision. A tool can support API tests, visual checks, CI, reporting, and integrations, and still be the wrong choice if nobody adopts it, the runs are flaky, or the billing model turns into a budget surprise. Start with the workflow, not the brochure The first question is not “What does this tool support?” It is “Where will this tool sit in our actual delivery flow?” A tool that looks great in a demo can still fail if it does not fit how your team writes tests, reviews failures, shares results, and ships code. If your team lives in GitHub PRs, Slack, and CI pipelines, then the evaluation should center on how quickly a test result shows up where developers already work. If your team has QA specialists, product owners, and client stakeholders, then reporting and handoff matter as much as assertion syntax. This is why feature checklists can mislead. Two tools may both claim browser automation, API coverage, and dashboards, but one might require a heavy framework rewrite while the other can be adopted incrementally. The latter is usually the better tool, even if it looks less impressive on paper. Checklist item one, can people actually use it next week? Adoption beats capability. If a tool needs a long onboarding program, a specialist only one person on the team understands, or a custom setup that no one wants to own, the tool becomes shelfware fast. Look at who will author tests, who will maintain them, and who will interpret failures. A tool that lets QA write quickly but gives developers a painful review experience can still become a bottleneck. A good evaluation asks for the smallest realistic test case. Take one happy-path flow, one negative case, and one flaky UI interaction, then see how far each tool gets you without custom glue. That is usually more useful than a vendor demo with polished sample scripts. Checklist item two, what happens when the tests get messy? E

After containerizing an application with Docker, the next logical step is deploying it on Kubernetes. Kubernetes helps automate application deployment, scaling, networking, and management of containerized workloads. Instead of manually running containers, Kubernetes ensures your application remains available and can easily scale when needed. In this guide, we'll deploy a Docker image of a Node.js application on Kubernetes using a Deployment and a Service. Prerequisites Before starting, make sure you have: Docker installed Kubernetes cluster running (Docker Desktop Kubernetes, Minikube, Kind, EKS, etc.) kubectl configured A Docker image pushed to Docker Hub In my case, the image was: madhavnaks/node-app:latest Why Kubernetes? Running a container using Docker is straightforward: docker run -p 3000:3000 madhavnaks/node-app:latest However, in production environments we need much more than simply running a container. Kubernetes provides: High availability Self-healing containers Load balancing Service discovery Horizontal scaling Rolling updates This makes it the industry standard for container orchestration. Understanding the Kubernetes Architecture for This Deployment For this deployment, we'll use two Kubernetes resources: Deployment A Deployment is responsible for: Creating Pods Maintaining desired replica count Recreating failed Pods automatically Managing updates and rollbacks Service A Service provides a stable network endpoint for Pods. Since Pod IPs change frequently, Services allow applications and users to communicate reliably with Pods. Deployment and Service Manifest Create a file named: app.yaml Add the following configuration: apiVersion : apps/v1 kind : Deployment metadata : name : node-app spec : replicas : 2 selector : matchLabels : app : node-app template : metadata : labels : app : node-app spec : containers : - name : node-app image : madhavnaks/node-app:latest ports : - containerPort : 3000 --- apiVersion : v1 kind : Service metadata : name : node-a

When you expose inventory or checkout endpoints to public-facing front-ends or third-party webhooks, safeguarding those APIs from brute-force scripts, scraping bots, and inventory hoarding algorithms becomes a critical requirement. Without defensive rate limiting, a single coordinated script can easily overwhelm your database connections. The Problem with Simple Counter Resets A common mistake when setting up basic API protection is using a rigid "Fixed Window" counter (e.g., allowing 100 requests per minute, resetting exactly at the turn of the clock). This creates a massive flaw where a developer can flood your server with 100 requests at 11:59:59 and another 100 requests at 12:00:01, effectively doubling your acceptable burst traffic and causing severe performance dips. To handle uneven burst traffic safely without crashing your database, the standard approach is implementing a token bucket algorithm. The Token Bucket Pattern The token bucket algorithm maintains a centralized bucket that holds a maximum capacity of tokens. Tokens are added back to the bucket at a constant, predictable rate over time. Each incoming API request consumes exactly one token. If the bucket is completely empty, the request is instantly rejected with a 429 Too Many Requests status code, protecting your core server threads. javascript // Quick Redis-based token bucket rate limiter concept async function isRateLimited(userId) { const key = `rate:${userId}`; const now = Date.now(); // Use a Redis multi-exec transaction to atomically check and update tokens const [tokens, lastRefill] = await redis.hmget(key, 'tokens', 'lastRefill'); // Calculate token replenishment based on time elapsed... // Return true if tokens <= 0, otherwise decrement tokens and update timestamp }

In Q1 2026, OpenAI and Anthropic moved enterprise customers from flat-rate plans to token-based billing. The change looks administrative, but it had a direct consequence for engineering teams: the real cost of AI became visible for the first time. The market's reaction over the following two months was enough to reopen a question many considered settled: does AI actually deliver measurable ROI? What happened when the bill arrived The most documented case is Uber. The company had encouraged all employees to use agentic tools as much as possible and even ranked AI usage internally on leaderboards. The result: the entire annual budget was consumed in four months. The response was a $1,500/month cap per employee per agentic coding tool (Claude Code, Cursor, and similar). At Brex, engineers were limited to $500/week in tokens; employees outside engineering received a $5/week cap. T-Mobile temporarily capped usage at $2,000/month per user with plans to migrate to a tiered system. One unnamed company, according to Ed Zitron in "AI Is Slowing Down" (June 2026), spent $500 million on Anthropic models in a single month due to absent spend controls. These are not isolated cases. A KPMG survey reported by the Wall Street Journal in June 2026 found that only 26% of companies have a comprehensive view of their AI costs; 50% have partial visibility; and 22% only find out what they owe after the bill arrives. Steve Chase, KPMG's global head of AI, told the Journal: "It's a new resource that needs to be managed that didn't exist quite that way, and we're seeing exponential growth." The structural problem behind the spending caps The spending caps are a symptom. The root cause, as Zitron details in the same article, is that the economics of generative AI require numbers that currently seem out of reach. Anthropics has made over $330 billion in compute commitments with Google, Amazon, and Microsoft, plus another $45 billion with CoreWeave and SpaceX. To cover those commitments, it nee

73 cryptographically signed npm packages from Microsoft were compromised last week with advanced credential-stealing malware that fires the moment a developer opens one in an AI coding agent. Claude Code, Gemini CLI, Cursor, VS Code — all trigger it. It's the second supply-chain attack in two months against the same Microsoft account. "The genius of this Miasma worm lies in how it adhered to legitimate workflows. It does not exploit any software vulnerability in GitHub or npm. Instead, it exploits the underlying trust model of the modern engineering ecosystem." — Cloudsmith What actually changed 73 official Microsoft npm packages were poisoned with the Miasma worm — a clone of TeamPCP's open-sourced Mini Shai-Hulud toolkit Malware executes automatically when any of the 73 packages are opened inside an AI coding agent The payload (28 KB) harvests credentials from AWS, Azure, GCP, Kubernetes, 90+ dev tool configs, and password managers , then spreads laterally through cloud infrastructure Attack vector: stolen Microsoft publisher credentials → bypasses the build pipeline entirely → malicious build published with valid SLSA provenance attestation Each infection gets a uniquely encrypted payload — meaning hash-based IOCs are useless for detection GitHub initially flagged packages as "terms of service violations" rather than malware; Microsoft only acknowledged possible malicious content 48 hours later The same Microsoft account was compromised in May 2026 (durabletask Python SDK on PyPI, 400k downloads/month) — and apparently wasn't fully remediated Why this one stings The supply-chain attack playbook has levelled up. SLSA provenance — the framework designed to give you cryptographic confidence that a package came from a legitimate build — was used against you here. Attackers stole a legitimate Microsoft OIDC token, published a malicious build with real provenance, and conventional scanners waved it through as a routine trusted update. The AI agent angle makes it worse.

Visibility Comes Before Optimization in IT Operations is a practical operating principle, not a slogan. The useful version of analytics, automation, and software operations is usually quieter than the marketing version. It is less about collecting everything or automating everything, and more about making the work easier to understand, review, and improve. The practical problem Teams often try to optimize before they can see the system clearly. That creates confident changes based on partial evidence, especially in infrastructure and telecom-adjacent workflows where signals are distributed. This is where many teams lose clarity. They have tools, charts, workflows, and activity, but the connection between evidence and decision is weak. When that connection is weak, software work becomes harder to evaluate. Teams still make decisions, but they rely more on memory, opinion, or urgency than on a reviewable operating picture. A smaller operating model Start with visibility: what is running, which state changed, where the weak signal appeared, and which workflow was affected. Then connect that signal to a decision or operational review. The important detail is restraint. A useful system does not need to track every possible action or automate every possible step. It needs to preserve the signals that help operators understand the situation and act with more confidence. That usually means naming the workflow, keeping the outcome visible, preserving enough context to explain the signal, and making uncertainty explicit instead of hiding it behind a polished interface. What to review Useful analytics separates normal activity from operational risk. It should make the next investigation smaller, not create another dashboard that requires interpretation from scratch. A reviewable system is easier to trust because it can explain its own state. It shows what happened, what changed, what remains uncertain, and which decision should move next. For WebmasterID, this is the practical

The Problem: The Chaos of Giant AI Code Diffs Autonomous coding tools can spin up full implementations, run scripts and commit hundreds of lines of code in seconds. But if you have managed a team of developers using them, or tried to build a complex feature solo, you have likely run into giant code diffs . A single vague prompt transforms into a massive, multi-file PR that takes a human tech lead hours to confidently review. Features get built but the step-by-step product rationale and architectural decisions are often lost inside ephemeral chat histories. The solution is enforcing strict workflow guardrails. I tried all major spec-driven development (SDD) workflows and what I found is they focus 90% on product shape and much less on the actual implementation. This is also the case of get-shit-done which I love for its pragmatism, low ceremony-driven yet solid at context and flexibility. But I needed something more specialized. Introducing Get Tasks Done I built Get Tasks Done from get-shit-done to provide a lightweight, deterministic state machine layer for AI-assisted development. It bridges the gap between high-level human intent and execution AI agents by turning specifications into granular execution tasks using leveraging a GitHub-native integration . Instead of a fluid, unpredictable implementation step, GTD structures development into explicit, auditable stages: Product Intent ➔ Markdown Specs ➔ Granular GitHub Issues ➔ Atomic PRs The Architecture: Guardrails for the Agentic Layer The system coordinates across five distinct layers: Planning Artifacts Local markdown planning templates enforce small, highly contained prompt boundaries. By keeping information tightly localized, context drift drops significantly. I extended it with a thorough task decomposition gate that ensures planning tasks are enough atomic to avoid drift (and even executed by cheaper models). Runtime Commands & State Deterministic tools manage how the agent reads the state machine, standard

TL;DR: Create React App builds bake environment variables at build time. ConfigMaps let you inject runtime configs into your container. Here’s how to bridge them so the same Docker image works across dev, staging, and production. The Problem You’ve built a React app with Create React App (CRA), Vite, or Next.js. You use .env files: js // api.js const API_URL = process.env.REACT_APP_API_URL; You build your Docker image: dockerfile FROM node:18 AS builder COPY . . RUN npm run build # REACT_APP_API_URL gets baked here FROM nginx:alpine COPY --from=builder /build /usr/share/nginx/html Then you deploy to Kubernetes. But now you want different API URLs for staging vs production. You could rebuild the image for each environment (bad – slow, wasteful). Or you could use a ConfigMap to inject values at runtime. ConfigMap to the Rescue A ConfigMap stores key-value pairs. Kubernetes can mount it as a file inside your pod. But React runs in the browser, not in the container’s filesystem. So how does the browser read a file from a ConfigMap? Simple: You serve a dynamic env-config.js file from your web server. Step-by-Step Solution Create a ConfigMap with your environment variables yaml # configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: react-env-config data: env-config.js: | window.__env = { REACT_APP_API_URL: " https://api.production.com ", REACT_APP_FEATURE_FLAG: "true" }; Apply it: bash kubectl apply -f configmap.yaml Modify your React app to read from window.__env Instead of reading process.env directly, use a runtime config: js // config.js export function getEnvVar(name) { // Runtime injection from window. env (provided by ConfigMap) if (window. env && window. env[name] !== undefined) { return window. env[name]; } // Fallback to build-time env vars (for local dev) return process.env[name]; } Use it in your components: js // api.js import { getEnvVar } from './config'; const API_URL = getEnvVar('REACT_APP_API_URL'); Serve the ConfigMap file via your web server U

I came to programming late — I didn't get into this world until I was past 35, and I'm 62 now, still writing code every day. This is a "build in public" post about a tool I just finished, and I'd genuinely love your feedback. The itch For years I watched infrastructure teams keep their AWS inventory in spreadsheets. It always worked — right up until it didn't. Nobody had time to keep it current, and every single one eventually drifted away from reality. Middleware EOL was the same story: a hand-maintained list, no alerts, no dashboard, quietly going stale. One day I asked the obvious question: we have tfstate, we have boto3 — why are we still doing this by hand? What I built SyncVey is a self-hosted web app that: Inventories your AWS resources into a System → Environment → Asset ledger (EC2, ECS, Lambda, RDS, S3, ALB, VPC, EBS), scanned live via boto3/AssumeRole Detects attribute-level drift between your tfstate and live AWS — including resources someone built by hand in the console that terraform plan never sees Tracks the app/middleware layer per environment and flags end-of-life runtimes The drift piece is the part I care about most. terraform plan only knows about resources Terraform already manages. The thing that actually bites teams is the resource someone spun up by hand in the console — plan is blind to it. SyncVey diffs your tfstate against the live AWS state, so those show up too. The stack (and why) Django + htmx + Postgres — server-rendered, no SPA, no Node build step MIT-licensed, no SaaS, no telemetry One docker compose up and your data stays inside your own infrastructure git clone https://github.com/MR-TABATA/SyncVey cd SyncVey docker compose up I deliberately leaned on htmx because, for a tool someone has to deploy and maintain themselves, "no frontend toolchain" matters more than a fancy client. I'd love your honest take It's AWS-only for now and very much a solo project, so I'm sure there are rough edges. I'm not an AWS specialist — I deliberatel

Cron's biggest problem isn't scheduling — it's silence. A cron job can fail every night for a month, and unless you're manually checking logs on the server, you won't know. No alert, no dashboard, no audit trail. Just a backup that doesn't exist when you need it, or a data sync that quietly stopped three weeks ago. Monitoring fixes this. But "cron job monitoring" means different things depending on the tool. Some watch for missing heartbeats. Some track full execution history. Some just page you when something breaks. This article compares six approaches — from writing your own monitoring scripts to using a fully managed scheduler with built-in observability — so you can pick the right one for your workload. Heartbeat Monitoring vs. Execution Monitoring Before comparing tools, understand the two fundamentally different approaches. Heartbeat monitoring (dead man's switch) is passive. Your cron job pings a monitoring URL after each run. If the ping doesn't arrive on schedule, you get an alert. This tells you whether a job ran — but not what happened . If the job runs but returns bad data, the ping still fires and the monitor stays green. Execution monitoring is active. The scheduler fires the job, captures the response, records the outcome, and alerts on failure. You get the full picture: status code, response body, duration, retry count, and a timeline of every execution. When to use each: Heartbeat monitoring makes sense when you're stuck with system cron. Execution monitoring makes sense when you're choosing a scheduler — you get monitoring, retries, and logging as part of the platform. Comparison at a Glance Tool Type Alerts Execution Logs Retries Free Tier DIY scripts Custom ⚠️ Whatever you build ⚠️ Whatever you build ⚠️ Whatever you build ✅ Free (your time) Healthchecks.io Heartbeat ✅ Email, Slack, webhooks ❌ No ❌ No ✅ 20 checks Cronitor Heartbeat + telemetry ✅ Email, Slack, PagerDuty ⚠️ Basic (duration, exit code) ❌ No ⚠️ 5 monitors Better Stack Uptime + heartb

IBM and HashiCorp have announced new LDAP secrets management capabilities in IBM Vault Enterprise 2.0, introducing a redesigned architecture to manage LDAP credentials, support password rotation, and automate the identity lifecycle. By Craig Risi

Cloud teams waste between 40% and 60% of their infrastructure budget on a false choice: committing to reserved capacity they won't fully use or chasing spot instance savings they can't. Introduction: The Cloud Cost Optimization Dilemma Cloud teams waste between 40% and 60% of their infrastructure budget on a false choice: committing to reserved capacity they won't fully use or chasing spot instance savings they can't operationalize. The decision between commitment discounts and spot instances is not a preference. It is a calculation with three variables: workload predictability, failure tolerance, and the operational cost of managing interruptions. Commitment discounts lock you into capacity for one or three years. You pay upfront or monthly for compute resources whether you use them or not. The mechanism is simple: cloud providers offer 30% to 72% discounts because they can forecast their own capacity planning when customers commit. You save money when your actual usage matches your commitment. You lose money when usage drops below the committed level because you still pay for idle capacity. Spot instances offer 70% to 90% discounts by selling unused cloud capacity at auction prices. The provider can reclaim these instances with 30 seconds to 2 minutes of notice. You save money when your workload can tolerate interruptions and you build automation to handle instance termination. You lose money when interruptions cause failed jobs that must restart from scratch, consuming more compute time than the discount saved. Most engineering teams pick one strategy and apply it everywhere. This creates two failure modes. Teams that over-commit pay for capacity during low-traffic periods. Teams that over-rely on spot instances spend engineering time rebuilding checkpoint systems and retry logic that costs more than the discount delivers. The correct approach is workload-specific. Measure your actual usage patterns for 30 days. Calculate the cost of interruption handling. Then a

Introduction Modern cloud-native systems generate an enormous amount of telemetry data every second. Applications, containers, Kubernetes clusters, APIs, databases, and infrastructure components continuously emit metrics, logs, and traces to help engineering teams understand system behavior and troubleshoot issues. While observability has become essential for operating distributed systems reliably, it has also introduced a new challenge: managing the scale, cost, and quality of telemetry. OpenTelemetry (OTel) has emerged as the industry standard for collecting and processing observability data. It provides a vendor-neutral framework for instrumenting applications and exporting telemetry to different observability backends. However, simply adopting OpenTelemetry is not enough. Without proper optimization strategies, organizations often face excessive telemetry ingestion costs, noisy dashboards, high-cardinality metrics, trace overload, and inefficient debugging workflows. This article explores practical approaches for optimizing observability using OpenTelemetry. It focuses on metrics, logs, and traces individually while also discussing broader optimization strategies across the telemetry pipeline. Understanding the OpenTelemetry observability pipeline OpenTelemetry provides a unified framework for generating, collecting, processing, and exporting telemetry data. At its core, the OTel ecosystem consists of SDKs, instrumentation libraries, collectors, processors, and exporters. Applications generate telemetry using OpenTelemetry SDKs or auto-instrumentation agents. This telemetry is then sent to the OpenTelemetry Collector, which acts as a centralized telemetry processing layer. The collector can receive telemetry from multiple sources, enrich it with metadata, apply filtering or sampling, and export it to one or more observability backends. The observability pipeline typically follows this flow: Application → OTel SDK → OTel Collector → Observability Backend The Open

In the hyper-competitive landscape of modern B2B outbound sales, speed-to-lead and outreach capacity are the ultimate drivers of pipeline volume . Yet, traditional Sales Development Representative (SDR) teams face a exhausting bottleneck: reaches and qualifications are limited by human bandwidth . A typical outbound SDR spends up to 80% of their day dialing numbers, navigating IVR phone trees, hitting voicemail, and dealing with incorrect contact records. When an inbound lead submits a form requesting a product demo, the average company takes 42 minutes to respond. By that time, prospect engagement has cooled by over 400%. To shatter this operational limit, modern revenue operations (RevOps) teams are transitioning from rigid auto-dialers and static voice bots to autonomous voice AI sales agents . By pairing the hyper-realistic conversational engine of ElevenLabs with the visual orchestration power of n8n , you can deploy a scalable, context-aware calling agent that handles inbound qualification and outbound follow-up calls in real-time. This technical blueprint provides an end-to-end guide to designing, securing, and deploying a production-grade Voice AI Sales Agent using ElevenLabs Conversational AI and n8n . We will cover how to manage conversation state, execute live database tool calls, secure webhook communication, route calls dynamically, and configure infrastructure to achieve sub-second response latency . The Architecture of an Enterprise Voice Agent Building a conversational voice agent requires a multi-layered system that operates in near real-time. When a human speaks over a telephony network, their voice must be digitized, transcribed, processed by a large language model (LLM), synthesized back into audio, and sent back down the line—all within a fraction of a second. To ensure stability, scalability, and absolute separation of concerns, our architecture decouples the telephony and voice generation layer from the logic and database integration layer . [

Microsoft used their Build 2026 event to announce new functionality for Microsoft Foundry. Citing Foundry as "the place where AI agents move from experiments to production systems," in a blog post, Nick Brady writes that the release brings “runtime, tools, memory, grounding, models, observability, and governance” that developers need for production agents, rather than just new model endpoints. By Matt Saunders

Something I keep noticing across teams and orgs that are actually getting value from AI versus the ones that aren't. The difference rarely comes down to the model or the algorithm. Most organizations are already drowning in data. Logs, metrics, alerts, reports, dashboards, tickets. The information exists. The bottleneck is what happens after the data shows up. How long does it take to interpret what the signal means? Who decides what to prioritize when three things need attention at once? How fast can the right people coordinate a response once a decision is made? That's where AI actually earns its keep. Not by replacing the human in the loop but by compressing the time between something happening and someone doing something useful about it. Signal to understanding to action. That's the chain that matters. Think about it in terms you deal with every day. A vulnerability gets disclosed. The CVE exists, the advisory is public, your scanner picked it up. None of that is the bottleneck. The bottleneck is figuring out which of your services are affected, who owns them, how bad the exposure actually is in your specific context, and getting a patch scheduled before someone exploits it. AI that helps you answer those questions in minutes instead of days is genuinely valuable. AI that and adds another dashboard to look at isn't. This applies across the board. Incident response, infrastructure management, risk assessment, customer systems, operational workflows. The teams getting real value aren't the ones with the fanciest models. They're the ones who figured out where their decision bottlenecks actually are and pointed AI at those specific gaps. The strategic advantage is rarely in the algorithm. It's in organizational responsiveness. How fast can you go from "something happened" to "we're handling it". The AI is just the thing that compresses that timeline. Where's the biggest decision bottleneck in your current workflow?

Terraform can provision an AWS EC2 instance and set up a Python virtual environment in a single, reproducible run — the whole workflow is declarative and version‑controlled. 📑 Table of Contents 💻 Terraform — How to Provision an EC2 Instance 🔧 AWS Provider — Configuring Credentials 🐍 Python Environment — Setting up a Virtualenv on the Instance 📦 Installing Python and venv 📦 Activating and Using the Environment 📦 User Data — Automating Installation with Terraform 🟩 Final Thoughts ❓ Frequently Asked Questions How do I store the Terraform state securely? Can I use a different Linux distribution for the EC2 instance? Is it possible to attach an Elastic IP to the instance? 📚 References & Further Reading 💻 Terraform — How to Provision an EC2 Instance A Terraform configuration file describes the desired state of AWS resources; applying it makes the real cloud match that state. First, install Terraform (version 1.5.0 or newer). The binary is a single executable, so the operating system loads it directly into memory and the process performs HTTP requests to AWS endpoints. $ terraform version Terraform v1.5.0 on linux_amd64 + provider registry.terraform.io/hashicorp/aws v5.12.0 Next, create a main.tf that declares an aws_instance resource. The provider block authenticates with AWS using either environment variables or a shared credentials file. # main.tf terraform { required_version = ">= 1.5.0" required_providers { aws = { source = "hashicorp/aws" version = "~> 5.12" } } } provider "aws" { region = "us-east-1" } resource "aws_instance" "app_server" { ami = "ami-0c02fb55956c7d316" # Amazon Linux 2 instance_type = "t3.micro" # User data will be defined later user_data = data.template_file.init.rendered tags = { Name = "terraform-ec2-python" } } Running terraform init contacts the provider registry, downloads the provider plugin, and stores it under .terraform . The generated .terraform.lock.hcl file records exact plugin checksums, guaranteeing that subsequent runs use the same

A pattern keeps repeating itself in AI projects. The model works. The demo works. The proof of...

We wired our LLM eval suite into Datadog over about four months. Most of the panels we built got deleted. These are the five that stayed, and the metrics that feed them. TL;DR: We run an LLM-as-judge eval suite on every PR that touches a prompt, and we ship the results to Datadog as custom metrics. The dashboard started with fourteen panels. We kept five. The one that catches the most real regressions is per-criterion pass-rate split out by judge criterion, not the single rolled-up pass-rate number, because an aggregate of 91 percent hid the fact that one criterion had dropped from 0.95 to 0.62. Below are the metrics we emit, the Python that submits them, the monitor config we alert on, and the panels we tried and dropped. Some context on the setup so the rest makes sense. We are a Series-C dev-tool startup. We have a handful of prompts in production that do real work (classification, extraction, a summarization step in an agent loop). Each one has an eval set of tagged examples, somewhere between 80 and 400 per prompt. The judge is a separate model call that scores each output against a rubric. We run the suite in GitHub Actions. The eval job emits metrics to Datadog at the end of every run. Backend service health was already in Datadog, so putting eval data next to it meant one place to look during an incident instead of two. 1. Emit per-criterion pass-rate, not just the rolled-up number This is the one that earns its place. Our judge scores each output against multiple criteria. For the extraction prompt it is four: correct fields, no hallucinated fields, format valid, no refusal. Early on we only emitted one number, prompt_eval.pass_rate, the fraction of examples that passed every criterion. That number is fine for a smoke test and useless for debugging. The problem showed up on a prompt change that looked clean. Overall pass-rate went from 0.93 to 0.91. Two points. Nobody would block a PR on two points. But underneath, the "no hallucinated fields" criterion had