标签：#Cloud

Part of my AWS learning journey — transitioning from Systems Engineer to Cloud/DevOps. This session covers Amazon EFS — shared storage that multiple EC2 instances can read/write simultaneously. 📋 Topics Covered # Topic Type 1 What is Amazon EFS Concept 2 EFS vs EBS vs S3 Concept + Interview 3 EFS Architecture (Mount Targets, AZs, NFS) Concept + Cert 4 Mounting — What It Means Concept 5 Mount Targets & Security Group Config Concept + Lab 6 EFS Storage Classes Concept + Cert 7 EFS Lifecycle Management & Policies Concept + Cert 8 EFS Performance Modes & Throughput Modes Concept + Cert 9 Benefits of EFS Concept 10 Security & Encryption Concept + Interview 11 EFS Pricing & Cost Optimization Concept 12 Lab: Launch 2 EC2 Instances Lab 13 Lab: Create & Configure EFS Lab 14 Lab: Mount EFS on Both Instances Lab 15 Lab: Demonstrate File Sharing Lab 16 Cleanup Checklist Lab 17 Assignment — Independent Repeat Practice What is Amazon EFS? EFS = Elastic File System — a fully managed, auto-scaling shared file system that multiple EC2 instances can mount and use at the same time , both reading and writing. Analogy: Think of EBS as a personal hard drive — it belongs to one laptop only. EFS is like a shared Google Drive folder that your entire team can open simultaneously from different computers, see the same files, and edit them in real time. Key characteristics: Uses the NFS 4.1 protocol (Network File System — standard Linux file sharing) Serverless — no capacity to provision, no servers to manage Auto-scales — grows from KB to PB automatically, shrinks when files are deleted Highly available — data replicated across multiple AZs in a Region Pay-as-you-go — billed per GB actually stored, no pre-provisioning EFS vs EBS vs S3 — The Big Comparison This is one of the most commonly tested comparisons in AWS certifications. Feature EBS EFS S3 Access Single EC2 instance only Multiple EC2 instances simultaneously Internet / API, unlimited clients Protocol Block storage NFS v4.1 HTTP/REST (

Artificial Intelligence is rapidly evolving from chatbots to autonomous agents capable of...

If your Terraform plans are slow, your blast radius is too wide, or multiple teams are stepping on each other's changes, it's time to split your monolith. See The Problem with Large Terraform States for how to diagnose whether you've reached that point. This guide walks through the process of breaking a monolithic Terraform state into smaller, focused states — and how Snap CD can manage the dependencies between them so you don't have to. The approach 1. Identify natural boundaries Look at your resources and group them by lifecycle and ownership. Common boundaries: Networking — VPCs, subnets, route tables, NAT gateways. Changes rarely, underpins everything. DNS — Zones, records. Usually owned by a platform team. Compute — Kubernetes clusters, VM scale sets, container services. Changes more often, depends on networking. Application infrastructure — Databases, caches, queues, storage accounts. Owned by application teams. Monitoring — Dashboards, alerts, log sinks. Changes frequently, depends on everything but nothing depends on it. A useful test: if two resources would never be changed in the same PR by the same person, they probably belong in different states. 2. Map the dependencies Before you move anything, draw the dependency graph. Which groups produce values that other groups consume? networking dns │ ▲ ▼ │ compute ──────────►─┘ │ ▼ application │ ▼ monitoring The outputs that cross these boundaries are what you'll need to wire up after the split. Typical examples: Networking → Compute: vpc_id , private_subnet_ids Compute → DNS: load_balancer_ip Compute → Application: cluster_endpoint , cluster_ca_certificate Application → Monitoring: database_id , cache_name 3. Use terraform state mv to migrate resources Terraform's state mv command lets you move resources from one state to another without destroying and recreating them. # Initialize the destination state cd modules/networking terraform init # Move resources from the monolith to the new state terraform state mv \

At some point every growing Terraform project hits a wall. Plans that used to finish in seconds now take minutes. Applies feel risky because hundreds of resources share a single blast radius. Colleagues avoid running terraform plan because it hammers cloud APIs hard enough to trigger throttling. The state file itself becomes a liability — large, slow to lock, and one bad write away from corruption. This guide covers the symptoms of an oversized state, the band-aids teams reach for, and the structural fix that actually works. How Terraform state works under the hood Every terraform plan does two things: Refresh — for every resource in state, Terraform calls the provider's API to read the current real-world status. A state with 500 resources means 500+ API calls, often more when resources have nested data sources. Diff — compare the refreshed state against the desired configuration and produce a change set. The refresh phase is the bottleneck. It's sequential per provider (parallelism helps across providers, not within one), and every resource pays the cost whether you changed it or not. Adding ten resources to a 500-resource state doesn't make plans 2% slower — it makes the refresh 2% slower on every single plan, for every engineer, forever. Symptoms of a state that's too large Slow plans The most visible symptom. Plan time scales with resource count because every resource is refreshed on every plan, regardless of whether its configuration changed. The exact speed depends on provider — AWS resources with complex nested structures (IAM policies, security group rules) are slower to refresh than simple ones, and Azure resources that require multiple API calls per refresh are worse still. These aren't edge cases — users regularly report 2,900-resource states taking 20–25 minutes to plan and 1,600-resource states taking 8+ minutes . Even starting Terraform with a large state can take minutes before a single API call is made . There's a long-standing proposal for terraform

Cost is everything. In just about every agentic conversation, the three things that come up for enterprises implementing AI workloads are: Cost Observability Security and as AI continues to throw everyone for a loop when it comes to cost management (e.g - Uber running out of the yearly token budget in one quarter), the ability to shrink resource (like hardware) usage will be crucial moving forward. In this blog post, you will learn how to cust costs by 90% using Agent Susbtrate in comparison to Agents running in k8s Deployments/Pods. The Cost Comparison Agents need a place to run. The "place to run" needs to be a platform that's easily managed, orchestrated, and has the ability to cluster resources. Resources like CPU, GPU, and memory need to be able to scale and expand. Without this, it's a matter of manually managing servers that Agents are running on and clients to interact with said server. That's why so many organizations choose Kubernetes to run Agentic. When running Agents per Pod, however, that can get costly very quick in terms of hardware (GPU, CPU, memory) and performance (can your cluster scale up and down quickly based on resource needs when it comes to Agents coming up and going down per use?). The tests in this blog post show: Always-on Agents running in k8s. Actors running in Workers via Agent Substrate And the comparison will be 50 always-on Pods in comparison to 50 Actors across 5-7 Workers (Pods). If there are 50 Agents running per Pod and 50 Agents running per Worker with 5-10 Actors per Pod, you can already imagine the hardware resource savings that can be accomplished. Right now, the majority of organizations start off with the "one Agent per Pod" approach as that's the fastest way to show value and get up and running. For the future, however, Agents in Actors via Agent Substrate will be how organizations deploy when they care about efficiency, optimization, and managing cost. Let's dive in from a hands-on perspective. Prerequisites To follow a

AWS launched Lambda MicroVMs, a new serverless compute primitive that runs each user session or AI agent in its own Firecracker virtual machine with hardware-level isolation, snapshot-based rapid launch, and state preservation for up to eight hours. Reddit community analysis found the minimum setup costs $3.03/day, roughly 9x Fargate spot pricing. By Steef-Jan Wiggers

Hey everyone 👋 If you run batch jobs, data pipelines, or any kind of AI and ML training on Kubernetes, you have probably hit this wall. Kubernetes is fantastic at deciding WHERE a pod should run, but it is surprisingly clueless about WHEN a job should start. 😅 You submit ten jobs, the cluster fills up, and the rest just sit there as Pending. No real queue, no priority, no fairness between teams. One noisy team can eat all your expensive nodes while everyone else waits. 🥲 That is exactly the gap Kueue fills, and today I want to walk you through it with a pile of hands on examples you can run on any cluster, even your homelab. 🏡 👉 Key takeaway up front: Kueue is a job level manager that holds your jobs in a real queue and only admits them when there is enough quota to actually run them. 🧪 Everything in this guide was tested against Kueue v0.18.1 using the v1beta2 API. I pinned every command and manifest to that version so you do not get surprised by API drift. 📋 What we will cover ✅ Why Kubernetes needs a queue ✅ The building blocks in plain language ✅ Installing Kueue ✅ Setting up quota with a ResourceFlavor, a ClusterQueue, and a LocalQueue ✅ Submitting a Job and watching it get queued and admitted ✅ Priority based admission ✅ Partial admission and elastic jobs ✅ Multiple resource flavors for x86 and arm ✅ Fair sharing between teams with cohorts ✅ Dedicated quota with a shared fallback ✅ Queueing a plain Pod ✅ Why this matters a lot for GPUs and your cloud bill 🤔 Why Kubernetes needs a queue Native Kubernetes scheduling is pod centric. The scheduler looks at one pod at a time and tries to place it. That works great for long running services. Batch workloads are different. They have a beginning and an end, they often need a fixed chunk of capacity, and they compete with other teams for the same nodes. Without a queueing layer you get: ✅ Jobs that fail or stay Pending when resources are tight ✅ No quota governance, so one team can starve the others ✅ No admission prio

The interesting part of OpenAI and Codex on AWS is not that another cloud menu got more model names. That part is useful. Enterprises want strong models. Developers want Codex closer to their infrastructure, data, and deployment machinery. The interesting part is that frontier AI is being pulled into the same boring machinery that already governs everything else companies run: procurement, IAM, billing commitments, region policy, audit logs, support contracts, data boundaries, and security review. That sounds like paperwork. It is also how enterprise software becomes real. model access was the easy problem For a while, AI adoption was framed as an access problem. Can we call the model? Can we get enough rate limit? Can we wire the SDK into our product? Can the coding assistant see enough of the repo to be useful? Those are real questions. They are not the end of the story. The next set is much more familiar to anyone who has operated software inside a company: which account owns this usage, which data can cross the boundary, who can create agents, which region runs inference, how the bill is allocated, and what evidence exists when an incident involves model output. That is the part where the demo becomes a platform. OpenAI on AWS matters because many companies already have that platform muscle in AWS. They have IAM, billing, private networking, audit trails, procurement paths, compliance evidence, cost allocation tags, and teams whose job is to make all of this survivable. Putting a frontier model behind that machinery does not make the hard parts disappear. It makes them legible. bedrock is a procurement surface Amazon Bedrock is usually described as a managed model service, which is true and also undersells the point. For enterprises, Bedrock is a procurement and control surface. If OpenAI models and Codex are available through Bedrock, a company can route adoption through an existing cloud relationship instead of creating a new vendor path for every team that wa

Building a Serverless Resume on AWS I rebuilt my resume as a live AWS application instead of a PDF. It's a static site backed by a real serverless pipeline: a visitor counter that reads and writes to a database, behind an API, deployed through infrastructure as code, with its own CI/CD pipeline pushing updates automatically. I did this through the Cloud Resume Challenge, and this post walks through how it's built and what I actually learned doing it. The Architecture The site itself is static: HTML and CSS, no server rendering anything on the fly. It's hosted in an S3 bucket, sitting behind CloudFront, with Route 53 pointing my custom domain at the CloudFront distribution. Nobody hits S3 directly. Every request goes through CloudFront first, which means consistent load times no matter where in the world someone's loading the page from, and it keeps the actual storage layer shielded from direct traffic. That's the whole story for the page itself. The visitor counter is a separate thing entirely, and it only starts once the page has already finished loading. Once the page renders, JavaScript running in the browser fires off a fetch to API Gateway. API Gateway invokes a Lambda function through a resource policy attached directly to it, that's a different kind of permission than the role Lambda itself uses, one controls who can call Lambda, the other controls what Lambda is allowed to do once it's running. The Lambda function uses its own execution role to read and write a single item in a DynamoDB table: the current visitor count. It increments that number, writes it back, and the result travels back through Lambda, through API Gateway, and into the page, where the count updates on screen. Every piece of this, the S3 bucket, CloudFront, Route 53, the IAM roles, the Lambda function, the DynamoDB table, API Gateway, was provisioned through Terraform. None of it was clicked together in the AWS console. And once it was built, GitHub Actions took over deployment entirely: p

In this episode, Heroku co-founder and Ink & Switch founder Adam Wiggins argues for a 'local-first' architecture that reconciles cloud-based collaboration with the performance and data ownership of local software. He explores the role of CRDTs and version control primitives in non-code domains, and examines how a hybrid AI future might leverage local models for core productivity tasks. By Adam Wiggins

Industry 4.0 data collection sounds simple until you look closely at the factory floor. In theory, the flow is clean: machine → gateway → cloud → dashboard In practice, it is usually less tidy. Factories may have PLCs, CNC machines, sensors, meters, inspection systems, production lines, and older equipment all working together. Some devices use Ethernet. Some still rely on serial interfaces. Some data is useful every second. Some data only matters when a machine changes state, crosses a threshold, or triggers an alarm. This is where an industrial edge gateway becomes useful. A gateway such as Robustel EG5120 can sit between factory equipment and upper-layer systems, helping collect selected machine or PLC data, handle it locally where needed, and forward useful information toward cloud or enterprise platforms. That does not mean the gateway replaces PLCs, SCADA, MES, or the cloud. It simply means factory data often needs a practical middle layer before it becomes useful somewhere else. Factory data is not one clean data stream One thing that gets underestimated in Industry 4.0 projects is how mixed the data sources can be. A PLC may provide equipment status, alarms, and process values. A CNC machine may expose cycle information or maintenance indicators. Sensors and meters may generate temperature, vibration, energy, or environmental data. Inspection systems may produce quality-related events or selected result data. A production line may generate throughput signals, downtime events, or operating states. These are all “factory data,” but they do not behave the same way. A machine fault may need quick attention. An energy reading may only need periodic reporting. A repeated sensor value may not need to be sent upstream every time. A quality inspection output may be useful as metadata, but not every raw file is practical to upload continuously.So the first question is not only: Can we connect this machine? A better question is: What data do we actually need, where sho

Anatomy of the Trust Gap Before we can talk about the solution, we need to talk honestly about how...

Why your Cloudflare Turnstile token works in the browser but 403s from requests You solved the Turnstile widget. You can see the token in the page. You copy it into your script, POST the form from requests, and the server hands you back a 403 — or a JSON body with "success": false. The token clearly worked a second ago in the browser, so what changed? Short answer: a Turnstile token is not a password you can carry around. It's a one-time, short-lived proof bound to a very specific context, and replaying it from a different context is exactly what it's designed to reject. Below is what that context is, how to tell which constraint you're hitting, and the fix for each. The real scenario You're automating a flow on a Cloudflare-protected site. There's a cf-turnstile widget on the form. You get a token one of two ways: you render the page in a real browser (Playwright/Selenium) and read cf-turnstile-response, or you hand the sitekey + page URL to a solving service and get a token back. Either way, you then submit the form with a plain HTTP client requests, httpx, axios) and it fails. The frustrating part: it's intermittent-looking. The reason it feels random is that there are four separate constraints, and you're usually tripping a different one each time. The four things a Turnstile token is bound to 1. It's single-use Once Cloudflare validates a token server-side (the siteverify call your target makes), that token is spent. Submit twice, retry, or test it once by hand, and the second use returns false. You get a fresh one per submission. 2. It has a short TTL Turnstile tokens expire fast — a few minutes. Solve early, do other work, submit later, and the token can be dead on arrival. The widget auto-refreshes in the browser precisely because tokens go stale; a script that grabs the token and sits on it loses that refresh. 3. It's bound to the sitekey and the page URL Multiple widgets. Some pages embed more than one Turnstile (login + newsletter). Solving the wrong site

Amazon has released AWS FinOps Agent in public preview, a managed service that automates several common FinOps workflows. The agent can investigate cost anomalies, correlate spend changes with AWS activity data, and integrate with tools such as Slack and Jira to route findings to resource owners. By Renato Losio

The decision to migrate an on-premises database to the cloud is rarely a simple one. It touches infrastructure, budget, team workflows, and application uptime all at once. Yet for most organizations, moving your on-premises database to the cloud is no longer a question of if but when — and more importantly, how to do it without bringing production systems to their knees. Whether you're running PostgreSQL on bare metal or SQL Server in a company-owned data center, the path to cloud-hosted data has predictable pitfalls and, when navigated carefully, significant payoffs. This guide walks through the full migration process: from pre-migration assessment to post-migration validation, with real configuration examples and the kind of hard-won detail that vendor documentation tends to skip. Why Organizations Make the Move On-premises databases carry hidden costs that compound over time. Hardware refresh cycles, licensing renewals, physical security, power, and cooling — these expenses are easy to underestimate when they're already baked into the operating budget. Cloud databases shift that burden to a consumption model, where you pay for what you use and scale without a procurement cycle. Beyond cost, there's the question of availability. Cloud providers like AWS, Google Cloud, and Azure run managed database services with built-in replication, automated failover, and point-in-time recovery that would take a dedicated DBA weeks to configure manually. A startup can get enterprise-grade durability out of the box. A mid-sized company can stop worrying about what happens when a hard drive fails at 2 a.m. Performance is another draw — though it comes with caveats. Managed cloud databases excel at read-heavy workloads and benefit from proximity to cloud-native application services. Latency-sensitive operations or highly custom storage configurations sometimes perform better on dedicated hardware. Knowing which category your workload falls into is the first real question to answer.

AWS has recently announced the AWS Workload Credentials Provider to automatically deliver and refresh certificates and secrets for applications. The open source tool reduces the need for custom automation, helps prevent outages caused by expired certificates, and works in both AWS and non-AWS environments. un By Renato Losio

Exam Guide: Developer - Associate 🏗️ Domain 2: Security 📘 Task 3: Manage Sensitive Data In Application Code Managing Sensitive Data In Application Code is about keeping secrets out of your code, classifying data properly, and building applications that handle sensitive data safely. You need to know when to use Secrets Manager vs Parameter Store, how to mask PII in API responses and logs, and how to isolate data in multi-tenant applications. The ability to choose the right secret management service, implement data sanitization, and enforce tenant-level data boundaries, is vital. 📘Concepts Data Classification Understand data sensitivity levels and how each should be handled: Classification Examples Handling Requirements PII (Personally Identifiable Information) Name, email, SSN, phone number, address Encrypt at rest and in transit, mask in logs and API responses, restrict access PHI (Protected Health Information) Medical records, insurance IDs, lab results HIPAA compliance, encryption required, audit trail mandatory Financial Credit card numbers, bank accounts, transaction data PCI DSS compliance, tokenization, never store full card numbers Public Marketing content, public API docs No special handling needed 💡 If a scenario mentions compliance or audit trail , think encryption with KMS (for CloudTrail logging) and Secrets Manager (for automatic rotation). If it mentions PII in logs , think data masking and sanitization. Secrets Manager vs SSM Parameter Store Both store configuration and secrets. Feature Secrets Manager SSM Parameter Store Automatic rotation Yes (built-in for RDS, Redshift, DocumentDB) No (you build it yourself with Lambda) Cost $0.40/secret/month + $0.05 per 10,000 API calls Free (Standard tier), $0.05/advanced parameter/month Cross-account access Yes (via resource policy) Yes (advanced parameters only) Max size 64 KB 4 KB (Standard) / 8 KB (Advanced) Versioning Automatic (AWSCURRENT, AWSPREVIOUS labels) Yes (version history) Encryption Always encrypt

Vercel has released Eve, an open-source framework for building, deploying, and operating AI agents in production. The framework uses a filesystem-based project structure to organize agent instructions, tools, skills, subagents, communication channels, and scheduled tasks, enabling developers to define agent behavior while reducing the amount of supporting infrastructure they need to implement. By Daniel Dominguez

Most enterprise RAG failures are not model failures. They are infrastructure failures. The demo works because the PDF is clean, the user is friendly, the permissions are simple, and nobody is measuring drift, latency, access control, source quality, or hallucination risk. Production RAG needs more than a vector database: Data pipelines that know what changed Identity-aware retrieval Source quality scoring Prompt and response guardrails GPU / inference cost controls Observability for retrieval, latency, grounding, and failed answers Human approval for high-risk actions The real question is not: Which LLM should we use? The better question is: What infrastructure makes this AI answer trustworthy enough for business use? Discussion question: If you were building an enterprise RAG system today, which layer would you harden first: data quality, access control, evaluation, observability, or cost governance? Tags: Enterprise AI, RAG, LLMOps, Cloud Architecture, AI Infrastructure, MLOps, Responsible AI, Generative AI.

👋 Hi DEV Community! I'm the creator of Cloud Compass , a newsletter dedicated to making cloud computing easier to understand. If you've ever felt overwhelmed by the constant stream of cloud updates, new services, documentation, and buzzwords, you're definitely not alone. I originally published this as the welcome issue of my newsletter, and I'm sharing it here because I hope it helps developers, students, and anyone starting their cloud journey. ☁️ Welcome to Cloud Compass Cloud news, concepts, and insights without the overwhelm. The cloud is moving fast. Let's make sure you're not left behind. Welcome to Cloud Compass — your guide to staying current with cloud computing while building real cloud knowledge, without feeling overwhelmed. Let's be honest. Keeping up with cloud technology is exhausting. Cloud providers release new services, features, and updates almost every week. Between official documentation, blog posts, YouTube videos, LinkedIn posts, and countless tutorials, it's difficult to know: What actually matters? What should you learn first? Where do you even begin? That's exactly why I started Cloud Compass . This isn't a newsletter written by AI or filled with copied announcements. It's written by someone who genuinely enjoys learning cloud computing and wants to make it easier for everyone else—whether you're: A developer trying to stay current A student entering the cloud world for the first time A professional who wants to understand cloud technology without spending hours reading documentation The goal is simple: Show up regularly with something that's actually useful. "I want Cloud Compass to feel less like reading tech news and more like having a conversation with someone who already read everything and saved you the time." Here's what you can expect 📰 Cloud News Roundup The most important updates from across the cloud industry—not just AWS, Azure, and Google Cloud, but the wider cloud ecosystem. No endless lists of links. No unnecessary hype. Just