标签：#AWS

AWS describes a specification-driven approach for composing flexible data workflows by separating intent from processing logic. Architecture uses declarative specifications, reusable processing capabilities, and validation before execution. AWS reports that the approach can reduce dataset onboarding from weeks to days while supporting traceability, versioning, data classification, and governance. By Leela Kumili

In addition to sending billions of dollars to states, Meta will make substantive changes to its platforms as part of a landmark settlement.

Building a Serverless Weather Pipeline on AWS: A Step-by-Step Walkthrough This is a build log for someone who's used AWS a bit — deployed a Lambda from the console, poked around S3 — but hasn't touched CDK, Step Functions, EventBridge Scheduler, or GitHub's OIDC setup before. I'll explain each concept the first time it comes up, and show the actual code behind every piece, roughly in the order I built it. Here's what it ends up doing: every 10 minutes, EventBridge Scheduler kicks off a Step Functions workflow that pulls current weather for five cities in parallel from a free public API, reshapes the results into JSON Lines, drops them into S3 in a partitioned layout, and makes them queryable in Athena with plain SQL. No crawler, and no AWS credentials sitting anywhere in the GitHub repo that deploys it. kasukur / serverless-weather-pipeline AWS Serverless Weather Pipeline Serverless Weather Data Pipeline A small but complete serverless data pipeline on AWS walkthrough: EventBridge Scheduler → Step Functions → Lambda → S3 → Glue/Athena , deployed by GitHub Actions with no AWS access keys stored anywhere (authentication is via GitHub's OIDC provider). flowchart TD A["EventBridge Scheduler (every 10 min)"] --> B["Step Functions state machine"] B --> C["PrepareCities (Pass)"] C --> D["ForEachCity (Map, concurrency 4)"] D --> E["FetchWeather (Lambda -> Open-Meteo public API)"] E -.-> F["retries transient errors (up to 2 attempts)"] E -.-> G["FetchFailed (Pass): per-city failure absorbed here, other cities continue"] E --> H["TransformWeatherData (Lambda, pure function, no AWS calls)"] H -.-> I["splits successes vs failures"] H -.-> J["builds JSON-Lines body + partitioned S3 key"] H --> K["LoadToS3 (Lambda, writes to S3 via boto3)"] K --> L["S3 (processed/dt=YYYY-MM-DD/hour=HH/*.jsonl)"] L --> M["Glue Data Catalog table (partition projection -- no crawler)"] M --> N["Athena (query with plain SQL)"] D -.-> … View on GitHub Table of Contents What we're building, and why eac

LLMs can now call tools, but turning their output into a trustworthy event stream is still a puzzle. We wire Claude’s function‑calling to an SNS FIFO topic, giving you ordered, deduplicated notifications that downstream Lambda functions can consume with zero‑loss guarantees. Why SNS FIFO Is a Good Fit for LLM‑Generated Events When an LLM decides to “publishAlert”, you usually want the alert to be processed exactly in the order it was generated . Imagine a fire‑alarm system that first warns about a smoke detector, then follows up with a sprinkler‑activation command. If those two messages arrive swapped, you could end up turning on sprinklers before the fire is even confirmed. FIFO stands for First‑In‑First‑Out . An SNS FIFO topic guarantees that messages sharing the same MessageGroupId are delivered to subscribers in the exact order they were published. This is different from the default “standard” SNS topics, which deliver messages quickly but without ordering guarantees. In plain English: SNS FIFO is like a single‑lane road with a traffic light that lets cars (messages) pass one after another, never overtaking. Key terms (first use) Term Meaning Function calling A feature where the LLM can invoke a pre‑defined tool (a piece of code) instead of just returning text. FIFO topic An SNS topic that preserves the order of messages that belong to the same logical group. MessageGroupId An identifier that tells SNS which messages belong together for ordering. MessageDeduplicationId A token that prevents the same message from being delivered twice within a 5‑minute window. Lambda A serverless compute service that runs code in response to events (like an SNS message). Because the LLM can generate many alerts rapidly, using a FIFO topic means you can treat the AI as a deterministic producer rather than a chaotic chatterbox. The downstream Lambda sees the alerts in the same sequence the model emitted them. Setting Up Claude’s Function Calls to Publish to SNS Before you can send

Have you ever been coding and then gotten into that flow state? You know where hours pass by , and it feels to you it's only ben a few minutes? Me too. One thing that really helps me get into that state is music. So I create my own music Lo-Fi server called compile and chill. As a part of this project, I created three radio stations. Each station can generate a 16:9 scene with Amazon Bedrock , compose an instrumental loop with ElevenLabs, and turn an illustration into a six-second video through OpenRouter. Generated files live in private Amazon S3 storage and return to the browser through the Nuxt server. I also added a Stream Deck API interface! This tutorial shows how to build this radio station from start to finish. The complete source code is available in the Compile & Chill repository . Watch the full video on YouTube . Prerequisites You need the following tools for the complete build: Node.js 22.19 or newer. The locked Nuxt 4.5.2 release requires Node 22.19+, 24.11+, or 26+. npm 10 or newer. An AWS account and a configured AWS Command Line Interface (AWS CLI) profile. The AWS Serverless Application Model (AWS SAM) CLI for the private storage stack. Access to Stability AI Stable Image Ultra through Amazon Bedrock in us-west-2 . An ElevenLabs API key for music generation. An OpenRouter API key for animated scenes. The provider credentials are optional. Without them, the UI, bundled scene, station switching, player, and Focus Block timer still work. The identity running the app needs bedrock:InvokeModel plus bucket-scoped permissions for s3:GetObject , s3:PutObject , s3:DeleteObject , s3:DeleteObjectVersion , and s3:ListBucketVersions . Use a role or profile scoped to the station bucket rather than an administrator identity. For this project I included infrastructure as code with SAM to help setup the AWS parts. It's also included in the repo. Steps 1. Run the station without credentials Pull down the repo and get started! git clone https://github.com/ErikCH/comp

AWS Professional Services just published production data on a multi-agent system that compresses infrastructure-as-code development from weeks to minutes. The system chains four specialized agents (discovery, IaC generation, governance, operations) using Amazon Bedrock AgentCore primitives. This is not a demo. It is a deployed enterprise migration workflow with real customer proof points. The interesting part is how AWS routes tasks between agents without creating circular dependencies, and how they instrument handoffs when a single migration spans four agents with different failure modes. Architecture: Four Agents, One Workflow The system decomposes cloud migration into four agent roles: Discovery Agent : Scans existing infrastructure, builds dependency graphs, identifies migration candidates IaC Generation Agent : Converts discovered resources into Terraform or CloudFormation templates Portfolio Governance Agent : Validates generated IaC against organizational policies, cost budgets, security baselines Post-Migration Operations Agent : Monitors deployed resources, handles drift detection, executes remediation Each agent is a Bedrock Agent with tool access scoped to its domain. The discovery agent cannot deploy infrastructure. The IaC generation agent cannot read production credentials. The governance agent has read-only access to policy repositories. AgentCore orchestrates handoffs using a state machine pattern. When the discovery agent completes a scan, it writes structured output (JSON schema with resource metadata, dependencies, and migration readiness scores) to an S3 bucket. The IaC generation agent subscribes to that bucket via EventBridge and begins template generation only after the discovery agent marks the scan as complete. State Management and Handoff Primitives The key orchestration primitive is a migration manifest stored in DynamoDB. Each migration project gets a manifest with these fields: project_id : Unique identifier for the migration current_sta

How We Cut AWS Staging Costs by 87% With EventBridge Scheduler No code changes. No Lambda functions. No complex scripts. Just 4 schedulers and a realization that nobody uses staging at 3am. Here's a question every engineering team should ask themselves: "When was the last time someone actually used our staging environment at 2am?" For us? Never. Not once. Yet we were paying for it — EC2 running, ECS Fargate tasks spinning, compute burning money — every single hour of every single day, including weekends, holidays, and the 21 hours per day when nobody on our team was even awake. That's the hidden tax of staging environments. And most teams never fix it because the solution feels complicated. It isn't. This is how we cut our staging compute costs by 87.5% — using AWS EventBridge Scheduler, zero Lambda functions, and zero lines of application code. The Problem: Staging Was Running 24/7 For No Reason Our staging environment had two resources running around the clock: EC2 instance — our staging app server ECS Fargate service — our backend API container Our team actively uses staging for roughly 3 hours a day . That's it. The math was embarrassing: Running: 24 hours/day Used: 3 hours/day Wasted: 21 hours/day = 87.5% of compute going nowhere Monthly cost breakdown: EC2 + ECS Fargate (24x7): ~$19.18/month EC2 + ECS Fargate (3hr/day): ~$2.40/month Monthly saving: $16.78 Yearly saving: $201.35 Reduction: 87.5% $201/year saved on staging compute alone — with 45 minutes of setup and zero application code changes. Multiply that across dev environments, QA clusters, review apps, and load test environments. The savings compound fast. The Solution: AWS EventBridge Scheduler Most engineers reach for Lambda when they need to automate AWS tasks on a schedule. That works — but it means writing code, managing runtimes, setting up CloudWatch Logs, and maintaining a function forever. EventBridge Scheduler is the better tool here. It lets you call any AWS SDK action directly on a cron sche

The AWS Community Builders program opens applications once a year, typically in early January, and closes within about two weeks. That's a narrow window. If you're serious about the 2027 cycle, you have roughly four months from now to build the contribution track record that gets you selected. I wrote my personal story about getting into the program from Cameroon. This post is different. It's a practical, no-fluff guide covering how the program works, what the application actually asks, what reviewers evaluate (based on patterns from people who've been accepted and rejected), and how to prepare starting today. What the AWS Community Builders Program Actually Is AWS Community Builders is a global program that recognizes people who share AWS knowledge publicly. Not AWS employees. Not necessarily experts. Engineers, students, content creators, and community organizers who consistently write, build, speak, or contribute to open source around AWS services. The key word is consistently . This is not a certification you study for. It's recognition of a public track record of helping others learn and build on AWS. The program sits below the AWS Heroes program in AWS's community ladder. Heroes are veterans with years of visible impact. Community Builders is the accessible entry point, and for most engineers reading this, the realistic first target. It's free to apply. Membership runs in yearly cycles with renewal based on continued activity. The Categories (Pick One That Matches Your Work) When you apply, you select a technology category. For 2026 the categories were: AI Engineering : Building generative AI applications with Amazon Bedrock, prompt engineering, RAG, fine-tuning, agents Cloud Operations : Observability and configuration (CloudWatch, Systems Manager, Config, Service Catalog) Containers : ECS, EKS, Fargate, App Runner Data : Databases and analytics (DynamoDB, RDS, S3, OpenSearch, Redshift, Athena) Dev Tools : CI/CD, CDK, build pipelines, Application Composer Fro

A reference guide compiled from deploying two Node.js/Docker apps to AWS EC2, covering the real issues hit and how they were fixed. 1. Getting Connected Q: How do I SSH into my EC2 instance? chmod 400 your-key.pem ssh -i your-key.pem ubuntu@YOUR_ELASTIC_IP Type yes when asked about the fingerprint the first time. Q: chmod 400 doesn't seem to work / I get "bad permissions" / "Permission denied (publickey)" This happens when your .pem key sits on a Windows drive mounted into WSL (e.g. /mnt/c/Users/you/Downloads ). NTFS doesn't honor Linux permission bits properly. Fix: copy the key into WSL's native filesystem first. mkdir -p ~/.ssh cp "/mnt/c/Users/you/Downloads/your-key.pem" ~/.ssh/your-key.pem chmod 400 ~/.ssh/your-key.pem ssh -i ~/.ssh/your-key.pem ubuntu@YOUR_ELASTIC_IP Q: My key filename has spaces in it — how do I reference it? Wrap it in quotes: ssh -i "Terminal Key Pair.pem" ubuntu@YOUR_ELASTIC_IP Q: How do I know which actual instance/IP I'm connected to? TOKEN = $( curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600" ) curl -s -H "X-aws-ec2-metadata-token: $TOKEN " http://169.254.169.254/latest/meta-data/instance-id curl -s -H "X-aws-ec2-metadata-token: $TOKEN " http://169.254.169.254/latest/meta-data/public-ipv4 Compare this to what the AWS Console shows for your instance — it's easy to accidentally SSH into an old instance if an Elastic IP got reassigned. 2. Domain Name / HTTPS Without Buying a Domain Q: I don't want to buy a domain — can I still get real HTTPS? Yes — use sslip.io . Any hostname like YOUR_IP.sslip.io automatically resolves to that IP with zero signup. Let's Encrypt (via Certbot) will issue a real, trusted certificate for it just like a paid domain. Q: Why can't I just use the raw IP with HTTP? Clerk (auth) and Razorpay (payments) both require HTTPS with a real hostname in production/live mode. Plain http://ip will not work with either. Q: I later bought a real domain — how do I switch o

AWS has released aws-bench, an open-source benchmark for evaluating AI agents on real AWS tasks such as misconfigurations and infrastructure provisioning. Unlike traditional benchmarks, it uses real resources in disposable AWS accounts, scoring agent performance through automated verifiers. By Gianmarco Nalin

When I became an AWS Community Builder, one of the things I was most excited about was getting the opportunity to experiment with AWS. Let me Introduce .... eventinary.com A Completly free event management software Like most developers, I had a long list of things I wanted to build. AI applications, serverless projects, agents, APIs — there was no shortage of ideas. The AWS credits made it easier to experiment without constantly thinking about the cost of every service I turned on. But after a while, I started asking myself a different question. What if I used those credits to build something that could actually give back to the community? That question eventually led me to Eventinary . I've always enjoyed technical meetups and community events. A meetup may only have a few dozen people in a room, but something interesting happens there. Someone learns about a technology for the first time. Someone meets another developer. Someone gets inspired to build something. Someone gives their first technical talk. The event may last only a few hours, but the impact can last much longer. Then I started looking at what happens behind the scenes. Organizers have to create the event, manage registrations and RSVPs, keep track of attendees, coordinate speakers, prepare schedules, communicate updates, and somehow keep everything organized. For larger events, the number of tools and spreadsheets can grow quickly. I thought, why not build a platform that makes this easier? That became Eventinary. At first, it was just another idea. Then I started building it. As the platform grew, I realized I didn't want to build another simple invitation or RSVP website. I wanted to create a proper digital event platform that could support the entire event experience. Today, Eventinary can help organizers with things like: Event creation and digital event pages RSVP and attendee registration Speaker and session management Event schedules and itineraries Guest and attendee management Event informat

Logitech increased prices by up to 25 percent last year.

Most developers assume running a real web platform on AWS costs a fortune. Mine doesn't. HomeServerLab — a free AWS learning platform with an AI assistant, tutorials, OAuth login, and an Apps Marketplace — runs for less than $1/month in infrastructure costs. Here's exactly how. The stack and what it costs AWS Lambda Every route on the site is handled by a single Lambda function written in Python. No EC2, no always-on server, no idle costs. Lambda charges per invocation and per GB-second of compute — at my traffic levels, it stays well within the free tier and costs virtually nothing beyond it. API Gateway HTTP API The entry point for all requests. HTTP API is significantly cheaper than REST API on AWS — $1 per million requests. At my current traffic, this rounds to zero. DynamoDB Handles rate limiting, user sessions, chat history (with TTL), and OAuth state. On-demand pricing means I pay per read/write, not for provisioned capacity. Again, within free tier at my scale. Amazon Bedrock (Nova Micro) Powers the built-in AI assistant. Nova Micro is one of the cheapest foundation models available on Bedrock — and with a 25 messages/day rate limit per user, costs stay negligible. Cloudflare (Free plan) Sits in front of everything: DNS and proxying, CDN and caching (reduces Lambda invocations), WAF and bot protection, DDoS mitigation, and Cloudflare R2 for the Apps Marketplace (10GB free tier). All of this on the free plan — $0/month. CloudFront (Free tier) Sits between Cloudflare and Lambda as an additional layer. 1TB of data transfer and 10 million requests/month free. More than enough. The real cost The only real cost is Bedrock — and even that is minimal with rate limiting in place. Everything else stays within free tiers at my traffic levels. Total: less than $1/month. The key insight Serverless means you pay for what you use, not for what you reserve. Combined with Cloudflare's free plan absorbing most of the traffic before it even hits AWS, the actual billable usage

The lawsuit alleges that Oura rings are unable to measure any of the physiological signals needed to assess sleep quality or determine sleep stages.

I recently worked through a hands-on Amazon S3 practical series covering the features I would actually expect to use while working with AWS storage. Instead of only documenting definitions, this post focuses on what I configured, the commands I used, how I verified the behavior, and what I observed when something went wrong . For the concepts behind these practicals, I have already covered S3 in two detailed sessions: Session 1: AWS S3 Deep Dive — Objects, Encryption, Bucket Policies & Everything In Between Session 2: AWS S3 — Versioning, Static Hosting, CORS, Object Lock & More This post is the practical companion to those two sessions. The concepts are covered there; here I focus on actually building, testing, verifying, and troubleshooting the S3 features. The labs covered: S3 bucket configuration and lifecycle management Bucket policies with IAM, EC2 and HTTPS-only access SSE-KMS encryption with CloudTrail verification Pre-signed URLs AWS CLI s3 sync S3 Versioning and version recovery Static website hosting S3 CORS S3 Object Lock Note: This is a practical write-up, so I have intentionally kept the focus on implementation and verification rather than turning it into a generic S3 theory article. 1. S3 Bucket Configuration and Lifecycle Management Objective Create an S3 bucket with a secure baseline and configure a lifecycle rule that automatically transitions objects to lower-cost storage classes over time. Configuration For the lab: Block Public Access remained enabled. Bucket Versioning was enabled. Lifecycle rule: s3-lab-lifecycle The rule applied to all objects. Current objects transition to: Standard-IA after 30 days Glacier Flexible Retrieval after 90 days The lifecycle flow was: Day 0 ↓ Object uploaded ↓ Day 30 → Standard-IA ↓ Day 90 → Glacier Flexible Retrieval Result The lifecycle rule was successfully created and enabled, and the S3 console confirmed the configured transition periods. What this demonstrates Instead of manually moving old objects, S3 Life

Serverless on AWS isn't "just use Lambda." It's a design philosophy: let AWS manage the infrastructure, pay only for what you use, and build with managed services that scale independently. But the patterns that work in serverless are fundamentally different from traditional architectures — and the anti-patterns are expensive to learn the hard way. This guide covers the patterns that work in production, the anti-patterns that waste money or cause outages, and the decision framework for when serverless is the right (or wrong) choice. The Serverless Building Blocks ┌─────────────────────────────────────────────────────────────────────┐ │ AWS SERVERLESS STACK │ ├─────────────────────────────────────────────────────────────────────┤ │ COMPUTE │ Lambda | Fargate (serverless containers) │ │ API │ API Gateway (REST/HTTP/WebSocket) | AppSync (GraphQL)│ │ ORCHESTRATION │ Step Functions | EventBridge Scheduler │ │ MESSAGING │ SQS | SNS | EventBridge │ │ STORAGE │ S3 | DynamoDB | Aurora Serverless │ │ STREAMING │ Kinesis | DynamoDB Streams | MSK Serverless │ │ AUTH │ Cognito | IAM | Lambda Authorizers │ │ OBSERVABILITY │ CloudWatch | X-Ray | Application Signals │ └─────────────────────────────────────────────────────────────────────┘ Key principle: In serverless, you compose applications from managed services. Lambda is the glue between them — not the application itself. Pattern 1: Synchronous API (Request/Response) The most common serverless pattern: HTTP API backed by Lambda. Client → API Gateway → Lambda → DynamoDB / Aurora Serverless │ Response ← ─ ─ ─ ─ ─ ─ ┘ Best Practices API Gateway HTTP API (not REST API) — cheaper, faster, simpler for most cases One Lambda per route (single responsibility) — not a monolith Lambda Keep Lambda warm — use Provisioned Concurrency for latency-sensitive endpoints DynamoDB for simple access patterns — scales with traffic, no connection pooling Aurora Serverless v2 for complex queries — but use RDS Proxy to manage connections When to Choose H

Security researchers at Wiz recently examined S3-compatible object storage services across six popular neoclouds, revealing significant security gaps compared to Amazon S3. While S3 has become the de facto standard for object storage, most services lack several of AWS's security protections. By Renato Losio

An e-commerce alert is not complete when an API accepts a message. It is complete when the application records a terminal delivery state, suppresses an invalid recipient, or escalates through a separately governed channel. Short answer: use a dedicated SMS API for a small critical-alert worker when template ownership and direct status control matter; keep AWS SNS when SMS belongs inside an existing cloud messaging stack, and prefer a callback-capable provider when escalation must begin in under a minute. That choice creates work. A direct API keeps the send path narrow, but polling, retries, dead-letter handling, and country-specific fallback rules remain application responsibilities. For critical alerts, those responsibilities need the same idempotency and audit discipline as a ledger entry: one intent, one durable identifier, and an append-only record of every state observation. No provider turns carrier delivery into exactly-once delivery. Implement the template control plane in Node.js Start with the contract, not the vendor. The application owns an immutable alert intent containing the business event ID, recipient, template version, jurisdiction, and escalation deadline. Template ownership is the decision axis: if compliance reviewers must approve and reproduce the exact text that was sent, keep the canonical template version in the application and treat a provider template ID as deployment metadata. If a provider must own localization or regulatory registration, record that provider template ID beside the application version rather than letting it become invisible configuration. A useful state machine separates accepted from a terminal delivery result. Persist the provider message ID after the initial send, schedule periodic status reads, and append each observation with its timestamp and request ID. A retry after HTTP 429 is transport recovery, not permission to create a second alert; honor Retry-After , use exponential backoff, and preserve the same idempote

Short version: An Application or Network Load Balancer costs ~$0.0225/hour, about $16/month, just to exist , plus capacity units. Classic Load Balancers run ~$18/month. Load balancers outlive the services behind them: the app gets torn down, the ALB keeps billing. Here's how to find load balancers with no real traffic or no healthy targets, and remove them safely. Why idle load balancers linger The hourly base charge is fixed - an ALB with zero requests bills the same ~$16/month as a busy one. Load balancers are usually created early (with an app or an IaC module) and deleted last, if ever. A handful of abandoned ALBs from old environments is real, recurring money. Step 1 - List load balancers and their traffic aws elbv2 describe-load-balancers \ --query 'LoadBalancers[].{Name:LoadBalancerName,Type:Type,ARN:LoadBalancerArn}' \ --output table For an ALB, check request volume over the last 7 days (the metric dimension is the tail of the ARN, e.g. app/my-alb/50dc6c495c0c9188 ): aws cloudwatch get-metric-statistics \ --namespace AWS/ApplicationELB \ --metric-name RequestCount \ --dimensions Name = LoadBalancer,Value = app/my-alb/50dc6c495c0c9188 \ --start-time " $( date -u -d '7 days ago' +%Y-%m-%dT%H:%M:%SZ ) " \ --end-time " $( date -u +%Y-%m-%dT%H:%M:%SZ ) " \ --period 86400 --statistics Sum \ --query 'Datapoints[].Sum' Near-zero request counts over a week is a strong idle signal. (For NLBs, use the AWS/NetworkELB namespace and ActiveFlowCount .) Step 2 - Check for empty or unhealthy target groups A load balancer with no healthy targets is doing nothing useful: for tg in $( aws elbv2 describe-target-groups \ --load-balancer-arn <lb-arn> \ --query 'TargetGroups[].TargetGroupArn' --output text ) ; do echo "== $tg ==" aws elbv2 describe-target-health --target-group-arn " $tg " \ --query 'TargetHealthDescriptions[].TargetHealth.State' --output text done Empty output (no targets) or all unhealthy alongside near-zero requests is a confident "delete me." Step 3 - Delete saf

En el post anterior exploramos qué es un Data Lake y por qué son tan importantes en el ecosistema de datos actual. Ahora es momento de ensuciarnos las manos y ver exactamente qué servicios de AWS necesitamos para construir un Data Lake completamente serverless y cómo orquestarlos. Los Servicios Fundamentales Un Data Lake serverless en AWS se construye sobre cinco pilares fundamentales que trabajan en conjunto para crear una solución escalable y costo-eficiente: Storage Procesamiento Catalogo Seguridad Explotación Amazon S3 - El Corazón del Storage S3 no es solo nuestro sistema de archivos, es la piedra angular del Data Lake. Aquí almacenamos tanto los datos crudos como los procesados, y su organización es crucial para el rendimiento y los costos. Estructura de carpetas de un data lake estandar: data-lake-bucket/ ├── raw/ # Datos sin procesar │ ├── year=2024/ │ ├── month=12/ │ └── day=15/ ├── processed/ # Datos transformados │ ├── bronze/ # Limpieza básica │ ├── year=2024/ │ ├── month=12/ │ └── day=15/ │ ├── silver/ # Transformaciones de negocio │ ├── year=2024/ │ ├── month=12/ │ └── day=15/ │ └── gold/ # Datos listos para consumo │ ├── year=2024/ │ ├── month=12/ │ └── day=15/ └── athena-results/ # Resultados de queries Notarás que todo el data lake se encuentra en un mismo bucket, esto es lo más recomendable ya que S3 tiene un límite de 100 bucket que podemos crear por cuenta (no importa la región, ya que S3 es un servicio global) Configuraciones clave en S3: Versionado habilitado para auditoría y rollback Lifecycle policies para optimizar costos (Standard → IA → Glacier) Server-side encryption con KMS para seguridad si es necesario. Cross-region replication para disaster recovery AWS Glue - El Motor de Transformación Glue es suite de servicios de data serverless que maneja tanto el descubrimiento de esquemas como las transformaciones de datos. Componentes principales: Glue Jobs : Herramienta predilecta para ejecutar ETLs, nos permite procesar y transformar los dato