I compared the real cost of running LLMs on AWS - here's when each option makes sense

AWS gives you three ways to run LLM inference in production. I've deployed all three for clients and the decision always comes down to the same variables: volume, team size, and how much you value your weekends. Here's the short version. The three paths Bedrock — Fully managed, pay-per-token. You call an API, you get tokens back. No GPUs, no cold starts, no 3am pages about OOM pods. SageMaker Endpoints - Semi-managed. You bring your model (or a fine-tuned one), deploy it on dedicated instances, and handle autoscaling. Pay per hour whether you're serving requests or not. Self-hosted on EKS — Full control. vLLM or TGI on GPU spot instances with Karpenter. Cheapest per token at scale, most operational overhead. The cost crossover that matters This is the table I keep coming back to with every client: Volume Bedrock (Haiku) SageMaker (g5.xlarge) EKS (g5.xlarge spot) 1K req/day ~$36/mo ✓ ~$1,015/mo ~$674/mo 50K req/day ~$1,800/mo ~$1,015/mo ~$674/mo ✓ 500K req/day ~$18,000/mo ~$6,090/mo ~$2,022/mo ✓ The crossover point where self-hosting beats Bedrock: 10,000–20,000 requests/day . Below that, Bedrock wins on simplicity alone. Above it, you're leaving serious money on the table. The hidden cost nobody models upfront Teams prototype on Bedrock (smart move — it's the fastest path to production). But the cost curve isn't linear. At 10K requests/day it's cheap. At 50K it's "we need to talk to finance." At 500K it's a rearchitecture project. The mistake is not choosing Bedrock at low volume. The mistake is not planning the exit path before you need it. Quick decision framework You should pick... When... Bedrock No ML infra team, <50K req/day, need frontier models (Claude, Llama) SageMaker Fine-tuned models, predictable traffic, need dedicated VPC EKS self-hosted >100K req/day, open-source models, dedicated platform team What I actually recommend Use a hybrid. Most production systems I've deployed use: Bedrock for complex reasoning and customer-facing chat (low volume, high qua