开发者

We were running a cloud-hosted platform on AWS EKS , with EC2 worker nodes managed by us, MongoDB Atlas for NoSQL workloads, AWS RDS for relational databases, and Amazon ElastiCache for Redis for caching and temporary data. Over time, the infrastructure had grown the way most real systems grow: more services, more data, more backups, more images, more snapshots, and more “temporary” resources that were no longer temporary. The platform worked, but the cloud bill was higher than it needed to be. So we started cutting waste, improving the application, and resizing the infrastructure around how the system actually behaved. The result: around $120,000 in annual savings , without sacrificing reliability. The Problem Was Not One Big Thing When we started reviewing the infrastructure, it was clear that there was no single expensive resource causing the entire problem. The cost came from many places at once. Some services were using more CPU and memory than they needed. Some microservices did not really need to be separate anymore. Some databases were oversized for their actual usage. Some storage had accumulated over time. Some backups and snapshots were kept longer than necessary. Some resources were simply unused. That is usually how cloud costs grow. Not because of one bad decision, but because of hundreds of small decisions that were reasonable at the time and never revisited later. So instead of looking for one magic fix, we approached the problem from multiple angles: application code, architecture, databases, Kubernetes resources, storage, backups, caching, and non-production environments. The Optimizations 1. Making the Application Use Fewer Resources One of the most important parts of the optimization was improving the application itself. It is easy to look at cloud cost as an infrastructure problem only, but inefficient code directly affects infrastructure cost. If the application uses too much CPU or memory, the platform needs more pods, larger nodes, bigger ins

TL;DR We built a serverless pipeline that ships FSx for ONTAP audit logs to Honeycomb, where its high-cardinality query engine turns file access data into actionable insights. Two delivery paths verified: [Path A: Direct] FSx for ONTAP → S3 Access Point → EventBridge Scheduler → Lambda → Honeycomb Events Batch API [Path B: OTel Collector] FSx for ONTAP → S3 Access Point → EventBridge Scheduler → Lambda → OTel Collector → OTLP → Honeycomb Why Honeycomb for file access logs? Because file access data is inherently high-cardinality : thousands of users × millions of file paths × dozens of operations × multiple SVMs. Traditional log tools force you to pre-aggregate or sample. Honeycomb lets you query the raw events at full resolution. ┌──────────────────────────────────────────────────────┐ │ Honeycomb Query Engine │ │ │ │ "Show me which users accessed /vol/finance/* │ │ between 2am-4am last Tuesday" │ │ │ │ → BubbleUp: auto-detect anomalous dimensions │ │ → Heatmap: visualize access density over time │ │ → GROUP BY user, path, operation — no pre-indexing │ │ │ │ 20M events/month FREE │ └──────────────────────────────────────────────────────┘ This is Part 10 of the Serverless Observability for FSx for ONTAP series. Why Honeycomb for File Access Logs? Most observability tools index a fixed set of fields. When you have high-cardinality dimensions — like file paths ( /vol/data/project-alpha/2026/Q1/report-final-v3.docx ) or Active Directory usernames — you hit index bloat, slow queries, or forced sampling. Honeycomb's columnar storage handles this natively: Capability Traditional Logs Honeycomb Query by arbitrary field Pre-index or full scan Instant (columnar) GROUP BY high-cardinality field Expensive / limited Native BubbleUp (anomaly detection) Manual investigation Semi-automatic (select time range, BubbleUp identifies differing dimensions) Heatmap visualization Requires pre-aggregation Raw events For FSx for ONTAP audit logs, this means you can ask questions like: "Which

I've noticed more and more people, usually developers and tech nerds, adding a guestbook to their personal websites. First, if building a personal website is becoming more common, that would be amazing. I love that "small web" vibe. Second, the guestbook idea is awesome. I really hope it's a thing. submitted by /u/kixxauth [link] [留言]

Is it first time happening or what? https://preview.redd.it/m4qemuh3lc4h1.jpg?width=1142&format=pjpg&auto=webp&s=d91c07b578207c3ba341cc89deb9dd035c6034d6 submitted by /u/eliassrm [link] [留言]

If you've ordered glasses from them, you can go to: https://www.zennioptical.com/myAccount/myOrders My page just spins. I was curious where my glasses order was after a couple weeks of not receiving them. The network tab shows 30+ css files being downloaded for a very simple website, 10+ trackers and advertising js scripts, and the page still won't load. How about this: SELECT * FROM orders WHERE customer_id = :id ORDER BY order_date Then you can render my most recent orders on the server and at least render some HTML with relevant data. Ok, at scale that might now work. I understand that. Zenni isn't Amazon or Google, but they probably get many requests. In that case we could set up 1+ load balancers that simply forward the request to a sharded server based on userid to balance the load. It's absolutely crazy that we could handle thousands of requests per second in the early 2000s with a few servers in a colo facility and these days everyone has to pretend their facebook or myspace that needs to analyze complex graph connections between people. I just want to see what the status of my order is. It's not that hard. submitted by /u/DrAwesomeClaws [link] [留言]

A survival guide for when everything goes wrong in production. There's a moment every engineer who works with Kafka experiences. You check the producer. Messages are sending. You check the consumer. Nothing. The consumer group shows zero lag because there's nothing to lag behind — as far as the consumer knows, the topic is empty. But it's not empty. The messages are there. Somewhere. In some partition, at some offset, behind some configuration you set six months ago and forgot about. Kafka doesn't lose messages. But it's very good at hiding them from you. Consumer Lag: The Number Everyone Watches Wrong Consumer lag is the difference between the latest offset in a partition and the offset your consumer group has committed. Simple concept. Dangerous in practice. The mistake: treating lag as a single number. Lag is per-partition. If you have 30 partitions and one consumer is stuck on partition 17 while the others are healthy, the total lag looks manageable. But partition 17's data is hours behind, and whatever downstream system depends on that data is serving stale results. Monitor lag per partition. Tools like Burrow, Kafka Exporter for Prometheus, or even kafka-consumer-groups.sh --describe break it down. If one partition's lag is growing while others are stable, you have a stuck consumer, a hot partition, or a poison message. A poison message is a record your consumer can't process — malformed data, unexpected schema, null where it shouldn't be null. The consumer throws an exception, the offset doesn't commit, and it retries the same message forever. Lag grows. The consumer looks "alive" because it's processing — just not making progress. The fix: dead letter queues. After N retries, move the message to a separate topic, commit the offset, and move on. Alert on the dead letter topic. Investigate later. Don't let one bad record block millions of good ones. Rebalance Storms: The Silent Killer Consumer rebalancing is Kafka's mechanism for redistributing partitions acro