标签：#engineering

Metron was the Greek word for a measure: the standard you judge a thing against. It gives us metric, and metronome. Pick the wrong metron and everything you count against it comes out wrong, no matter how carefully you count. Ask most teams how they know AI adoption is working, and the answer is some version of: we’re shipping more. More PRs, more tickets closed, more velocity on the board. None of those numbers were ever measuring what people thought they were measuring. They were proxies, and bad ones, for whether the team was creating value. AI didn’t break that. It just made the proxies lie louder. An agent can produce PRs faster than any team can review them. It can close tickets all day. None of that tells you whether the thing built was worth building, or whether it fixed the actual problem, or whether anyone downstream is better off. Burning tokens isn’t the same thing as creating value. It just looks the same on a dashboard built to reward the first thing. The same DORA numbers I pointed at earlier are blunt about it: individual output climbs while delivery throughput and stability drop. Same bottleneck, counted at the wrong end . This is Theory of Constraints in one sentence: speeding up a step that wasn’t the bottleneck doesn’t speed up the system. It just piles more work in front of whatever the real bottleneck is. Implementation used to be the constraint. Now it isn’t. Review is. Coordination is. Making sure everyone agrees on what “done” even means is. Agents made the fast part faster and left the slow part exactly where it was, except now it’s buried under more output arriving to be reviewed by fewer people who understand any of it. Measuring individual speed after that shift is measuring the wrong clock. The team can be faster and worse off in the same quarter. The instinct, when this gets noticed, is to add another skill to the agent. Automate the review step too. Automate the coordination. Whatever’s slow, throw a capability at it. That doesn’t fix

Your data warehouse has a table called orders . It has columns like amount , status , created_at , and customer_id . Now three people ask "What was Q1 revenue?" The analyst writes SELECT SUM(amount) FROM orders WHERE created_at BETWEEN '2026-01-01' AND '2026-03-31' . The data engineer adds WHERE status = 'completed' . Finance excludes refunds and trial conversions. Three queries, three numbers, one question. Nobody is wrong. They just defined "revenue" differently. Multiply this by every metric in your organization, every team that queries the warehouse, and every tool that displays a number. That's the problem. A semantic layer solves it by defining each metric once, in one place, and serving that definition to every consumer. What is a semantic layer? A semantic layer is a metadata layer between your data warehouse and every tool that queries it. It defines business metrics, maps them to SQL, and exposes them through APIs. Instead of every consumer writing its own query, they all reference the same definition. When someone asks for "revenue," the semantic layer knows that means: SUM ( CASE WHEN status != 'refunded' AND type != 'trial' THEN amount ELSE 0 END ) That definition lives in one place. Dashboards, APIs, AI agents, and ad-hoc queries all use it. Change the definition once and every consumer gets the updated calculation. No Slack thread asking "which number is right." No detective work tracing a wrong number back to a stale query in a notebook somewhere. The core components of a semantic layer: Metrics (measures). The numbers you aggregate: revenue, order count, average deal size. Each metric has a fixed SQL definition. Dimensions. The columns you filter and group by: date, status, category, region. Dimensions define the axes of analysis. Relationships (joins). How tables connect: orders belong to customers, products belong to categories. Defined once, reused by every query. Access rules. Who can see what. Row-level security, tenant isolation, role-based ac

"We need real-time analytics" is one of the most common requests in data engineering. It's also one of the most misunderstood. When the VP of Sales says "real-time," they usually mean "faster than the dashboard that refreshes overnight." When the CTO says it, they might mean sub-second event streaming. The gap between those two definitions is a 6-month infrastructure project. Most teams don't need true real-time. They need fast enough. And "fast enough" is achievable with pre-aggregation caching at a fraction of the complexity and cost of a streaming architecture. What is real-time analytics? Real-time analytics means querying data with minimal latency between when an event happens and when it's visible in your analytics. The spectrum: Freshness Latency Architecture Use case True real-time < 1 second Event streaming (Kafka, Flink) Fraud detection, stock trading, live monitoring Near real-time 1-60 seconds Micro-batch or streaming Operational dashboards, alerting Frequent refresh 1-60 minutes Scheduled refresh + caching KPI dashboards, AI agent queries Batch Hours to daily Scheduled ETL Board reports, monthly summaries Most analytics use cases fall in the "frequent refresh" category. Revenue by region doesn't need sub-second freshness. Active users in the last hour doesn't need event streaming. A pre-aggregation cache that refreshes every 15 minutes covers 90% of what teams call "real-time." When you actually need real-time True real-time analytics (sub-second latency from event to query result) is worth the infrastructure investment when: Fraud detection. Every second of delay is potential fraud that slips through. Live monitoring. Server health, API error rates, active user counts for live products. Trading and pricing. Financial instruments where stale data means wrong prices. Live events. Streaming metrics during a product launch, marketing campaign, or live broadcast. If you're in one of these categories, you need an event streaming architecture: Kafka, Flink, M

Ruth Linehan explains how migrating high-performance caching services from Kotlin to Rust shattered internal preconceptions around delivery velocity and engineering overhead. She discusses the ergonomics of the Rust borrow checker, shares how compile-time safety shortens the developer feedback loop, and profiles how tools like Criterion and flamegraphs optimize concurrent code paths. By Ruth Linehan

A three-person agency received a $14,000 AWS bill in one day after attackers extracted static access keys and burned Claude invocations on Bedrock. Combined with May's DN42 incident, where an autonomous agent provisioned $6,531 of oversized infrastructure in 24 hours, practitioners warn that cloud billing lags roughly a day behind agent-speed spend. By Steef-Jan Wiggers

Every organisation that has run a significant software system for more than a few years has felt a version of the same thing: a change that should have taken days takes months, nobody can quite explain why, and the explanation that eventually gets offered — the domain is complex, the requirements changed, the previous team was careless — is almost never checked against an alternative approach for the software architecture or alternative framework choices, because the alternative was never built. There is no possible comparison to determine the solution chosen is a good one and there is no benchmark to measure "fit for purpose." This is the unfalsifiability problem, and it is worth stating plainly before anything else in this piece, because it is the reason the cost described below is so rarely traced back to its actual cause. Every system is built once. There is no version of your platform built the other way, running alongside it, that anyone can compare it to. So when a system works, the approach that produced it gets read as validated. When a system becomes expensive to change, the cost gets attributed to anything except the structural decision that caused it — because that decision was made years ago, by people who may have moved on, and there is no control group to prove that the structure was the variable that mattered. That absence of a control group is not a minor academic point. It is the reason a specific, avoidable pattern of cost has been able to spread through the industry for decades, get taught in courses, get validated in interviews, and still never be clearly named as a mistake. This article is an attempt to name it — and to offer something more useful than a diagnosis: a way to check, this week, whether it applies to you. The Villain: Process Over Product Ask almost any team building a significant piece of software what the goal of the project is, and the honest answer, more often than anyone would like to admit, is not "build the best-fitting prod

Last week an AI-generated GPU kernel ran 18.71× faster than an optimized PyTorch baseline. The model—Fable 5—didn't just edge past the human implementation. It lapped it. Claude Opus 4.8 reached 14.4×. GLM-5.2 hit 11.14×. GPT-5.5 managed 4.34×. Fable's kernel was in a different tier entirely. The exciting read: AI is starting to improve the low-level machinery that makes AI itself cheaper and faster. Specialized performance work that once required rare expertise just got dramatically easier to explore. The uncomfortable read: what happens when the best implementation is also the one nobody on your team would have written—or can fully explain? That question is about to land on every engineering team that ships AI-generated code. The Benchmark Problem A benchmark shows the kernel ran fast under tested conditions. It doesn't show: How it behaves across different GPU hardware How it handles numerical edge cases What happens under months of production changes Whether it degrades gracefully when inputs shift The person who wrote it can't answer these questions either. The AI generated this code through a process that doesn't leave a reviewable chain of reasoning. There's no commit message that says "I chose this approach because X." So the reviewer's job just got harder—not easier. The Real Shift I've been watching this pattern across engineering teams this year. The argument is moving from "can AI generate working code?" to "can our org absorb generated code without breaking quality, morale, or judgment?" The GPU kernel story makes the tension concrete: One side says the code ran, it was measured, it won. Stop moving the goalposts. The other side says somebody still has to know where it can fail and take responsibility when it does. Both are right. AI can make implementation cheaper while making proof more expensive. Senior engineers may write less code but spend more time designing adversarial tests, checking assumptions, planning rollbacks, and deciding whether an impr

​I. The Terminal Velocity of Disembodied Intelligence ​The current paradigm of artificial intelligence is approaching a fundamental cognitive ceiling. We have built vast, sprawling minds, yet we have trapped them in sensory deprivation chambers. Today’s Large Language Models and industrial robotics operate under a fatal flaw: the reliance on linear, arbitrary tokenization. Traditional AI chops raw text into disconnected, non-semantic fragments, wasting immense computational memory tracking the mere phonetic order of spelling and grammar. ​Simultaneously, industrial machines navigate the physical world through siloed sensor pipelines. They process light, space, and kinetics as heavy, raw digital arrays, relying on brute-force algorithmic equations to stitch reality together. This is not intelligence; it is computational exhaustion. To achieve true artificial general intelligence, the machine must stop reading about the world and begin to inhabit it. ​II. The Biological Imperative ​Nature solved the hardware-software integration problem millions of years ago. Human biology remains the ultimate benchmark for flawless, metabolic efficiency. Our somatosensory system does not wait for a central brain to read a text string before pulling a hand away from a fire; it processes physical feedback in milliseconds through decentralized neural highways. ​Human cognition leverages cross-modal integration within the angular gyrus, allowing visual data to be instantly cross-referenced with kinesthetic feedback to trigger anticipatory motor reflexes. Furthermore, biological intelligence is inherently tied to physical survival. We manage thermodynamics through a centralized hypothalamic thermostat, protecting the body from cellular destruction, while routing ambient light data to deep-brain structures to drive a natural circadian rhythm. True AGI requires this exact synthesis of abstract thought and biological mechanics. ​III. The Logographic Paradigm Shift ​To bridge the gap between

"We're using Kafka with exactly-once semantics, so we don't have to worry about duplicates." I've heard this in architecture reviews, design docs, and postmortem explanations. It represents a misunderstanding of what Kafka's exactly-once guarantee actually covers, and the gap between the promise and the reality has caused real production incidents. What Kafka's Exactly-Once Actually Covers Kafka's exactly-once semantics (EOS), introduced in 0.11.0, operates at two levels: Producer idempotence ( enable.idempotence=true ): The producer assigns each message a sequence number. The broker deduplicates messages with the same producer ID and sequence number. This prevents duplicates caused by producer retries — the message lands in the Kafka partition exactly once, regardless of retry count. Transactions ( transactional.id ): Allows a producer to write to multiple partitions atomically. Either all writes commit or none do. Combined with isolation.level=read_committed on consumers, readers only see committed transactions. Together, these give you exactly-once message delivery within the Kafka cluster. What Exactly-Once Does Not Cover Here's the boundary that engineers miss: Kafka's exactly-once guarantee is scoped to the Kafka cluster. The moment your consumer does anything outside Kafka — writes to a database, calls a REST API, publishes to a cloud queue — you're outside the transaction boundary. Consider a typical consumer: consumer . poll ( records ); for ( record : records ) { database . save ( process ( record )); // External write — outside Kafka transaction } consumer . commitSync (); If the application crashes after database.save() but before commitSync() , Kafka re-delivers the message. The consumer reprocesses it. The database now has two writes for the same event. Enabling producer idempotence on the consumer's Kafka writes does not fix this. The Patterns That Actually Give You End-to-End Safety Idempotent Consumers Design consumer processing logic to be idempote

Risk stratification sounds like a data-science buzzword until you have to build the thing. For a Medicare Advantage plan, it's a concrete pipeline: take a population of members, score each one's clinical and financial risk, and rank them so care management and documentation teams know who to touch first. Here's how I'd architect it. The core idea Population health risk stratification = scoring + segmentation. You compute a per-member risk signal, then bucket members into tiers (e.g., rising-risk, high-risk, catastrophic) so finite resources go where they move outcomes and revenue most. The mistake teams make is treating it as a single ML model. In practice you want a layered signal: a stable, explainable base (RAF + chronic conditions) plus optional predictive overlays. Explainability matters because care managers won't act on a black-box score, and auditors won't accept one. Step 1: Build the member feature record { "member_id" : "SYNTH-77310" , "age" : 73 , "hccs" : [ "HCC37_1" , "HCC85" , "HCC18" ], "raf" : 1.842 , "gaps" : [ "a1c_overdue" , "no_pcp_visit_180d" ], "utilization" : { "ed_visits_12m" : 3 , "inpatient_12m" : 1 } } The RAF here is your defensible, model-grounded risk anchor under CMS-HCC V28. Everything else is supplemental signal. Step 2: Score and tier def risk_tier ( member ): base = member [ " raf " ] util = 0.15 * member [ " utilization " ][ " ed_visits_12m " ] \ + 0.30 * member [ " utilization " ][ " inpatient_12m " ] score = base + util if score >= 3.0 : return " catastrophic " if score >= 1.8 : return " high " if score >= 1.0 : return " rising " return " stable " Keep the weights transparent and tunable. The point isn't a perfect model; it's a defensible, reproducible ranking your operational teams trust. Step 3: Make "rising-risk" actionable The tier that quietly drives the most ROI is rising-risk — members trending toward high cost who still have open documentation and care gaps. Surface their specific gaps (overdue labs, undocumented chroni

Everyone talks about prompts memory RAG But production issues were actually loops false completion replay retries wrong tool non deterministic execution Here are the top 7. That's what led us to build Failproof AI not because we wanted another framework, but because we kept seeing the same reliability problems across every framework.

i've been that person. standing in front of leadership with an 18-month architecture diagram, explaining why we need six months of infrastructure before a user touches a single feature. and it made sense. for 25 years it made sense. writing boilerplate was expensive. every feature came with a tax — database migrations, routing config, auth wiring. build a shared platform first, pay that tax once. the roadmap justified the investment. then i saw a stat that wouldn't leave me alone. roughly 60% of features on a six-month roadmap are obsolete by launch. not slightly off. obsolete. the customer's problem shifted. the market moved. you spent six months building a precise answer to a question nobody asks anymore. the longer you invest before showing something real, the more expensive it is to admit you were wrong. so you don't. you ship the wrong thing and call it "on schedule." i've done it. i've watched it happen. AI didn't create this problem. but agents are making it impossible to ignore. the 82-point gap mckinsey's 2025 survey: 88% of organizations use AI. only 6% see real bottom-line impact. that 82-point gap isn't about tools. everyone has the same tools. but something shifted in their may 2026 report. they describe agents working overnight — enriching requirements, generating code, packaging outputs for morning review. they call it the "24-hour sprint." leading organizations see 3-5x productivity with 60% smaller teams. a product owner logs in at 9am and finds a feature went from requirements to tested code overnight. nobody worked late. agents did. that's not autocomplete. that's a different delivery model. and here's what most teams miss: it only works when the work is small, bounded, and complete. agents need to know where a task starts and ends. horizontal platform architectures don't give them that. the codebase is the prompt jeremy d. miller built wolverine for .NET. in june 2026 he wrote: "the structure of your codebase is now, effectively, part of the prom

The cohesion paper series is now published in full — five papers that build a chain from the concept of cohesion to the Independent Variation Principle (IVP) . The chain: On the Nature of Cohesion — defines cohesion as a $2k$-tuple: for $k$ partitioning rules, $k$ (purity, completeness) pairs. Proves the knowledge-embodiment theorem: maximal cohesion under a rule coincides with exact knowledge embodiment under that rule. Shows that every published algorithmic cohesion metric measures a structural proxy (method-call overlap, shared-field density), not cohesion as defined by a principle. DOI: 10.5281/zenodo.20785752 Causal Cohesion — instantiates the schema under one concrete rule — change-driver-assignment identity: elements belong together iff $\Gamma(e_1) = \Gamma(e_2)$. Develops the metric $H_\text{causal}(M) = (\text{purity}(M), \text{completeness}(M))$, a two-dimensional score that fills one slot of the $2k$-tuple. DOI: 10.5281/zenodo.20785881 Four Necessary Conditions for Optimal Modularization — from the schema plus the objective of minimizing change propagation, proves four conditions — Admissibility, Element Form, Separation, Unification — are necessary and jointly exhaustive, uniquely pinning the $\Gamma$-equality partition $E / \tilde{\Gamma}$. DOI: 10.5281/zenodo.21362420 Why Minimizing Change Propagation Minimizes Maintenance Cost — decomposes total maintenance cost into access, alignment, cognitive, and domain-fixed components. Proves that minimizing change propagation cost is equivalent to minimizing total maintenance cost under an explicit coefficient condition, justifying the objective paper 5 assumed. DOI: 10.5281/zenodo.21362542 The Independent Variation Principle — synthesizes the chain into a single structural principle and examines the premises (change drivers, functional model, change isolation), preconditions (driver independence, decisional autonomy), and scope boundary. DOI: 10.5281/zenodo.21362618 Two derivations Last month's preprint — Der

Meta recently open-sourced Brain2Qwerty v2, a noninvasive Brain–Computer Interface (BCI) that can decode sentences from thoughts using electroencephalography (EEG) or magnetoencephalography (MEG) signals from the brain. In evaluations, the system achieved a word accuracy rate 61% on average, compared to 8% for other non-invasive methods. By Anthony Alford

Drawing from the enduring adaptability of HTML and HTTP, Seph Gentle proposes embedding self-contained schemas directly into file headers, ensuring data remains readable without external definitions. His experimental format prioritises forward, backwards, and sideways compatibility, enabling data format evolution without central coordination or data loss By Olimpiu Pop

TL;DR Traditional SaaS streaming requires exporting sensitive data to a vendor cloud, creating security risks and egress costs. BYOC reverses this model by running the data plane inside the customer’s cloud while the vendor manages the control plane. This keeps data within the enterprise perimeter while still providing a managed platform. Condense builds on this model with AI-driven automation, unified monitoring, and marketplace deployment, enabling private, compliant, and cost-efficient real-time data streaming. The enterprise data landscape is currently defined by a conflict between real-time AI data streaming utility and the strict requirements of data sovereignty . For years, the standard SaaS model forced a compromise. To access premium analytics, companies had to export sensitive telemetry to a vendor cloud. This created massive cloud egress costs and introduced significant security vulnerabilities. Bring Your Own Cloud (BYOC) for data streaming platforms has emerged as the professional solution to this dilemma. It allows a business to keep data within its own perimeter while benefiting from a fully managed, high-performance ecosystem. The BYOC Architecture: Privacy by Design An experienced analyst views BYOC as a clean separation of concerns. The architecture splits the environment into two distinct layers to ensure raw data never leaves the authorized environment. SaaS Control Plane: This is the management layer hosted by the provider. It handles the brain of the operation. It manages orchestration, user access, and pipeline configuration without ever seeing the actual data packets. Private Data Plane: This is the muscle. The managed Kafka clusters , Kubernetes (K8s) nodes, and storage engines like ClickHouse live inside the customer Virtual Private Cloud (VPC) . By keeping the data plane inside the customer perimeter, telemetry collection remains private. This architecture is the most direct path to satisfying internal security audits and global regulatory

Here is a reliable test: find a prompt that isn't working. Read it carefully. Now ask yourself — at which specific sentence did the model get permission to do what it did wrong? You will almost always find it. A hedged instruction. A missing constraint. An ambiguous scope. The model did not misunderstand you — it followed the most statistically probable interpretation of what you wrote. That interpretation was not the one you intended. These are not beginner mistakes. They are structural patterns that reappear at every experience level, because they look reasonable when you write them and only reveal themselves in the output. TL;DR: Prompts fail because they hand interpretive control to the model on dimensions where you had a specific requirement. Each of the seven mistakes below is a different way of doing that — and each has a specific, testable fix. Mistake 1: Placing Critical Instructions in the Middle of the Prompt Language models process all tokens simultaneously through attention mechanisms , but the effective weight any individual token receives depends heavily on its position. Instructions near the beginning and end of a prompt receive disproportionately more attention weight than those in the middle. This is not a quirk — it is a consequence of how positional embeddings interact with self-attention across long contexts. This effect is well-documented. The "Lost in the Middle" study (Stanford / UC Berkeley, 2023) showed that retrieval accuracy from long-context windows degrades significantly for information placed in the middle — even in capable models. The same mechanism applies to instruction prompts: GPT-4o and Claude 3.5 Sonnet both exhibit measurably lower constraint adherence for instructions buried mid-context compared to those at the leading or trailing position. Open-weight models including DeepSeek-V3 and Llama 3 display the same positional bias — this is not a proprietary model quirk, it is a structural property of the transformer architecture. T

Leave a carton of milk on the counter and it spoils in a day. Put the same carton in a refrigerator and it lasts a week or more. Nothing about the milk has changed — the same bacteria, the same enzymes, the same chemistry. What changed is temperature, and temperature does not nudge reaction rates gently. It controls them with an exponential lever. A swing of just a few degrees can stretch shelf life from hours to days. This article explains the equation behind that lever — the Arrhenius equation — what each term means physically, how to use it to compare rates at two temperatures, and the mistakes that quietly corrupt activation-energy estimates. Why this calculation matters Almost any process that involves chemistry running over time depends on the temperature-rate relationship. Food spoilage, drug degradation, battery aging, polymer curing, corrosion, and the cracking reactions in a refinery all speed up or slow down with temperature in the same exponential way. Engineers who design accelerated life tests rely on it directly: they run a product hot for weeks to predict how it behaves cold for years. The reason a quantitative model is essential is that intuition fails here. A linear guess — "twice as hot, twice as fast" — is badly wrong. Reaction rate climbs far faster than temperature does, and how much faster depends on the activation energy of the specific reaction. Without the Arrhenius equation you cannot convert an oven-shelf test into a real-world prediction, and you cannot tell whether a 5 C process drift matters or not. The core formula Svante Arrhenius proposed the relationship in 1889, building on earlier work by van 't Hoff. It states that the rate constant k of a reaction depends on temperature as: k = A * exp( -Ea / (R * T) ) Here A is the frequency factor (sometimes called the pre-exponential factor), Ea is the activation energy in J/mol, R is the universal gas constant 8.314 J/mol K, and T is the absolute temperature in kelvin. The physical picture

A panel on data ownership challenged the definition of "ownership," arguing it must extend beyond simple account control to include structural independence, interoperability, and community governance. Speakers like Zenna Fiscella, Paul Frazee, Boris Mann, and Robin Berjon emphasised the need for shared standards, unbundled platforms, and better tools to support user sovereignty. By Olimpiu Pop

DoorDash details the architecture behind Ask DoorDash, its AI-powered conversational shopping assistant, combining LLMs, specialized AI agents, MCP-based tooling, and an intelligence layer with persistent consumer memory and live backend data. Early results show up to 24% higher checkout conversion, 17% larger baskets, and improved intent accuracy using memory-backed sessions. By Leela Kumili