今日已更新 264 条资讯 | 累计 23847 条内容
关于我们

标签：#rust

找到 309 篇相关文章

AI 资讯

What building an LLM inference engine from scratch taught me about compiler design

the insight that started this project hit me while i was finishing a bytecode-compiled language i'd written in C i'd spent months building a hand-written lexer, a single-pass Pratt compiler, a stack VM with 35 opcodes, and a mark-and-sweep garbage collector. and right near the end i had this realization: an LLM inference engine is the same problem. it's a graph-compile plus memory-plan plus kernel-schedule problem. i'd just built one so i decided to find out if that was actually true the project the result is ignis, a from-scratch LLM inference engine in Rust. i used it specifically to see how far the compiler analogy held up. the dependency count ended up at 2: memmap2 (to mmap the weight blob off disk) and fancy-regex (for one look-ahead in the BPE tokenizer). everything else is hand-written, because the whole point was to understand what's actually happening the compiler analogy holds up better than i expected the interesting part of any inference engine isn't loading the weights or doing matrix math. it's what happens between "here's a compute graph" and "here's an efficient execution plan." that's a compiler problem ignis builds an SSA (static single assignment) IR of the entire Qwen2 forward pass. every operation in the transformer (the RMSNorm layers, the SwiGLU activations, the attention projections, all of it) becomes a node in the graph with explicit data dependencies then fusion passes run over the graph. the intuition is simple: if operation B always and only reads the output of operation A, you can merge them into one op and eliminate the intermediate buffer. in practice this fused 49 RMSNorm ops and 24 SwiGLU ops, bringing the total from 435 operations down to 362 that part felt expected. the liveness analysis surprised me the liveness analysis after fusion, the graph still needs activation buffers: scratch memory to hold intermediate results as the plan executes. the naive approach allocates one buffer per node. the smarter approach asks: which buffer

2026-06-27 原文 →
AI 资讯

96% of cuBLAS, no `unsafe`: what cuTile Rust proves

GPU programming usually asks Rust developers to surrender the borrow checker at the launch boundary: references collapse into raw pointers, and aliasing, synchronization, and stream lifetimes become hand-managed invariants. A new NVIDIA Labs paper argues that trade is unnecessary. How cuTile Rust Extends the Borrow Discipline to GPU Dispatch cuTile Rust is a tile-based DSL that carries Rust's ownership and borrowing rules across the host-to-GPU launch boundary — not just through host code. Introduced in "Fearless Concurrency on the GPU" (arXiv:2606.15991), submitted by NVIDIA researchers Melih Elibol, Jared Roesch, Isaac Gelado, Eric Buehler, and Michael Garland , it lets you author the kernel itself in idiomatic, memory-safe Rust rather than wrapping hand-written unsafe CUDA. The mechanism is type construction, not a runtime lock. Before launch, mutable output tensors are partitioned into provably disjoint tiles; each tile program then receives an exclusive &mut view of its slice, while inputs arrive as shared & references . Because the partitions cannot overlap, the kernel is single-threaded in its semantics and data-race-free by construction, yet still compiles to massively parallel GPU code. As Melih Elibol put it, "each tile program gets an exclusive &mut view of its memory, plus the inputs as shared references" (source: users.rust-lang.org ). Explicit unchecked types remain available for local opt-out when you need lower-level control. The safety story would be academic if it cost throughput, but the reported numbers say otherwise. On an NVIDIA B200, cuTile Rust reaches 7 TB/s on memory-bound element-wise operations and 2 PFlop/s on GEMM — roughly 96% of cuBLAS, and within measurement noise of cuTile Python . End to end, the companion Qwen3 inference engine Grout reaches 171 generated tokens/s for Qwen3-4B on an RTX 5090 and 82 tokens/s for Qwen3-32B on a B200 in batch-1 decode . Those are the authors' own measurements on specific hardware — independent reprod

2026-06-27 原文 →
AI 资讯

The Illusion of Microservices: Why the Modular Monolith is Once Again the Gold Standard in Architecture

Throughout my career, transitioning between CTO roles and, more recently, focusing purely on distributed systems architecture and high-performance engineering, I've seen many architectural patterns rise and fall. But few have caused as much silent damage to company bottom lines as the premature adoption of microservices. Over the last decade, the industry bought into the idea that, in order to scale, you needed to split your system into dozens (or hundreds) of independent services. The practical result I find in most companies? The creation of the dreaded "Distributed Monolith." The Anatomy of Waste: Networks vs. Memory The hard truth we need to face with maturity is that microservices primarily solve problems of organizational scale (Conway's Law), not necessarily performance. If your engineering team isn't the size of Netflix or Uber, prematurely fragmenting your codebase is shooting yourself in the foot. Technically, what happens when we break down a monolith without the proper domain boundaries? We trade extremely fast and cheap local function calls (resolved in the processor's L1/L2 Cache) for slow and expensive network calls (TCP/IP). We start spending an absurd amount of computational time on constant JSON serialization and deserialization, and the AWS bill explodes with internal traffic costs (egress/ingress) between Availability Zones (AZs). You haven't scaled your application; you've merely added network latency and infrastructure complexity. The Return of the Modular Monolith True seniority in software engineering isn't about mastering the most complex architecture of the moment, but having the wisdom to know when not to use it. That's why the Modular Monolith has consolidated itself as the initial gold standard for new projects and restructurings. In a well-designed Modular Monolith (and languages with strong type systems and strict scope control, like Rust, shine absurdly well here), you maintain the logical separation of domains. Modules are independen

2026-06-26 原文 →
← 上一页 下一页 →