标签：#gpu

I'm training an AlphaZero-style agent (Gumbel MuZero via DeepMind's mctx ) to lay out working factory modules for Factorio: the network places machines, belts and inserters on a grid, and the reward comes from an exact throughput verifier. Everything runs in JAX on a single RTX 5070: 128 environments in one batch, an action space of A = 1729 (3 entity types × 144 cells × 4 rotations + "done"), and a small 474k-parameter conv net in bf16. While benchmarking training configurations I hit this: MCTS simulations per move training throughput XLA compile time 16 143 episodes/s 5 s 32 47 episodes/s 13 s 64 9 episodes/s 60 s Doubling the simulation budget should roughly double the cost — each simulation is one network call plus some tree bookkeeping. Instead, 16→32 costs ×3 and 32→64 costs ×5 . Something in the search was superlinear, and this post is the story of finding it in the compiled HLO and fixing it by rewriting one ~80-line function ( PR #116 ), with bitwise-identical search results. Ruling out the network First, components in isolation (batch 128): one network evaluation takes ~0.8 ms , and the rest of recurrent_fn (environment step + observation + legal-action mask) adds almost nothing on top — the whole function is also ~0.8 ms. So at 64 simulations the network accounts for roughly 50 ms per move. But a full policy step at 64 simulations costs 362 ms . Hundreds of milliseconds were going somewhere else. To localize them I benchmarked three variants of the same policy step: full — production setup; no-net — network replaced by constant logits, real environment; tree-only — no network and no environment: recurrent_fn returns the embedding unchanged. Nothing left but mctx's own tree machinery. sims full no-net tree-only 8 10.7 ms 3.7 ms 3.8 ms 16 20.9 ms 8.3 ms 8.3 ms 32 64.7 ms 34.0 ms 33.7 ms 64 362.1 ms 124.7 ms 125.0 ms The pure tree machinery is superlinear all by itself. Per simulation it costs 0.47 → 0.52 → 1.05 → 1.95 ms as the budget goes 8 → 16 → 32 → 64

A practical framework for test planning, hardware scheduling, artifact traceability, failure classification, and evidence-based quality gates Disclaimer: The views expressed in this article are my own. The architecture, examples, terminology, and code snippets are generalized for educational purposes and do not describe or disclose any employer’s proprietary systems, confidential information, or internal implementation details. A software change can compile successfully, pass unit tests, and still introduce a serious GPU regression. The failure may appear only on one GPU generation. It may depend on a particular driver, firmware revision, operating system, graphics API, or workload. A change may preserve functional correctness while quietly reducing performance. It may also cause an intermittent failure that disappears when the test is rerun. This is why GPU validation cannot be treated as conventional CI/CD with a GPU runner attached to the end of the pipeline. A dependable GPU validation platform must coordinate: Software and firmware artifacts Hardware configurations Test coverage GPU resource scheduling Failure classification Performance baselines Engineering evidence It must do all of this while operating under an important constraint: compatible GPU capacity is limited and expensive. The objective is not simply to run more tests. It is to produce reliable evidence quickly enough to support engineering decisions. Why conventional CI/CD is not enough A conventional application pipeline often resembles: Commit ↓ Build ↓ Unit tests ↓ Integration tests ↓ Deployment A GPU validation pipeline is more multidimensional: Code or configuration change ↓ Build software and firmware artifacts ↓ Determine affected GPU configurations ↓ Reserve compatible hardware ↓ Prepare the driver and runtime environment ↓ Run functional, stability, and performance tests ↓ Collect logs, traces, metrics, and crash artifacts ↓ Classify failures and compare results with baselines ↓ Make a mer

You don't need to know anything about Go to read this. The game is just the fixed yardstick. The story is a hardware benchmark: the same program, the same problem, the same settings — only the machine changed, from a 2017 GPU cluster to a single 2026 graphics card. That makes it a rare clean measurement of one decade of progress. What "solving" means here There are two very different things a computer can do with a board game. It can play it well — that's what AlphaGo did. Or it can solve it: mathematically prove the outcome under perfect play from both sides, leaving no doubt. Solving is the hard one. You explore an enormous tree of "if I play here, they play there…" move sequences until you have an airtight proof. Each node in that tree is one position examined. The target here is a single 7x7 opening called JA . In 2023, a NeurIPS paper ( Game Solving with Online Fine-Tuning , Wu et al.) proved its verdict — the attacker cannot win — using a cluster of twelve GTX 1080Ti GPUs running 384 parallel workers. The solver is guided by a neural network that estimates how hard each branch is, and crucially that network is fine-tuned online — it keeps learning during the solve. I rebuilt that exact solver (same code, same problem, same initial model, same search settings) and ran it on one RTX 5090 . It reached the identical proof . Everything but the hardware was held fixed, so the two runs line up as a generation-vs-generation benchmark — and it doubled as a full shakedown of the new Blackwell workstation. The numbers 1x RTX 5090 (2026) 12x GTX 1080Ti (2017) ratio Worker slots 24 384 1/16 the parallelism Per-slot throughput 284 nodes/s 141 nodes/s 2.01x faster Search work to proof 1.01B nodes 1.73B nodes 0.59x (41% less work) Avg work per sub-job 4,189 nodes 6,136 nodes shallower proofs Live model updates 4,007 208 19.3x more Wall-clock time 41.4 h 8.9 h 4.64x slower Verdict loss (proven) loss identical The single card finished slower in wall-clock time (41 h vs 9 h) — b

A practical introduction to WebGPU, WGSL, render pipelines, compute shaders, and the future of high-performance graphics on the web. Your browser can stream 4K video, run a complete code editor, render complex 3D scenes, and host multiplayer games. But for years, web developers accessed the GPU through an API based on an older generation of graphics programming. WebGPU changes that contract. WebGPU is not simply a faster version of WebGL. It is a new approach to graphics and parallel computation on the web—one built around explicit pipelines, modern GPU architecture, compute shaders, predictable resource management, and a shader language designed specifically for the browser. This article expands on the progression presented in the uploaded High Performance Graphics: Introduction to WebGPU material: why WebGPU matters, how it differs from WebGL, how WGSL works, how the rendering pipeline is constructed, and how compute shaders extend the GPU beyond graphics. WebGPU is not WebGL 3.0 This is the first mental model to correct. WebGPU does not build on WebGL. WebGL exposes a browser-friendly version of the OpenGL ES programming model. WebGPU instead uses concepts associated with modern GPU APIs and provides a portable abstraction over the graphics capabilities available on the user’s system. WebGPU and WGSL are W3C standards for accessing GPU acceleration from web applications. The API supports both graphics rendering and general-purpose parallel computation. ( W3C ) WebGL │ └── OpenGL ES-style state machine WebGPU │ ├── Explicit pipelines ├── Explicit resource bindings ├── Command encoding ├── Compute shaders └── Modern GPU execution model The difference is architectural, not cosmetic. In WebGL, you frequently change global rendering state and then issue a draw call. In WebGPU, you describe the pipeline and resources more explicitly, record commands, and submit those commands to the GPU. WebGL mental model Change state ↓ Change more state ↓ Bind resources ↓ Draw WebGPU

The AI hardware landscape has shifted significantly in 2026, with NVIDIA, AMD, and Intel all competing for developers who need GPUs capable of running local large language models and AI inference workloads. Choosing the right GPU for AI workloads requires looking beyond marketing numbers and focusing on the specifications that actually affect real-world performance. Memory capacity, memory bandwidth, and software ecosystem maturity consistently matter more than theoretical compute peaks when running transformer models locally. This comparison covers the most relevant workstation and prosumer GPUs available in mid-2026, including NVIDIA's Blackwell architecture (RTX 50-series), AMD's Radeon AI Pro R9700, and Intel's Arc Pro B70. The goal is to provide a practical reference for developers deciding which hardware best fits their model sizes, software stack, and budget constraints. Which GPU specifications matter for AI workloads Marketing materials from GPU vendors emphasise AI TOPS and tensor performance, but these metrics rarely tell the complete story for local inference. The specifications below are ranked by their actual impact on running large language models. VRAM capacity VRAM is typically the first limiting factor when running LLMs locally. A model cannot execute entirely on the GPU if it does not fit into available memory. Once model weights spill into system RAM, inference performance drops dramatically. Approximate VRAM requirements for common model sizes: Model Size Recommended VRAM 7B 8-12 GB 14B 16 GB 32B 24-32 GB 70B 48-64 GB 120B+ Multiple GPUs For most homelab users, moving from 16 GB to 32 GB of VRAM provides a substantially larger practical benefit than increasing raw compute performance. A 32 GB GPU capable of running an entire model will often outperform a theoretically faster 16 GB GPU forced to offload tensors into system memory. Memory bandwidth Memory bandwidth determines how quickly model weights can be streamed into compute units. Large tran

Concept design for a building is slow and expensive. A homeowner planning an extension, or a contractor trying to win a job, is stuck between two bad options: pay a drafter $500–2,000 for a concept package, or fight SketchUp's learning curve for a week. Meanwhile the actual idea — "a 4-bed duplex with a garage and a palm out front" — fits in one sentence. So I built Forge3D Spaces : you type that sentence, and a few seconds later you're walking through a furnished 3D house in your browser — with measured floor plans, DXF for AutoCAD, and a cost estimate that come out of the same model. No install. Here's how it works under the hood. The pipeline: sentence → structured plan → building The naive approach — "ask an LLM to emit a 3D scene" — falls apart fast. Models are bad at spatial consistency; walls don't meet, rooms overlap, doors float. So the LLM never touches geometry directly. It emits a structured program , and a deterministic solver turns that into a watertight building. The prompt becomes a spec. A strict JSON-schema call (OpenRouter, json_schema response format with every field required) turns "4-bed duplex with a garage" into a room program: room types, target areas, adjacencies, storeys. A slicing-tree solver lays it out. This is the old floorplanning trick from chip design — recursively split a rectangle with horizontal/vertical cuts until every room has its area. A squarify pass keeps rooms from collapsing into corridors. The output is exact rectangles with real dimensions, guaranteed non-overlapping and gap-free. Walls, openings, roof, furniture get generated from the solved plan. Every door and window is placed by rule, not by vibes. Because the plan is a real data structure, the 2D floor plan, the 3D model, the elevations, and the bill of quantities are all views of the same thing . Drag a wall and they all move together. Nothing drifts out of sync, because there's nothing to sync — it's one model. The rendering: WebGPU, and the fallback you actually

Have you ever wondered why your most personal health queries need to travel across the globe to a centralized server just to get a simple answer? In an era where privacy-preserving AI is becoming a necessity rather than a luxury, the paradigm of Edge AI is shifting the landscape. By leveraging WebLLM and the raw power of WebGPU , we can now execute high-performance Large Language Models (LLMs) directly within the browser sandbox. No API keys, no server costs, and most importantly—zero data leakage. Today, we are building a private health consultation bot that runs 100% client-side. Why Browser-Native LLMs? 🥑 Before we dive into the code, let’s talk about why this matters. Traditional AI architectures rely on heavy GPU clusters. However, with the advent of the WebGPU API, we can tap into the user's local hardware. This approach offers: Ultimate Privacy : Data never leaves the browser. Cost Efficiency : $0 server bills for inference. Offline Capability : Once the weights are cached, you're good to go. If you are interested in more production-ready examples and advanced architectural patterns for decentralized AI, I highly recommend checking out the deep dives over at WellAlly Tech Blog . The Architecture: From Weights to Wasm To make this work, we use TVM (Apache TVM) as the compilation stack, which allows models to run on different backends, and WebLLM as the high-level interface for the browser. Data Flow Diagram graph TD A[User Input] --> B[React Frontend] B --> C[WebLLM Worker] C --> D{WebGPU Support?} D -- Yes --> E[TVM.js Runtime] D -- No --> F[Fallback/Error] E --> G[IndexedDB Model Cache] G --> H[Local GPU Inference] H --> I[Streamed Response] I --> B Prerequisites 🛠️ To follow this tutorial, ensure you have: A browser with WebGPU support (Chrome 113+, Edge, or Arc). Node.js and npm/pnpm installed. The tech_stack : React , WebLLM , TVM , and Vite . Step 1: Setting Up the WebLLM Engine First, we need to initialize the MLCEngine . Since LLMs are heavy, we should

Linux 7.2 Improves Multi-GPU Displays, M3 Support, Mesa Rusticl Defaults Arm Mali Today's Highlights This week's hardware and driver news highlights include critical Linux 7.2 kernel updates for multi-GPU display detection and initial support for Apple M3 Pro/Max/Ultra SoCs. Additionally, Mesa's Rusticl OpenCL implementation now defaults to enabling Arm Mali Panfrost driver support, simplifying GPGPU access on embedded devices. Linux 7.2-rc3 Improves Multi-GPU Display Detection (Phoronix) Source: https://www.phoronix.com/news/Linux-7.3-rc3-Multi-GPU-Fix This update for the Linux 7.2-rc3 kernel targets a persistent issue within multi-GPU setups on x86_64 systems: inconsistent display detection. The patch specifically addresses scenarios where certain graphics cards, particularly in configurations mixing integrated and discrete GPUs or multiple discrete cards, would fail to initialize displays correctly or report their presence erratically to the operating system. This is a crucial fix for users and developers deploying workstations with diverse GPU hardware, ensuring more reliable and stable display outputs without manual configuration workarounds. The improvement lies in refining the kernel's ability to probe and correctly identify active display outputs across various GPU architectures. It directly impacts system boot times and user experience by reducing potential black screens or incorrect display layouts. For enterprise and professional users relying on multiple monitors or specific GPU setups for tasks like rendering or scientific computing, this kernel patch is a significant quality-of-life enhancement, removing a long-standing friction point in Linux graphics stack stability. This contributes to the broader goal of making Linux a more robust platform for high-end graphics and compute workstations. Comment: This is a welcome fix for anyone who's wrestled with inconsistent display outputs on multi-GPU Linux machines; it often means less time debugging Xorg conf

Bryan Oliver discusses the frontier of AI infrastructure: chaos engineering for large-scale GPU clusters. He shares how engineering leaders can handle complex topologies, network protocols like RDMA, and NUMA misalignments. Discover seven practical fault-injection strategies to maximize multi-million dollar hardware efficiency and build robust observability loops. By Bryan Oliver

The company is taking a modular approach to designing these chips, anticipating that their needs will change as AI evolves rapidly by the time the chips are in production.

TL;DR Short version: no. I dropped a much older GPU ( Quadro P2000, 5GB, Pascal, 2016 ) next to an RTX 3090 (24GB, Ampere) on the same box, ran the same context-length ladder (8K→128K) through Ollama and vLLM on qwen3-coder:30B-A3B , and got zero extra usable context in either engine — and a 74% decode-speed hit for the trouble. Ollama hits the identical Chunk too big wall at ctx=65536 whether the P2000 is there or not. vLLM refuses tensor-parallel across the two cards entirely — not a VRAM problem, a flat compute-capability rejection ( Minimum capability: 75. Current capability: 61. ) that fails in 40 seconds, before any memory profiling. And the one real, measured effect of adding the P2000 to Ollama: decode speed goes from 76 → 19.5 tok/s at ctx=49152 once the P2000 gets pulled in as an actual compute device. Full narrative version — the two-stage collapse, the prompt-cache validation bug caught mid-sweep, the CUDA13-silently-drops-Pascal finding — is on Medium .## The setup ardi (dual Xeon E5-2680 v4, 128GB RAM, openSUSE Leap) has a Quadro P2000 sitting in a second slot next to the RTX 3090 this whole series has run on so far. Same model as phase 1 ( qwen3-coder:30B-A3B ), same box, four legs: {Ollama, vLLM} × {3090 only, 3090+P2000 tandem}, priced through HomeLab Monitor against real GPU power draw. Ollama: same wall, extra tax ctx 3090 only decode tok/s tandem decode tok/s P2000 VRAM (tandem) 8,192 124.3 122.0 6 MB / 0% 24,576 108.2 70.0 62 MB / 0% 32,768 99.4 61.0 62 MB / 0% 49,152 75.7 19.5 3,580 MB / 55% 65,536 fatal: Chunk too big fatal: identical Chunk too big — Two separate costs, not one: decode already falls behind at ctx=24576 while the P2000 is still basically idle (62MB, 0% util) — some scheduling overhead just from having a second visible device. Then the real collapse hits at ctx=49152, when the P2000 actually gets pulled into the compute path (3.58GB, 55% util) and decode craters to 19.5 tok/s . Same context ceiling either way, worse speed the wh

The moment you press Enter, billions of mathematical operations begin. Let's follow that journey. Every day, millions of people ask ChatGPT, Gemini, Claude, or other AI assistants questions. The answer appears almost instantly. But have you ever wondered what actually happens after you press Enter? Why can't a normal CPU answer these questions quickly? Why do companies spend billions on GPUs? Let's take a journey from your keyboard to the AI's brain. Imagine This... Suppose your office receives 10,000 letters. You have two choices. Option 1: One super-fast employee He opens one letter after another. Very fast. But still... One at a time. This is a CPU. Option 2: 10,000 employees Each opens one letter simultaneously. The work finishes almost instantly. This is a GPU. The difference isn't that each employee is smarter. There are simply many more workers working together. CPU vs GPU Think of it like this. CPU = CEO making decisions. GPU = Thousands of factory workers building products simultaneously. Why CPUs Are Amazing Your CPU performs tasks like Opening Chrome Playing music Running Windows Calculating taxes Managing memory Running applications These jobs require decisions branches conditions interrupts This is logical thinking. CPUs are built for this. Why GPUs Exist Originally GPUs were invented for games. Imagine rendering one image. A 4K monitor contains over 8 million pixels. Each pixel needs calculations. Every frame. 60 times every second. Instead of calculating one pixel at a time... GPU calculates millions together. Gaming accidentally created the perfect hardware for AI. AI Doesn't Think Like Humans LLMs don't "think" in English. They perform mathematics. Lots of mathematics. Almost everything inside an LLM becomes... Matrix × Matrix Vector × Matrix Addition Multiplication Normalization Softmax That's all mathematics. Billions of times. Why Matrix Multiplication Matters Imagine two tables. Table A 1 2 3 4 5 6 7 8 9 Multiply with Table B 2 4 6 8 1 3 Every n

GPU programming usually asks Rust developers to surrender the borrow checker at the launch boundary: references collapse into raw pointers, and aliasing, synchronization, and stream lifetimes become hand-managed invariants. A new NVIDIA Labs paper argues that trade is unnecessary. How cuTile Rust Extends the Borrow Discipline to GPU Dispatch cuTile Rust is a tile-based DSL that carries Rust's ownership and borrowing rules across the host-to-GPU launch boundary — not just through host code. Introduced in "Fearless Concurrency on the GPU" (arXiv:2606.15991), submitted by NVIDIA researchers Melih Elibol, Jared Roesch, Isaac Gelado, Eric Buehler, and Michael Garland , it lets you author the kernel itself in idiomatic, memory-safe Rust rather than wrapping hand-written unsafe CUDA. The mechanism is type construction, not a runtime lock. Before launch, mutable output tensors are partitioned into provably disjoint tiles; each tile program then receives an exclusive &mut view of its slice, while inputs arrive as shared & references . Because the partitions cannot overlap, the kernel is single-threaded in its semantics and data-race-free by construction, yet still compiles to massively parallel GPU code. As Melih Elibol put it, "each tile program gets an exclusive &mut view of its memory, plus the inputs as shared references" (source: users.rust-lang.org ). Explicit unchecked types remain available for local opt-out when you need lower-level control. The safety story would be academic if it cost throughput, but the reported numbers say otherwise. On an NVIDIA B200, cuTile Rust reaches 7 TB/s on memory-bound element-wise operations and 2 PFlop/s on GEMM — roughly 96% of cuBLAS, and within measurement noise of cuTile Python . End to end, the companion Qwen3 inference engine Grout reaches 171 generated tokens/s for Qwen3-4B on an RTX 5090 and 82 tokens/s for Qwen3-32B on a B200 in batch-1 decode . Those are the authors' own measurements on specific hardware — independent reprod

About a year ago, I turned my gaming PC into a local AI Lab. And yes, the most important word in that sentence is LOCAL . Let me tell you the story of how I sacrificed my gaming hours to build several tools, and now I'm going to tell you about this one that I use every single day. The Problem: Token bankruptcy Day to day, all of us developers who work with Artificial Intelligence share the same headache: tokens and rate limits . We're all victims of the high prices that come with constantly running inference with AI agents like Claude Code, Codex, or Gemini CLI (yeah, I love working from the terminal, I LOVE CLIs). While I was building AI systems (agent orchestration, LLM fine-tuning ), I was burning through way too many tokens. I tried tweaking the prompts and cleaning up the junk in my context, but the real devourer of my quota showed up when I had to learn a new tool. I was implementing solutions in QGIS (QGIS is a free, open-source Geographic Information System (GIS) software that allows users to create, edit, visualize, analyze, and publish geospatial data on maps) for a project and I didn't know the interface 100%. Like any dev facing something new, I leaned on AI agents: I'd take a screenshot, send it over, and ask for explanations. Here's an important fact that hurt my wallet: A screenshot on my MacBook (Full HD resolution of 1920x1080) burns about 258 tokens per tile on models like Claude. That adds up to roughly 1,548 tokens per image (sounds like a lot, and yeah my friend, it is way too much when we're talking about context). Now imagine sending dozens of these images a month trying to understand a complex interface as a 2x dev (99x, I'd say, in this new AI era). I was eating through my hourly Claude allowance just doing visual queries, leaving me with no quota left to generate the actual code I really needed for my development. The Epiphany (and the Hardware) One day, during a forced break thanks to a Claude rate limit , I looked over at my Gaming PC. I

On April 29, 2026, Nathan Sobo published the Zed 1.0 announcement post on Zed's blog. The post landed on Hacker News at 2,047 points and 663 comments — the highest-engagement HN story in the present cache by a substantial margin. The launch announcement is a milestone marker after five years of development, roughly a million lines of Rust, and a custom GPU-accelerated UI framework called GPUI that the Zed team built from scratch rather than building atop Electron, Chromium, or any other browser engine. The structural argument Zed has been making for the last several years is condensed in one sentence from Sobo's post: "Instead of building Zed like a web page, we built it like a video game, organizing the entire application around feeding data to shaders running on the GPU." The video-game framing is not metaphorical. Zed's editor surface is composited by feeding glyph atlases, syntax-tree-derived color spans, and pane-layout geometry into GPU shaders the way a video-game engine composites its frames. The reason this matters is the reason the Zed team gave for starting over from the Atom era: Atom was built as a fork of Chromium, and the same team that built Atom is the team that spawned Electron. The Atom-Electron-VSCode lineage is, in the historical-causal sense, Zed's own. Sobo's post is unusually direct about the inherited limitation: "Electron eventually became the foundation of VS Code (which today seems to be forked into a new AI code editor every other week). Web technology offered an easy path to shipping flexible software, but it also imposed a ceiling. No matter how hard we worked, we couldn't make Atom better than the platform it was built on." The 1.0 announcement is, in part, a statement that the rebuild from scratch has finally cleared that ceiling. Who built it Zed's three co-founders all worked on Atom at GitHub before founding Zed Industries in 2021. Nathan Sobo led the Atom team from 2011 to 2018; he also co-led Teletype for Atom, one of the first

How GPU scheduling complexity and MLOps integration are forcing platform teams to rearchitect Kubernetes clusters before operational debt becomes insurmountable. As AI workloads consume roughly 40% of enterprise Kubernetes clusters by 2026, the platform's default scheduler is proving fundamentally mismatched with the topology-aware, gang-scheduled demands of GPU-intensive training and inference. Platform engineering teams that invest now in purpose-built GPU scheduling layers, multi-tenant partitioning, and FinOps-driven autoscaling will separate themselves from organizations drowning in 30-45% GPU utilization rates and mounting infrastructure costs. Why the Default Kubernetes Scheduler Fails GPU Workloads Kubernetes was designed for stateless, CPU-bound services, and its pod-by-pod bin-packing scheduler has no native awareness of GPU topology, NUMA boundaries, or NVLink interconnect bandwidth. This becomes a critical failure point with NVIDIA H100 SXM5 nodes, where achieving full-bandwidth tensor parallelism requires all 8 GPUs on a node to be scheduled as a single atomic unit. The default scheduler cannot guarantee this co-placement, meaning distributed PyTorch FSDP or MPI training jobs frequently land on suboptimal node configurations, wasting expensive NVLink bandwidth and forcing teams to over-provision GPU capacity. Idle GPU memory stranded across partially-utilized nodes is the primary driver behind the 30-45% utilization rates reported in 2025 surveys by Gradient Dissent and Weights and Biases, representing millions of dollars in annual wasted spend for mid-to-large enterprises running mixed AI workloads. Building the GPU Scheduling Stack: Volcano, KAI Scheduler, and MIG Platform teams are converging on a layered scheduling architecture that replaces or augments the default Kubernetes scheduler with GPU-aware primitives. Volcano has become the dominant choice for distributed training workloads, using its PodGroup abstraction to enforce gang scheduling across

A short essay on why the developer community should invest as much effort as possible into LLMs that are free from corporations and states. ML researchers and hardware engineers both need to contribute here. The latter may even be more important, because whether users can run advanced LLMs on personal hardware depends on breaking NVIDIA's monopoly. This essay is highly political, especially in the opening sections. Keep that in mind. Corporate AI Will Be Closed and Unaccountable by Default The other day, almost at the same time as the release of Fable 5, Anthropic's Dario Amodei published an article called "Policy on the AI Exponential", where he discussed what the world should do with powerful AI-based systems. All sections except the first contain fairly reasonable proposals, or at least proposals worth discussing. I will not consider them here. The real core is in the first section. In that first section, he effectively proposes a system in which the state would be required to license advanced AI systems, measured by the amount of compute used, and even ban the release of models that are not considered safe for society. In practice, this repeats a story as old as the world: a large corporation wants to regulate the market so smaller companies do not interfere with its ability to earn mountains of money, all under noble-sounding pretexts. And the point is not that Amodei is some villain. He is simply an entrepreneur who wants to earn as much money as possible. Any large corporation would prefer not to let smaller companies near the feeding trough in its field. Anthropic is merely saying this openly, and that is all. In effect, AI Big Tech wants a future where all non-AI companies become its serfs, mortally dependent on intelligence delivered through Anthropic's API, or OpenAI's, or Google's, and so on. In practice, those AI companies would hold the revenue of all these other companies in their hands. Without them, the whole economy around those companies would cru

CUDA for AMD Lemonade, Intel Arc Pro Linux Gains, XPU Manager 2.0 Today's Highlights Today's top GPU news highlights include AMD's Lemonade SDK gaining NVIDIA CUDA support, significant performance improvements for Intel Arc Pro GPUs on Linux 7.1, and the major 2.0 overhaul of Intel's XPU Manager for better GPU management on both Windows and Linux. AMD's Lemonade SDK For Local AI Adds NVIDIA CUDA Support (Phoronix) Source: https://www.phoronix.com/news/AMD-Lemonade-10.7-Released AMD has released a new version of its Lemonade SDK, a powerful local AI server solution designed to leverage AMD's diverse hardware ecosystem, including their CPUs, GPUs, and NPUs. The most significant update in this release is the addition of NVIDIA CUDA support. This integration allows developers to utilize NVIDIA GPUs within their Lemonade-powered local AI deployments, bridging a critical gap in cross-platform AI development. The inclusion of CUDA support is a strategic move, enabling Lemonade to tap into NVIDIA's extensive CUDA ecosystem and a vast array of pre-optimized models and libraries. This means that applications built with Lemonade can now seamlessly target a wider range of hardware, offering unprecedented flexibility for developers working with local AI. For users, it provides the choice to deploy their AI models on either AMD or NVIDIA hardware using a single, unified SDK, expanding the potential reach and efficiency of their AI workloads. Comment: This is a massive step for cross-vendor AI development. Being able to use AMD's Lemonade SDK to deploy local AI models and then seamlessly target NVIDIA GPUs via CUDA truly unifies the AI backend landscape for diverse hardware setups, making it incredibly practical for hybrid environments. Intel Arc Pro B70 Showing Off Some Performance Wins With Linux 7.1 (Phoronix) Source: https://www.phoronix.com/review/linux-71-arc-pro-b70 Recent testing by Phoronix indicates that Intel's Arc Pro B70 discrete GPUs are demonstrating notable perform

In 2025 Google Cloud added G4 , powered by NVIDIA's RTX PRO 6000 Blackwell Server Edition GPUs to their offering, allowing them to offer hardware not only for AI applications, but also for other applications, such as rendering, simulations or gaming. A single G4 instance with one accelerator ( g4-standard-48 ) comes equipped with 48 CPU cores, 180 gigabytes of RAM and 96 gigabytes of GPU memory. This is a lot of resources for a single cloud workstation, that only the most demanding workstreams would utilize. Most professionals who require a graphics accelerator to do their job, don't really need this much compute power for day to day tasks. It wasn't financially reasonable to pay for a G4 instance, when you weren't utilizing all the resources you paid for. If only there were smaller machine types… If only you could share that one very powerful GPU between multiple virtual machines… Introducing fractional VMs! During Google Cloud Next 2026, Google announced GA for fractional G4 VMs and was the first provider to bring vGPU functionality to RTX PRO 6000 accelerators. vGPU stands for virtual graphical processing unit . Just like VMs (virtual machines) are a way to split one physical computer into smaller, independent systems, vGPU allows for a single physical accelerator to be split into 2, 4 or 8 virtual accelerators! The new fractional machine types ( g4-standard-24 , g4-standard-12 , g4-standard-6 ) now allow you to perfectly match the compute capabilities to your needs! Who is it for? The existence of those new machine types makes it much more cost-efficient to move many GPU-dependent tasks to the cloud. Replacing physical workstations in offices with cloud infrastructure is not a new thing , but till now, Google Cloud didn't offer a good platform for those who needed workstations to process images, post-process videos, simulate physics or render 3D graphics. Those users now can get exactly the hardware they need, allowing their companies to move away from maintaini

The generation side is shipping fast (TileGym, AutoKernel, KernelEvolve). The validation-side surface for “what the kernel actually did at runtime” has not kept pace. TL;DR In the past nine months, three significant releases have landed for auto-generation of CUDA kernels: NVIDIA TileGym , RightNow AutoKernel, and Meta’s KernelEvolve. Each ships training infrastructure for kernel generation. Validation infrastructure (what the generated kernel actually did at runtime, on a real workload, in a production-shaped environment) has not kept the same pace. eBPF traces are the ground-truth layer that closes the gap. What “validation” means at the kernel level Two distinct validation surfaces: Pre-launch: the generated CUDA C compiles, the PTX assembles, the kernel passes a numerical-equivalence test against a reference. Standard compiler / unit-test territory. Generation frameworks ship this themselves. Post-launch: the kernel ran, returned, took N microseconds, used M registers per thread, hit X cache miss rate, and did or did not serialize the rest of the stream behind it. This is the layer that an eBPF trace plus standard CUDA driver counters can answer for any kernel, generated or hand-written. Auto-generation pipelines do not by default close the post-launch loop. They demonstrate “the kernel works in our test setup”. They do not demonstrate “the kernel does not regress p99 latency on production inference traffic”. What an eBPF trace adds to a generated kernel Once a generated kernel is in a real workload, the same trace surface used for any CUDA kernel applies: launch latency from cudaLaunchKernel , sync stalls from cudaStreamSynchronize , host-side overhead from the dispatcher, host scheduling preemption while the GPU is busy. None of those signals are visible to a generation framework that evaluates kernels in isolation. -- post-launch validation: did the new generated kernel regress p99? SELECT kernel_name , COUNT ( * ) AS launches , AVG ( duration_ns ) / 1 e3 AS