Dev.to
How to Monitor AI Agents in Production
TLDR Monitoring AI agents in production requires distributed tracing: a single user request fans out into 10 or more internal operations, and logs alone cannot show you which step is slow, failing, or burning your token budget. OpenTelemetry's gen_ai.* semantic conventions give you standardized span attributes for LLM calls, tool invocations, and agent steps. Some are stable today; others are still experimental. Auto-instrumentation libraries (OpenLLMetry, OpenInference, OpenLIT) cover most agent frameworks with two to three lines of initialization code. You do not change your agent code. Traces ship to OpenObserve over OTLP. From there you get SQL-queryable trace data, token usage dashboards, cost attribution by agent and model, and alerting on latency and cost anomalies. OpenObserve also exposes an MCP server. You can query your live agent traces from a Claude or GPT session without opening a dashboard. Why Agents Are Harder to Monitor Than a Single LLM Call A single LLM call is straightforward to observe. One HTTP request, one response, one latency number. You can log the input and output and call it done. An agent is different. When a user sends a message, the agent calls an LLM to decide what to do, invokes a tool, processes the result, calls the LLM again, possibly calls another tool, and eventually returns a response. That one user message becomes ten or more internal operations. Some of those operations call external APIs. Some retry. Some spawn sub-agents. Without distributed tracing, you see none of this structure. You know the response took 8 seconds. You do not know whether the LLM took 7 of those seconds or whether a tool made three retries before timing out. Four categories of problems appear in production agents that you cannot debug without traces: Latency. Which step is slow? The LLM call? The tool execution? A retry loop the agent entered because the tool returned ambiguous output? Cost. Which agent, which task, which model is consuming tokens? A s
Manas Sharma
2026-05-28 14:18
👁 10
查看原文 →
Dev.to
I Analyzed 1,000 AI-Generated Blog Posts for Quality. Here's the Data.
Last year, I was doing something that felt increasingly absurd: manually reading AI-generated content to decide if it was "good enough." PostAll — the content automation tool I've been building — was producing hundreds of blog posts per week for clients. And I had no systematic way to evaluate quality at scale. I was spot-checking. Vibes-checking, really. That doesn't work at volume. So I built a programmatic quality analysis pipeline, ran it over 1,000 AI-generated posts, and let the numbers tell me what my gut was missing. The findings surprised me. A few of them genuinely changed how I think about AI content quality. What I Actually Measured First, a definition of terms, because "quality" is almost meaninglessly vague in this space. I broke quality into five measurable dimensions: Readability — Flesch-Kincaid grade level and reading ease score Keyword density — Target keyword frequency and distribution across the post Grammar error rate — Errors per 1,000 words, caught via LanguageTool's API Factual accuracy — Claims that could be verified programmatically (dates, statistics, named entities cross-referenced against a knowledge base) Structural consistency — Presence of expected elements: intro hook, subheadings, conclusion, CTA I used 1,000 posts across three categories: SaaS product descriptions, long-form "how-to" articles (1,200–2,000 words), and listicles (500–900 words). All were generated by PostAll using GPT-4o, with various prompting strategies. The Setup The analysis pipeline isn't complicated, but the piece that makes it useful is the batch processing layer: import anthropic import language_tool_python import textstat from dataclasses import dataclass from typing import Optional import json @dataclass class QualityReport : post_id : str flesch_reading_ease : float flesch_kincaid_grade : float grammar_errors_per_1000_words : float keyword_density : float structural_score : int # 0–5 based on element presence flagged_claims : list [ str ] overall_score :
Aakash Gour
2026-05-28 14:16
👁 11
查看原文 →
Dev.to
From Forgotten Repo to Live App: How I Finished Photremium.com Using GitHub Copilot
This is a submission for the GitHub Finish-Up-A-Thon Challenge What I Built Photremium is an all-in-one, lightning-fast web utility platform engineered for high-performance image processing. Built to eliminate the friction of clunky, ad-heavy design tools, it provides users with instantaneous, client-side and serverless tools like high-fidelity background removal, image resizing, custom QR code generation and many more. As a software engineering student, this project represents my vision of creating a modern production platform that prioritizes raw speed, high usability, and robust SEO architectural patterns. Live Platform: photremium.com GitHub Repository: itsaminaziz/photremium.com Demo The Live Application Experience the full toolset live right now at photremium.com . Key Features in Action Feature Implementation Speed / Processing Compress IMAGE Client-side Canvas / Web Workers Instantaneous local compression Resize IMAGE Client-side React & HTML5 Canvas Real-time pixel/percent adjustment Crop IMAGE Client-side UI & Visual Crop Editor Instantaneous browser-based cropping Convert to JPG Client-side File Readers (Bulk Upload) Instant batch conversion via browser Convert from JPG Client-side Canvas (PNG/GIF compiler) Multi-format local generation QR Code Generator Vector-based SVG/Canvas rendering Instant download generation QR Code Scanner Client-side WebRTC Camera / File API Real-time local camera processing Blur Face Hybrid Client-side Face Detection Instant local privacy overlay mapping Remove Background (AI) Cloud-based Serverless / Cloudflare Edge < 2 seconds (Any device image processing) Watermark IMAGE Client-side Layer Composition Instantaneous text/graphic stamping The Comeback Story The Before (A Half-Baked Local App) Photremium started as an ambitious prototype on a local machine. While the fundamental image-processing utilities worked locally, the project hit a massive wall when it came to global deployment and production readiness. It was plagued with
Muhammad Amin Aziz
2026-05-28 14:15
👁 10
查看原文 →
Dev.to
Custodial vs trust-minimized: two settlement layers for the agent economy
"Settlement layer for the agent economy" is suddenly a crowded sentence. In the last few weeks, two very different things have started competing for it. OKX Agent Payments Protocol (APP) announced in May 2026 — backed by AWS, Alibaba Cloud, Uniswap, Paxos, QuickNode, with ecosystem support from Base, Ethereum Foundation, Solana, Sui, Aptos, Optimism — describes itself as the settlement layer where AI agents pay each other. AEON's $8M pre-seed (May 20, YZi Labs) described itself the same way. There is a list now, and it is getting longer. We build one of the things on this list, and we think the framing is wrong. These are not rivals jostling for the same slot. They are two different layers, and they answer two different versions of the same question: what does an agent need to settle a trade? This piece walks through the two layers, where each is the right answer, and why an honest comparison gets you further than picking sides. What APP and other custodial-venue protocols actually do A useful way to read OKX APP — and similar moves coming from the larger exchanges — is to treat them as a venue layer . An exchange already holds inventory across many chains. It already has the risk engines, the liquidity, the legal arrangements with banks and partners. Adding an agent-facing API on top of that machinery is a relatively short walk. What an agent gets, in exchange, is breadth. Many chains, many assets, batched and netted settlement against deep internal books, and fast execution because the exchange is just moving entries in its own ledger. For an agent whose job is "find a price somewhere and execute now," that is a powerful primitive. What the agent gives up, in exchange, is a counterparty. At any moment between deposit and withdrawal, the agent's balance is the venue's promise to pay. That promise is normally good. The agent has no way to verify, from inside its own logic, that it still is. This is not a criticism of OKX APP. It is the structural shape of any venue-
Baris Sozen
2026-05-28 14:11
👁 9
查看原文 →
Dev.to
Treasure Hunting at Scale: Why Our Cache-Aside Cache Cost Us 40% in Tail Latency During Black Friday
The Problem We Were Actually Solving During load testing at 50k concurrent hunters hitting the hunt endpoints, p99 latencies stayed under 200ms. But at 270k concurrent users in production, the hunt page suddenly took 1.8 seconds to load, triggering cascading 502s from our CDN. The error surfaced in Datadog as hunt_page_render_time_bucket{le=2.0} = 42% while le=0.5 dropped to 18%. The fingerprints were identical across three regions: high latency correlated exactly with Redis cache miss rate spiking from 12% to 48% during the hunt start window. Our cache-aside pattern with a 30-second TTL was amplifying miss storms. We discovered that the treasure hunt start time was synchronised by marketing campaigns. When the clock struck 10:00:00 UTC, 270k users hit the endpoint within 30 seconds. Each request would check the cache (miss), fetch from PostgreSQL, render the page, and write the cache entry. But PostgreSQL couldnt keep up with 9k queries per second during that window, causing query queueing and connection exhaustion. The Redis layer, designed for 150k ops/sec, was not the bottleneck. The database was. What We Tried First (And Why It Failed) Our first attempt was to increase Redis TTL from 30 seconds to 5 minutes. This reduced cache misses from 48% to 24%, and p99 latency improved to 650ms. But at 320k concurrent users, the latency still spiked to 1.4s because the underlying database queries were still hitting the same table with the same indexes. The Redis layer was masking symptoms, not solving the root cause. Next, we tried database read replicas. We spun up three read replicas and routed hunt queries to them using a weighted service mesh. This worked for a few minutes, but then we hit replication lag. The replicas fell 800ms behind primary, causing hunt pages to display stale treasure locations. Our operators started getting customer complaints about seeing the wrong treasure coordinates. We rolled back within 15 minutes. We even tried increasing PostgreSQL share
Lillian Dube
2026-05-28 14:11
👁 8
查看原文 →
HackerNews
Rust Will Save Linux from AI, Says Greg Kroah-Hartman
signa11
2026-05-28 13:50
👁 2
查看原文 →
HackerNews
AI Is Starting to Hit Power Grid Limits Simple, Crédible, Ouvre La Discussion
latentframe
2026-05-28 13:42
👁 2
查看原文 →
Reddit r/artificial
Best image generatir
So this a2e.ai website allows you to generate any image and for free. You get a good amount of credits amidst signing up Referal link: https://video.a2e.ai/?coupon=LgQi submitted by /u/No_Restaurant_5461 [link] [留言]
/u/No_Restaurant_5461
2026-05-28 13:35
👁 4
查看原文 →
Reddit r/webdev
breakpoints
Just starting a discussion on breakpoints. Not like this hasn't been beaten to death before :D (͡(¬‿¬)) Tailwind sm: 40rem - roughtly 640px xs: is commonly set to 30rem or 320px for the smallest iphone width folding phones might be a use case for even smaller viewports.. idk, i cnat afford one so mauybe that's not how they work? supposedly they can be 280px there's smart glasses. no idea. VR. no idea. windows on desktop, people like me get really annoyed when some content of a window goes off screen or it transforms into a mobile version of the site just because i shrunk the window when i wanted to keep using the desktop site. (oh you and your tiling window manager, lol) submitted by /u/xThomas [link] [留言]
/u/xThomas
2026-05-28 13:27
👁 4
查看原文 →
Wired
Altra Running Promo Codes June 2026
Score big savings on Altra Running shoes with up to 50% off sale styles, 20% off select models, 10% off your first order when you sign up, plus free standard delivery on every purchase.
Adrienne So
2026-05-28 13:00
👁 8
查看原文 →
Reddit r/webdev
Choosing the best tool for a beginner browser-based 3D prototype on Greek Mythology for Design and Tech MDP for HSC in NSW Aus.
I’m working on a school Design & Technology major project and need help finding a subreddit to choose the best tool. The project is a browser-based low-poly 3D educational prototype about Greek mythology. It has a small Mount Olympus hub which is basically a decorated hub area with the one portal to the one playable myth scene: Theseus and the Minotaur in Daedalus’ Labyrinth which is essentially like a maze where you can explore/follow the myth through some mediums like cutscenes and some small puzzles. My constraints are: needs to run in a browser should work on normal school laptops/devices low-poly/simple 3D style minimal coding experience must be realistic for a student prototype ideally easy to share with a link More context is that essentially I have my whole design planned out with multiple eventualities based on the constraints on the tool I end up using and so I essentially just need a medium to construct my project in which is why I have left it so late to ask for advice on what tool to use to construct this prototype. I am confident with my extensive designs and a decently functioning prototype of this one playable scene and hub area along with potential future designs that I will show the design process adequately for a band 6 hopefully. submitted by /u/Bigbrainmc [link] [留言]
/u/Bigbrainmc
2026-05-28 12:13
👁 4
查看原文 →
Dev.to
I built an MCP server that gives AI persistent memory of your SQL database
A while ago I tried to build a local coding assistant. I downloaded Qwen3, fired it up on my MacBook with 16GB of RAM, and within a day realized the output quality was nowhere close to Claude or GPT-5. The model could fit . It just couldn't compete . So I changed the question. If I can't make the model smarter on my hardware, can I make what I feed it smarter? Where the tokens actually go I started watching where my Claude / Cursor / Copilot sessions actually spent their tokens. The surprise: most of it wasn't reasoning. It was lookup . Every fresh chat about my company's database re-discovered the same things: What does status = 3 mean? (cancelled) How does orders join to users ? ( orders.user_id → users.id ) What's that cryptic JobStatus enum? (a dozen integer codes nobody remembers) The model figured it out, the session ended, and tomorrow it figured it out again . Same tokens, same latency, every single time. The expensive part of working with an AI wasn't the thinking — it was re-teaching it things it had already learned yesterday. There's a lot of attention right now on trimming AI output tokens (talk like a caveman, strip the pleasantries, etc.). But in my workflow the bigger leak was on the input side: paying full token cost every session to re-establish context that never changed. "Memory" isn't a feature, it's an architecture question AI clients are starting to bolt on "memory" features. But they're proprietary, opaque, and locked to one tool. Claude's memory doesn't help Cursor. Cursor's doesn't help Copilot. You can't inspect it, you can't share it with a teammate, and you can't diff it. What I actually wanted was an explicit, inspectable, shareable context layer that any AI client could read deterministically — same answer every time, same file my team could hand off. I picked the highest re-learn cost in my world to start with: SQL databases. Enter amnesic amnesic is an open-source MCP server that gives any AI client persistent semantic memory of your
Suraj Goyal
2026-05-28 11:58
👁 6
查看原文 →
Dev.to
No-Code Strategy Builder: Turning a Trading Idea Into Testable Rules
Most trading ideas start as vague thoughts. "Buy when RSI is oversold and price bounces from support." It sounds reasonable. But the moment you try to test or automate it, the ambiguity becomes obvious. What exactly counts as oversold? How is support defined? What qualifies as a bounce? When do you exit? Without precise answers, the idea cannot be tested, measured, or executed consistently. This gap between intuition and execution is exactly what no-code strategy builders are designed to close. Why vague trading ideas fail Most traders think in concepts rather than rules. "Buy the dip." "Trade strong momentum." "Enter when the trend looks healthy." These ideas feel intuitive, but they are unusable in practice unless translated into explicit logic. Without clear definitions, you cannot backtest a strategy, cannot repeat decisions consistently, and cannot diagnose why results change over time. Ambiguity leads to second-guessing. Second-guessing leads to inconsistent execution. Inconsistent execution makes performance impossible to evaluate. What a no-code strategy builder actually does A no-code strategy builder is a visual system that forces clarity. Instead of writing code, you select indicators, define conditions, combine logic using AND/OR rules, specify entries and exits, and then test the strategy on historical data. Conceptually, it works like assembling building blocks. Each block represents a condition such as "RSI below 30" or "price above moving average." When combined, those blocks form a complete, testable trading system. The key benefit is precision. From idea to testable strategy The transformation follows a predictable workflow. You begin with a loose idea, such as buying when a stock is oversold and starting to recover. You then break that idea into components. What defines oversold? What signals recovery? How do you enter? How do you exit? How much do you risk? Once those questions are answered, the idea becomes a set of explicit rules. For example,
Reyaz
2026-05-28 11:55
👁 12
查看原文 →
Dev.to
v0 by Vercel Review: AI-Generated React Components That Actually Ship
I opened v0, typed "a settings page with profile editing, notification preferences, and a connected accounts section," and watched it generate a fully functional three-tab settings interface in under 40 seconds. The component used shadcn/ui primitives, Tailwind utility classes, and TypeScript types — the exact stack I would have chosen if I had written it from scratch. I copied the code, pasted it into my Next.js project, changed two import paths, and it rendered correctly on the first try. This is the v0 value proposition distilled: generate UI components that look like a senior frontend developer wrote them, then paste them into your real project without rewriting half the output. After generating 15 components across two weeks of real product work, I can confirm that v0 delivers on this promise more reliably than any general-purpose AI coding tool I have tested. But its scope is narrower than the marketing suggests, and understanding where v0 stops being useful is as important as knowing where it excels. The shadcn/ui Advantage v0 is built on top of shadcn/ui, and this is the single most important fact about how it works. shadcn/ui is not a component library in the traditional sense — it is a collection of copy-pasteable React components built on Radix UI primitives with Tailwind styling. When v0 generates a component, it uses these primitives directly, which means the output is consistent, accessible, and composable. The practical benefit is that v0-generated components integrate with your existing project without introducing a new design system. If you already use shadcn/ui — and a large and growing percentage of Next.js projects do — the generated components reuse your existing Button, Card, Dialog, and Input primitives. v0 just assumes you have them installed and generates code that expects them. If you do not have shadcn/ui in your project, v0 prompts you to run the initialization command before generating anything, which takes about 30 seconds. This archite
pickuma
2026-05-28 11:55
👁 6
查看原文 →
Reddit r/artificial
Recommended NotebookLM alternatives
I really like NotebookLM, especially for dumping PDFs/slides/long YouTube videos into one place and asking questions about them. But I’m starting to feel like it’s very “research workspace” first, which makes sense. It’s great when I already have sources and I want to understand them. Less great when I want something more flexible for actual learning, especially on mobile. The things I’m looking for: - handles PDFs, slides, articles, and long You Tube videos - lets me chat with the material / summarize / ask follow-up questions - has more output styles than just one default format - ideally lets me change voice, tone, length, and depth - works well on mobile - can translate or help me learn across languages - good for topics beyond school research, like communication, social skills, history, humanities,career stuff, etc. - bonus if it helps plan what to learn next instead of just summarizing one source A few I’ve looked at so far: Quizzify seems good if your main use case is active recall. It’s more of a quiz/practice-test focused, which is useful because summaries can trick you into thinking you learned something. My brain absolutely falls for this. The downside is that it feels more school/study-tool specific. BeFreed for the audio learning side. It’s not really a NotebookLM clone, but that’s kind of why I like it. You can paste a PDF, article, You Tube link, or just prompt a topic, then it turns it into a personalized audio learning path. You can adjust the voice, style, depth, and length, and the mobile experience is much better for learning while walking/commuting. I’ve used it more for history, communication, social skills, and career-type topics than pure school research. Elephas looks interesting for Mac users because it can do document Q&A and writing locally. That might be helpful if connection issues are the annoying part. But from what I can tell, it’s more of a doc chat / writing assistant than a flexible learning app. Gamma / Canva / Napkin seem strong
/u/HoseaJacob
2026-05-28 11:49
👁 5
查看原文 →
Dev.to
GitHub Copilot Workspace Review: Task-Level AI Coding in the Browser
I tested GitHub Copilot Workspace on 12 real tasks across three repositories in May 2026 — a mix of bug fixes, feature additions, and documentation updates. My goal was to figure out whether the spec-first, browser-based workflow actually produces useful code, or whether it is a demo that falls apart when you ask it to do real work. The answer sits somewhere between impressive and frustrating, with the tool's success rate varying dramatically based on task size, repository maturity, and how well you write the initial specification. The Spec-First Workflow Forces Better Communication Copilot Workspace changes the AI coding interaction in one fundamental way that I have not seen elsewhere: you do not start with code, you start with a specification. When I opened Workspace on a Next.js project and typed "add rate limiting to the API routes using the existing rate-limit.ts utility," the system did not immediately generate code. It spent roughly 15 seconds reading my repository, then produced a three-step implementation plan: (1) import the rate-limit utility in each route, (2) wrap the route handler with the rate limiter, (3) add a test for the rate-limited behavior. I could approve the plan as-is, reject individual steps, or add revision notes before any code was written. On this particular task, the plan was correct and I approved it. Workspace then executed each step, modified five route files, and produced a draft pull request with a clear description and a summary of what changed. The entire process — from typing the specification to having a reviewable PR — took 4 minutes and 12 seconds. This planning phase is not window dressing. On a different task where I asked Workspace to "add WebSocket support to the chat feature," it read the repository and surfaced during planning that the project was deployed on Vercel's serverless functions, which do not support persistent WebSocket connections. It suggested using Vercel's Edge Functions with a third-party real-time serv
pickuma
2026-05-28 11:36
👁 6
查看原文 →
Dev.to
Azure Container Apps Express: The Agent-First Platform You've Been Waiting For
I've been running AI workloads on Azure Container Apps for over a year. Every time I spin up a new agent backend, the ritual is the same: create an environment, configure networking, set scaling rules, wire up health probes, then deploy the actual container. For a prototype agent that might live for a week, that's too much ceremony for what you get. ACA Express, which hit public preview in May 2026, kills most of that ceremony. And a separate but related announcement, Docker Compose for Agents, brings MCP gateways and model serving to standard ACA environments. They solve different problems and run on different infrastructure, but together they cover the full spectrum of agent deployment on Azure. Let me break down both. ACA Express: What It Actually Is Express is a new environment tier within Azure Container Apps. You bring a container image. Express handles provisioning, HTTPS, scaling (including scale-from-zero with subsecond cold starts), and resource allocation. No environment to manually provision through the portal. No networking to configure. No scaling rules to write. Under the hood, Express is built on ACA Sandboxes, a platform primitive that uses prewarmed pools to deliver that subsecond startup. This isn't the standard ACA cold-start experience with a fresh coat of paint. It's a different architecture. The tradeoffs are real. Express is HTTP workloads only, consumption CPU only. No GPU. No VNet integration. No Dapr. No service discovery between apps. No managed identity at runtime. No health probes. If you need any of those, standard ACA environments are still there. But for stateless HTTP agent backends, Express is dramatically faster to deploy and cheaper to run. Here's what it takes to get a container running: # Create an express environment az containerapp env create \ --name my-express-env \ --resource-group rg-my-agents \ --environment-mode express \ --logs-destination none # Deploy your app az containerapp create \ --name my-agent-api \ --resource
Brian Spann
2026-05-28 11:34
👁 11
查看原文 →
Dev.to
Vibe Coding Is Fun Until Production
🚀 The Golden Age of “Just Ship It” A few months ago, I started building side projects differently. Instead of: Planning architecture Reading documentation Writing every function manually I started doing this: “Build me a responsive dashboard with authentication, dark mode, PostgreSQL integration, and Stripe payments.” And somehow… It worked. AI tools can now generate: APIs UI components Database schemas Docker configs Tests Documentation We’ve entered the era of vibe coding . And honestly? It feels amazing. What Is “Vibe Coding”? Vibe coding is when you: Describe what you want Let AI generate most of the implementation Keep iterating through prompts Instead of engineering every detail manually, you steer the vibe of the application. Tools making this popular: Cursor GitHub Copilot Claude Windsurf ChatGPT Replit AI You become less of a code writer and more of a: reviewer editor product thinker debugger At least in theory. The First Few Days Feel Like Magic The productivity boost is unreal. You can build in hours what used to take days. Things that once required: Stack Overflow endless documentation tabs debugging sessions at 2 AM …now happen through prompts. You feel unstoppable. Then Production Arrives And production is where the vibes end. Because production doesn’t care if the demo looked cool. Production cares about: edge cases reliability security scalability maintainability observability This is where AI-generated code starts exposing cracks. Problem #1: The Code Looks Right This is the dangerous part. AI code is often: clean formatted nicely modern-looking confident But hidden underneath: unnecessary complexity duplicated logic subtle bugs bad abstractions Problem #2: Hallucinated Architecture AI is very good at generating: components snippets isolated features It is much worse at: long-term architecture consistency scaling systems over time You start noticing: 4 different API patterns duplicated utilities random folder structures inconsistent state management
Sripadh Sujith
2026-05-28 11:32
👁 6
查看原文 →
Dev.to
NVIDIA CUTLASS: High-Performance CUDA Templates for AI Linear Algebra
If you've trained a transformer in the last three years, your GPU spent most of its wall-clock time inside a matrix multiplication. The kernels doing that work were probably written by cuBLAS, generated by a compiler stack like Triton, or hand-assembled on top of NVIDIA's CUTLASS templates. CUTLASS is the one most people don't see directly, but it sits underneath a surprising amount of modern AI infrastructure — from FlashAttention to vLLM to several internal kernels inside PyTorch. What CUTLASS actually is CUTLASS — CUDA Templates for Linear Algebra Subroutines — is a header-only C++ template library NVIDIA publishes on GitHub under Apache 2.0. It is not a drop-in replacement for cuBLAS. cuBLAS gives you a closed-source binary with a stable API: you call cublasGemmEx and you get a tuned kernel. CUTLASS gives you the building blocks to write your own kernel, with control over tile sizes, data layouts, epilogues, and how the kernel decomposes work across the GPU's memory hierarchy. That control is the point. If you're building a custom inference engine and your projection layer needs to fuse a GEMM with a SiLU activation and a residual add, cuBLAS can't fuse the epilogue for you — you'd launch the GEMM, then a separate elementwise kernel, paying twice for global memory traffic. With CUTLASS, the epilogue is a template parameter. You write the fusion once, instantiate the template, and the compiler emits a single kernel. This is why CUTLASS shows up wherever standard cuBLAS shapes don't fit — unusual data types like FP8, custom epilogues, sparse or grouped GEMMs, attention-shaped matrix products. Anywhere the stock library doesn't have what someone needs and the performance ceiling matters, you tend to find a CUTLASS kernel. CUTLASS is not a productivity library. It is a kernel-author's library. If you're writing PyTorch model code, you'll never import cutlass directly — you'll consume kernels that were built on top of it. The audience here is people who write the ker
pickuma
2026-05-28 11:32
👁 9
查看原文 →
Reddit r/MachineLearning
Training GPT-like model on non-language series [R]
I am responsible for a research project that is supposed to train a GPT-like model (Transformer-decoder) with 100M, 250M and 500M model variants. # params ## training dataset - 750M tokens - vocabulary is ~15k to ~100k tokens (depends on tokenizer settings) - ~3% of the vocabulary is used in ~50% of the training tokens (similar to language, where most of the vocabulary is used very sparsely) ## training hyper-params - optimizer = AdamW - lr = 1e-3 (works the best compared to 1e-2 and 1e-4) - betas = [0.9, 0.95] - effective batch size = 4M tokens - epoch = 16 - warmup steps ~200 (approx 1 epoch) ## model hyper-params - 16 layers (but variants with up to 48 layers were tested) - embedding = flexible to yield 100M, 250M and 500M model - MLP size = 4*n_embd - 16 attention heads - context window = 1000 # Issue The model seems to fail to learn the basic auto-regressive behavior. It often gets stuck on generating a single token (no repetition penalty, no sampling yet). Is training GPT-like models still a black magic? Is there some trick to this? *Disclaimer*: I will add/edit the parameters above as people ask clarifying questions. submitted by /u/gartin336 [link] [留言]
/u/gartin336
2026-05-28 11:31
👁 5
查看原文 →