标签：#machinelearning

Every "run this model locally" guide tells you to grab a Q4 GGUF and move on. That advice is fine right up until you try a long-context run and your machine starts swapping. The weights are the part everyone budgets for Quantization maths is straightforward. A model's weight footprint is roughly params x bits / 8 : Quant Bits/param 12B model Quality note Q8_0 ~8.5 ~12.8 GB Near-lossless, rarely worth it Q6_K ~6.6 ~9.9 GB Very close to Q8 Q4_K_M ~4.8 ~7.2 GB The usual sweet spot Q3_K_M ~3.9 ~5.9 GB Noticeable degradation Below Q4 the loss stops being subtle. Instruction-following degrades before raw perplexity does, which is why benchmark numbers can look fine while the model quietly stops respecting your system prompt. The KV cache is the part that bites Here is what the guides skip. The KV cache scales with context length , and it is not quantized by default: kv_bytes ~= 2 (K and V) x layers x kv_heads x head_dim x seq_len x dtype_bytes The practical consequence: a model that loads in 7 GB can need well over twice that at long context. Grouped-query attention helps a lot — kv_heads is much smaller than attention heads — but the term still grows linearly with sequence length while your weights stay fixed. Two knobs matter more than picking a fancier quant: --ctx-size : do not allocate 128K if your prompts are 8K. You are reserving memory you will never touch. KV cache quantization ( q8_0 for K/V): roughly halves cache memory for a quality hit most workloads never notice. Underused. A decision order that works Start at Q4_K_M Set context to what you actually use, not the model maximum If you are still tight, quantize the KV cache before dropping to Q3 Only move up to Q6/Q8 if you have headroom left over That ordering matters: dropping to Q3 to buy context is the most common mistake, and it trades a permanent quality loss for memory you could have gotten from the cache instead. Per-quantization benchmarks and deployment notes for the Gemma 4 family are collected at ge

A few weeks ago, I set out to build a small portfolio project: a Streamlit app that could take any tabular dataset, understand something about its structure, and give honest guidance on how to model it. I called it ContextLens . I expected it to be a practical exercise in Python, machine learning, and deployment. What I didn't expect was how closely it would connect with the same questions I work with every day in my PhD research on context-aware intelligent systems. The problem I started with Most introductory machine-learning tutorials follow a familiar sequence: Load a CSV. Choose a model. Train it. Check the accuracy. What often gets skipped is the layer of judgment that should come before any of that: Is this actually a classification problem or a regression problem? Is the target so imbalanced that accuracy becomes misleading? Is that "ID" column secretly leaking the answer into your model? Are there duplicate rows, missing values, high-cardinality categories, or too many features for the number of available observations? Experienced practitioners make these judgments almost automatically. But that reasoning usually remains invisible—it sits in someone's head rather than inside the system, where another person can inspect it. ContextLens is my attempt to make that layer visible. Upload a dataset, and it profiles the data, flags structural risks—missingness, duplicate rows, likely identifier columns, class imbalance, and high-dimensional settings—and adapts its evaluation guidance to what it finds before training a single model. The point is not simply to train a model. The point is to ask whether the modelling process makes sense in the first place. Why I call it "context-aware" rather than "AI-powered" I was deliberate about this distinction, just as I have been throughout my PhD work, and it turned out to be the most important design decision in the whole project. ContextLens does not claim to be intelligent in the way a human expert is. It does not hide its

I wanted to add a background-removal tool to my site's image cluster that stayed true to the 100% client-side processing principle I already use for PDFs and image conversions. The path there was anything but linear: a library dropped over a licensing problem, a carefully chosen model that turned out more limited than expected, and a bug that froze the entire page — not just the tool — during computation. Here's the full build, including the parts that didn't work the first time. The starting problem: what's actually feasible for free? The initial idea was broad: remove backgrounds, and maybe unwanted objects too. The two tasks have very different difficulty levels. Removing objects requires inpainting — plausibly reconstructing the erased area — which in practice still means heavy generative models, impractical to run client-side with good quality on an average device. Removing a background , on the other hand, is a segmentation problem: separating a subject from its surroundings. That has much lighter models available, runnable entirely via WebAssembly with no server involved at all. So: background removal only, object removal shelved for later. The AGPL trap The first library that looked like a perfect fit turned out to be distributed under AGPL , a strong copyleft license. Free to use — but with a real catch for anyone embedding it in a public, closed-source web service: AGPL can require releasing the full source of the project that embeds it, under the same license. "Free for the end user" and "safe to drop into a closed-source commercial product" are two different questions, and it's worth answering the second one before writing integration code, not after deploying it. Before wiring any "free" AI library into a commercial project, check the exact license, not just the price tag. AGPL, GPL, and other strong copyleft licenses are fine for personal or internal tools, risky for a public closed-source product. The fix: switch to Transformers.js — Hugging Face's li

Claude Opus/Sonnet Voice Mode, Open-Weight Model Cost Savings, & GitHub AI Agent Security Today's Highlights This week's top stories focus on major commercial AI model updates, practical tools for cost-effective LLM deployment, and critical security vulnerabilities in AI-powered developer tools. Anthropic expands its multimodal voice capabilities to more powerful Claude models, while a new 'Show HN' project promises significant cost reductions with open-weight models. Claude’s voice mode is now available for Opus and Sonnet (The Verge AI) Source: https://www.theverge.com/ai-artificial-intelligence/970065/anthropic-voice-mode-claude-opus-sonnet-haiku-ai Anthropic has rolled out its voice mode capability to its more powerful Claude Opus and Sonnet models, extending a feature previously exclusive to the faster, lighter Haiku model. This enhancement allows developers to integrate advanced multimodal conversational AI into their applications, enabling real-time voice interactions with a higher degree of intelligence and nuance than previously possible. For instance, developers can now build voice agents that not only understand complex spoken queries but also provide sophisticated, context-aware responses, leveraging the deep reasoning and comprehensive knowledge base of Opus and Sonnet. This update significantly expands the potential for developers to create more natural and intuitive user experiences across various domains, from customer service and educational tools to interactive creative assistants. By making Opus and Sonnet accessible via voice, Anthropic is addressing a key demand for richer human-computer interaction, pushing the boundaries of what commercial AI APIs can offer in terms of multimodal capabilities. This move facilitates the creation of next-generation applications where seamless voice interaction is paramount, without sacrificing the underlying intelligence of the AI model. Comment: This is a huge step for building more capable voice-first applicat

When engineers talk about visual regression or website change monitoring, pixel-level diffing algorithms (like pixelmatch or Euclidean RGB distance) are usually the default solution. However, in real-world web environments, pixel-by-pixel comparisons fundamentally fail under normal user interactions and dynamic rendering conditions: Elastic Layout Shifts: A single 20px dynamic banner inserted at the top of a page pushes every subsequent DOM element down, causing 100% of the downstream pixels to fail a pixelmatch test, even if the content itself hasn't changed. Sub-Pixel Anti-Aliasing Jitter: Operating systems (macOS vs. Linux vs. Windows) render font glyphs with subtle sub-pixel anti-aliasing variations, creating thousands of false-positive pixel deltas. Semantic vs. Cosmetic Changes: Changing a single word in a paragraph should trigger a localized alert, but a minor color gradient shift in a hero image shouldn't trigger an emergency notification. At PageWatch.tech , we solved this by combining classical Structural Similarity (SSIM) , ORB Feature Alignment , and Siamese Neural Networks (SNN) for latent-space semantic comparison. In this article, I will dive into the mathematics, neural network architectures, and TypeScript implementation of our computer vision diff pipeline. 🧮 1. Beyond Pixel Comparison: Structural Similarity Index (SSIM) Unlike raw Mean Squared Error (MSE), SSIM measures visual change based on human perception across three dimensions: Luminance , Contrast , and Structure . Mathematically, the SSIM between two image windows $x$ and $y$ is defined as: $$\text{SSIM}(x, y) = \frac{(2\mu_x\mu_y + C_1)(2\sigma_{xy} + C_2)}{(\mu_x^2 + \mu_y^2 + C_1)(\sigma_x^2 + \sigma_y^2 + C_2)}$$ Where: $\mu_x, \mu_y$ are the local pixel mean intensities. $\sigma_x^2, \sigma_y^2$ are the local variances. $\sigma_{xy}$ is the covariance between $x$ and $y$. $C_1, C_2$ are stabilization constants. TypeScript Implementation of SSIM Window Sliding Below is a snippet of how

TL;DR : most production AI tasks are not LLM tasks. To triage my email, I replaced a 7-billion-parameter model with a tiny classifier in Go. The rule fits in one sentence. Rules first, a small model next, the LLM only as a last resort. The result: no GPU, sub-millisecond inference, and a cloud call that became rare. Here is how, with the real numbers. This article is for developers who put an LLM in production and pay the bill. Not a demo. Most AI tasks are not LLM tasks In 2026, the default reflex is to wire a big model into everything. A question comes in, you call the LLM. But many tasks do not need it. Filing an email under "work" or "newsletter" is classification. A problem solved for twenty years, long before LLMs. To classify is to pick a label from a short, stable list. To generate text is something else. The first job needs a small model. The second earns a big one. The rule I defend fits in one sentence. Put the LLM last. The setup I built an agent that triages my inbox. It is a daemon. It reads new messages and files each one into a category: work, notification, newsletter, promo, and a few more. Nothing secret, just my real mailbox, with years of mail. The first version handed every email to a local LLM. A 7-billion-parameter model, Qwen 2.5 7B, served by Ollama on a GPU. Ollama is a tool that runs an LLM on your own machine. It worked. But the price was heavy. A GPU on all the time. One more container to watch. And an absurd slowness for the question asked. One day I asked myself: does deciding "is this a newsletter?" really need 7 billion parameters? No. The answer sits in two or three words from the sender and the subject. So I rethought the whole thing. Three layers, from cheapest to most expensive Every email goes through three layers, in order. It stops at the first one that can answer. Deterministic rules. Instant, exact, no cost. A small model. Sub-millisecond, on CPU. The LLM. Only if the small model is unsure. The routing code fits in a few lin

Most code that calls an AI model works like a conversation: ask, wait a second, get a reply. Fine-tuning doesn't. You hand off a job and walk away, checking back every few seconds to see if it's finished. DSPy is a framework for building programs that call language models. Instead of hand-writing and endlessly tweaking prompt strings, you declare what you want in terms of inputs and outputs, and DSPy turns that into the actual prompt. It can even optimize those prompts for you automatically, so getting a better result doesn't mean rewording things by hand. Here's something I didn't know starting out: DSPy already knows how to talk to almost any AI model, Together AI included. Asking a question and getting an answer back is handled by a shared layer that works for everyone, so no new code is needed there. Fine-tuning (the "hand off a job and walk away" thing from the top) is the exception. Every company does fine-tuning its own way, so DSPy needs a small custom piece, called a Provider, to handle each one. Building the Provider for Together AI is what my PR does. Why does this matter? Together AI is one of the cheaper, more popular places to fine-tune open-source models like Llama, so a lot of people building with DSPy end up wanting to use it. Before this, they had to step outside the framework: fine-tune on Together by hand, then wire the finished model back into their DSPy program themselves. With the provider in place, fine-tuning becomes a first-class option. You point DSPy at your training data, and it handles the upload, the job, the waiting, and hands back a model you can drop straight into the rest of your pipeline. That is the whole point of a framework, taking a fiddly manual process and making it one clean step, and adding a provider is how that gets extended to one more company. The Provider does one job from start to finish: take your training examples and hand back a fine-tuned model. Under the hood, that's five steps: Check your training data is in a

We spent the last months consolidating seven separate sequence classifiers into one multi-head model, our apex model, so to speak, and since the weights are now public, I wanted to share what worked and what surprised us. Setup: a shared mmBERT-small encoder with seven task heads, binary injection (BCE), document class (7-way), tool type (14-way), tool operation (6-way), tool data-flow tags (3× BCE, multi-label), intent routing (5-way), and threat type (7-way). The part that needed care: our training rows only carry labels for a subset of tasks, so absent tasks are masked out of the loss entirely. We ended up writing a self-test that asserts absent-task gradients are exactly zero, which caught two subtle bugs, and I'd recommend it to anyone doing similar masking. About 5k synthetic/real multi-task rows help the heads co-train; the test sets stay 100 % real data. Held-out results per head: injection F1 0.962, documents 0.980, tool type 0.957, tool operation 0.945, tool tags 0.958, routing 0.916, threat 0.952. Quantization: both the unified model and the dedicated single-task variants ship quantized -edge builds (ONNX INT8 + INT4 embeddings, from 96 MB) with measured parity benchmarks in the repos, the worst head loses 0.012 against FP32. Was it worth it vs. seven dedicated models? We released both variants, so you can judge for yourself, the dedicated models score marginally higher on most tasks, but the unified one does one encoder pass instead of up to seven. Our weak spot: routing, at 0.916. The intent classes overlap semantically ("write code that analyzes my data" is that code or analytics?), and I suspect the ambiguity is genuinely in the data. If you have ideas beyond relabeling, let me know :) Weights and per-head metrics: https://huggingface.co/patronus-studio submitted by /u/PatronusProtect [link] [留言]

Hey everyone, I'm not in college anymore. Is it possible to do research with a professor or any research lab while working a full time job? If yes, what's the best way to reach out and get involved? Also if anyone looking for someone to work with on a research project or something similar, can dm me. submitted by /u/VastThen1742 [link] [留言]

Hey all! Is anyone else going to be at KDD in Jeju? Would love to connect with fellow attendees. I work on interpretability, fairness, and editing of text-to-image models, so I'd especially love to meet people working in these areas. But honestly, we can chat about anything: research, the conference, life, or just grab a coffee/drink. I land in Jeju on the night of the 8th, so hmu if you're around and want to link up! submitted by /u/Deep-Inevitable-1977 [link] [留言]

- Tutorial: https://ordinaryintelligence.substack.com/p/how-to-build-an-ai-slop-detector - Notebook on GitHub: https://github.com/Buzzpy/Python-Projects/blob/main/AI-slop-detector.ipynb submitted by /u/gamedev-exe [link] [留言]

Reviews are released! Lets discuss them here! submitted by /u/Forsaken-Lab-7010 [link] [留言]

...may the odds be in your favor. On a more serious note, as an Area Chair for Neurips, I can tell the incentives that they placed this year are kinda working (risk of rejecting a reviewer's paper if they are not being responsible). I've had the least number of reviewers to chase/emergency reviewers to recruit since I've started ACing for major conferences (so maybe 5ish years). Hopefully, reviewers will also be active in discussions... submitted by /u/GuestCheap9405 [link] [留言]

I'm training an AlphaZero-style agent (Gumbel MuZero via DeepMind's mctx ) to lay out working factory modules for Factorio: the network places machines, belts and inserters on a grid, and the reward comes from an exact throughput verifier. Everything runs in JAX on a single RTX 5070: 128 environments in one batch, an action space of A = 1729 (3 entity types × 144 cells × 4 rotations + "done"), and a small 474k-parameter conv net in bf16. While benchmarking training configurations I hit this: MCTS simulations per move training throughput XLA compile time 16 143 episodes/s 5 s 32 47 episodes/s 13 s 64 9 episodes/s 60 s Doubling the simulation budget should roughly double the cost — each simulation is one network call plus some tree bookkeeping. Instead, 16→32 costs ×3 and 32→64 costs ×5 . Something in the search was superlinear, and this post is the story of finding it in the compiled HLO and fixing it by rewriting one ~80-line function ( PR #116 ), with bitwise-identical search results. Ruling out the network First, components in isolation (batch 128): one network evaluation takes ~0.8 ms , and the rest of recurrent_fn (environment step + observation + legal-action mask) adds almost nothing on top — the whole function is also ~0.8 ms. So at 64 simulations the network accounts for roughly 50 ms per move. But a full policy step at 64 simulations costs 362 ms . Hundreds of milliseconds were going somewhere else. To localize them I benchmarked three variants of the same policy step: full — production setup; no-net — network replaced by constant logits, real environment; tree-only — no network and no environment: recurrent_fn returns the embedding unchanged. Nothing left but mctx's own tree machinery. sims full no-net tree-only 8 10.7 ms 3.7 ms 3.8 ms 16 20.9 ms 8.3 ms 8.3 ms 32 64.7 ms 34.0 ms 33.7 ms 64 362.1 ms 124.7 ms 125.0 ms The pure tree machinery is superlinear all by itself. Per simulation it costs 0.47 → 0.52 → 1.05 → 1.95 ms as the budget goes 8 → 16 → 32 → 64

When choosing a university for a masters in ML/DL, what is more important if someone wants to go into research and an eventual PhD. Is it the ranking/prestige factor of the university or the strength of the research groups in the university? Should an admission decision be made hoping that I will get to work with X/Y professor or lab? submitted by /u/Hot_Version_6403 [link] [留言]

Reviews drop today. This thread is for reactions, celebrations, commiserations, and anything useful in between. First: if you got good reviews, say so. There's a norm in these threads where only the bad news gets aired, and it skews everyone's sense of what's normal. Post your wins. Second , the thing worth repeating every cycle: the review process is noisy, and that noise is measured, not folklore. The NeurIPS consistency experiments (2014, repeated 2021) found that a large fraction of accepted papers would have been rejected by an independent second committee. Reviewer assignment, load, and luck of the draw account for a lot. A score is a weak signal about your work and a strong signal about the process that produced it. That cuts both ways. It's not a license to dismiss every criticism as noise — it's a reason to weight reviews by the quality of the argument rather than the number attached to them. The reviewer who found a real hole in your evaluation did you a favor, even if the tone was rough. The one who clearly skimmed did not, regardless of the score. So: prioritize the reviews that make the paper better. Fix what's fixable, contest what's genuinely wrong, and concede the rest gracefully in the rebuttal. Things worth discussing: Reviews that caught something you'd missed Rebuttal strategy — what's worth contesting vs. conceding, and when new experiments actually shift a score Patterns you're seeing this cycle (missing baselines, compute comparisons, ablation depth, reproducibility asks) Framing a response when a reviewer has clearly misread the submission Backup plans: ICLR, AISTATS, workshops Please paraphrase rather than paste review text, and no speculation about reviewer or AC identities. To anyone who got bad news: this doesn't define your research impact. Plenty of heavily-cited work took two or three cycles to land somewhere. Rejection is a scheduling problem. How did everyone do? submitted by /u/Afraid_Difference697 [link] [留言]

Paper: https://arxiv.org/abs/2607.19058 Code (GitHub): https://github.com/nuemaan/skewadam Hi everyone, I just published a preprint on a new optimizer designed to tackle the massive VRAM bottleneck in Mixture-of-Experts (MoE) training. If you've trained MoEs, you know that optimizer state is usually the largest single line item in the memory budget. AdamW, for example, spends 50.6 GB of state memory just to update a 12.6 GB model. I built SkewAdam to fix this by using a tiered state allocation . Instead of treating all parameters equally, it allocates precision based on parameter behavior: Backbone (5% of params): Momentum + Factored 2nd moment Experts (95% of params): Factored 2nd moment only Router (<0.01% of params): Exact 2nd moment The Hardware Results: Optimizer state memory drops from 50.6 GB to 1.29 GB (a 97.4% reduction). Peak training memory drops from 81.4 GB to 31.3 GB. This allows a 6.78B MoE to fit comfortably on a single 40GB GPU without sacrificing convergence or router stability. submitted by /u/Kooky-Ad-4124 [link] [留言]

As I was reading interp papers, I found myself copy-pasting passages back and forth to Claude to parse through them. Eventually just vibe-coded a tool to annotate and discuss papers in place. https://paper-reader.dev - select a passage, a formula, or a figure, and explain the selection with the full paper as context. You can also select a citation to get a brief overview of the cited paper without switching context. Repo is at github.com/tumanian/paper-reader if anyone curious (mostly Claude, some Cursor, some me - built on vercel and supabase). Please be gentle, this runs on my own API key with a modest cap, so don't be too enthusiastic. Hoping this can be useful to someone, and genuinely looking for feedback, especially on where the explanations are wrong or unhelpful — that's the part I can't fully self-evaluate. submitted by /u/tumanian [link] [留言]

Every team shipping an agent has the same meeting. Someone asks "is it ready?" and the room splits. One person saw a great demo. Another watched it invent a refund policy an hour ago. The argument runs in circles because nobody agreed what "ready" means, so the loudest opinion wins and the agent ships on a vibe. Making an AI agent production-ready is not a moment of confidence. It is a bar you write down before you build, then measure against. This post is about that bar: why agents need a different one than the services you already ship, and how to define it so "is it ready?" becomes a number instead of an argument. Why "production-ready" breaks for agents For a normal service, "production-ready" is settled. Correct output for valid input, handles errors, meets a latency target, has tests and a rollback. You know the shape of done. An agent breaks three of those assumptions at once: It is non-deterministic. The same input can produce different output, so "correct" becomes "acceptably right, often enough." Its failure surface is open-ended. A function fails in ways you enumerated; an agent fails in ways you never imagined, because it composes language, tools, and judgment on the fly. Its worst case is not a 500 error. It is a confident wrong answer that looks right, which is far more expensive than a crash, because a crash at least tells you it failed. So the honest question is not "is the agent correct." It is "is the agent acceptably wrong, safely, within budget, and repeatably enough to trust." That question has four parts, and each is a line on your bar. The four lines of the bar Write these down before you build. If you cannot fill them in, you do not have a spec, you have a wish. Task success. On a fixed set of real tasks , not the happy-path demo, what fraction must the agent complete correctly? Pick the number. 85 percent means one in seven users gets a wrong answer. Acceptable for this job, or fireable? Decide on purpose. Failure acceptability. Not all wron

I've been building an AI that learns to play the classic Snake game through reinforcement learning. The goal is to reach high scores while keeping training time as low as possible. The current version averages 86 points (87 is the maximum) after less than 10 hours of training on a single free Google Colab T4 GPU. To keep training fast, it runs 4,096 Snake games directly on the GPU, combines GPU-native environment simulation with PPO + GAE, and uses a spatially-preserving CoordConv architecture that maintains the full game grid throughout training. I'm sure there's still room to improve. If you've worked on reinforcement learning or efficient training systems, what would you try next? Better exploration, reward design, network architecture, or something else? Repository: ( https://github.com/siddhartha399/PPO-CoordConv-Snake ) I'd really appreciate any feedback or criticism. submitted by /u/Due_Highlight_9341 [link] [留言]