标签：#learning

A lesson in overcorrection, from an AI-assisted builder who is scared stiff of shipping spaghetti and...

I have submitted a paper to acml 2026 journal track, the official date of release of review is 27 August, but I have not heard anything from them, if anyone received the review then let me know I will write to program chairs. Thanks submitted by /u/Jealous_Key_4030 [link] [留言]

Sliding Window Attention with sinks, one of the simplest existing fixes for the quadratic-cost problem in LLMs, holds up as well or better than the linear-attention variants labs have been spending post-training compute to produce. That is the claim of a [new arXiv preprint]( https://arxiv.org/abs/2608.28444 ) by Alexia Jolicoeur-Martineau, Rhea Sanjay Sukthanker, Pashmina Cameron and Emy Gervais. On the long-context reasoning benchmarks the paper singles out, the gap is not close. "SWA achieves massively higher performance (2 to 10 times higher than linear attention)," the abstract reports, naming Needle-in-a-Haystack and BABILong as the two tasks. The pitch is that the whole post-training-to-linear pipeline has been benchmarked against the wrong thing. "This line of research has not been properly compared to simpler baselines," the authors write. Their alternative needs no post-training, runs fast, and holds memory low. The recommendation is blunt: "we strongly recommend switching to SWA instead of post-training linear models." Linear attention, the abstract concedes, "may have shown some promise, but they likely require to be trained from scratch or extensive post-training in order to even match SWA." --- submitted by /u/Justgototheeffinmoon [link] [留言]

We noticed our anti-money laundering models were performing suspiciously well. After digging into standard baselines on dynamic graphs, we found widespread temporal leakage in message-passing. If you train a GNN on a static snapshot of a dynamic graph, your model is likely cheating by seeing future edges during training. We got sick of reviewing papers with broken evals, so we released SynthFin-AML v10.0 (100k nodes, 1.2M edges) to force strict causal boundaries. The Temporal Leakage Trap Standard transductive random splits fundamentally fail on financial transaction networks because they violate the arrow of time. If Node A sends funds to B on Day 2, and B to C on Day 10, a standard 2-hop GNN will pull the Day 10 edge into the loss calculation for Day 2. The model literally looks into the future to compute embeddings. The Fix: 3-Snapshot Architecture To stop the model from cheating by looking ahead at the transaction graph, we enforced a strict 3-snapshot point-in-time split: Train Graph (Edges ≤≤ Day 7) Val Graph (Edges ≤≤ Day 8) Test Graph (Edges ≤≤ Day 10) By physically disjointing the temporal windows, we bound the receptive field of the GNN to the true causal horizon. Graph vs Tabular Reality Check Most synthetic datasets suffer from distribution leakage, where fraud transaction amounts are statistically separable from normal retail traffic. We killed the "amount split cheat" by ensuring fraud and retail transaction amounts share the exact same lognormal distribution (μ=8.517,σ=0.8 μ =8.517, σ =0.8). With tabular leakage fixed, we benchmarked a tuned LightGBM against GraphSAGE to see if the GNN overhead actually pays off for AML. We engineered 11 point-in-time graph features (Weighted PageRank, neighbor volume aggregates) for the tree model. Results (PR-AUC on strict temporal split): LightGBM (11 features): 0.848 GraphSAGE (Inductive): 0.881 Spoiler: GraphSAGE barely beats trees here unless your edge features are incredibly dense. The gap isn't astronomical, b

Have you ever wondered why we keep learning advanced things that probably might not be applied properly where we came from? As someone who came from a developing country where resources are not being served on a gold plate. In fact, even if you have all the necessary knowledge to make a change but one thing comes up with no answer, how can we implement our knowledge gained abroad with no funding and no equipment to help us contribute to the blooming of our beloved country? I guess our parents worked hard to actually send us abroad, not to return to our country but instead to find a way to make a living where God destined us to go. This is not because they hate our homeland, but they feel there is no way things can change where corruption and unemployment reign to a high degree. Therefore, this is the time where advanced technologies must not be seen as burdens in a developing country. Share answers on this particular spec. Thanks!

RamaLama runs large language models as OCI containers, so a single command ( ramalama run smollm:135m ) pulls a model and starts talking to it, with no Python environment to babysit. I spent an afternoon putting it through its paces on an Apple Silicon Mac (Apple M4 Pro, 48 GB RAM, macOS 26.6) with Docker 29.4 provided by OrbStack. This guide is what I actually saw: the install, the first model, an OpenAI-compatible server, and the one macOS-specific catch that isn't obvious from the docs. Every command and number below is from that run, on RamaLama 0.24.0. What is RamaLama? RamaLama is an open-source CLI from the container-tooling community that treats models like container images. Instead of assembling an inference stack yourself, it pulls a hardened OCI image containing llama.cpp (or vLLM/MLX) plus your chosen model and runs it with Podman or Docker. If you've used Ollama the ergonomics feel familiar ( run , serve , list , pull ), but the runtime and model live inside containers you can inspect and sign, and weights come straight from Hugging Face, Ollama, or any OCI registry. Installing RamaLama on macOS With Homebrew it's one command: brew install ramalama That pulled RamaLama 0.24.0 and, notably, its own copy of llama.cpp , ggml , and libomp as dependencies. Hold onto that detail; it matters for GPU acceleration later. Confirm the install: ramalama version # ramalama version 0.24.0 You also need a container engine running. I used Docker through OrbStack; Podman works too and is RamaLama's default on Linux. Running your first model The headline command: ramalama run smollm:135m "In one sentence, what is a Linux container?" Passing a prompt as an argument gives you one-shot output instead of dropping into a chat REPL. On first run this pulled the RamaLama container image, downloaded the model, and answered. smollm:135m resolves to hf://HuggingFaceTB/smollm-135M-instruct-v0.2-Q8_0-GGUF , a 138 MB, 8-bit quantized GGUF from Hugging Face. First-run wall-clock was 2

This is the time of year when the number of cold emails I receive about PhD positions tends to ramp up quite a bit. In many countries, this cold emailing is essentially part of the normal recruitment process, so there is nothing inherently wrong with doing this. However, there are a few things you definitely shouldn't be doing: Massive emails. The probability of me reading your email is inversely proportional to its length. Emailing everyone. Find supervisors that work in areas you are actually interested in. I do relatively foundational ML research (i.e., not associated with a specific application domain), but the majority of emails I get from prospective students are essentially "I want to apply ML to domain X". In many cases this does not constitute an ML research direction; you'd be better off finding a supervisor with expertise in domain X, which is where most of the impact will be. Generic research interests. If the most specific research interests you can give are "Machine Learning, LLMs, and AI" then I assume you only have a surface-level familiarity with the field, and are not ready for a PhD. Passing off workshop papers as conference papers. This has become a much more common thing in the last couple of years. It's a big red flag; I am not going to take on someone who is dishonest. Excessive AI use. Using them for fixing up grammar is fine, but if you outsource your thinking to LLMs then your research direction will be the same as everyone else who outsources their thinking to LLMs. This tends to result in something that would be an okay bachelor's thesis project, but nothing more than that. I get a lot of LLM emails, so determining if you are in this cluster is very easy. Summarise my paper. I already know what's in it, I don't need a summary. I care more about how you think you could build on it, or do something related. Don't use LLMs for this; see above point. Ignoring instructions on my website. Check prospective supervisors' websites for how you shou

I'm sharing this new tabular data diagnostic tool (Entropic Scree). It can be used to estimate these properties of your high-d, real-world, dirty dataset: The informational volume of the signal (i.e., helps you assess whether the signal is strong enough to survive the dataset's idiosyncratic volume). The overall signal-to-idiosyncratic volume ratio (SNR). The intrinsic rank. Provides an exploratory map that allows for the identification of decoupled sub-networks of variables. The linear sufficiency (i.e., does the dataset align with the linear assumptions of standard PCA?). Instead of evaluating linear variance, rank order, or Euclidean distance like traditional PCA variants, this new method evaluates a transformed mutual information metric. Relative to these baselines, it is less reliant on strong parametric or distance assumptions, making it appropriate to apply more broadly. It also serves as a practical diagnostic of the theory explored in the From Garbage to Gold framework, which describes when and why uncurated, error-prone data can be used directly to create accurate prediction models. There is a preprint that presents the full technical details, and Python and R packages will be released soon. Right now though, the original function is already available in R (see Quick Start R Function Code below). Entropic Scree Preprint: https://doi.org/10.5281/zenodo.22028087 Entropic Scree GitHub: https://github.com/tjleestjohn/entropic-scree From Garbage to Gold Preprint: https://arxiv.org/abs/2603.12288 Let me know how it goes if you give it a try... or if you have any questions or comments of course. ############ # Quick Start R Function Code. # To load the function, copy and paste the following into your R console, then hit enter. ############ # 1. Define the direct URL to the raw function script on GitHub url <- "https://raw.githubusercontent.com/tjleestjohn/entropic-scree/main/Entropic.Scree.v1.0.0%20-%20ENLI.R" # 2. Define what you want to name the file on your co

Hi, I'm making posters for ECCV 2026. Does anyone have any ML/CV posters they thought were really well done? Would love to see some cool examples. Thanks submitted by /u/National-Resident244 [link] [留言]

Paste a long document into ChatGPT and hit enter. Nothing happens for a second or two. Then the answer starts appearing, word by word, at a steady pace until it finishes. You have seen this hundreds of times. Most people never think about it. But those are two completely different things happening inside one request, running on two different parts of the same GPU, limited by two different bottlenecks. Once you know what they are, a lot of confusing things about serving AI models stop being confusing. Including why a faster GPU sometimes makes no difference at all. This is part two of a series. Part one covered the split between VRAM and cores. This post covers what each of them does when a request arrives. The example we will follow I am going to trace one question the whole way through: what is the capital of france Simple, short, and it produces a short answer. Perfect for watching the machinery. Before anything happens, the model server has already loaded the model's weights into VRAM. That happened at startup and they stay there. Nothing about your request loads a model. Your words get chopped into tokens, which are just pieces of words turned into numbers, and placed in VRAM alongside the weights. Now the work begins. Phase one: prefill, or "the pause" Before the model can write anything, it has to read everything you gave it. Here is the important part, and it is the opposite of what most people assume: it reads your entire prompt at once. Not word by word. All six words go to the cores together, in a single pass. That is possible because your whole prompt is already there. Nothing has to wait for anything else. So thousands of cores fire simultaneously and chew through all of it in one burst. flowchart LR A["what is the capital of france<br/>all 6 tokens together"] --> C W["16 GB of model weights"] --> C C["GPU CORES<br/>every core busy"] --> O1["First word: 'The'"] C --> O2["Notes saved to VRAM<br/>the KV cache"] Two things come out of that single pass: The

When I first started learning HTML, I thought it would be the easiest part of web development. The basic tags seemed straightforward, and creating a simple page didn't take long. But as I kept learning, I realized that HTML is more than just putting elements on a page. Understanding semantic HTML , proper document structure, forms, accessibility, and knowing which element to use in different situations takes practice. For example, it can be tempting to use for almost everything, but learning when to use elements like <section>, <article> , <nav> , or <header> makes a big difference in writing meaningful markup. I'm curious about other developers' experiences: What HTML concept confused you the most when you were starting out, and what helped it finally click? I'd love to hear different perspectives, especially from people who have been working with HTML for a while.

After spending the past two weeks redoing all the models around the house, I realized it might make a good topic to chat about. I know that everyone and their brother has their own way to figure out what models they want to run on their hardware, but I figure that my own criteria might help some of the Mac users out there, so I'm tossing it into the mix as well. Picking which models to even compare When a new model comes out, the first thing I always do is check what folks are saying: huggingface discussions, reddit comments, etc. Benchmarks are useful, but I want to know what happens when people actually use the thing. Is the tokenizer broken or llama.cpp/mlx implementation bugged? Does it follow instructions? Overthink? Hallucinate a bunch? Discussion comment sections are a treasure trove of info. After that, I go peek at the model on Artificial Analysis. I know AA isn't everybody's favorite way to judge models, but honestly it has a pretty solid litmus test for whether the model will be good for me or not. In particular, there are a specific few benchmarks there which line up really well with what I need: strong context reasoning hallucination rate how many output tokens the model used to get its scores That last one is one of the most important. Combined with the "Humanity's Last Exam" score and overall intelligence, it gives me an idea of how much output the model had to produce to actually land on its current spot on the leaderboard. On a Mac, I really care about that. Remember: our compromise with Macs is getting stupidly large amounts of VRAM at the price of everything being a lot slower than NVidia GPUs, especially as context and token generation sizes increase. So if one model gets a slightly better score by generating dramatically more tokens, you gotta keep that in mind. Really long-winded thinking sessions could make a model almost unusable. You might think "I don't mind waiting for quality", but then suddenly find yourself just going to ChatGPT because

First of all, I warmly welcome everyone out there in the DEV Community. [Completely open for discussion — drop your thoughts below.] From my perspective, it feels like everyone is racing towards AI/ML. The moment someone says they want to become an AI/ML Engineer, the conversation immediately shifts towards: Python → ML → Deep Learning → LLMs → Latest AI Tools And thinking about it, well, it’s quite understandable too. AI is one of the most exciting areas in technology right now. BUT, I have a question… Why are we starting to treat AI/ML Engineering as something completely different from Software Engineering? I often see people following an extremely narrow path towards AI/ML while completely skipping the fundamentals of Software Engineering. Backend development gets ignored. Databases, networking, operating systems, system design — all of them get ignored. And afterwards: APIs, deployment, testing, distributed systems… All of these seem quite trivial, right? Because the end goal is simply to create or automate something with AI. But it’s quite clear to me that AI can’t possibly live by itself. For any AI model to thrive, we need data. That data needs storage and pipelines. A model needs an application around it. That application needs APIs. Those APIs need backend infrastructure. And now we have an actual system. That system needs to be monitored for bugs, optimized for CPU and memory efficiency, refactored when necessary, maintained over time, and tested against new use cases. So thinking about all of this: How does one even fathom becoming an AI/ML “Engineer” without understanding what they are actually engineering into and working on? Maybe AI/ML Engineering and Software Engineering aren’t two completely different entities. Maybe they are different components of the same system. Now, I’m not saying: “You should become an expert in everything.” Specialization is indeed important. But specialization doesn’t necessarily mean abandoning the fundamentals that the spe

For Job Postings please use this template Hiring: [Location], Salary:[], [Remote | Relocation], [Full Time | Contract | Part Time] and [Brief overview, what you're looking for] For Those looking for jobs please use this template Want to be Hired: [Location], Salary Expectation:[], [Remote | Relocation], [Full Time | Contract | Part Time] Resume: [Link to resume] and [Brief overview, what you're looking for] ​ Please remember that this community is geared towards those with experience. submitted by /u/AutoModerator [link] [留言]

You've probably seen the term RAG everywhere lately — "RAG pipeline," "RAG chatbot," "build your own RAG app." It sounds complicated, but the idea behind it is actually pretty simple. In this article, I'll explain RAG in plain language, then walk through how it works using a real project I built: Guidely , an internal knowledge assistant that answers questions using a company's own documents. The Problem RAG Solves Large language models (like GPT or Claude) are trained on a huge amount of general knowledge, but they don't know about your specific data — your company's internal docs, your product manuals, your onboarding guides. They also can't be retrained every time a document changes; that's slow and expensive. RAG solves this without retraining the model at all. Basically: RAG means: before answering a question, first go find the relevant pieces of your own documents, and hand those to the AI along with the question. That's it. "Retrieval" (finding the right information) + "Augmented Generation" (the AI answers using that information). Instead of the AI answering from memory alone, it answers using facts you hand it in the moment. The Three Core Pieces Let's break down the three things you need to make this work: chunking , embeddings , and vector search . 1. Chunking — Breaking Documents Into Pieces You can't hand an AI model an entire 200-page document and ask it to search through it efficiently. So the first step is splitting documents into smaller, manageable pieces called chunks . In Guidely, I used a token-window chunker — it splits text based on a fixed number of tokens (roughly, pieces of words) per chunk, rather than just splitting by paragraph or sentence. This matters because: Chunks that are too big waste space and slow things down. Chunks that are too small lose context and produce confusing answers. A token-window approach gives you consistent, predictable chunk sizes, which makes the next steps more reliable. 2. Embeddings — Turning Text Into Numbe

Third-year PhD student, NLP / interpretability. I want a reality check from people doing similar work. I started using Claude Code for the boring parts: argparse boilerplate, plotting, config wrangling. Over the last few months the scope has crept. It now writes most of my experiment scaffolding, refactors my dataloaders, does first-pass debugging on training runs, and drafts the analysis scripts. I mostly read diffs and say yes. The output is fine. My throughput is up. The thing bothering me is that I no longer hold my own codebase in my head. When a result looks off, I used to have an instinct about which line was lying to me. Now I go hunting like it’s someone else’s repo. I catch bugs later than I used to, and I catch them by reasoning about the numbers rather than by knowing the code. I don’t think the tool is the problem. I think I delegated a layer that was doing more for my understanding than I gave it credit for. Questions for people further along or in the same spot: Roughly what fraction of your research code do you write yourself now? Is there anything you deliberately refuse to hand off? (For me I think the eval harness and anything defining a metric should stay mine, but I keep breaking my own rule.) Does anyone have a workflow that keeps the speedup without the detachment? Reading the diff line by line is not cutting it. Not looking for a “tools are just tools” answer. I’m asking about the specific feeling of not owning your own experiments anymore. submitted by /u/NeatFox5866 [link] [留言]

In the last two blogs, I shared how AI failed to solve a few issues in programming and the value of self-search; today, I am going to share the opposite. The main goal is to show how you can learn from AI and use it as effectively as possible. This all started when I was using trigger.dev and got the following error: Node.js 21 detected without native WebSocket support. Suggested solution: For Node.js < 22, install "ws" package and provide it via the transport option: import ws from "ws" new RealtimeClient(url, { transport: ws }) using trigger.dev The error clearly asks me to either install the ws package or update Node.js. But since I did not have a full experience with trigger.dev I could not figure out how to do that. My approach to debugging is based on methods: Checking resources (AI and Google) Following instincts With this error, I went with AI first, asking ChatGPT about it; then I tried Googling it (which used to work before the AI age), but I could not find any data. With ChatGPT, I gave it two extra points to help it get the right answer; I shared that I am using trigger.dev, added the web resources, and asked for a solution based on my tech stack. By giving ChatGPT context and a web search, it was able to find the config page on trigger.dev and get the results I wanted. With this experience and the ones I had before, the most important thing when using AI was the context and knowledge I had to provide. The in-depth knowledge can help the user and AI to find the optimal solution, yet going blindly might lead you to a black hole without knowing how to return.

I found this GitHub link, and the HTML file contains ~7k papers. Some are anonymized, and the details seem pretty accurate. It looks like these might actually be the accepted papers. https://github.com/xll0328/NIPS26- Can someone confirm whether this list is legit? I’m hoping it’s just a coincidence since it seems way too early. submitted by /u/Feuilius [link] [留言]

As I have started my Data analysis and science curriculum, Microsoft excel is the first tool we have learnt. In my life before getting into this course, I used to underestimate the power of Microsoft excel. Definition of Microsoft Excel This is a spreadsheet software developed by Microsoft that allows you to collect, organize, analyze, calculate and visualize data efficiently. The Excel Interface When open Microsoft excel, you first an interface made of various tools We start with the ribbon. The ribbon contains a toolbar that contains all the commands organized into tabs such, Home, Insert, Page Layout , Formulas, Charts and Data There is a formula Bar that content and formulas appear. In addition, we have Quick access toolbar that contains icons for Save, Undo and Redo. A row is a horizontal line of cells that runs from left to right across the grid while a column is a vertical line of cells that runs from top to bottom down the grid. A cell is where row and column intersect and form a single box. Cleaning A Dirty Dataset As we learnt about Data cleaning with Excel on dataset that was a bit very disorganized. We first started with sorting that data which basically means arranging the data in specific order. The sorting can be Text sorting which is A-Z or Z-A, Number sorting which is from largest to small and vice versa, Date sorting which is Oldest to Newest and vice vera. Next was filtering which allows you display data that meet certain criteria and hide the rest temporarily. Next, Number formatting changes how numbers display without altering their actual value. Conditional Formatting highlights cells automatically based on rules or criteria, helping spot trends or outliers. The dataset we were cleaning had a lot of duplicates, so it was important for us to remove the duplicates. Text functions for Cleaning Data Text functions are used to remove extra spaces , standardize text ,extract parts of strings or find and replace sub strings. text functions include UPP

Changelogs and pricing pages ship the model. Your feed just argues about it later. I used to treat AI Twitter like a release channel. Bad idea. The timeline is commentary. The drop is usually a quiet line on a docs page. Last month a new model ID showed up on an API pricing table before anyone I follow wrote a thread. I was not clever. That URL was already on a watch. The feed still spent the afternoon debating vibes. What belongs in a real brief When a lab ships, I want five boring facts: The model name and the ID your code will call Price per million tokens (input and output) Context window and any rate-limit changes Deprecations or aliases that reroute old names Where it lives (API only, chat app, open weights, or all three) A launch blog is optional. Those five lines are the brief. Where the news actually appears Social posts trail the docs. I keep pages, not accounts. OpenAI: API changelog, deprecations, pricing. Anthropic: news, platform release notes, pricing. Google: Gemini API changelog and pricing. Open weight: Hugging Face org pages I actually deploy from. Discord is faster for some open-weight labs. Fine. I still want the model card and the price before I rewrite a prompt. Monday rituals die by Tuesday I tried opening three changelogs every Monday. Skim. Close tabs. Feel responsible. It works until a midweek price cut or a silent alias change. Then you learn from an invoice spike or a broken eval. Google Alerts on "new GPT" or "Claude release" is noise. You get essays, not the SKU. Screenshot watchers catch layout shifts on marketing pages. Sometimes useful. I usually need the sentence that changed on the pricing table. What I leave running I paste the docs URLs I already trust into a website change alert and ask for a one-line brief: new models, price cuts, deprecations, alias moves. AyeWatch is what I use for that. Free Preview is $0 (3 topics, 6 lifetime runs). Pro is $9 a month. When something fires, I get a short summary, open the page, and copy the