标签：#machinelearning

Somewhere in the DataLens build, my teammate and I hit the wall every "zero-dependency" project eventually hits: the anomaly detector needed a neural net, and the rulebook said no third-party packages. No NumPy. No pandas. No scikit-learn. Just Python 3.14's standard library. Our first reaction was denial. You cannot build an ANN without a matrix library — everyone knows that. numpy.dot() is basically load-bearing infrastructure for machine learning in Python. We spent an embarrassing amount of time trying to convince ourselves some obscure math submodule secretly did vectorized linear algebra. It doesn't. There is no shortcut. If you want matrix multiplication in pure stdlib Python, you write nested for loops and you like it. What we normally would have installed In any other project, this is a two-second decision: pip install numpy , import it, move on with your life. Matrix ops, broadcasting, vectorized activation functions — all free. Neither of us had ever really had to think about how A @ B works under the hood, because neither of us had ever had to write it ourselves. What it actually took to replace it An autoencoder needs: matrix multiplication, transpose, element-wise activation functions (sigmoid, ReLU), and gradient computation for backprop. Without NumPy, every one of those is a hand-rolled function operating on nested Python lists. Matrix multiply becomes three nested loops instead of one line. A forward pass that would be a single .dot() call turns into a small file of helper functions: matmul() , transpose() , add_bias() , sigmoid() , sigmoid_derivative() . We split it — one of us built the forward pass and activation functions, the other took backprop and the training loop — and then spent a good while debugging the seam where the two met. The genuinely hard part wasn't the math — it was performance. Pure Python loops over lists of lists are slow, and profiling a dataset with a few thousand rows through even a small autoencoder made that obvious fas

How to get rejected by IEEE T-PAMI with 'Excellent' scores?[D] submitted by /u/cussealin [link] [留言]

Hi everyone! We just released Rustuna (GitHub: https://github.com/optuna/rustuna/ ), a high-speed, memory-efficient implementation of Optuna built in Rust. Optuna-Compatible Design: Keeps the familiar API and concept of Optuna. Zero Python Dependencies: Mitigating the risk of supply chain attacks. Lower Memory Footprint: Optimized memory management natively in Rust. For details, please check out the following blog post. https://medium.com/optuna/announcing-rustuna-cc82a6815bf7 submitted by /u/c-bata [link] [留言]

Our research team has been exploring an alternative approach to achieving interactivity and better responsiveness with LLM systems. One of the team members wrote up a post about it: https://research.yandex.com/blog/the-kv-cache-as-an-agent-runtime The post sums up the overall idea of modifying models inference state (KV-cache) for achieving a more interactive LLMs. This idea was used in our lab's previous papers Hogwild! Inference , and AsyncReasoning , the post also contains a preview of the future work in this direction, where a Qwen3.8-27B agent is playing a DOOM env interactively using similar techniques. We think that its interesting whether model inference/runtime design is itself an under-explored axis of agent capabilities, alongside models and the harness (e.g. harness is too abstract, changing model is too costly, do we need something in between?) submitted by /u/_puhsu [link] [留言]

Hello all, I'm a radar signal processing engineer and i trained a 5-class classifier (car, large_vehicle, two_wheeler, pedestrian, pedestrian_group) on RadarScenes radar point clouds. The input vector is a per-scan histogram (16 bins) and the network is a 3-layer MLP. The loss function is a class-weighted cross-entropy loss. This work is based on "Histogram-based Deep Learning for Automotive Radar" paper. I scoped the project to be one scan only. Accumulation of multiple scans is the next step. Data Class Imbalance: two-wheelers and large_vehicles has a low number of occurences. Aggregated Classes: two_wheeler mixes bicycles and motorized variants; large_vehicle merges trucks, buses, and trains together due to data scarcity. Sequence Bias: Long tracks of slow-moving objects can skew a particular data split velocity distribution, causing high F1 score variance across folds. Ablation studies I tried with bigger MLPs, alternative feature encodings, and different histogram binning, all moved performance less than the variation caused by changing the train/validation/test split. I measured that split sensitivity across 6 folds, keeping the same proportions. Changing the histogram to per-instance statistics (mean/median/std) slightly degraded performance. Main findings Macro F1 rises from 0.381 to 0.764 as the naturally occurring number of radar detections per instance increases from 1 to 5. I trained the model normally using all available detections, then bucketed its existing validation predictions by each instance's detection count and computed macro F1 per bucket. The classes car and pedestrian has the best performance and two_wheeler has the worst. A car is often confused as large vehicle when the car was wider than usual or had a unusually high rcs (which can happen due to multipath for example). The two_wheeler is often confused as pedestrian because their vr_compensated distributions overlap, which is the the model's single most important feature for these two cla

Is LfD and BC research being effected by recent advances in (so-called) Frontier LLMs? Or is research in LfD and BC sort of going along in an independent direction from these? Are you seeing any use from ViTs or VLAs? Any other recent advances you would like to bring up? submitted by /u/moschles [link] [留言]

One thing that has bothered me about LLM benchmarks for a while is that most of them are essentially snapshots. A model is evaluated, a score is published, and we tend to talk about that score as if it describes a relatively stable object. But with API-served models, the thing behind the model name can change over time: serving infrastructure changes, provider configurations change, versions change, and sometimes behaviour changes without an obvious public version transition. So we started approaching benchmarking as a longitudinal measurement problem rather than a leaderboard problem. We continuously evaluate models across coding, multi-turn reasoning and tool use, while also running lightweight probes at a higher frequency. The important part for us is not simply asking "which model scores highest?", but: Is the model behaving differently from its own previous baseline? Is the change larger than its normal repeated-call variability? Did the benchmark configuration itself change? Is the effect concentrated in a particular task? Is it correlated across models from the same provider? Is an apparent degradation actually an availability/infrastructure issue rather than a capability change? One historical analysis covered 31,352 repeated score observations across 49 models . The standard deviation of within-day scores was 2.80 points , while the standard deviation of between-day daily medians was 8.43 points . That is roughly a 3:1 difference. I don't think this result by itself establishes that providers are changing models day-to-day - there are too many possible confounders for that conclusion. Task composition, sampling, missingness, provider behaviour and methodology changes all matter. But it was enough to convince us that temporal variation deserves to be measured rather than treated as noise around a permanent leaderboard score. Our current approach therefore keeps benchmark configurations versioned and only compares longitudinal observations produced under comp

Ever woke up feeling like a truck hit you, only to realize your Apple Watch had been screaming "Warning!" via your data for the last 24 hours? Heart Rate Variability (HRV) is the "canary in the coal mine" for our bodies. It's a powerful metric that tracks the variation in time between each heartbeat, serving as a direct window into your Autonomic Nervous System. In this guide, we are going to build a real-time HRV anomaly detector using wearable data analysis , Scikit-learn , and AWS Lambda . By applying machine learning to time-series health data, we can identify physiological stress, potential infections, or overtraining before physical symptoms even manifest. If you’ve been looking to dive into anomaly detection in time-series or want to master health data engineering , you’re in the right place! The Architecture: From Heartbeat to Alert 🛠️ To achieve real-time monitoring, we need a pipeline that moves data from your wrist to a cloud-based inference engine. Here is the high-level flow: graph TD A[Apple Watch / Wearable] -->|Sync| B(Apple HealthKit) B -->|Webhook/Hook| C[AWS API Gateway] C --> D[AWS Lambda - Inference] D -->|Fetch History| E[(DynamoDB / S3)] D -->|Isolation Forest| F{Anomaly?} F -->|Yes| G[Push Notification / Alert] F -->|No| H[Log & Silent] Prerequisites 📋 Before we start coding, ensure you have the following: Python 3.9+ Scikit-learn & Pandas for data crunching. AWS Account (for Lambda deployment). An app to push HealthKit data (like Health Auto Export or a custom Swift hook). Step 1: Understanding the Data 📊 HRV data is tricky because it’s highly personalized. What is "low" for an athlete might be "high" for someone else. This is why we use Isolation Forest , an unsupervised learning algorithm that excels at detecting outliers in multi-dimensional datasets without needing labeled "sick" vs. "healthy" days. Step 2: Building the Anomaly Detection Logic Let's write the core logic using Scikit-learn . We’ll use the Isolation Forest algorithm becaus

When I first started working in scientific machine learning, I understood the physics much better than the coding. Every time I wanted to try a new physics-informed neural network problem, I had to start almost from scratch: changing the PDE, updating boundary conditions, modifying the architecture, tweaking the training schedule, debugging errors, and generating plots—all by hand. That frustration pushed me to build PINNStudio . It is a free, open-source no-code GUI designed to eliminate boilerplate code so you can focus entirely on the physics. Instead of rewriting a new script for every problem, you can define your setup directly through the interface: PDE Definitions & coupled multi-output PDE systems 1D or 2D domains with boundary and initial conditions Network architecture & custom training schedules Forward problems (solving known PDEs) or Inverse problems (estimating unknown parameters from data) What happens next? PINNStudio automatically generates the code (built on top of DeepXDE), runs the model, streams the training log, and displays live loss curves and solution plots directly inside the app. It also includes built-in templates for classic equations like Heat, Allen-Cahn, and Cahn-Hilliard. GitHub (Open Source): https://github.com/AsfandyarKhan72/PINNStudio Quick Install: pip install pinnstudio My hope is that this will be helpful for students and researchers with limited coding experience, as well as experienced PINN users who just want a faster workflow. I’d love to get your feedback, feature suggestions, or bug reports! Huge thanks to Lu Lu and the DeepXDE team for creating the foundation that made this possible. submitted by /u/Impossible-Jello2749 [link] [留言]

I'm currently learning Machine Learning through a course, and I want to start building projects alongside it. My main goal right now is simply to build several good ML projects and get familiar with the complete project development process . I want to use AI coding tools such as Cursor, Claude Code, or GitHub Copilot to speed up development, but I'm unsure about the right way to vibe-code an ML project . For example, should I: Give the AI the complete project requirements and let it build the project? First create the architecture/pipeline myself and then let AI implement it? Build the project step-by-step and ask AI to implement each stage? Let AI handle things like data cleaning, EDA, preprocessing, and boilerplate while I focus on the ML decisions? Give AI a detailed specification before starting? Ask AI to review and improve the code after it generates it? Use one long conversation/context for the entire project, or separate prompts for different stages? How should I handle debugging and modifying AI-generated ML code? Basically, what is the best workflow for vibe-coding an ML project from start to finish? I'm not trying to replace learning ML with AI — I'm already studying the concepts separately. I just want to use AI effectively to build projects faster without ending up with a messy or poorly structured project . I'd especially like to hear from people who have built ML projects using Cursor/Claude Code/Copilot: What workflow do you personally follow, and what mistakes should I avoid? Also, please suggest any good communities where I can see how other people are building ML projects and discuss AI-assisted development. Thanks! submitted by /u/TusharKharade_ [link] [留言]

I ported MoE expert expansion to llama.cpp 🚀 Run MoE models with MORE routed experts than the native top-K (8->x), adaptive threshold, 99→50% influence decay, layer range. Runtime-only, all backends. Tested on Qwen 3.6 35B A4B+ https://github.com/vagrillo/llama.cpp/blob/moe-expansion/docs/moe-expansion.md submitted by /u/Specific-Tax-6700 [link] [留言]

Hello all, TL;DR: point density, not model architecture, was the real bottleneck for a 5-class radar-only classifier on RadarScenes. Going from 1 to 5 points per instance roughly doubles macro F1 (0.381 → 0.764), while a whole set of architecture and feature changes all landed inside a measured noise floor. Real failure case attached: a stationary two-wheeler misread as a pedestrian. Setup I'm a perception / radar signal processing engineer getting into ML on radar data. Trained a 5-class classifier (car, large_vehicle, two_wheeler, pedestrian, pedestrian_group) on RadarScenes radar point clouds only, no camera or lidar. Per-instance histogram (16 bins) encoding into a 3-layer MLP. Main result: point density is the ceiling Macro F1 goes from 0.381 to 0.764 just by increasing points per instance from 1 to 5. Same trained model, nothing else changed. Why: a single point can't carry a size or velocity-spread signature. large_vehicle's F1 is 0.037 at n=1 vs 0.995 at n=11+. Some classes still work at n=1 (car separates cleanly on RCS/Doppler alone), others don't (two_wheeler and pedestrian collapse to the same near-zero-velocity signature when sparse). Ablation studies Wider/deeper networks, six alternative feature encodings, different bin edges, all landed inside the noise floor I measured with a 6-fold split sensitivity check (same train/val/test proportions, sequences reassigned per fold). Closest thing to an exception: swapping the histogram for explicit per-instance statistics (mean/median/std) actually made things slightly worse (0.658 vs baseline's 0.686), and pedestrian's own F1 fell outside its class-specific noise floor. Data caveats RadarScenes is naturalistically collected, not balanced: common classes get broad coverage, rare ones don't. two_wheeler merges two physically different speed regimes (bicycle vs a much rarer motorized variant). large_vehicle merges large_vehicle/truck/train/bus, RadarScenes' own recommended scheme, mostly forced by data scarcity i

I feel that reproducibility is now a lost cause in machine learning research for three reasons: Many research is moving towards the physical AI territory, where you need expensive hardwares or even entire laboratories with high-speed cameras, in order to perform an experiment. You truly have no idea if the experiment can be reproduced and have to trust the demo. But demos are not perfectly reliable. Plus people are incentivized to only show the part of the demo that works. The entire system can fall apart the moment the recording stops. You have big AI companies releasing various tools, which they claim to solve a host of problems with certain amount of accuracy or efficiency. Unless you work at those companies there is really no proof of that and you will have to take their words on it. They have strong financial incentive to blow-up those figures. There is no solid way to check it either because the problem that they solve are so vague and subjective. We need to address the elephant in the room which is that people are incentivized to produce non-reproducible work to prevent their lunch being eaten by their competitors or looking bad. That's why some of us will probably never get a reply when we email the authors for their code. So what now? Maybe everything will be OK because we can contrast it with scientific progress in earlier parts of history, e.g., building the atomic bomb or sending people to the moon. These projects had low "outside reproducibility" but high "internal reproducibility". Plus all these work were mathematical in nature and carefully checked. But I don't think many areas of machine learning research is like that. What do you think? Should reproducibility be abandoned? If not how is it best implemented going forward? submitted by /u/NeighborhoodFatCat [link] [留言]

If you use agentic workflows with custom skills or rules (Cursor rules, Claude Code slash commands, OpenCode, etc.), you have probably run into the routing trade-off: Stuff every skill definition into the system prompt (destroys your context window and degrades instruction-following). Use an LLM router turn to classify the user prompt (costs money, wastes 1,000+ tokens, and adds 2+ seconds of network latency). To solve this, I built Routed ; an open-source, local-first hybrid router for agent skills that runs 100% offline on your CPU. GitHub: https://github.com/bshea-1/Routed License: MIT https://i.redd.it/5ca68gffgxnh1.gif How it Works Under The Hood Routed indexes your installed skill directories and evaluates prompts through a 4-part hybrid scoring pipeline: * Dense Vector Embeddings (60%): Runs quantized ONNX models (Arctic Embed S / MiniLM) locally on CPU. * Lexical BM25 (25%): Okapi BM25 for strict keyword relevance. * Exact / Alias Match (10%): Direct command and alias matching. * Metadata (5%): Recency and usage heuristics. The entire lookup completes in under 20ms without sending a single byte of prompt data over the wire. Supported Environments Routed auto-detects and injects adapters into: Cursor, Claude Code, LM Studio, Ollama, Antigravity IDE, Windsurf, OpenCode, Continue, Codex, and I just dropped support for MCP Servers!! And although v1.0 dropped last night, I just shipped v1.1.0 with two major additions based on early feedback: Model Context Protocol (MCP) Server ( routed mcp ): Instead of loading 20+ tool schemas into your GPU's context window, your local model only sees a single route_skill tool. Routed executes on CPU, selects the exact skill needed, and injects only that schema on demand. Native Multilingual Understanding: The embedding pipeline now natively understands input across 100+ languages (German, Spanish, French, Japanese, etc.) and automatically decomposes compound nouns (like German Speicherleck ), mapping prompts directly to the cor

AACL-IJCNLP 2026 acceptance results will be released in a few hours. Feel free to share your thoughts and feelings! How did you do? submitted by /u/Starscream-11813 [link] [留言]

Modern price monitoring systems need to do more than tell you that a price changed. A single abnormal listing, a scraped error, or a temporary outlier can make a traditional threshold-based detector fire an alert when nothing meaningful happened. In this project, I built a lightweight real-time price anomaly detector in Python that combines: A rolling median baseline Median Absolute Deviation (MAD) Robust Z-scores Short-term percentage returns Trend confirmation Alert cooldowns The goal is simple: detect meaningful price movements without overreacting to noisy observations. Note: This project monitors retail prices from Google Shopping results through SerpApi. It is a retail-price monitoring example, not a financial exchange-data feed. What we're building The pipeline looks like this: ┌──────────────────────┐ │ SerpApi / Shopping │ └──────────┬───────────┘ │ ▼ ┌──────────────────┐ │ Price Extraction │ │ + Validation │ └────────┬─────────┘ │ ▼ ┌────────────────────┐ │ Rolling Price │ │ History │ └────────┬───────────┘ │ ┌───────────┼───────────┐ ▼ ▼ ▼ Median MAD Return % │ │ │ └───────────┼───────────┘ ▼ Robust Z-score │ ▼ Trend Confirmation │ ▼ Signal Engine │ ▼ Alert Cooldown The implementation is intentionally small and interpretable. The complete engine is built around a single PriceAlertEngine class and a compact AnomalyResult data structure. Why not just use standard deviation? A common first implementation is: price > mean + 3 * standard_deviation The problem is that standard deviation is sensitive to extreme observations. Suppose your historical prices are: 990, 995, 999, 1001, 1005 Then one bad observation such as: 1500 can distort the mean and standard deviation. That can move your detection boundary away from the actual market behavior you are trying to model. For a noisy retail environment, a more robust baseline is useful. That's where median and Median Absolute Deviation come in. 1. Building a rolling median baseline Instead of storing an unlimited stre

I've been working on a practical implementation of Sliding Window Attention (SWA) for pretrained Hugging Face causal LLMs. The idea is simple: instead of allowing every generated token to attend to the complete historical KV cache, maintain a bounded cache consisting of: attention sinks + recent sliding window I implemented this as a reusable inference layer rather than modifying or retraining the model. GitHub: https://github.com/oraby8/SWA The implementation currently includes: bounded KV cache circular/ring-buffer storage attention sinks streaming prefill chunked attention masking autoregressive decoding Full Attention vs SWA benchmarking TTFT / TPOT / throughput measurements KV-cache memory measurements One interesting result from my Qwen2.5-7B experiment: Context Full KV SWA-64 16K ~923 MB ~3.5 MB 32K ~1.84 GB ~3.5 MB 64K OOM ~3.5 MB At 16K, SWA-64 also reduced TPOT from ~38.4 ms to ~30.5 ms in this setup. However, there is an important trade-off: tasks requiring information far outside the active window can degrade. I'm currently investigating how much of this is inherent to SWA versus implementation/model-specific behavior. I'm sharing the implementation mainly to get feedback from people working on LLM inference, KV-cache optimization, and long-context models . I'd be particularly interested in: Which model architectures should I validate next? What failure cases should I benchmark? What would make this useful for existing HF inference workflows? Are there cache/attention implementation details I may be overlooking? Feedback and experiments are very welcome. submitted by /u/ahsaor8 [link] [留言]

Originally published on hexisteme notes . "Will it fit on our hardware?" is the wrong first question. It's the one everyone asks, because it's free to answer — the thing either loads or it doesn't. Throughput costs you a measurement. So the capacity gate passes, and it feels like the decision is made. The measurement Mac Mini M4, 24GB unified memory, ~120GB/s memory bandwidth. A 27B model, IQ4_XS quantized, 15GB on disk. Capacity gate: pass. Metal's recommendedMaxWorkingSet is 17.76GB, the model is 15GB, ollama ps reports 100% GPU resident. No swap, no spillover. By every "does it fit" criterion this is a clean win. Generation: 5.6 tokens/second. That's not a usable interactive worker. It's barely a usable batch worker. And nothing about the capacity check hinted at it. The arithmetic that would have told me in advance Autoregressive generation reads the entire model's weights once per token. So: ceiling ≈ memory bandwidth ÷ bytes touched per operation = 120 GB/s ÷ 15 GB = 8 tokens/second Measured 5.6 against a ceiling of 8. Ratio 0.70. That ratio is the whole verdict. When measured throughput is a large fraction of the arithmetic ceiling, you are bandwidth-bound , and you now know something concrete: the bottleneck is not your configuration, not memory pressure, not thermal throttling. It's how fast bytes move. Rule of thumb I now use: ratio ≥ 0.5 → bandwidth-bound, and size-reduction fixes are dead. Why "just quantize harder" doesn't work The natural move when capacity is tight is to shrink. Lower quantization, smaller batch, heavier compression. It's the reflex, and in a bandwidth-bound regime it's close to useless. I was considering Q3_K_M at 13.8GB. Run the same division: 120 ÷ 13.8 = 8.7 tokens/second (up from 8) Under 9% more throughput. For a real drop in output quality, because quantization error doesn't scale linearly with size the way bandwidth does — you give up more than you get, every time, in this regime. I killed that plan without downloading anythin

building a missing data infrastructure and started benchmarking long multi-session conversations (LoCoMo). I know the data looks like: people, facts, claims, events, timestamps, relations. So I extract those into a graph. I did not look at the QA pairs while building extractors or retrieval rules. No “if question contains X, fetch fact #173.” Recall is very high and it keeps working on new conversations in the same format. Is this classical overfitting, or just schema-aware engineering? What is the cleanest test that would convince you it isn’t leakage. submitted by /u/chaachans [link] [留言]

I run Knipsmig , a QR-code photo sharing service for weddings and parties. Guests scan a code and upload straight from the phone, no app. Most of those uploads are 12 MP and print fine. A meaningful slice are not: photobooth captures at 1080x810, WhatsApp forwards at 1600x1200, screenshots, old scans someone re-uploaded. Those end up in the printed photo book looking soft. So I added an "Improve resolution" option to the editor. It adds up to 4x the pixels, and the whole thing runs on my own server. No API, no vendor, nothing leaves the box. This post is about why I went local and what it took to make that work inside a Rails app. Why not just call an image API I already have Gemini and OpenAI keys configured in the app for other things, so the lazy path was obvious. I did not take it, for three reasons. The generative models redraw the image. They don't upscale, they regenerate. Faces drift. These are guests' faces at someone's wedding, and "your aunt looks slightly different now" is not a feature. A super-resolution network stays faithful to the input: it only adds pixels consistent with the ones already there. Privacy paperwork. Every third-party processor I add has to go into the DPA. Guests' photos leaving the server to be fetched by a vendor is a real change, not a footnote. Running locally means the data processing agreement doesn't change and the existing opt-out for third-party AI stays about third parties. Cost. Per-image API pricing on a bulk action over hundreds of photos adds up fast. CPU time on a job lane I already pay for is free at the margin. The model I went with realesr-general-x4v3 from the Real-ESRGAN project (BSD-3-Clause). It's the compact SRVGGNet variant: about 1.2M parameters, roughly 5 MB as an ONNX file, and around 10x faster on CPU than the full RRDBNet x4plus. Quality is more than fine for event snapshots. Getting it into a usable shape was a one-off: export the release weights with the repo's pytorch2onnx.py script using dynamic H/W a