标签：#mac

This article provides a step by step comparison of three Gemma 4 deployments on a single AWS hosted GPU enabled system. A suite of Python MCP tools is built to simplify management of each deployment, and one benchmark harness is shared across all three so that the runtime is the only variable. https://github.com/xbill9/gemma4-dev The whole exercise cost under three dollars, and that is the part worth keeping. Nineteen instances and about four and a half instance-hours bought three serving sweeps, nine timed boots and a handful of A/B restarts. It also bought five wrong claims, each caught by measuring instead of reasoning. On hardware where a run is expensive, the cheapest of those five would have shipped with a caveat attached. What is this project trying to Do? Three rigs in this monorepo serve google/gemma-4-E2B-it on an AWS G5g instance. One runs vLLM, one runs a pure JAX port, one runs PyTorch with transformers. The hardware is identical and only the runtime slot moves, so this should be the cleanest A/B available. For months it was not, because each rig measured itself with its own harness and quoted its own number. Three harnesses computing three statistics is not a comparison. Prerequisites An AWS account with G-family quota in us-east-1 . Each g5g.2xlarge is 8 vCPU, so 16 vCPU of spot quota runs two at once. A subnet, a security group opening TCP 8000, and an instance profile carrying AmazonSSMManagedInstanceCore plus read on the Hugging Face token secret. A Hugging Face token in Secrets Manager. It is fetched at boot into a root-only EnvironmentFile and never appears in user data. boto3 and the standard credential chain. No AWS CLI shell-outs, no inbound SSH rule, and no private key anywhere in the flow. AWS EC2 G5g Instance g5g.2xlarge — 8 vCPU, 16 GiB host Host CPU AWS Graviton2, aarch64 GPU 1x NVIDIA T4G, Turing, SM 7.5 GPU memory 15,360 MiB per nvidia-smi ; AWS lists 16,384 nominal G5g is the only family AWS ships that puts an NVIDIA GPU behind a Gravi

This article provides a step by step comparison of three Gemma 4 deployments on a single AWS hosted GPU enabled system. A suite of Python MCP tools is built to simplify management of each deployment, and one benchmark harness is shared across all three so that the runtime is the only variable. https://github.com/xbill9/gemma4-dev The whole exercise cost under three dollars, and that is the part worth keeping. Nineteen instances and about four and a half instance-hours bought three serving sweeps, nine timed boots and a handful of A/B restarts. It also bought five wrong claims, each caught by measuring instead of reasoning. On hardware where a run is expensive, the cheapest of those five would have shipped with a caveat attached. What is this project trying to Do? Three rigs in this monorepo serve google/gemma-4-E2B-it on an AWS G5g instance. One runs vLLM, one runs a pure JAX port, one runs PyTorch with transformers. The hardware is identical and only the runtime slot moves, so this should be the cleanest A/B available. For months it was not, because each rig measured itself with its own harness and quoted its own number. Three harnesses computing three statistics is not a comparison. Prerequisites An AWS account with G-family quota in us-east-1 . Each g5g.2xlarge is 8 vCPU, so 16 vCPU of spot quota runs two at once. A subnet, a security group opening TCP 8000, and an instance profile carrying AmazonSSMManagedInstanceCore plus read on the Hugging Face token secret. A Hugging Face token in Secrets Manager. It is fetched at boot into a root-only EnvironmentFile and never appears in user data. boto3 and the standard credential chain. No AWS CLI shell-outs, no inbound SSH rule, and no private key anywhere in the flow. AWS EC2 G5g Instance g5g.2xlarge — 8 vCPU, 16 GiB host Host CPU AWS Graviton2, aarch64 GPU 1x NVIDIA T4G, Turing, SM 7.5 GPU memory 15,360 MiB per nvidia-smi ; AWS lists 16,384 nominal G5g is the only family AWS ships that puts an NVIDIA GPU behind a Gravi

YOLO26 ships a depth-estimation model — dense, full-resolution, per-pixel regression, a task architecturally much closer to image restoration than to detection. I wanted to know whether the backbone+neck weights it learns through depth training transfer to a different dense-regression task (deraining), compared with training the same architecture from scratch. The deraining model that came out of it is a useful byproduct, but the transfer result is the part I think is worth discussing. What I inherited from YOLO26-depth The CSPDarknet backbone and PAN-FPN neck, unchanged. The depth decoder's multi-scale fusion (project the P3/P4/P5 pyramid to a common width, progressively upsample-and-add P5→P4→P3). That part isn't depth-specific — it's just feature fusion — so RGBHead reuses it. What I changed / added Replaced the 1-channel Depth head with a new RGBHead . The config change is one line; RGBHead itself is a new restoration decoder, not a re-pointed depth head. Reconstruction tail that continues to full input resolution (deraining needs pixel-exact output; depth stops at 1/4 res). Skip connections from the stride-2 and stride-4 backbone layers into the tail, so fine detail has a path that doesn't route through an 8×-downsampled bottleneck. Residual output — the head predicts a correction added to the input (NAFNet/Restormer-style), not the image directly. LayerNorm in the head's own conv blocks; the backbone and neck stay on BatchNorm (folds into conv at TensorRT export, and keeps the model loadable from the whole YOLO26 pretrained zoo, not just the depth checkpoint). How it was trained and measured ClearView as an external lib — its mixed synthetic+real rain recipe, Charbonnier loss, and 10-test-set protocol — so the numbers land on ClearView's own model-zoo scale, not a benchmark I made up. Released scales: nano (5.25M) and small (12.13M). Loading the YOLO26-depth checkpoint into this architecture matches 468/468 backbone+neck tensors exactly — only the new RGBHead

After following various arXiv papers and researcher discussions on X/bluesky about latent reasoning and continual learning, one idea which resonates strongly is that path forward (towards AGI) may depend less on generating ever-longer chains of thought and more on finding architectures that can reason beyond the token stream. LLMs routinely reach correct answers through flawed or fabricated CoT steps, and produce perfectly logical steps that end in wrong answers (Kambhampati, 2025). The trace doesn't track the computation which clarifies that verbalized CoT is an imitation of reasoning and not the mechanism itself. The alternative mechanism which gets the most attention is latent reasoning: instead of verbalizing every intermediate result, the model repeatedly transforms its continuous hidden state and decodes only the answer. I’m breaking latent reasoning down into at least five distinct families: Continuous thoughts in autoregressive LMs: Coconut (Hao et al., 2024) feeds the model's own final hidden state back in as the next input embedding while Soft Thinking (Zhang et al., 2025) reasons in a continuous concept space. Theory here argues a single continuous state can hold several search frontiers at once and expand them in parallel (Zhu et al., 2025) . Compressed discrete non-linguistic tokens: Abstract-CoT (Ramji et al., 2026) swaps verbal rationales for a short sequence from a learned vocabulary. It is non-linguistic, but still serial and externally decoded, the discrete end of the spectrum . Recurrent depth and looped models: recurrent-depth LMs (Geiping et al., 2025) and looped Transformers (Saunshi et al., 2025; Zhu et al., 2026) reapply a shared block to a latent state. Mostly framed as parameter efficiency and test-time-compute scaling, not as a new reasoning interface . Task-trained recursive solvers: HRM (Wang et al., 2025) and TRM (Jolicoeur-Martineau, 2025) recursively refine latent and candidate-answer states. Their ARC pipelines are transductive: eval

The Brave Wanderer: I made Claude play a Pokémon it never read a guide for Full timeline video of this 2,000-turn run (game frames + a live cost counter on the left, the model's real-time thinking log on the right): https://youtu.be/ewyM7mzGzTM At the end of the first article in this series , I made a promise. Fable 5's fluency in FireRed owed half its credit to the walkthroughs it had memorized — it wrote down "Oak's Parcel," an item the game hadn't shown it yet, 141 turns early. So the only honest exam is a new exam paper: "Same harness, same model, a map it cannot recite — I'll post the numbers." This article is those numbers. The exam paper is Pokémon Team Rocket Edition — the Chinese fan translation of the Spanish community hack Pokémon Edición Team Rocket, released in January 2026. You play a Team Rocket recruit working your way up from the Five Island base. Five story rounds, four regions; the Kanto chapter alone is labeled 30-35 hours for a human player. And most importantly: this game is essentially absent from the model's training data . No guide to recite. Just the screen and itself. There's also a lovely narrative twist: the hack sets your home base inside the original FireRed's Five Island Rocket Warehouse — the enemy hideout you raid late-game as the hero in the official version. Same map, opposite allegiance. Rules unchanged: vision only, one screenshot plus its own notes per turn, one button-press tool, a 2,000-action cap. The result, up front 8 hours 43 minutes, 2,000 turns, $113.44. It reached the middle of the prologue's first mission — roughly 40-60 minutes of human play time. It taught itself plenty: menus, battles, catching, the save flow, all from scratch; after losing to a fellow recruit it wrote a revenge battle plan into its notes, ground levels, and actually won the rematch; it even induced map rules like "dark blue water can't be surfed, light blue can," and maintained a dead-ends list and an NPC-interview checklist in its notes. One deta

Please post your questions here instead of creating a new thread. Encourage others who create new posts for questions to post here instead! Thread will stay alive until next one so keep posting after the date in the title. Thanks to everyone for answering questions in the previous thread! submitted by /u/AutoModerator [link] [留言]

Hey everyone, My brother and I just released TontaubeV1, a 2.9B-parameter open-weight TTS model focused on expressive speech, long-form generation/narration, and low-latency local inference. It is primarily aimed at English and German and supports zero-shot voice cloning from up to one minute of reference audio. It builds on DualCodec, a multi-codebook discrete audio codec. It was trained on 7 languages and ~200k hours of audio (mostly tested in English and German). I wanted to make a post to highlight two choices that worked well for us and seem less common in current TTS models: 1. Character-level tokenization We start from a Qwen3-1.7B checkpoint for our semantic codebook model. Many modern, and especially LLM-based, TTS models use the tokenizer from the backbone model, add special/audio tokens, and train the model on predicting the next token. We experimented early on with character-level tokenization and found that it generally worked better than using the original BPE tokenizer from Qwen. We still use the tokens emitted by the Qwen tokenizer, but force it to tokenize spoken text as a sequence of individual characters. When experimenting with forcing Qwen to predict text in this mode, we found that it was still able to answer questions correctly, which suggested to us that language understanding was retained even with this unusual representation. We did this because a) context length usually is not as much of an issue for TTS as it is for regular LLMs, since we do not require huge reasoning budgets and use chunking, and b) it makes the character-to-sound mapping much simpler internally. Speech is a lot about syllables and short character sequences. When using the regular BPE tokenizer, we found that the model went out of distribution more often and was more likely to encounter a sequence of tokens that was rare or absent from the TTS training data. Complex sequences of special characters can be particularly confusing because they may tokenize into combinations

Expectations for Tim Cook were almost impossibly high when he stepped in to replace Apple's visionary cofounder in 2011. He inherited a company on a blockbuster run, after Steve Jobs returned and revitalized the Mac, launched the iPod, and oversaw the launch of the iPhone. Now, after 15 years in the CEO role, Tim Cook […]

I'm exploring Hidden Markov Models (HMMs) as a baseline method for "dataset exploration/discovery" where I have a bunch of unstructured data with no annotations, and wish to gain insights about the structure and semantics of the data within. I was wondering if there are more modern (deep learning based or otherwise) approaches which have completely superseded HMMs for such tasks. submitted by /u/fullgoopy_alchemist [link] [留言]

I own two laptops. A 2020 Intel MacBook Air, 8GB RAM, no unified memory, gifted by my sister. And a...

Since 2021, our team at Autor has shipped over 50 AI products across healthcare, fintech, logistics, and SaaS. Some of them are running in production right now, handling thousands of automated calls per month. Others failed spectacularly — and those are the ones that taught us the most. Where This Comes From I started Autor in Toronto as a one-person AI development shop. The original thesis was simple: companies needed custom AI but couldn't hire fast enough to build it themselves. Four years and 50+ products later, we're a senior-only studio with a production voice AI platform (Loquent) serving healthcare and dental clients 24/7. Along the way, we've impacted over 5 million users, helped clients raise more than $10 million in funding, and shipped across 10+ countries. This isn't a highlight reel. This is the unvarnished list of things I got wrong, figured out the hard way, or wish someone had told me before I wrote my first line of production AI code. 1. Your First AI Product Should Be Boring Our first few products were ambitious. Multi-modal pipelines, complex reasoning chains, novel architectures. Most of them took twice as long as estimated and required constant babysitting in production. The products that actually made money and kept clients happy? A straightforward document classifier. A simple intent router. A basic FAQ bot with good fallback logic. I used to think "boring" meant "not innovative." Now I know boring means "reliable enough that I don't get paged at 3am." Our most successful product, Loquent, handles healthcare scheduling calls. It's not doing anything architecturally exotic. It picks up the phone, understands what the caller needs, books or reschedules an appointment, and hangs up. The magic isn't in the model — it's in the 200+ edge cases we've handled around it. If you're building your first AI product, pick the most boring version of your idea and ship that. You can add complexity later. You cannot add reliability later. 2. Prompt Engineerin

I have submitted a paper to acml 2026 journal track, the official date of release of review is 27 August, but I have not heard anything from them, if anyone received the review then let me know I will write to program chairs. Thanks submitted by /u/Jealous_Key_4030 [link] [留言]

Sliding Window Attention with sinks, one of the simplest existing fixes for the quadratic-cost problem in LLMs, holds up as well or better than the linear-attention variants labs have been spending post-training compute to produce. That is the claim of a [new arXiv preprint]( https://arxiv.org/abs/2608.28444 ) by Alexia Jolicoeur-Martineau, Rhea Sanjay Sukthanker, Pashmina Cameron and Emy Gervais. On the long-context reasoning benchmarks the paper singles out, the gap is not close. "SWA achieves massively higher performance (2 to 10 times higher than linear attention)," the abstract reports, naming Needle-in-a-Haystack and BABILong as the two tasks. The pitch is that the whole post-training-to-linear pipeline has been benchmarked against the wrong thing. "This line of research has not been properly compared to simpler baselines," the authors write. Their alternative needs no post-training, runs fast, and holds memory low. The recommendation is blunt: "we strongly recommend switching to SWA instead of post-training linear models." Linear attention, the abstract concedes, "may have shown some promise, but they likely require to be trained from scratch or extensive post-training in order to even match SWA." --- submitted by /u/Justgototheeffinmoon [link] [留言]

We noticed our anti-money laundering models were performing suspiciously well. After digging into standard baselines on dynamic graphs, we found widespread temporal leakage in message-passing. If you train a GNN on a static snapshot of a dynamic graph, your model is likely cheating by seeing future edges during training. We got sick of reviewing papers with broken evals, so we released SynthFin-AML v10.0 (100k nodes, 1.2M edges) to force strict causal boundaries. The Temporal Leakage Trap Standard transductive random splits fundamentally fail on financial transaction networks because they violate the arrow of time. If Node A sends funds to B on Day 2, and B to C on Day 10, a standard 2-hop GNN will pull the Day 10 edge into the loss calculation for Day 2. The model literally looks into the future to compute embeddings. The Fix: 3-Snapshot Architecture To stop the model from cheating by looking ahead at the transaction graph, we enforced a strict 3-snapshot point-in-time split: Train Graph (Edges ≤≤ Day 7) Val Graph (Edges ≤≤ Day 8) Test Graph (Edges ≤≤ Day 10) By physically disjointing the temporal windows, we bound the receptive field of the GNN to the true causal horizon. Graph vs Tabular Reality Check Most synthetic datasets suffer from distribution leakage, where fraud transaction amounts are statistically separable from normal retail traffic. We killed the "amount split cheat" by ensuring fraud and retail transaction amounts share the exact same lognormal distribution (μ=8.517,σ=0.8 μ =8.517, σ =0.8). With tabular leakage fixed, we benchmarked a tuned LightGBM against GraphSAGE to see if the GNN overhead actually pays off for AML. We engineered 11 point-in-time graph features (Weighted PageRank, neighbor volume aggregates) for the tree model. Results (PR-AUC on strict temporal split): LightGBM (11 features): 0.848 GraphSAGE (Inductive): 0.881 Spoiler: GraphSAGE barely beats trees here unless your edge features are incredibly dense. The gap isn't astronomical, b

RamaLama runs large language models as OCI containers, so a single command ( ramalama run smollm:135m ) pulls a model and starts talking to it, with no Python environment to babysit. I spent an afternoon putting it through its paces on an Apple Silicon Mac (Apple M4 Pro, 48 GB RAM, macOS 26.6) with Docker 29.4 provided by OrbStack. This guide is what I actually saw: the install, the first model, an OpenAI-compatible server, and the one macOS-specific catch that isn't obvious from the docs. Every command and number below is from that run, on RamaLama 0.24.0. What is RamaLama? RamaLama is an open-source CLI from the container-tooling community that treats models like container images. Instead of assembling an inference stack yourself, it pulls a hardened OCI image containing llama.cpp (or vLLM/MLX) plus your chosen model and runs it with Podman or Docker. If you've used Ollama the ergonomics feel familiar ( run , serve , list , pull ), but the runtime and model live inside containers you can inspect and sign, and weights come straight from Hugging Face, Ollama, or any OCI registry. Installing RamaLama on macOS With Homebrew it's one command: brew install ramalama That pulled RamaLama 0.24.0 and, notably, its own copy of llama.cpp , ggml , and libomp as dependencies. Hold onto that detail; it matters for GPU acceleration later. Confirm the install: ramalama version # ramalama version 0.24.0 You also need a container engine running. I used Docker through OrbStack; Podman works too and is RamaLama's default on Linux. Running your first model The headline command: ramalama run smollm:135m "In one sentence, what is a Linux container?" Passing a prompt as an argument gives you one-shot output instead of dropping into a chat REPL. On first run this pulled the RamaLama container image, downloaded the model, and answered. smollm:135m resolves to hf://HuggingFaceTB/smollm-135M-instruct-v0.2-Q8_0-GGUF , a 138 MB, 8-bit quantized GGUF from Hugging Face. First-run wall-clock was 2

This is the time of year when the number of cold emails I receive about PhD positions tends to ramp up quite a bit. In many countries, this cold emailing is essentially part of the normal recruitment process, so there is nothing inherently wrong with doing this. However, there are a few things you definitely shouldn't be doing: Massive emails. The probability of me reading your email is inversely proportional to its length. Emailing everyone. Find supervisors that work in areas you are actually interested in. I do relatively foundational ML research (i.e., not associated with a specific application domain), but the majority of emails I get from prospective students are essentially "I want to apply ML to domain X". In many cases this does not constitute an ML research direction; you'd be better off finding a supervisor with expertise in domain X, which is where most of the impact will be. Generic research interests. If the most specific research interests you can give are "Machine Learning, LLMs, and AI" then I assume you only have a surface-level familiarity with the field, and are not ready for a PhD. Passing off workshop papers as conference papers. This has become a much more common thing in the last couple of years. It's a big red flag; I am not going to take on someone who is dishonest. Excessive AI use. Using them for fixing up grammar is fine, but if you outsource your thinking to LLMs then your research direction will be the same as everyone else who outsources their thinking to LLMs. This tends to result in something that would be an okay bachelor's thesis project, but nothing more than that. I get a lot of LLM emails, so determining if you are in this cluster is very easy. Summarise my paper. I already know what's in it, I don't need a summary. I care more about how you think you could build on it, or do something related. Don't use LLMs for this; see above point. Ignoring instructions on my website. Check prospective supervisors' websites for how you shou

I'm sharing this new tabular data diagnostic tool (Entropic Scree). It can be used to estimate these properties of your high-d, real-world, dirty dataset: The informational volume of the signal (i.e., helps you assess whether the signal is strong enough to survive the dataset's idiosyncratic volume). The overall signal-to-idiosyncratic volume ratio (SNR). The intrinsic rank. Provides an exploratory map that allows for the identification of decoupled sub-networks of variables. The linear sufficiency (i.e., does the dataset align with the linear assumptions of standard PCA?). Instead of evaluating linear variance, rank order, or Euclidean distance like traditional PCA variants, this new method evaluates a transformed mutual information metric. Relative to these baselines, it is less reliant on strong parametric or distance assumptions, making it appropriate to apply more broadly. It also serves as a practical diagnostic of the theory explored in the From Garbage to Gold framework, which describes when and why uncurated, error-prone data can be used directly to create accurate prediction models. There is a preprint that presents the full technical details, and Python and R packages will be released soon. Right now though, the original function is already available in R (see Quick Start R Function Code below). Entropic Scree Preprint: https://doi.org/10.5281/zenodo.22028087 Entropic Scree GitHub: https://github.com/tjleestjohn/entropic-scree From Garbage to Gold Preprint: https://arxiv.org/abs/2603.12288 Let me know how it goes if you give it a try... or if you have any questions or comments of course. ############ # Quick Start R Function Code. # To load the function, copy and paste the following into your R console, then hit enter. ############ # 1. Define the direct URL to the raw function script on GitHub url <- "https://raw.githubusercontent.com/tjleestjohn/entropic-scree/main/Entropic.Scree.v1.0.0%20-%20ENLI.R" # 2. Define what you want to name the file on your co

Hi, I'm making posters for ECCV 2026. Does anyone have any ML/CV posters they thought were really well done? Would love to see some cool examples. Thanks submitted by /u/National-Resident244 [link] [留言]

Paste a long document into ChatGPT and hit enter. Nothing happens for a second or two. Then the answer starts appearing, word by word, at a steady pace until it finishes. You have seen this hundreds of times. Most people never think about it. But those are two completely different things happening inside one request, running on two different parts of the same GPU, limited by two different bottlenecks. Once you know what they are, a lot of confusing things about serving AI models stop being confusing. Including why a faster GPU sometimes makes no difference at all. This is part two of a series. Part one covered the split between VRAM and cores. This post covers what each of them does when a request arrives. The example we will follow I am going to trace one question the whole way through: what is the capital of france Simple, short, and it produces a short answer. Perfect for watching the machinery. Before anything happens, the model server has already loaded the model's weights into VRAM. That happened at startup and they stay there. Nothing about your request loads a model. Your words get chopped into tokens, which are just pieces of words turned into numbers, and placed in VRAM alongside the weights. Now the work begins. Phase one: prefill, or "the pause" Before the model can write anything, it has to read everything you gave it. Here is the important part, and it is the opposite of what most people assume: it reads your entire prompt at once. Not word by word. All six words go to the cores together, in a single pass. That is possible because your whole prompt is already there. Nothing has to wait for anything else. So thousands of cores fire simultaneously and chew through all of it in one burst. flowchart LR A["what is the capital of france<br/>all 6 tokens together"] --> C W["16 GB of model weights"] --> C C["GPU CORES<br/>every core busy"] --> O1["First word: 'The'"] C --> O2["Notes saved to VRAM<br/>the KV cache"] Two things come out of that single pass: The

After spending the past two weeks redoing all the models around the house, I realized it might make a good topic to chat about. I know that everyone and their brother has their own way to figure out what models they want to run on their hardware, but I figure that my own criteria might help some of the Mac users out there, so I'm tossing it into the mix as well. Picking which models to even compare When a new model comes out, the first thing I always do is check what folks are saying: huggingface discussions, reddit comments, etc. Benchmarks are useful, but I want to know what happens when people actually use the thing. Is the tokenizer broken or llama.cpp/mlx implementation bugged? Does it follow instructions? Overthink? Hallucinate a bunch? Discussion comment sections are a treasure trove of info. After that, I go peek at the model on Artificial Analysis. I know AA isn't everybody's favorite way to judge models, but honestly it has a pretty solid litmus test for whether the model will be good for me or not. In particular, there are a specific few benchmarks there which line up really well with what I need: strong context reasoning hallucination rate how many output tokens the model used to get its scores That last one is one of the most important. Combined with the "Humanity's Last Exam" score and overall intelligence, it gives me an idea of how much output the model had to produce to actually land on its current spot on the leaderboard. On a Mac, I really care about that. Remember: our compromise with Macs is getting stupidly large amounts of VRAM at the price of everything being a lot slower than NVidia GPUs, especially as context and token generation sizes increase. So if one model gets a slightly better score by generating dramatically more tokens, you gotta keep that in mind. Really long-winded thinking sessions could make a model almost unusable. You might think "I don't mind waiting for quality", but then suddenly find yourself just going to ChatGPT because