标签：#mac

I've been working on a practical implementation of Sliding Window Attention (SWA) for pretrained Hugging Face causal LLMs. The idea is simple: instead of allowing every generated token to attend to the complete historical KV cache, maintain a bounded cache consisting of: attention sinks + recent sliding window I implemented this as a reusable inference layer rather than modifying or retraining the model. GitHub: https://github.com/oraby8/SWA The implementation currently includes: bounded KV cache circular/ring-buffer storage attention sinks streaming prefill chunked attention masking autoregressive decoding Full Attention vs SWA benchmarking TTFT / TPOT / throughput measurements KV-cache memory measurements One interesting result from my Qwen2.5-7B experiment: Context Full KV SWA-64 16K ~923 MB ~3.5 MB 32K ~1.84 GB ~3.5 MB 64K OOM ~3.5 MB At 16K, SWA-64 also reduced TPOT from ~38.4 ms to ~30.5 ms in this setup. However, there is an important trade-off: tasks requiring information far outside the active window can degrade. I'm currently investigating how much of this is inherent to SWA versus implementation/model-specific behavior. I'm sharing the implementation mainly to get feedback from people working on LLM inference, KV-cache optimization, and long-context models . I'd be particularly interested in: Which model architectures should I validate next? What failure cases should I benchmark? What would make this useful for existing HF inference workflows? Are there cache/attention implementation details I may be overlooking? Feedback and experiments are very welcome. submitted by /u/ahsaor8 [link] [留言]

Originally published on hexisteme notes . "Will it fit on our hardware?" is the wrong first question. It's the one everyone asks, because it's free to answer — the thing either loads or it doesn't. Throughput costs you a measurement. So the capacity gate passes, and it feels like the decision is made. The measurement Mac Mini M4, 24GB unified memory, ~120GB/s memory bandwidth. A 27B model, IQ4_XS quantized, 15GB on disk. Capacity gate: pass. Metal's recommendedMaxWorkingSet is 17.76GB, the model is 15GB, ollama ps reports 100% GPU resident. No swap, no spillover. By every "does it fit" criterion this is a clean win. Generation: 5.6 tokens/second. That's not a usable interactive worker. It's barely a usable batch worker. And nothing about the capacity check hinted at it. The arithmetic that would have told me in advance Autoregressive generation reads the entire model's weights once per token. So: ceiling ≈ memory bandwidth ÷ bytes touched per operation = 120 GB/s ÷ 15 GB = 8 tokens/second Measured 5.6 against a ceiling of 8. Ratio 0.70. That ratio is the whole verdict. When measured throughput is a large fraction of the arithmetic ceiling, you are bandwidth-bound , and you now know something concrete: the bottleneck is not your configuration, not memory pressure, not thermal throttling. It's how fast bytes move. Rule of thumb I now use: ratio ≥ 0.5 → bandwidth-bound, and size-reduction fixes are dead. Why "just quantize harder" doesn't work The natural move when capacity is tight is to shrink. Lower quantization, smaller batch, heavier compression. It's the reflex, and in a bandwidth-bound regime it's close to useless. I was considering Q3_K_M at 13.8GB. Run the same division: 120 ÷ 13.8 = 8.7 tokens/second (up from 8) Under 9% more throughput. For a real drop in output quality, because quantization error doesn't scale linearly with size the way bandwidth does — you give up more than you get, every time, in this regime. I killed that plan without downloading anythin

building a missing data infrastructure and started benchmarking long multi-session conversations (LoCoMo). I know the data looks like: people, facts, claims, events, timestamps, relations. So I extract those into a graph. I did not look at the QA pairs while building extractors or retrieval rules. No “if question contains X, fetch fact #173.” Recall is very high and it keeps working on new conversations in the same format. Is this classical overfitting, or just schema-aware engineering? What is the cleanest test that would convince you it isn’t leakage. submitted by /u/chaachans [link] [留言]

I run Knipsmig , a QR-code photo sharing service for weddings and parties. Guests scan a code and upload straight from the phone, no app. Most of those uploads are 12 MP and print fine. A meaningful slice are not: photobooth captures at 1080x810, WhatsApp forwards at 1600x1200, screenshots, old scans someone re-uploaded. Those end up in the printed photo book looking soft. So I added an "Improve resolution" option to the editor. It adds up to 4x the pixels, and the whole thing runs on my own server. No API, no vendor, nothing leaves the box. This post is about why I went local and what it took to make that work inside a Rails app. Why not just call an image API I already have Gemini and OpenAI keys configured in the app for other things, so the lazy path was obvious. I did not take it, for three reasons. The generative models redraw the image. They don't upscale, they regenerate. Faces drift. These are guests' faces at someone's wedding, and "your aunt looks slightly different now" is not a feature. A super-resolution network stays faithful to the input: it only adds pixels consistent with the ones already there. Privacy paperwork. Every third-party processor I add has to go into the DPA. Guests' photos leaving the server to be fetched by a vendor is a real change, not a footnote. Running locally means the data processing agreement doesn't change and the existing opt-out for third-party AI stays about third parties. Cost. Per-image API pricing on a bulk action over hundreds of photos adds up fast. CPU time on a job lane I already pay for is free at the margin. The model I went with realesr-general-x4v3 from the Real-ESRGAN project (BSD-3-Clause). It's the compact SRVGGNet variant: about 1.2M parameters, roughly 5 MB as an ONNX file, and around 10x faster on CPU than the full RRDBNet x4plus. Quality is more than fine for event snapshots. Getting it into a usable shape was a one-off: export the release weights with the repo's pytorch2onnx.py script using dynamic H/W a

Hi All, Was reading AIStats' website and it seems like abstract submission is due in 3 weeks. Does anyone know where to find the LaTex template for 2027? It seems like very little information is available on their website. Another question, is a Quant Finance paper a better fit for AIStats or ICLR? Some background about the paper: Rejected by UAI with 76654, had some errors with proofs had to fix it by re-writing 9 pages during rebuttal. AC rejected the paper saying the changes were too substantial and unable to be fully verified during rebuttal period. Resubmitted the fixed paper to a finance conference, won best paper award (best paper for this conference usually end up in journals like JQFA, which is just 1 tier below the big 3 in finance), had the chief editors of a Q1 finance/math journal in the conference verbally offering he will take this paper if we submit it to his journal. Unfortunatley my department requires at least 1 Comp Sci paper to graduate, so my plan is to try and get this paper accepted into a Comp Sci conference, then submit an extension to that Q1 Finance/Math journal. Rejected again at ICDM, despite having all positive scores. Our AC meta-review was blank so we still do not know why we were rejected. All of our emails receieved no reply. I am torn between ICLR or AIStats to re-submit this paper to. My worries are: In comp sci venues we frequently get comments like "this paper lacks novelty. The method is just XXXXX, the math is just XXXXX." But I had a scroll through at previous year's AIStats papers for key words like finance and there were none. It seems like AIStats is very pure stats, not that applied. My co-author is worried that the math in our paper is not hardcore enough. We have never submitted to neither venues in the past. Would be nice to get some advice. submitted by /u/d_edge_sword [link] [留言]

I ran a side-by-side ML text-processing and model-training workflow using Fable 5.1 vs. Astra (both on xhigh), and the results could not have been more different. Warning, long post. TL;DR -- Astra codes more agentically, Fable more coherently. Fable writes better and follows directions better. Astra's final outcome was slightly better, and its scientific rigor/reproducibility was noticeably stronger. Both models improved their F1/Accuracy by 0.02-04 after human feedback on their approach, demonstrating that neither have mastered the AI/ML text processsing, vectorization, and model training process completely. Astra is a better coder, writing a stricter evaluation protocol (70/15/15 train/val/test vs. Fable's basic 80/20) that selected its model using a held-out validation set vs. Fable's simpler test F1-based selection. It also debugged more deeply, as both models hit a gensim 4.4 compiled-kernel bug: Fable tried to figure it out, failed, and just hid the stderr notices on affected runs (though told me it had done so), while Astra root-caused it aggressively, then fixed the environment by downgrading gensim alongiside compatible NumPy/SciPy dependencies. Astra wrote hardened training-run.py code the forced the uv venv it rebuilt without changing my default one, SHA-256'd the corpus to ensure reproducibility on later runs, output a split manifest and run-summary.json, and rendered a headless browser for QA with screenshots (not sure this was necessary, but impressive overkill all around). Fable's builder script was ephemeral, living only in tmp, and less intense overall. Astra deployed subagents more effectively, making use of my pre-built notebook-reviewer and citation-checker agents, the former of which caught a real bug via review (sentence-final word-loss tokenization defect) and fixed it, retaining a regression test in the process. Fable overlooked this issue because, for some reason, it did not call the subagents I had available (which is surprising, usually i

A researcher has reported a jailbreak of GPT-6 Astra within a day after release. The attack is described as combination of TIP (Task-in-Prompt) attack from ACL 2025 paper with four other unnamed techniques. TIP attacks exploit the model’s reasoning/instruction-following behaviour by hidding the harmful objective inside another task, like solving a cipher or executing a Python code. For GPT-6, the researcher says the original minimal TIP attack was no longer sufficient and had to be reworked. They have reportedly disclosed the details privately to OpenAI rather than publishing the jailbreak. The same researcher reported jailbreaking GPT-5 within an hour of its release a year ago. Source: screenshot/post from the researcher ; their ACL 2025 TIP paper linked in the original post. submitted by /u/Asleep-Requirement13 [link] [留言]

I run seven agents over the same domain. They have never once sent each other a message. That was not the plan. The plan was the thing everybody builds first: a coordinator that hands work between specialists, agents that call each other, a shared conversation they all append to. It worked in the demo and it fell apart the moment the work got real. What replaced it is boring and it has held up: every agent writes claims to one shared record, and nothing else. No agent reads another agent's reasoning. No agent can call another agent. The record is the only channel. Here is why, and what it cost. What breaks in the group-chat design Three things, roughly in the order they hurt. Context grows without bound. If agents converse, every agent needs everyone else's output in its window to participate. Six specialists means each one is reading five other monologues. Your token spend goes quadratic in the number of agents and the marginal agent makes the others measurably worse. Errors laminate. Agent B reads agent A's output as input. If A was confidently wrong, B does not treat it as a claim to be weighed — it treats it as context, which is to say, as true. By the time it reaches F you have a well-reasoned conclusion resting on a hallucinated premise, and nothing in the transcript flags where the floor gave way. You cannot answer "why." Six weeks later someone asks why the system concluded X. The honest answer is "there was a conversation." That is not an answer you can act on, and it is not an answer that survives an auditor. Agents as authors, not as callers The reframe that fixed it: an agent is not a function other agents invoke. An agent is an author with a domain of authority . Each of mine owns a slice of the problem and may only make claims inside it: Agent Domain Claims it may make Verification What is true about the thing today Observed facts, source records, reconciled geometry Design What it should become Plan gaps, code compliance, takeoffs Recovery What can be

Just received an email about the automatic reference/citation checker. Did anyone receive a follow up email about whether the checker was included in the paper's decision making too, along with the general instructional email? submitted by /u/Emergency_Plate241 [link] [留言]

Abstract Language models spend most of their attention on a small fraction of context, yet they read the entire KV cache to find the few tokens that matter. If the user asks about a previous detail in a 1M-token conversation, global attention layers must scan the full context to generate each token of the reply. A prominent approach mitigates this cost by pre-selecting relevant tokens via lightweight proxy scores, but this extrinsic scoring still incurs O(N) per step. We take an intrinsic approach motivated by the simple question: wouldn't the model already know which parts of the context are relevant? To this end, we introduce Declarative Attention (DA), a protocol that elicits the model to declare where it needs to attend within its chain-of-thought, partitioning generation into three modes: <global> (full context), <focus> (a specific region), and <local> (recent output only). The inference engine parses these declarations like tool calls and skips most of the KV cache read. Under zero-shot evaluation across 15 long-context tasks, DA on off-the-shelf models (Gemma-4-31B, Qwen-3.6-27B) significantly reduces total attended tokens during decoding (52.0%, 31.1%) with modest accuracy drops (1.27pp, 2.75pp) that shrink with model scale. DA unlocks a new axis of sparse attention, with further potential under training-based methods that future work can explore. Subjects: Computation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG) Cite as: arXiv:2609.02737 [cs.CL] (or arXiv:2609.02737v1 [cs.CL] for this version) https://doi.org/10.48550/arXiv.2609.02737 Focus to learn more submitted by /u/eigenlaplace [link] [留言]

submitted by /u/Winter_Mistake_3185 [link] [留言]

เมื่อ Benchmark โกหกคุณ, SWE-Bench ProMax กับคะแนนจริงที่โมเดลเก่งสุดทำได้แค่ 41.2% โดย Nokka (นก-กา), นักเขียนอิสระสายเทคโนโลยี ผู้เขียนบทความอธิบายเทคโนโลยีให้คนทั่วไปเข้าใจ 30+ บทความบน dev.to | 5 กันยายน 2026 บทความนี้เขียนโดย AI (glm-5.3 via ollama-cloud) ผ่าน Hermes Agent ภายใต้การควบคุมและตรวจสอบคุณภาพโดยมนุษย์, Nokka (นก-กา), อ้างอิงจาก paper วิจัย SWE-Bench ProMax บน arXiv ฉบับเต็ม เลข 90% ที่ทุกค่าย AI โชว์กันในปี 2026 อาจไม่ได้แปลว่าโมเดลเก่งขนาดนั้น paper วิจัยใหม่บน arXiv เปิดตัว benchmark ชื่อ SWE-Bench ProMax แล้วชี้ตัวเลขที่ต่างจากกระแสโดยสิ้นเชิง: เมื่อวัดด้วยงาน refactor จริงข้ามหลายไฟล์หลายภาษา โมเดลที่ดีที่สุดทำได้แค่ 41.2% [1] แต่ก่อนจะไปถึงตัวเลขใหม่ เรื่องที่ทำให้ paper นี้คุ้มอ่านที่สุดคือส่วนแรก: มันอธิบายว่าทำไมคะแนน benchmark เดิมถึงเชื่อไม่ได้เท่าที่คิด และนี่คือเรื่องเล่าแบบนั้นครับ ก่อนอื่น, ทำความเข้าใจศัพท์ SWE-bench Verified : benchmark เก่าที่ใช้กันทั้งอุตสาหกรรม วัดจาก issue จริงบน GitHub ของโปรเจกต์ Python Gold patch : เฉลยที่นักพัฒนาตัวจริงเคย commit ไว้ใน repository สาธารณะ Data contamination : โมเดลเคยเห็นเฉลยระหว่างเทรนจากข้อมูลสาธารณะ ทำให้ "จำ" ได้แทนที่จะ "คิด" Refactoring : การปรับโครงสร้างโค้ดใหม่โดยพฤติกรรมภายนอกเป็นเหมือนเดิมทุกอย่าง ถ้าให้อุปมา: สนามเดิมเหมือนข้อสอบที่ปล่อยเฉลยไว้ในห้องสมุดสาธารณะมาหลายปี นักเรียนคนไหนตั้งใจท่องห้องสมุดก็ทำข้อสอบได้เกือบเต็ม ส่วนข้อสอบใหม่ของ ProMax คือเขียนโจทย์สดจากงานจริงที่ยังไม่มีใครเคยเฉลยในที่สาธารณะ และบังคับให้แก้ทั้งเมือง ไม่ใช่เพียงบ้านหลังเดียว สามปัญหาที่ทำให้ benchmark เดิมเชื่อไม่ได้ จากการตรวจสอบ (audit) ที่ paper อ้างถึง ปัญหาของ SWE-bench Verified มีสามชั้นที่ซ้อนกัน [1] ชั้นที่หนึ่ง: test พังเกือบ 60% ในบรรดาโจทย์ที่โมเดล "ยังแก้ไม่ได้" เกือบ 60% มีปัญหาอยู่ที่ test เอง แบ่งเป็น 35.5% ที่ test แคบเกินไป (โมเดลแก้ถูกแล้วแต่ถูกตัดเพราะวิธีไม่ตรงกับที่คนเขียน test คิด) และ 18.8% ที่ test กว้างเกินไป (ตรวจสิ่งที่โจทย์ไม่เคยบอกให้ทำ) หนักสุดคือ OpenAI ถอด benchmark นี้ออกจากการใช้งานไปแล้วทั้งหมด ชั้นที่สอง: โมเดลจำเฉลยได้ งานวิจัยพบว่าโมเดล frontier สามารถขึ้นมาซ้ำเฉลยจา

From what I've seen online so far, the description of these systems is roughly: They asked the model (often Aster) to generate statements in LEAN and then submit those to a LEAN compiler to be checked. Based on the results of attempting the LEAN compilation, they somehow add those statements as fact. When the full proof in LEAN compiles, the system is finished. I can imagine trying to jam as much of a proof as possible into the context window but some of the papers these systems have produced are hundreds of pages. To me this would indicate that somehow the paper is being built piece by piece and being assembled before being submitted to LEAN. This resonates with the part of my understanding that after checking LEAN compilation there's some kind of management of "facts." I would like to try to implement my own janky version and see if it can answer a question I have about higher dimensional geometry. I'm struggling to find a meaningful way to compose larger ideas from smaller ones. I can imagine it's relatively simple if you know what to do. What things have you seen? Do you have any ideas you haven't seen that might be interesting to try? Is this a fool's errand because you really need huge amounts of hardware to do anything meaningful? I would welcome any thoughts or links on the matter, cheers submitted by /u/tough-dance [link] [留言]

an observation : GPT-5-class models are genuinely capable(They are) of doing a substantial fraction of knowledge work, why haven’t we seen a noticeable productivity shock in the real economy yet? Is AI actually less economically useful than the benchmarks suggest—or are organizations simply too slow, constrained, and inefficient to turn model capability into measurable output? Are we confusing “AI can do the task” with “AI can replace the economic system built around the task”? If GPT-5 is already this capable, what exactly is the bottleneck preventing that capability from showing up in GDP and productivity statistics? My take : There is no question that these models are genuinely impressive. The question is whether that intelligence is actually translating into measurable economic productivity. People are already asking whether models like GPT-6 or equivalent. Claude, and Gemini will replace large sections of white-collar workers. I think there is a much simpler question we should ask first: if these models are already so capable (to me they definitely are capable enough) at a huge range of knowledge work, why haven't we seen a correspondingly obvious increase in productivity? I'm not even talking about GPT-6 or whatever comes next. It's probably too early to judge a newly released model. I'm talking about the current generation—GPT-5 and its equivalents from Google and Anthropic. These systems are genuinely good. They can write, summarize, analyze documents, explain technical concepts, generate code, reason through problems, conduct research, manipulate information and perform a remarkable range of tasks that previously required educated human labour. And yet, looking at the world around us, something feels strange. Where is the enormous productivity shock? Why don't we see a dramatic effect on GDP growth? Why don't we see massive increases in output per knowledge worker? Why don't organizations appear to be accomplishing dramatically more with the same number of

Onur Satici explains how Vortex, an open-source columnar file format under the Linux Foundation, revolutionizes high-throughput data loading. He details how cascading lightweight encodings, layout-based segment pruning, and zero-copy memory pipelines eliminate CPU/NVMe bottlenecks to stream S3 data straight to GPUs at speeds up to 60 Gbps without requiring upfront data reprocessing. By Onur Satici

Macs are fantastic computers, but you’ll need one of these great displays to complete your work-from-home setup.

Benchmark scores: https://preview.redd.it/dgumcg67ggnh1.png?width=1378&format=png&auto=webp&s=fae8fb006ef46fcdebb0876717fc977a905baa89 https://openai.com/index/gpt-6-astra/ Above, GPT-6 uses a harness for ARC-AGI-3, and is at about 60% without one: https://preview.redd.it/bym9wajephnh1.png?width=615&format=png&auto=webp&s=72cb425fb037ce68a68dcb433e7748b27dc96c41 Prior to the launch, OpenAI President Greg Brockman said "I think it’s not unreasonable to feel that we are now in the AGI era". GPT-6 is now joining a growing list of models that greatly exceed the human baseline on GDPval-AA v2: https://preview.redd.it/to7tdvbn4inh1.png?width=1419&format=png&auto=webp&s=02c413ab031cd943087684c5c573ce1e524b917d If we have AGI, why do human knowledge/remote workers still have jobs? Is it just a matter of time until the economy replaces a large number of humans with LLMs, or are LLMs lacking something that these benchmarks fail to measure? submitted by /u/we_are_mammals [link] [留言]

Has anyone else received an AAAI-27 desk rejection related to modifications to the title or abstract between the abstract-registration deadline and the full-paper deadline? What I’m trying to understand is how the modification rule is being applied in practice. The AAAI-27 modification guidelines say that the title and abstract can still be edited after abstract registration, while warning against substantive changes, and describe rejection in terms of changes that make the submission describe qualitatively different research. In my case, almost everything was identical. The modifications were incredibly minor. The rejection notice says that the decision is final and appeals will not be considered. Did this happen to anyone else? submitted by /u/Dansilly [link] [留言]

Hi everyone, I just quickly wanted to share a paper I was working on for around a year now. I created this summary website with key results: https://flogrammer.github.io/moljepa/ TL;DR: its a multimodal JEPA model for molecules. There will be more work to do to improve performance and I would be happy about feedback and ideas :) submitted by /u/TerribleAntelope9348 [link] [留言]

Three weeks from decisions even. I wonder what percentage is industry and VC funded AI labs looking to mingle and recruit. submitted by /u/alrojo [link] [留言]