标签：#m

Every "run this model locally" guide tells you to grab a Q4 GGUF and move on. That advice is fine right up until you try a long-context run and your machine starts swapping. The weights are the part everyone budgets for Quantization maths is straightforward. A model's weight footprint is roughly params x bits / 8 : Quant Bits/param 12B model Quality note Q8_0 ~8.5 ~12.8 GB Near-lossless, rarely worth it Q6_K ~6.6 ~9.9 GB Very close to Q8 Q4_K_M ~4.8 ~7.2 GB The usual sweet spot Q3_K_M ~3.9 ~5.9 GB Noticeable degradation Below Q4 the loss stops being subtle. Instruction-following degrades before raw perplexity does, which is why benchmark numbers can look fine while the model quietly stops respecting your system prompt. The KV cache is the part that bites Here is what the guides skip. The KV cache scales with context length , and it is not quantized by default: kv_bytes ~= 2 (K and V) x layers x kv_heads x head_dim x seq_len x dtype_bytes The practical consequence: a model that loads in 7 GB can need well over twice that at long context. Grouped-query attention helps a lot — kv_heads is much smaller than attention heads — but the term still grows linearly with sequence length while your weights stay fixed. Two knobs matter more than picking a fancier quant: --ctx-size : do not allocate 128K if your prompts are 8K. You are reserving memory you will never touch. KV cache quantization ( q8_0 for K/V): roughly halves cache memory for a quality hit most workloads never notice. Underused. A decision order that works Start at Q4_K_M Set context to what you actually use, not the model maximum If you are still tight, quantize the KV cache before dropping to Q3 Only move up to Q6/Q8 if you have headroom left over That ordering matters: dropping to Q3 to buy context is the most common mistake, and it trades a permanent quality loss for memory you could have gotten from the cache instead. Per-quantization benchmarks and deployment notes for the Gemma 4 family are collected at ge

The eight-episode series stars Michelle Yeoh and Hunter Schafer.

This climate phenomenon could cause a massive global temperature spike and cause economic losses of $10 trillion by 2032.

The different IMAX filming and projection formats have a dramatic impact on the theater viewing experience.

I'm a marketer. When I land on a developer tool's website, I don't open the pricing page. I don't read the features. I don't watch the demo. I ask one question: "Can I explain what this product does in 10 seconds?" If the answer is no, you've already lost me. I've worked with enough SaaS products to know that most of them don't have a product problem. They have a communication problem. Developers spend weeks building a feature. Marketing spends days trying to explain it. Users spend three seconds deciding whether it's worth their time. That's a brutal mismatch. The best products I've seen don't try to sound intelligent. They try to sound obvious. You read the headline and instantly think, "I know exactly who this is for." That's incredibly hard to achieve. And it's usually the result of dozens of conversations between product, engineering, support, and marketing. So here's my hot take: A feature isn't finished when it's merged into main . It's finished when a complete stranger understands why it exists. As a marketer, that's the lesson building products has taught me. Curious to hear from developers: Have you ever built something technically impressive that users simply... didn't understand?

In response to a report that Nothing is planning to "exit 12 markets as global shipments decline," Nothing cofounder Akis Evangelidis said the company is "reorganizing" and laying off some of its staff, but that "the reported numbers are way overblown." Evangelidis disputed a claim about underwhelming sales for Nothing's Phone 4B, saying it "sold […]

Roku has increased prices across its streaming hardware, with the cheapest HD Streaming Stick now priced at $39.99 instead of $29.99, as first reported by The Desk. The price hike affects all of Roku's other devices as well, with some increasing by as much as $50. Here's a rundown of the changes: Roku is currently […]

"We can quash the subpoenas, or you could withdraw the subpoenas,” judge told US.

A U.S. citizen has asked a court to throw out the government's claim that he gave over a passcode to border authorities that wiped his phone's data, opening up fresh questions about a person's constitutional rights at the U.S. border.

submitted by /u/vitiral [link] [留言]

New delay on Copernicus satellite pics comes as US ramps up war with Iran again.

After teasing the series with some first-look images yesterday, Amazon is finally properly unveiling its Blade Runner streaming series. Called Blade Runner 2099, the show hits Prime Video on November 25th with all eight episodes dropping at once. We also got the first trailer for the series, which is full of Blade Runner staples: a […]

submitted by /u/xoner2 [link] [留言]

Anduril is said to be raising a new round of funding that may push its valuation up to about $100 billion, per Reuters.

Google's AlphaFold can help ID what parts of a gene editing protein enable mistakes.

This week, I covered a fascinating effort to preserve organs outside the body. There’s a huge shortage of donor organs, and one of the main reasons is time—they survive only a matter of hours outside the body, even when they’re kept on ice. Doctors dream of organ banks—stores of human organs that can be preserved…

submitted by /u/tamanikarim [link] [留言]

Meta is upgrading its AI chatbot with new productivity features in a bid to compete with rivals like Gemini, ChatGPT, and Claude. The update will allow Meta AI to tap into your calendar to help you plan events and generate daily briefings, as well as perform in-depth research that you can steer as it progresses. […]

The offline messaging app surged in popularity in India amid protests in New Delhi.

The post-apocalyptic tale takes place after the events of The Legend of Korra.