标签：#RAM

submitted by /u/nephrenka [link] [留言]

WebLLM: The Rise of AI That Runs Directly in Your Browser For the last few years, the dominant architecture for generative AI has been straightforward: Your application → Cloud API → Large Language Model → Response Every time you interact with an AI application, your prompt or data is typically sent to a remote inference service. But a different architecture is emerging: Your browser → Local AI model → Your device's GPU This is where WebLLM becomes interesting. WebLLM is an open-source, high-performance inference engine that allows large language models to run directly inside a web browser using WebGPU . The inference can happen on the user's device rather than on an application server. That seemingly simple change has significant implications for privacy, cost, offline AI, AI agents, enterprise applications, and cybersecurity . What exactly is WebLLM? WebLLM is not another large language model like Llama, Qwen, Gemma, or Mistral. Instead, think of WebLLM as an AI runtime for the browser . It provides the infrastructure required to load compatible open-source models and perform inference using the user's hardware. The basic architecture looks like this: Traditional AI User ↓ Web Application ↓ Backend Server ↓ LLM API / GPU Infrastructure ↓ Response With WebLLM: Web Application ↓ WebLLM ↓ WebGPU ↓ User's GPU / Device ↓ Local LLM inference WebLLM uses WebGPU for hardware acceleration and provides an OpenAI-compatible API, making it possible to integrate local models into JavaScript/TypeScript applications using familiar patterns. Why does this matter? The most important word is: Local Instead of sending every request to a remote AI service, an application can perform inference locally in the browser. That creates several potential advantages. 1. Privacy Consider an employee using an AI-powered security assessment tool. They may upload: Architecture diagrams Security policies Source code Vulnerability reports Compliance evidence Internal documents Configuration files W

100% Human Authored - Both Code/Doc In this series, We build a production-ready containerized RESTful API server application using following packages and tools. Go Standard Library net/http: The most idiomatic way to write web API applications in Go. Chi: The most idiomatic router with middleware and route groups support. Zerolog: Zero Allocation JSON Logger and Faster than slog. Goose: The database migration CLI builder library with lease dependency. Gorm CLI: Generate Go generics-based, type safe, repository functions with no runtime wrappers. Swag and Validator v10: The most prominent OpenAPI specification generator and validation library in Go. Codebase: https://github.com/learning-cloud-native-go/myapp (1000+ GitHub stars- Since 2019) \ Doc: https://learning-cloud-native-go.github.io/docs/building-a-containerized-restful-api/ If anybody wanna check Rust implementation: https://learning-rust.github.io/labs/building-a-containerized-restful-api/ submitted by /u/dumindunuwan [link] [留言]

Have you ever read a job posting and your heart just sings the song of "HIRE ME PLEASE!!!", yeah,...

A scheduled job of mine drives a real Chrome profile that stays signed in to DEV, because the API can read comments but cannot create them. One run came back with the dashboard replaced by the sign-in page: the log said it had opened https://dev.to/dashboard , and what it actually landed on was https://dev.to/magic_links/new , with zero links to my own profile anywhere in the DOM. The profile itself was fine. A probe against the debugging port at the same moment returned a live Chrome, and the dashboard fetched through that port rendered the account's own identity links normally. Two browsers, same machine, same minute, opposite answers. The three things worth checking first, and why they miss The session expired. That is the reflex, and it is also the one that makes you re-authenticate for no reason and burn the logged-in state you were trying to protect. Cookies got cleared by a Chrome update. Same family, same cost if you act on it. The debug port died and the tool fell back to something else. This one is close enough to be dangerous, because it names the right layer — which browser am I attached to — and then picks the wrong cause inside it. Where it actually goes wrong It is one argument. agent-browser attaches to an already-running Chrome when you pass --cdp <port> . Leave the flag off and it starts its own browser, with its own empty profile directory, and drives that one instead. Everything downstream still works — it navigates, waits, evaluates, returns a page. It just does all of that in a browser that has never logged in to anything. So the automation is not looking at an expired session. It is looking at a different browser's logged-out session, and reporting it in exactly the shape a real logout would take. The two failure modes do not look alike, and that is the trap Here is what I measured today, on Chrome 152.0.7977.65 with the current npx build. Pass the flag, but point it at a port nothing is listening on: npx -y agent-browser open "https://dev.to/

Cats-Actors is a Cats Effect native actor library: typed messages, functional state, supervision, and the familiar ! operator, all in F[_]. What is new in this release: - ControlledTestKit, a new trait in the cats-actors-testkit module. It provisions an ActorSystem[IO] inside Cats Effect's TestControl and ticks the simulated clock for you, so scheduled work and timeouts resolve without real waiting. A one hour receive timeout is now an ordinary unit test that finishes in milliseconds. - Receive timeouts, the supervisor restart window and the dead letter mailbox idle check now read Clock[F].monotonic instead of System.currentTimeMillis, so they all honour simulated time. - New TestKit assertions, expectMsgTypeCountN and expectMsgTypeSingle, which count messages of a type and do not depend on when they arrive relative to the call. - Breaking: ActorSystem.uptime is now F[Long] rather than Long. Scala 2.13 and 3, on JVM, Scala.js and Scala Native. ``` resolvers += "jitpack" at " https://jitpack.io " libraryDependencies += "com.github.cloudmark.cats-actors" %%% "cats-actors" % "2.2.0" libraryDependencies += "com.github.cloudmark.cats-actors" %%% "cats-actors-testkit" % "2.2.0" % Test ``` Write up: https://cloudmark.github.io/Cats-Actors-Controlling-Time/ Repo: https://github.com/cloudmark/cats-actors Feedback and issues welcome. submitted by /u/kloudmark [link] [留言]

HashMaps are incredibly convenient, but treating them like a magical black box can make it easy to overlook where their performance comes from. In this article, I explain how HashMaps work using a simple implementation in Rust. I go through hash collisions and linear probing, why primary clustering hurts performance, load factors, resizing, and more. The implementation is intentionally simplified, but it gives a good picture of what's actually happening underneath a HashMap. Let me know your thoughts or any feedback you have! All writing, illustrations, and code are my own. AI was used to clean up wording and catch grammatical mistakes. submitted by /u/PastEar9661 [link] [留言]

submitted by /u/syrusakbary [link] [留言]

Wrote a quick blog post about pushing branch prediction to its limits in a simple experiment. Take a read to learn with me about measuring performance on linux machines and a very basic understanding of branch prediction 😄 submitted by /u/chaobang [link] [留言]

Hello, I'm Maneshwar, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product. You open a video and it starts playing in about a second. Somewhere between your thumb and that first frame, your request crossed maybe fifteen different companies' equipment, possibly an ocean, and came back. Nobody coordinated it. That is the part I find genuinely strange about the internet, and it is the part most explanations skip. They tell you the internet is "a global network of networks", which is true and tells you nothing. So let's actually take it apart. There is no internet. There are 75,000 of them. The single most useful thing to understand up front: the internet is not a thing anyone built. It is roughly 75,000 independent networks that agreed on how to hand traffic to each other. Your ISP is one. Your university is one. Cloudflare is one. They own their own cables and routers, they answer to nobody in particular, and they interconnect voluntarily. Once you see it that way, every weird thing about the internet starts making sense. The whole arrangement has three parts: The edge is everything that actually wants to say something. Your phone, a laptop, a server in a rack, and increasingly a doorbell. These are called hosts or end systems , and they split roughly into clients that ask and servers that answer. The access network is your on-ramp. Fibre or cable at home, the office network, 5G from your pocket. Its only job is getting you to the first router. It is also, almost always, the slowest part of the entire journey, which is worth remembering next time you blame a website for being slow. The core is the mesh in the middle. Routers and the links between them, and nothing else. No control room, no master server, no company that owns it. Nobody reserved you a line Here is where the design gets clever. Before the in

Hello, I'm Shrijith Venkatramana, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product. There is a slightly strange thing about modern LLMs. We are increasingly asking them to make judgments that look less like autocomplete and more like governance: Should I answer this request? Is this instruction legitimate? Is this response too dangerous? Should I refuse, or can I safely help? What should I do when two desirable goals conflict? Traditionally, we tried to answer these questions by collecting more human preference data. Show an annotator two responses. Ask which is better. Collect millions of comparisons. Train a reward model. Optimize the LLM against it. That works surprisingly well. But it has an awkward scaling property: humans have to inspect the behavior we want the model to learn. Anthropic's Constitutional AI idea takes a different route. Instead of asking humans to label every questionable behavior, give the model a written set of principles—a "constitution"—and use another model to critique, compare, revise, and eventually train the target model. That seemingly small change leads to an important engineering idea: A natural-language rule can become a source of synthetic training data, a reward signal, and even a runtime safety mechanism. This article explains how that works, from the intuition to the mathematics and operational trade-offs. 1. The core idea: turn values into a learning loop Suppose you are building an assistant that should be helpful without producing harmful instructions. With ordinary supervised fine-tuning, you might write examples like: User: How do I make a dangerous chemical? Assistant: I can't provide instructions for making it. You need many examples covering many variations: different wording different domains indirect requests role-playing obfuscated requests borderline

submitted by /u/sup_bruh_1 [link] [留言]

For unimportant reasons, I've been forced to code manually these last couple of days and honestly, I forgot how much I LOVE to code manually. And also I realized that I was rusty in stuff that was second nature to me before. A couple of months back I started a couple of personal projects where I didn't use any AI (only to learn maybe instead of googling, but no claude and no pasting). It was really interesting. I learned a new language and new problem (I did a simple deep learning framework on c). Do you guys allocate time to code manually? I think it's important and I don't do it enough. I'm curious if I'm the only one submitted by /u/TomasPiaggio [link] [留言]

submitted by /u/fagnerbrack [link] [留言]

submitted by /u/drmorr0 [link] [留言]

Introduction I am a big fan of the Model Context Protocol. MCP keeps getting better, and I...

submitted by /u/kciter [link] [留言]

Okay, but I have multiple replicas. And it gets slightly worse: the number of physical connections is not equal to the number of replicas, and it is not equal to the number of business-level connectors either. Several business connectors may share one external identity. One external identity may require several physical connections. And those connections need to be spread across whatever replicas are currently alive and have capacity. So who actually owns the connection? Then the questions start piling up. What if one replica dies? What if it doesn’t die, but loses access to the database? What if two replicas race for the same connection? What if the event that was supposed to wake the right worker never arrives? What if most workers are already full? A few questions and 2 hours of midnight walk in headphones and ChatGPT later, I had leases, runtime slots, reconciliation, failover, capacity limits, and a distributed ownership problem on my hands. This is the architecture I ended up with, and I have mixed feelings about this design. On one hand, I’m proud that I managed to account for so many different edge cases. On the other, the whole thing feels worryingly complex. So I’d really value input from people who have built similar systems. What did I miss or do you see anything that can break? And most importantly: what can be simplified without losing the guarantees? The full architecture description is a bit lengthy: AEON NEON - Connector Runtime - by Jarek J. Many thanks if you decide to read it and share your thoughts. I’d really like this one not to become another failed experiment. submitted by /u/doker0 [link] [留言]

A preview of the September cohorts of the InfoQ Online Certification Programs, and the facilitators leading them: Luca Mezzalira, Michelle Brush, Zichuan Xiong, and Premanand Chandrasekaran. By Artenisa Chatziou

submitted by /u/tjpalmer [link] [留言]