AI 资讯

Half a year ago, I wanted to see for myself what can we currently have with local LLMs. I went down the rabbit hole, learned quite a lot in the process, and shared my results in an article . The results were pretty discouraging: even with 32 GB VRAM, the best models I could run were both too slow and too dumb. At the same time, what you could get for free from inference providers was actually decent - and much faster. I remember my conclusion: "Let's wait for the next generation of models, which looks very promising. If we can run something comparable to full-size Qwen3-Coder-480B locally, that would be year of the Linux Desktop age of fully capable local LLMs. And now this day has arrived. Models Half a year later, I'm revisiting this question. And this time, the whole situation has turned upside-down. Almost none of the providers still have free tier, and anything that's still free is barely good enough even for the simplest tasks. And is rate-limited all over. And on the local side, the next Qwen lineup is out. So, that's what I'm going to be looking at. Once again, I have two RX6800's, 16 GB each, and 64 GB RAM. On one hand, this is more VRAM than any "normal person" can have with one GPU - unless you've got something specifically for AI, like an unified-memory Mac or a DGX Spark. On the other hand, RX6800 is "pre-AI" - anything newer will have much better performance thanks to tensor processors. Qwen3.6-27B : This is a dense model, so basically you can't run it at all on anything less than 32 GB VRAM. It's the slowest one, but also the best one if you can run it. Its accuracy is claimed to be on par with Claude 4.5 Opus, and better than Qwen3.5-397B-A17B . This is what I've been waiting for. It runs reasonably fast on my setup, so it's very much usable both in terms of performance and accuracy. Qwen3.6-35B-A3B : This one is MoE, and it's pretty small, so it's the fastest one. It's good for anything that doesn't require too much (i.e. for agentic tasks that don'

GitHub said this week that Copilot CLI, when it runs inside a GitHub Actions workflow, will accept the built-in GITHUB_TOKEN for authentication. Per the July 2 changelog, the previous path required creating and storing a personal access token. The operational read is small and precise: one fewer human-owned credential to mint, rotate and inherit. The exact scope of the change The changelog covers a narrow surface. It applies to Copilot CLI when invoked from a GitHub Actions workflow, and it swaps the required credential from a PAT to the workflow's ambient GITHUB_TOKEN . GitHub does not describe changes to how Copilot CLI authenticates outside Actions, and this post will not extrapolate to those contexts. If your Copilot CLI usage lives on a developer laptop or in another CI system, nothing in this announcement moves for you. Why the PAT was the wrong credential to leave in the loop A personal access token has almost none of the properties you would want from an automation credential. It does not expire on a job boundary. It carries a person's identity, not the workflow's. It sits in Actions secrets long enough to outlive the engineer who created it. And its scopes were chosen by that engineer, at that moment, often wider than the job actually needs. GITHUB_TOKEN is the opposite shape. Actions mints it at the start of a job, scopes it through the workflow's permissions: block, and revokes it when the job ends. If the token leaks, the window for abuse is the runtime of the job, not the years until somebody remembers to rotate it. When the person who wrote the workflow leaves, the pipeline does not silently break because a token expired with their account. For scripted Copilot CLI calls that had to be wrapped in a PAT, that is the whole win. The tool authenticates against the workflow instead of against a human. Wiring it up The workflow-side pattern is the same one every GITHUB_TOKEN -consuming step already follows: declare permissions: explicitly at the job level, k

We’ve all been there. You start your morning feeling like a Productivity God, sitting straight and typing at 120 WPM. Fast forward four hours, and you've morphed into a literal shrimp, face inches away from the monitor, hunting for a missing semicolon. 🦐 In this era of remote work, real-time posture correction and computer vision for health have become more than just "cool projects"—they are spinal lifesavers. Today, we’re going to build a desktop application using MediaPipe , WebRTC , and Electron that monitors your neck angle and sends a desktop notification the moment you start slouching. By leveraging MediaPipe Pose and TensorFlow.js , we can calculate the Forward Head Posture (FHP) ratio with surgical precision directly in the browser environment. The Architecture 🏗️ Before we dive into the code, let’s look at how the data flows from your webcam to that "Sit up straight!" notification. graph TD A[Webcam Feed] -->|MediaStream| B(WebRTC API) B -->|Video Frames| C[MediaPipe Pose Model] C -->|Landmarks| D{Geometry Engine} D -->|Calculate Ear-Shoulder Angle| E{Threshold Check} E -->|Angle > 30°| F[Electron Main Process] F -->|Trigger| G[System Desktop Notification] E -->|Healthy| H[Continue Monitoring] style G fill:#f96,stroke:#333,stroke-width:2px Prerequisites 🛠️ To follow along, you'll need the following tech stack: MediaPipe Pose : For high-fidelity body tracking. WebRTC : To capture the video stream from your webcam. Electron : To wrap our logic into a desktop app that runs in the background. TensorFlow.js : The backbone for running ML models in JavaScript. Step 1: Setting up the Video Stream (WebRTC) First, we need to grab the camera feed. In a modern browser environment (or Electron's Chromium), we use navigator.mediaDevices.getUserMedia . async function setupCamera () { const videoElement = document . getElementById ( ' input_video ' ); const stream = await navigator . mediaDevices . getUserMedia ({ video : { width : 640 , height : 480 }, audio : false }); v

If you've ever built a feature that extracts text from PDFs, an Arabic-speaking user has probably filed this bug: "the words come out in reverse order." Not the letters — the words . Every line reads last-word-first. I spent the better part of a year fixing this class of bugs while building Confileo , a free PDF toolkit with first-class Arabic support. Here's what's actually going on, because almost every explanation online is wrong or incomplete. The four distinct failure modes People say "Arabic breaks" as if it's one bug. It's four: 1. Visual vs logical order (the reversed-words bug) A PDF doesn't store text the way a Word file does — it stores positioned glyph runs : "paint these shapes at these coordinates." For left-to-right scripts, the paint order happens to match the reading order, so naive extraction works by accident. Arabic is right-to-left. Many PDF generators emit the glyph runs in visual order — the order they appear on screen, left to right. A naive extractor concatenates the runs as stored and produces every line word-reversed. The text was never "reversed" in the file; your extractor just assumed paint order == reading order. Fix: reconstruct logical order using glyph positions + the Unicode Bidirectional Algorithm (UAX #9), not the content-stream order. Libraries like PyMuPDF already return text in logical order — a common mistake is "fixing" that output by reversing it again, which is how you get double-reversed text. Rule of thumb: never reverse Arabic yourself. If it looks backwards, your rendering layer lacks bidi support; the data is usually fine. 2. Disconnected letters (the ransom-note bug) Arabic letters are contextual: ع renders differently in initial, medial, final and isolated positions, and letters join. That joining is applied at render time by a shaping engine (HarfBuzz being the standard). If any step of your pipeline round-trips text through a non-shaping renderer — a canvas library, a barebones PDF writer, an image caption filter

Second in a series on using AI to review authorization — not to spray reports. Companion reference: AuthZ Smell Catalog . 1. Why AuthZ review is not vulnerability spraying The cheapest thing an AI can do in security is generate suspicion. Point a model at a codebase and it will hand you fifty "possible IDORs" before you finish your coffee. Almost all of them are wrong — guarded three lines up, scoped at the data layer, or protected at a boundary the model never saw. That flood is exactly why several bug bounty programs spent 2026 tightening or pausing: they were drowning in confident, plausible, wrong reports. So this review inverts the usual loop. The AI's job is not to find bugs — it is to over-generate hypotheses cheaply . My job is to kill them. What survives that killing is the only thing worth a human's time, and the record of what died is more useful than the record of what lived. The artifact of an honest review is therefore not a finding. It's a kill table . 2. Target and scope Target: Ory Kratos — an open-source identity and user-management server (login, registration, recovery, verification, sessions, self-service settings). Source-available, Apache-2.0. Why Kratos: it is exactly the shape where authorization goes wrong — multiple identities, a public API and an admin API, and (in Ory's hosted product) multi-tenancy. If a boundary is fragile, this is where it shows. Scope of this write-up: source reading only , on the public repository, single-tenant OSS build. No hosted target was touched. Nothing here is an undisclosed finding — the point is the method and the boundary design , and where relevant, how the design held against the hypotheses I tested. This maps to the reproduction tiers we track: everything below is repo_only , and I say so explicitly rather than implying it reaches a live product. What this review does and does not claim. In this limited, repo-only review, the hypotheses I tested were killed. This is not a claim that Kratos has no vulner

Your ads, managed by AI that gets smarter daily. Discussion | Link

A process that took 2–3 weeks and produced a 40-page document is the kind of thing everyone wants to "AI-ify." Here's what most people get wrong about it: they reach for the LLM first. In regulated lending, the LLM is the last and smallest part of the pipeline. The hard parts are: ingesting messy documents reliably, extracting data into a canonical model, validating and cross-reconciling it, and doing the actual financial calculations - and those calculations must be done in deterministic code, never by a language model that might quietly round a number wrong. The LLM drafts prose. Code does the math. A human owns the risk decision. That ordering isn't a limitation - under OSFI E-21 it's the only design that's allowed to exist. The teams that win with AI in banking aren't the ones using the biggest model. They're the ones who know which 10% of the problem the model should touch. Where have you seen an LLM used for something that should have been deterministic code? FinancialServices #AI #Lending

EDRSR — the Unified State Register of Court Decisions — is effectively all of Ukraine's judicial practice in open access. Today Qdrant holds **44M+ vectors : criminal (19M), civil (14.3M), commercial (5.1M), misdemeanors (5.6M). Vectorization of civil cases (CPC, justice_kind=1) — the largest cohort at 33.7M documents — runs on a dedicated EC2 instance (r6a.xlarge, 32 GB RAM, 2 TB gp3). Here's what's under the hood: models, pipeline, cost, rakes, and current status. Why Vectorize Courts When a lawyer searches "is there case law on recovering bank prepayment fees" — they don't want to open 40 decisions and read them through. They want the system to surface the top 5 most relevant ones, pull out key paragraphs, and show how courts reasoned. Full-text search (FTS) over keywords doesn't give that — it returns every document containing the word "fee", and there are thousands. For this semantic task you need vector representations of text. The model turns a paragraph from a decision into a point in a 1024-dimensional space; semantically similar paragraphs sit near each other. A kNN search in Qdrant returns the top K nearest, and an LLM composes the answer from exactly those relevant fragments. The only problem: the register is big. Very big. Scale Our prod database holds full texts of decisions starting from 2006. Breakdown by procedural type: Civil (CPC) — 33.7M documents. The largest category. Consumer, housing, labor, family. Criminal (CrPC) — 12M+ Administrative (CAS) — 14M+ Commercial (CC) — 6M+ Misdemeanors (CUaP) — 6M+ The Qdrant collection edrsr_decisions on a dedicated EC2 currently holds 44M+ vectors (122 segments, on_disk=true): | Proceeding type | justice_kind | Vectors | |—|—|—| | Criminal (CrPC) | 2 | 19,036,347 | | Civil (CPC) | 1 | 14,328,427 | | Misdemeanors (CUaP) | 5 | 5,579,432 | | Commercial (CC) | 3 | 5,098,662 | | Total | | 44,042,868 | Civil cases processed: 14.3M out of 33.7M — that's 42%. After CPC completes there will be roughly 63M+ vectors in

In production we have ~1.5 TB of full-text court decisions and their vector embeddings, plus another ~550 GB of other legal data: registries, legislation, business entities, a Spanish case law corpus, EU-Lex. If we take this corpus and train an MoE model the size of DeepSeek V3, scaled to 860B parameters, on GCP — what comes out? We break down the dataset, architecture, compute cost, and the properties such a model would have on Ukrainian law. What's in the Dataset The entire corpus is what's already running in SecondLayer's production. No extra scrapes, no Common Crawl, no noise. EDRSR — the dataset core, ~1.5 TB. The Unified State Register of Court Decisions of Ukraine. 96.2 million full-text decisions (1,079 GB in PostgreSQL TOAST), 471 GB of vectors in Qdrant (voyage-3.5, 1024-dim), 28 GB of metadata (court, judge, date, case category, proceeding type, statute code). Breakdown by jurisdiction: civil 33.7M, administrative 14M+, criminal 12M+, commercial 6M+, misdemeanors 6M+. Largest annual cohort — 2024 (115 GB of TOAST text). OpenReyestr — 43 GB. Ukrainian public registries: 16.7M legal entities (EDR), ownership structures (beneficiaries, shareholders), debtors (State Enforcement Service), NAIS registries. This is the foundation for SneakyPiper — our due-diligence platform — but here it serves as raw corpus for the model. Legislation — ~40 GB. The Constitution, major codes (Civil, Criminal, Criminal Procedure, Civil Procedure, Commercial Procedure, Administrative Procedure, Labor, Tax, Customs), laws, and secondary legislation. All structurally annotated: articles, parts, clauses, revision dates with effective-date tracking. This isn't flat text: we know that Article 124 of the Constitution took effect on a specific date, carries particular references, and is cited in a precise number of decisions. Supreme Court review practices + lu_court_decisions — ~25 GB. SC plenary decisions, practice overviews, Grand Chamber rulings. This is the most valuable slice — the

Local LLM Deployment, Agent Handbook, & LLM Cost Reduction: Applied AI Workflows Today's Highlights This week's highlights cover practical guides for running state-of-the-art LLMs locally and building AI agents, alongside an innovative technique to significantly cut LLM API costs for code processing. These resources focus on actionable insights and frameworks for real-world AI application development. Jamesob's guide to running SOTA LLMs locally (Hacker News) Source: https://github.com/jamesob/local-llm This GitHub repository provides a comprehensive, hands-on guide for setting up and running state-of-the-art Large Language Models (LLMs) on local hardware. It meticulously covers the necessary tooling, dependencies, and configuration steps required to get various open-source LLMs operational without relying on cloud APIs. The guide emphasizes practical considerations for local inference, including hardware requirements, model quantization techniques, and performance optimization for different architectures, directly addressing production deployment patterns. It serves as an invaluable resource for developers and researchers looking to experiment with LLMs, develop applications offline, or reduce costs associated with cloud-based inference by leveraging local compute. The guide offers concrete details and actionable steps, making it an essential resource for anyone aiming to implement LLMs in a controlled, private, or cost-effective environment. Comment: This guide is fantastic for anyone wanting to get serious about local LLM development. It covers the nitty-gritty details of setting up your environment and getting models like Llama-3 running efficiently on consumer hardware, which is crucial for privacy and cost savings. 60% Fable cost cut by converting code to images and having the model OCR it (Hacker News) Source: https://github.com/teamchong/pxpipe The pxpipe project introduces an innovative technique to drastically reduce API costs when processing code with Lar

One quiet handwritten page a day. No feed, no AI. Discussion | Link

I'm building Guildly, a Slack-like interface where you can run a company of AI employees. I recently used Fable to revamp the whole website into a cool retro pixel style. Comments URL: https://news.ycombinator.com/item?id=48780222 Points: 5 # Comments: 4