今日已更新 144 条资讯 | 累计 37695 条内容
关于我们

标签:#m

找到 11240 篇相关文章

AI 资讯

Routing LLMs by task verifiability: a small experiment (n=120, 3 models) inspired by Karpathy's framework [D]

Full disclosure: this is directional, not a paper. n=120 tasks, one internal evaluator, not peer reviewed. I work at an LLM infrastructure company. This experiment was done on my own time and is not a company claim. Karpathy's framework classifies tasks by verifiability. Can output be mechanically checked? High verifiability tasks like code compilation and structured JSON extraction are safer because the verifier catches errors. Low verifiability tasks like creative writing are riskier. I wondered if high verifiability tasks are also easier in practice. Can a weaker model do them as well as a frontier model if the verifier catches mistakes? Setup was 120 tasks across four categories. Code unit tests, structured extraction, multi hop reasoning, creative summarization. Three models: Claude Sonnet 4.6, GPT 5.5, local Mistral 3 8B via vLLM 0.6.3. Pass rate for the first two, human rating 1 to 5 for the last two. Results were messy. Code unit tests: Sonnet 4.6 94%, GPT 5.5 91%, Mistral 3 8B 87%. With one retry Mistral 3 hit 95%. That surprised me. I expected the gap to be bigger. Structured extraction: Sonnet 4.6 97%, GPT 5.5 94%, Mistral 3 8B 89%. With retry 96%. Also closer than I expected. But here is where it got weird. Sonnet 4.6 initially scored worse than GPT 5.5 on structured extraction, which made no sense. Turns out our JSON schema had an ambiguous nested array that confused Claude's tool use parser. Fixing the schema brought Sonnet to 98%, but I kept the original numbers in the table because the mistake is part of the story. Your verifier is only as good as your schema. Multi hop reasoning: Sonnet 4.6 78%, GPT 5.5 71%, Mistral 3 8B 51%. Retry didn't help. The model would hallucinate reasoning paths consistently. This is where the capability gap was real. Creative summarization: Sonnet 4.6 4.2 out of 5, GPT 5.5 3.9 out of 5, Mistral 3 8B 3.1 out of 5. Expected. Interpretation: high verifiability tasks seem simpler in the sense that weaker model plus verifier ca

2026-06-11 原文 →
AI 资讯

Debugging the Google Maps Duplicate Loading Bug in React

Originally published on clintech.me If you've integrated Google Maps into a React app and seen Autocomplete randomly stop working, Directions silently fail, or the API throw google is not defined on second render — you've hit the duplicate loading bug. Here's exactly what caused it in my case and how I fixed it. The setup that broke things While building delivery address flows at POLOM — a production e-commerce platform — I integrated Google Places Autocomplete across 20+ screens. I had the Maps JavaScript API loading in two places: A provider.tsx for global script loading across the app A useLoadGoogleMaps hook inside a shared component This caused race conditions. The Autocomplete and Directions APIs were initialising before the script fully resolved in some renders, silently failing in others. The failure wasn't consistent, which made it harder to catch. The fix Step 1 — Remove the global load Delete the script tag or next/script call in provider.tsx . There should be exactly one place the Maps API loads. Step 2 — Centralise in a hook Move all loading logic into a single useLoadGoogleMaps hook using dynamic loading. If you're on Next.js, next/script with strategy="afterInteractive" inside the hook is the right approach. Step 3 — Guard before initialising if ( ! window . google ?. maps ) return ; Check that the API is fully available before attempting to attach Autocomplete or Directions . Don't assume the script load event means every namespace is ready. Step 4 — Scope your ref correctly Bind the autocomplete instance to inputRef.current explicitly. If the component remounts, re-initialise the binding — don't assume the previous instance is still attached. The result One load, one source of truth, no race conditions. Autocomplete and Directions worked consistently across all 20+ screens without reinitialising on every render. Security — the step most developers skip Restrict your API key at the Google Cloud Console level: HTTP referrers: whitelist your domain onl

2026-06-11 原文 →
AI 资讯

Copilot Chat Goes GA in PRs — But Multi-Repo Visibility Is Still Missing

GitHub moved Copilot Chat's richer pull request experience to general availability this week — side-by-side chat with diffs, inline editing, and context-aware answers without leaving the review view. Previously in public preview, it is now live for all Copilot license holders. It is a real improvement for reviewing changes inside a single pull request. But it highlights a gap that per-PR AI tooling structurally cannot close: knowing what is open across the rest of your organisation. The Problem That Lives Outside the PR Most engineering teams don't work in one repository. They ship across services, libraries, and infrastructure — often with related PRs open in multiple repos simultaneously. A reviewer approving a payments service change without knowing that a dependent auth-service PR is still in draft is reviewing without full context. This is not a quality-of-feedback problem. It is a visibility problem. No amount of intelligence surfaced inside a PR tells you what is happening across your repositories. Gartner's 2026 assessment of AI coding agents makes the point clearly: the bottleneck has shifted from generating code to reviewing, securing, and governing it. Better per-PR AI raises the floor on feedback quality. The teams that pull ahead will be the ones who also solve the coordination layer — which PRs are open, which are stale, which are blocked on a dependency in another repo. What Changes With Better In-PR AI GitHub's GA release makes the review experience faster and less disruptive for individual PRs. That matters. But as per-PR intelligence becomes table stakes, the differentiator shifts toward cross-repo awareness: who is waiting for review, what related work is in flight, and where the actual bottlenecks in the delivery pipeline are. Engineering leaders should be watching PR age distribution and review load across all repositories — not just the ones that happen to be open in a browser tab right now. For teams already dealing with multi-repo sprawl, Cod

2026-06-11 原文 →
AI 资讯

Your Agent Returns 200 and Lies. Verify Before You Trust

A success gate verifies an AI agent's claimed success before your system accepts it. SuccessGate runs three read-only checks — schema/contract, claim-vs-evidence against the actual tool-call trace, and an optional post-condition probe — and turns a silent 200 into an explicit REJECTED with reasons. It's stdlib Python, needs no API key, moves nothing, and ships with a self-test you run in one command. Here's the failure that started this for me. An agent in a CRM workflow reported {"status": "sent", ...} for an invoice. Clean run. Green dashboard. 200 OK. The invoice went to a customer id that wasn't on our allow-list — a near-miss hallucination the model was completely sure about. Nothing crashed. No exception, no stack trace. We found it days later, downstream, the expensive way. That's not a rare bug. It's the default failure mode of agents in production, and it has a name now: silent-success drift . Cycles' writeup put it bluntly — "200 OK Is the Most Dangerous Response in Production" : "The most dangerous failures look like success." And the measurements back it up. The Berkeley Function-Calling Leaderboard (BFCL v3) puts frontier-model structurally invalid tool calls at 2–5% even on clean benchmark prompts — higher in noisy production ( via Future AGI ). The arXiv paper Agent Behavioral Contracts reports that across 1,980 sessions , contracted agents caught 5.2–6.8 soft violations per session that uncontracted baselines missed entirely. So the question is not how do I see failures sooner . It's how do I stop accepting a success the agent never actually achieved. TL;DR A 200 and a "status": "done" are claims, not proof. Agents return both while doing the wrong thing — or nothing. Observability is tracking : it tells you a call happened. It can't tell you the result was correct. That's a control problem. verify() runs three checks before you accept success: (1) schema/contract (shape, types, enums/allow-list), (2) claim-vs-evidence (did the agent actually call th

2026-06-11 原文 →
AI 资讯

You can just tell the Instagram algorithm what you want now

Instagram is going to let you tweak what its algorithm shows you on your main feed. With the Your Algorithm feature, "you can now see the topics we think you're interested in, and change them, across all the major parts of Instagram," according to Instagram boss Adam Mosseri. Right now, the feature will only surface […]

2026-06-11 原文 →