今日已更新 244 条资讯 | 累计 29978 条内容
关于我们

标签:#m

找到 8848 篇相关文章

AI 资讯

Your eval's confidence interval assumes independent examples. Yours are clustered.

Every binomial confidence interval you have ever computed on an eval pass rate, Wald, Wilson, Clopper-Pearson, all of them, rests on one assumption: each example is an independent draw. Most eval sets violate it. You have 40 questions generated from the same 8 documents, or 200 turns from the same 30 conversations, or 150 examples that are really 50 cases with 3 paraphrases each. Those are not 200 independent observations. And when you feed a correlated set into a formula that assumes independence, the interval comes out too narrow, which means you declare differences significant that aren't. I want to walk through why, put a number on how much it matters, and show the fix, because this one is invisible: the code runs, the interval prints, and it is quietly wrong. Why clustering shrinks your real sample size Independent examples each carry their own information. Correlated examples carry overlapping information. If five questions come from the same document, and the model either understands that document or doesn't, those five outcomes move together. You did not learn five independent things about the model. You learned something closer to one and a half. The survey-statistics name for this is the design effect (Kish, "Survey Sampling," 1965). For clustered data it is approximately: Deff = 1 + (m̄ - 1) · ICC where m̄ is the average cluster size and ICC is the intra-cluster correlation, the fraction of total variance that lives between clusters rather than within them. Your effective sample size is: n_eff = n / Deff That is the number of independent examples your clustered set is actually worth. The number Take a realistic eval set: n = 200 examples, drawn from 40 source documents, so average cluster size m̄ = 5. Suppose the ICC is 0.3, which is unremarkable for "questions from the same document" (I have measured higher). Deff = 1 + (5 - 1) · 0.3 = 2.2 n_eff = 200 / 2.2 ≈ 91 Your 200-example eval is worth about 91 independent examples. The correct confidence interval

2026-07-29 原文 →
AI 资讯

Starting Terraria modding (again)

This is my first dev blog I'm making a terraria mod I'm not sure if i want to start right now but i am sure to start soon i already have some ideas so here are the ideas The Operator The operator is someone i have in idea for a while kind of the lore aspect is you work it, killing bosses and giving proof to the operator for certain rewards, at first it is an Npc but after moon lord you fight him. I might bring him back as the same dude but is occupied by the Fixer as a vessel which he is chained or has custom hand cuffs for the fixer to occupies the fixer without The Operator body, without the body dying. The Dulled One This is not my idea but a alternative version of it (Game: Craft-Wars Redux Roblox, Boss: Dulled Spectrum) for credits, so I really liked this boss idea but I'm really not sure if they did what I'm doing, but his power is to erase or turn into dust or "dull". To erase certain parts of the world either matter, or space i don't want to say time because i feel like that would be boring, it can either be like passive very weak erase which it can re-gain power. Second version is the Compacted version which does way more damage, maybe one shot if i do one shot then im adding middle attacks, but deplete the dust bar by a ton so if the boss is not careful or the player stops it then you can easily beat it. it might have a regen system where if the bar is full then it heals. Also hammer both the versions the game and mine has hammer. The lore is not very fleshed out right now but i will figure it out

2026-07-29 原文 →
AI 资讯

Building Local AI Agents in Java with Tools4AI and Ollama: An Insurance Claims Use Case

Tools4AI is a 100% Java agentic AI framework that turns any annotated Java method into an AI-callable action. Ollama runs open models like Llama 3.1 and Phi-4 locally and exposes an OpenAI-compatible API. Point Tools4AI at http://localhost:11434/v1 and you get a fully offline, on-premise AI agent — no data ever leaves your network. In this tutorial we build an insurance claims triage agent that reads a claimant's free-text incident report, routes it to the right business action, extracts structured data, gates high-value payouts behind a human approval, and records a compliance audit trail. Who is this for? Java developers, solution architects, and engineering leaders in regulated industries (insurance, banking, healthcare) who want agentic AI without sending sensitive data to a third-party API . Table of Contents Why local AI agents matter for insurance Insurance runs on personally identifiable information (PII) : names, addresses, policy numbers, medical details, vehicle data, and loss descriptions. Sending that data to a hosted LLM API creates regulatory, contractual, and reputational risk. At the same time, claims teams are drowning in unstructured text — First Notice of Loss (FNOL) reports, adjuster notes, emails, and call transcripts. A local AI agent solves both problems at once: Data never leaves your premises. The model runs on your own hardware via Ollama. Deterministic business logic stays in Java. The LLM decides what to do; your audited, tested Java code decides how . Human-in-the-loop and audit trails are first-class, so you can satisfy compliance reviewers. That combination — private inference plus governed execution — is exactly what Tools4AI + Ollama gives you. What is Tools4AI? Tools4AI ( io.github.vishalmysore:tools4ai on Maven Central) is a lightweight, pure-Java agentic AI framework and ADK. Its core idea is simple and powerful: Annotate a Java class with @Agent and its methods with @Action . Tools4AI scans the classpath, and at runtime it maps

2026-07-29 原文 →
AI 资讯

Run and Compare AI Evaluations with a CLI for Developers and Coding Agents

TL;DR: This walkthrough shows how developers and coding agents can use Quantiles , an open-source AI evaluation platform licensed under Apache 2.0, to quickly run, analyze, and compare AI evaluations locally. We'll use the SimpleQA Verified benchmark as an example throughout this post, letting you follow the commands, inspect the evaluation results, and configure your own model for the same workflow. Running an AI evaluation is rarely as simple as sending prompts to a model. Developers must connect datasets, model APIs, scoring logic, result storage, and comparison tooling before they can answer a basic question: did the system get better? When those pieces are spread across scripts, notebooks, and logs, every rerun becomes harder to reproduce and diagnose. A score alone cannot reveal whether the model changed or whether the dataset, prompt, scorer, or sample set changed with it. Quickstart: Run an example benchmark The Quantiles CLI is called qt on the command line. A simple curl ... | bash command supports macOS and Linux on X86-64 and Arm64 systems. First, use it to install the CLI: curl -fsSL https://cli.quantiles.io/install.sh | bash If you don't want to run code directly sourced from the internet, see the install.sh source code first. Next, let’s run a built-in benchmark from start to finish using a single command. SimpleQA Verified is a 1,000-prompt benchmark created by Google DeepMind and Google Research. It re-curates questions from OpenAI's SimpleQA benchmark to reduce problems such as incorrect labels, topical bias, redundant questions, and ambiguous source evidence. Each example includes a short factual question in problem , its reference answer , topic and answer-type metadata, and supporting URLs. Use the following command to run simpleqa-verified using the built-in Quantiles demo model, which doesn't incur any usage charges: qt run simpleqa-verified Results from the demo model are intended only to demonstrate the evaluation workflow because its output

2026-07-29 原文 →
AI 资讯

We Open-Sourced Both Halves of Our Security Stack — Detection and Deliberation

We Open-Sourced Both Halves of Our Security Stack — Detection and Deliberation AEGIS catches the threat. ENLIL decides what it means. Both are free, and we want to know if they actually help you. We've written before about each of these projects separately — AEGIS's post-quantum forensic logging and why ENLIL runs 9 LLMs in parallel instead of one . This post is about why they're actually one system, and why we're not gatekeeping either half of it. Two different jobs AEGIS is an intrusion detection/prevention system. It watches traffic, correlates signals across nine layers (from crypto-level filtering to adaptive moving-target defense), and decides — fast, locally, without calling out to anything — whether something is an attack. It's deliberately narrow: detection and containment, nothing else. No counterattacks, no active reconnaissance, one process, deployable on a standard VPS. ENLIL does the opposite job. It's slow by design — it convenes a council of independent LLMs, lets them reason in isolation (no model sees another's answer until synthesis), and produces a signed Decree: the majority view, the dissents, and a final verdict. It's built for the decisions where being fast and wrong is worse than being slow and right. Neither one replaces the other. AEGIS shouldn't spend 30 seconds deliberating about whether a SYN flood is a SYN flood. ENLIL shouldn't be making split-second network decisions. But there's an obvious seam between them: what happens when AEGIS catches something that isn't a simple yes/no — a pattern that's ambiguous, or severe enough that you want more than one model's opinion before you act on it? The Bridge That seam is a small connector: when AEGIS's detector layer flags something at high or critical severity, it fires a signed event at ENLIL's API. ENLIL convenes a tier of the council sized to the severity — a lighter tier for routine escalations, the full council (including the most expensive model) only for the things that deserve it. The

2026-07-29 原文 →