今日已更新 279 条资讯 | 累计 31547 条内容
关于我们

I filled my agent's wiki with contradictions. It never gave a wrong answer.

Wenyu Zhang 2026年08月14日 08:46 2 次阅读 来源:Dev.to

There's a comfortable assumption behind a lot of "agent + knowledge base" work: garbage in, garbage out. Feed an agent a messy, stale, duplicate-ridden wiki and it'll confidently tell users the wrong thing. So we invest in dedup, freshness, clean ingestion — to stop the agent from hallucinating. I built a small testbed to measure that assumption, and it's wrong. Or rather: it's wrong about how bad ingest hurts, and the real answer is more interesting — and harder to catch. The setup I built a tiny agent that navigates a wiki the way a person does: it has a wiki_search tool and a wiki_read tool, it decides what to look up, reads a page, and answers. No vector database, no RAG injection — the agent navigates . (This is increasingly how capable models prefer to work: they know what they need better than a query-time embedding match.) The wiki is ten markdown pages of deliberately synthetic facts — invented services, made-up numbers the model cannot possibly know from training ("the Orion canary ring holds for 45 minutes before auto-promoting"). That last part matters: because the facts are synthetic, an agent with no wiki genuinely can't answer, so any success is attributable to the wiki, not to the model reciting what it already knew. First, does the wiki help at all? Baseline (no wiki tools) scored 0/4 . Augmented (wiki tools) scored 4/4 . A clean +100-point lift. Good — the testbed works, and the tasks are honest. ( the runner ) Then I started degrading the ingest quality and measuring what happened. The degradation I built three versions of the wiki: clean — the authoritative pages only. stale-present — each answer's page now has a contradicting duplicate (a page claiming the canary holds for 30 minutes, not 45), ranked below the real page. This simulates auto-ingest pulling in an old copy. stale-outranks — the same contradiction, but keyword-stuffed so it outranks the authoritative page in search. This simulates the very common failure where a spammy or verbose st

本文内容来源于互联网,版权归原作者所有
查看原文