I Built an Open-Source Multi-Agent Fact-Checker — Here's How It Works

Problem Statement We have a misinformation problem. But more specifically, we have a speed problem. A journalist spots a suspicious claim. They search for sources. Cross-reference databases. Call experts. Write a verdict. Get it edited. Publish, maybe 6 hours later. Maybe 3 days later. Meanwhile, the original claim has been screenshot, reposted, quoted in newsletters, and cited in arguments across five platforms. I wanted to build something that closed that gap. Not a chatbot that guesses. A proper pipeline, one that retrieves real evidence, reasons from it, and tells you why it reached a verdict. That's what Sift is. What is Sift? Sift (Source Inspection & Fact-checking Tool) is an open-source multi-agent AI pipeline that takes any text, extracts every factual claim, retrieves grounded evidence, and returns auditable verdicts — TRUE, FALSE, or UNCERTAIN, with cited sources and full reasoning chains. Paste a news article. A politician's speech. A viral statistic. A WhatsApp forward. Sift breaks it into individual claims and fact-checks each one independently. Why Multi-Agent? The naive approach is to ask an LLM: "Is this claim true?" The problem: LLMs hallucinate. They have knowledge cutoffs. They're confidently wrong in ways that are hard to detect. And critically, they don't show their work. A single LLM call can't reliably handle the full pipeline of: Extracting structured claims from noisy text Retrieving dated, traceable evidence from live sources Reasoning across conflicting evidence without confabulating Adversarially reviewing its own conclusions for overconfidence Finding corrections when something is wrong Each of these is a distinct task that benefits from its own prompt, its own tools, and its own failure modes. That's why I built five separate agents, orchestrated with LangGraph. The 5-Agent Pipeline Agent 1 — Claim Extractor A single paragraph can contain 4-5 distinct factual claims. Generic LLMs miss them or conflate them. This agent uses LLaMA 3.3 70