今日已更新 76 条资讯 | 累计 32406 条内容
关于我们

标签:#insurance

找到 3 篇相关文章

AI 资讯

Idempotent File Anchoring: SHA-256 Dedup Before You Call the API

Building any intake pipeline, you'll hit the same problem eventually. Files arrive from multiple sources. Some you've already processed: re-uploads of the same document, copies from two different intake paths, items your worker errored on last run and re-queued. Call the anchoring API blindly and you end up with multiple proof records for identical bytes. The ProofLedger v1 API returns a duplicate_of field in its 201 response when it detects a hash it's already seen. But that's only half the solution. A network round-trip costs time and quota even when it comes back as a duplicate. Hash-based local deduplication is the other half. Here's how to build a worker that handles both layers. Hash Locally First The core pattern: compute the SHA-256 digest before making any API call. If you've seen this digest before, skip it. If you haven't, submit it. Two things you need: a persistent record of digests you've already anchored, and chunked hashing so large files don't blow memory. import hashlib import json from pathlib import Path SEEN_DB = Path ( " anchored_hashes.json " ) def load_seen (): if SEEN_DB . exists (): with open ( SEEN_DB ) as f : return json . load ( f ) return {} def save_seen ( db ): with open ( SEEN_DB , " w " ) as f : json . dump ( db , f , indent = 2 ) def hash_file ( path : str ) -> str : h = hashlib . sha256 () with open ( path , " rb " ) as f : for chunk in iter ( lambda : f . read ( 65536 ), b "" ): h . update ( chunk ) return h . hexdigest () 65536-byte chunks keep memory flat regardless of file size. The load_seen / save_seen pair gives you a persistent record that survives worker restarts. Submitting and Reading duplicate_of When duplicate_of appears in the API response, its value is the proof ID of the earliest anchor for that hash. That's the canonical ID. The new proof ID from this call is irrelevant. import requests API_URL = " https://proofledger.io/api/v1/proof " API_KEY = " sk_YOUR_KEY_HERE " def anchor_file ( file_path : str , seen : dict

2026-08-10 原文 →