Knowledge-and-Memory-Management v0.0.2: Portable Knowledge Collection and Memory Management
Welcome to the v0.0.2 release of Knowledge-and-Memory-Management, a tool designed for ingesting and managing knowledge from diverse sources. This release marks a clean release, stripping all hardcoded personal paths and replacing them with the portable $AGENT_HOME environment variable. For experienced developers, this version brings consistency and ease of deployment across environments without sacrificing the core functionality of knowledge collection and memory management. The project focuses on three primary collection pipelines: web, video, and articles. Each pipeline is modular, allowing you to configure, extend, or replace components based on your stack. The memory management layer ensures that collected data is indexed, stored, and retrievable via semantic search, making it practical for building personal knowledge bases or feeding into larger systems like agents or RAG pipelines. Knowledge Collection The collection module is source-agnostic at its core but ships with specialized handlers for common content types. Web collection uses a configurable web scraper that supports depth limits, domain filtering, and content extraction via readability algorithms. You can target specific sections, strip ads, and normalize HTML into markdown. The scraper respects robots.txt and supports session management for authenticated sites. Video collection transcribes audio using a local or remote ASR model. The pipeline extracts audio tracks, splits them into chunks, and generates timestamped transcripts. This is particularly useful for processing lectures, talks, or screencasts. The transcript is treated as a text document for further processing. Article collection handles RSS/Atom feeds and direct URLs. It parses feeds, fetches full content using readability engines, and deduplicates entries. Articles are converted into a consistent schema: title, author, published date, body text, and metadata. All collected data passes through a normalizer that converts content into a stand