标签：#RAG

If you're running local models through Ollama and tired of Python's overhead, check out langchain-rust . It's a full LLM framework in pure Rust that works great with local models: Ollama support — first-class integration with tool calling, vision, and streaming 9 vector store backends — InMemory, SQLite, Qdrant, ChromaDB, Redis, PGVector, MongoDB, Pinecone, FileVectorStore BM25 keyword search — with Chinese/English tokenization, no external dependency Hybrid retrieval — BM25 + Vector with RRF fusion for better recall GraphRAG — Knowledge graph construction + community detection, all local CorrectiveRAG — Self-correcting retrieval with hallucination detection Code Interpreter — LocalSandbox (subprocess), E2B cloud, or WASM sandbox LocalEmbeddings — Run embeddings without calling an API Plus: LangGraph workflows, MCP client/server, 7 memory types, guardrails, and 12+ built-in tools. Single binary, no virtualenv, no pip conflicts. Just cargo add langchainrust and go. GitHub: https://github.com/atliliw/langchainrust Docs: https://docs.rs/langchainrust

That's a rebuild I shipped. The system: a RAG assistant for fraud analysts — ask it "how do we handle card testing followed by a successful auth?" and it should answer from the team's own SOPs and case history. The complaint: the answers were wrong, therefore the model must be dumb, therefore procurement should buy a bigger model. The model was fine. It was answering perfectly — from garbage context. Walk the forensic trail with me, because every step is checkable on your own system this week. Exhibit A: the chunking was destroying meaning before anything was embedded The ingestion split SOP documents every N characters, mid-sentence. Which means half the vectors in the index encoded fragments like this: chunk_147 = " ...ing to a freight forwarder. In these cases, do NOT " chunk_148 = " cancel the order immediately. First verify the customer via " The policy — don't cancel, verify first — exists in no single chunk. An embedding can't encode a meaning that isn't in its input. Retrieval was being asked to find semantics the pipeline had already shredded. Fix one: chunk on structure (sections, paragraphs), never on character counts, with enough overlap that no rule straddles a boundary. Exhibit B: dense-only retrieval, bimodal queries Fraud analyst queries split into two populations: pattern questions ("high-value order, new account, rushed shipping") and identifier questions ("what's the SOP for decline code 4863?", "rule VEL-013 rationale"). The system was dense-only — and embeddings treat a rare token like 4863 as noise, so identifier queries retrieved similar-feeling chunks instead of the literal match. Half the query population was structurally doomed regardless of model quality. Fix two: hybrid retrieval — BM25 for the identifiers, embeddings for the patterns, reciprocal rank fusion to merge. Exhibit C: nobody could see any of this, because quality was a rumor No golden dataset. No retrieval metric. The system's accuracy was whatever the loudest anecdote said it

We live in an era where our wrists track every heartbeat, step, and sleep cycle. Yet, most of this "Quantified Self" data sits rotting in massive .xml or .json export files that are impossible to read. What if you could simply ask your AI, "How did my resting heart rate trend during the week I was stressed about the product launch?" In this tutorial, we are building a Quantified Self RAG (Retrieval-Augmented Generation) pipeline . We will take fragmented health data from Apple HealthKit and Google Health Connect, process it using DuckDB , and vectorize it into Pinecone using LangChain . By the end of this guide, you’ll have a production-grade Health Data RAG system capable of high-performance natural language queries over your personal biometrics. The Architecture: From Raw Logs to Vector Insights Handling health data at scale requires a robust ETL (Extract, Transform, Load) process. Vectorizing every single heart rate measurement (which can occur every few seconds) is inefficient and expensive. We need to downsample and summarize before embedding. graph TD A[Apple Health/Google Health] -->|Export XML/JSON| B[Raw Data Storage] B --> C{DuckDB Processing} C -->|Cleaning & Downsampling| D[Structured Parquet/JSON] D --> E[LangChain Document Loader] E --> F[OpenAI Embeddings] F --> G[Pinecone Vector Database] H[User: 'Why was my sleep poor last Tuesday?'] --> I[LangChain RAG Chain] G --> I I --> J[LLM Contextual Answer] Prerequisites 🛠️ To follow along, you'll need: Python 3.10+ Tech Stack : Pinecone , LangChain , DuckDB , OpenAI , and Pandas . An export of your health data (Apple Health export.xml or Google Takeout). Step 1: Efficient Data Crunching with DuckDB Apple Health exports are notoriously large XML files. Loading them directly into memory with standard Python is a recipe for a crash. We use DuckDB for its blazing-fast analytical capabilities to filter and downsample our data. import duckdb # Load and parse the XML (simplified logic) # Note: In a real scenario,

Introduction Artificial Intelligence has moved from being an experimental technology to becoming a core component of modern software systems. Companies today are integrating AI into customer support, analytics, automation, healthcare, finance, education, and enterprise applications. However, as organizations start building AI-powered solutions, one major question appears: “How do we make an AI model work specifically for our business needs?” Many teams immediately assume they need to train their own AI model. Others believe a well-written prompt is enough. Some organizations invest heavily in fine-tuning without understanding whether it is the right approach. The reality is that there is no single solution. Modern AI development usually revolves around three major strategies: Prompt Engineering Retrieval-Augmented Generation (RAG) Fine-Tuning Choosing the wrong approach can lead to higher costs, poor AI performance, security issues, and unnecessary complexity. This article explains the differences between these approaches and how businesses can select the right AI strategy. The Problem: Making General AI Models Business-Specific Large Language Models (LLMs) such as GPT, Claude, Gemini, and Llama are trained on massive amounts of publicly available data. They are excellent at: Understanding language Generating content Writing code Answering general questions Summarizing information ** However, businesses usually need AI systems that understand:** Internal company documents Customer information Product knowledge Industry-specific terminology Private databases Business processes For example: A hotel company wants an AI assistant that can answer: “What is our cancellation policy for premium customers?” A general AI model does not know this information because it was never trained on the company’s private policies. So the challenge becomes: How do we customize AI without rebuilding an entire model from scratch? This is where Prompt Engineering, RAG, and Fine-Tuning come

Reddit may still be considering ending its licensing deal with Google.

Every RAG tutorial I've read makes the same two assumptions: you have a GPU, and you can call a cloud API. For the environments I build for, both assumptions are wrong. I work on health information systems in the public sector. The stack has to run inside institutional infrastructure — no data leaves the network — and the hardware I get is whatever the procurement cycle produced two years ago. In practice that means Windows Server, CPU only, and open-weight models running locally. So I built a RAG stack that runs entirely on-premise, no GPU, no cloud, no Docker. It's open source at github.com/psychohub/rag-onpremise : ASP.NET Core 9 for orchestration, Ollama for local inference, Qdrant for vectors, Python for the ingest pipeline, Mistral 7B as the LLM, nomic-embed-text for embeddings. Getting it into production took longer than the design did, because five things broke that no tutorial had warned me about. This is the field report. The environment, and why it matters Before the lessons, it's worth being precise about the constraint, because it changes what "good" looks like. The stack has to run on a Windows Server, not a Linux workstation. Docker is not available on many of the target machines — either because it wasn't approved, because GPO policies restrict it, or because ops teams already run everything as Windows services and adding a container runtime is a new operational surface nobody wants to own. GPUs are aspirational. In the meantime, you have CPU inference and you have to make it work. None of this is exotic. It's the default reality in a lot of public sector, healthcare, and legacy enterprise environments. It's also the reality most RAG content on the internet quietly assumes away. The overall shape of the system: Documents (PDF / Word / Excel) │ ▼ [ Python ingest ] ├─ Text extraction (pdfplumber, python-docx, openpyxl) ├─ Chunking (500 tokens, 50 overlap) ├─ Embeddings (nomic-embed-text via Ollama) └─ Store (Qdrant, cosine similarity) │ User query │ │

We've all been there: staring at a blood test report from three years ago, trying to remember if that "slightly elevated" glucose level was a one-time thing or a trend. Our health data is scattered across messy PDFs, fitness tracker exports, and physical medical folders. In the era of AI, why are we still manually digging through folders? 📂 Today, we are building the Ultimate Personal Health Knowledge Base . By leveraging Retrieval-Augmented Generation (RAG) , we will transform fragmented medical reports and logs into a searchable, private, and intelligent second brain. We’ll be using LlamaIndex for orchestration, Unstructured.io for parsing those pesky PDFs, and ChromaDB for local vector storage. If you're looking for advanced architectural patterns or production-grade data engineering strategies beyond this tutorial, I highly recommend checking out the deep dives over at WellAlly Tech Blog , which served as a major inspiration for this build. 🚀 The Architecture 🏗️ The goal is to create a pipeline that ingests raw data, vectorizes it, and allows for Hybrid Search —combining semantic meaning with keyword precision (crucial for medical terms!). graph TD A[Raw Health Data: PDFs, CSVs, MD] --> B(Unstructured.io Parser) B --> C{Chunking & Cleaning} C --> D[Sentence-Transformers] D --> E[(ChromaDB Vector Store)] F[User Query: Is my cholesterol improving?] --> G[LlamaIndex Query Engine] E <--> G G --> H[LLM: Local or OpenAI] H --> I[Actionable Health Insight] Prerequisites 🛠️ To follow along, you’ll need a Python environment with the following stack: Unstructured.io : To handle "dirty" PDF and image-based reports. ChromaDB : Our lightweight, open-source vector database. Sentence-Transformers : To generate local embeddings without sending data to the cloud. LlamaIndex : The glue that connects our data to the LLM. pip install llama-index chromadb unstructured sentence-transformers llama-index-vector-stores-chroma Step 1: Ingesting Messy Medical Reports 📄 Medical reports are

Liquid syntax error: Unknown tag 'endraw'

Building a 100% Local RAG System on Kubernetes — No API Keys Required Ahmed Nafies Ahmed Nafies Ahmed Nafies Follow Jul 30 Building a 100% Local RAG System on Kubernetes — No API Keys Required # kubernetes # rag # llm # postgres 1 reaction Add Comment 8 min read

On a Tuesday morning in March, a chief executive asked a question that should have taken thirty seconds to answer: have we ever agreed to a liability cap below one million dollars? The answer existed. It was written down, signed, filed, and sitting on the shared drive the whole time. Finding it took three days, and not finding it in time cost forty thousand dollars. Every organization has a version of that Tuesday. The knowledge is real, it survived, and it is spread across a million files in a hundred formats, organized by whoever was closest to the filing cabinet that day. An organization knows more than anyone in it. The hard part is getting at it. Keyword search fails for a specific, fixable reason The obvious first fix is to index every word and search it. Type "liability cap," get every document containing "liability" and "cap." This fails, and it fails in ways worth naming precisely, because each failure points at what the real fix has to do. The contract does not say "liability cap." It says "limitation of liability." Two phrases, one meaning, zero shared keywords. Your search returns nothing and you conclude the document does not exist. The search bar cannot tell the difference between "we have no such contract" and "we have it, filed under different words." Matching words is not matching meaning. Search "termination" across an employee handbook and a supplier agreement and you get firing, contract expiry, and possibly a paragraph about ending a software license, ranked by nothing more meaningful than word frequency. People ask questions, not keywords. Nobody thinks in search terms. They think "have we ever agreed to a liability cap below a million?" A keyword engine has no idea that this is a question, let alone which words in it matter. What actually closes the gap The fix is to stop comparing words and start comparing meanings, which requires turning text into something you can measure distance in. An embedding model reads a passage and returns a list of

The former OpenAI researcher’s fund was forced to unwind public equities after leveraged public bets plummeted. But he still has cards to play.

If you just want the recommendation: for the ordinary AI-generated image an inference job hands back — a 2 to 8 MB PNG — do one plain object PUT into your S3-compatible storage and stop there, because multipart upload only earns its complexity when a single artifact is big enough that losing a transfer halfway through costs you real money to redo, which for my team starts somewhere north of 100 MB. Everything below is about that threshold, and about the operations bill you pick up the moment you cross it. I run the platform roadmap for a team that renders a few hundred thousand images a month, and I count pages before I count features, so read the rest with that bias in mind. Should I use multipart upload for large AI-generated images, or a single object PUT? Multipart solves two narrow problems: a payload too awkward for one HTTP round trip, and a transfer you refuse to restart from byte zero. A 6 MB PNG has neither problem. The shape of the flow is always the same wherever you run it. You start a multipart upload and get back an upload id, you push each part under that id, you collect the returned ETag and part number for every one of them, and you send the finished list back in a complete call that stitches the object together server-side. Parts have to be at least 5 MiB on Amazon S3 and on every S3-compatible store I've tested against, with the final part exempt, which already tells you the feature was designed for objects measured in hundreds of megabytes rather than for a batch of thumbnails. Where it genuinely pays off in an image pipeline is the long tail: a 4-gigapixel tiled upscale, a nightly ZIP export of a customer's whole render history, a raw latent archive somebody in research wants kept for a year. Those are the jobs where a dropped connection at 80% is a real incident and not a shrug. For everything else, one put is one line of code and one thing to monitor. There's a second cost that people underrate, and it's the one I'd argue about in a design re

What Is Retrieval Augmented Generation (RAG), and Why Does It Make AI So Much Less Confidently Wrong? You know that game show contestant who buzzes in before the host finishes reading the question, shouts "MOUNT EVEREST!" with absolute certainty, and then looks genuinely confused when the correct answer turns out to be "the Treaty of Westphalia"? That's been AI for most of its existence. Supremely confident, occasionally correct, and deeply committed to whatever pops into its head first. Now imagine that same contestant gets a new rule: before answering, they can phone a friend who has the exact relevant textbook already open to the right page. The friend reads them the actual answer, word for word, and then the contestant puts it in their own words for the judges. Suddenly, our buzzer-happy friend is getting questions right. That phone call is Retrieval Augmented Generation, and it's the reason AI chatbots have gotten weirdly more useful in the past year. The Old Way: Confidently Wrong at 200 Miles Per Hour Traditional large language models (big AI systems trained on tons of text) get trained on enormous dumps of text scraped from the internet, books, and whatever else researchers can feed them. Then the training ends. The model gets sealed off, frozen in time with whatever it learned. When you ask a question, these models generate answers by predicting the most plausible-sounding next words based on patterns they memorized during training. It's essentially very sophisticated autocomplete. The AI has no fact-checking mechanism. It doesn't "know" anything in the way you know your own phone number. It just knows what words tend to follow other words. This leads to what researchers politely call hallucinations, which is a fancy term for "making stuff up with tremendous confidence." The AI generates text that sounds authoritative and well-structured because it's learned the pattern of how authoritative text sounds. But the actual facts? Those might be completely invent

Use one key prefix per user, delete from the application every key you can name, and leave lifecycle rules to sweep the old temporary images nobody will ever ask for again. That's the whole design, and I've watched teams get it wrong in the same two ways for years: they either try to make the storage layer clever enough to know what a user is, or they hand the entire deletion problem to a lifecycle policy and then wonder why an account-deletion request took nine days to actually remove anything. I design data layers for a living, so I'm going to be blunt about the durability and consistency side of this rather than the upload-a-file-in-five-minutes side. Why the key layout matters more than the backend you pick Object storage has no folders. There's a flat keyspace and a delimiter convention, and every "folder" you see in a console is the UI grouping keys that share a prefix — which is good news, because it means the layout is yours to design and costs nothing to enforce. The layout I keep landing on is users/{userId}/generations/{yyyy-mm}/{uuid}.png , with a sibling users/{userId}/scratch/ prefix for renders that only exist so the browser can show a preview. Four properties come out of that shape, and they're the reason I don't get creative here. Listing a tenant's images is a single prefix query rather than a metadata scan, which matters because object stores generally don't let you search metadata server-side — you filter by prefix or you keep an index in your own database. Deleting an account becomes "enumerate one prefix, delete what's under it," so the compliance clock is something you control. The month segment keeps any single listing page from growing without bound, and it gives you a cheap way to write an age-based rule later. And the opaque UUID means the key never leaks a filename, a prompt, or an email address into a URL that might end up in a log or a referrer header. One thing I'd push back on if I saw it in review: don't put the user's email or usern

The RAG Security Gap Retrieval-Augmented Generation (RAG) has rapidly emerged as the foundational architecture for grounding enterprise AI agents in proprietary corporate knowledge. By pairing Large Language Models (LLMs) with high-density vector databases and knowledge graphs, organizations enable agents to answer complex queries, analyze financial records, and automate customer support workflows using live operational context. However, as agentic workflows transition from prototype sidecars to core infrastructure, exposing unstructured enterprise data to vector search pipelines introduces severe, unmonitored security surfaces. When an LLM retrieves document chunks from vector stores, traditional identity management frameworks break down. Role-Based Access Control (RBAC) configured in legacy SQL databases or cloud storage buckets does not natively translate into vector embedding spaces. If a vector store ingests documents without preserving fine-grained document-level Access Control Lists (ACLs) or cryptographic data lineage, autonomous agents operate in an over-permissioned context. The consequences of ungoverned RAG architectures are severe: Privilege Escalation via Context Injection: An employee with basic read access asks an agent a high-level query. The agent’s vector search retrieves chunked financial projections or executive emails that lack query-time authorization filtering, exposing confidential data in the generated response. Indirect Prompt Injection: Malicious actors embed hidden instruction payloads inside public or shared enterprise documents (e.g., hidden white text in a PDF invoice). When the RAG engine ingests and retrieves this chunk, the LLM executes the injected commands, hijacking the agent’s execution loop. Stale Context & Hallucination Loops: Vector databases retain outdated document embeddings indefinitely unless bound to stateful lifecycle policies. Agents grounding decisions on stale operational procedures generate hallucinated or legally

In my previous article , I built a fully local RAG assistant Ollama, ChromaDB, LangChain, all running in Docker. It answered technical support questions by searching through documentation and citing sources. It worked. But after using it for a while, I noticed something uncomfortable: it treated every question the same way . Ask it "how to close monthly payroll?" it searches the docs. Fine. Ask it "the server crashes at startup" it also searches the docs. Less fine. Ask it something completely outside the documentation it searches the docs. Useless. A real support technician doesn't do that. They first assess the situation, then decide what to do: look it up, run a diagnosis, or escalate to a human. My RAG had no such judgment. That's what this article is about how I evolved the system into an Agentic AI architecture using LangGraph, where the assistant first decides which strategy to use , then acts accordingly. The Core Limitation of Classic RAG Classic RAG is a linear pipeline. Every query follows the exact same path: Question → Embed → Retrieve → Prompt → LLM → Answer No branching. No decision-making. No memory between steps. This works perfectly for procedural questions where the answer lives in the docs. But technical support involves at least three distinct scenarios: Scenario Example Best strategy Procedural question "How do I create an account?" Search documentation Known error code "ERR-COMP-001 appears" Lookup error database Unknown incident "Server crashes, no idea why" Diagnose + escalate if needed A single RAG pipeline handles the first case well and the other two poorly. The solution is to add a layer of reasoning before retrieval. What Agentic AI Adds The shift from RAG to Agentic AI comes down to one thing: the system plans before it acts . Instead of one fixed pipeline, you have: Question ↓ Classifier (what kind of question is this?) ↓ ├── Procedural → RAG Agent (search docs) ├── Error code → Diagnostic Agent (lookup + LLM analysis) └── Complex → D

In geophysics you almost never get to see the thing you're studying. You get a seismic trace, a...

A follow-up to How Old Is My Data? The failure mode that gets worse when a machine is reading the data In a classic dashboard, stale data is a human problem: someone looks at a number that's six hours old and makes a slightly worse decision. Annoying, rarely catastrophic. Now hand that same data to a retrieval-augmented-generation (RAG) pipeline, or to an autonomous agent. The stakes change. The system doesn't pause to sanity-check the timestamp — it acts. And when the data it acts on is stale, three things are true at once: The answer is confidently wrong. There is no error to fire on — the query succeeded, the model responded, latency was normal. Every other signal on your dashboard is green. That's the worst combination in observability: a real failure that is completely invisible to the signals we currently emit. Where staleness hides in AI systems RAG: index vs. corpus. Your vector index was built from a corpus at some point in time. The corpus keeps changing — documents get added, edited, retracted. If the re-embedding job stalls or falls behind, the index quietly drifts out of date. The retriever still returns plausible chunks; the model still writes a fluent answer. It's just answering from a version of reality that no longer exists. The quantity you care about is the age of the index relative to its source — not the age of either one alone. Feature stores: online–offline skew. The features your model trained on and the features it serves on are supposed to match. When the online store lags the offline pipeline, predictions degrade in a way that looks like model drift but is actually data staleness wearing a costume. Agents: stale shared state. Multi-agent systems coordinate through shared memory, scratchpads, and context. An agent reasoning over state that another agent updated ten steps ago — but which never propagated — makes locally reasonable, globally wrong decisions. This isn't a new or exotic problem: it's exactly the regime that Age of Information t

Use a private bucket with short-lived presigned URLs when an avatar belongs to exactly one user, and reach for a public CDN-backed bucket only when the images are genuinely public and you'd rather pay for cache hits than for signatures. For a Node.js SaaS that is the entire decision, and everything after it is plumbing: which S3-compatible provider you point at, how long a signature should live, and what happens to the stored object on the day a user deletes their account. Avatars are small. That removes half the hard problems. The half that's left is the half I get paged for, because an avatar key is written by an untrusted client, read on nearly every page render, cached in three places you don't control, and referenced from a database row that has its own opinion about which object is current. So the questions I ask a storage vendor aren't about upload throughput. They're about whether a partial write can ever be visible to a reader, what the durability number is actually measuring, and how I reconcile the bucket with my user table after a failed deploy. I've never watched a team lose avatar bytes. I've watched several lose track of which bytes were current, which is the same outage with a friendlier root-cause section. How should a Node.js SaaS store private user avatars in object storage? Three moves, in this order. Create one private bucket for the whole tenant base, write each avatar under a key that carries a random component, and mint a presigned GET at display time instead of persisting any URL. Store the key in your database, on the user row, and nothing else, because keys are stable and signatures expire — a URL you saved last Tuesday is a support ticket waiting to happen. Serving the image then costs you one signing call per render, which you can cache in Redis for slightly less than the signature's own lifetime. That random component does more work than it looks like it does. Overwriting a fixed path like users/8821/avatar.png puts you in a read-modify

Not every arb opportunity is simultaneous. My bot uses a “legged” approach: it buys one side when it’s heavily underpriced, then waits for market sentiment to shift and buys the other side later for a total cost under $1.00. This strategy shines in volatile non-crypto markets (elections, sports playoffs, news-driven events). Careful inventory and timing controls turned it into a consistent contributor to the bot’s $130k+ track record. The sample source is in https://github.com/cryptomoonday/polymarket-arbitrage-bot