标签：#selfhost

I recently rebuilt my homelab from scratch. The goal was simple: one machine, everything containerised, zero exposed ports, GPU-accelerated local AI, and a fully automated backup setup. No cloud subscriptions for the tools I use every day. This is the full technical breakdown — what I'm running, how it's wired together, and the hard-won fixes that cost me hours so you don't have to repeat them. What I'm Running Eight services, 26 containers, one machine: Service Purpose Portainer Docker management UI Uptime Kuma Service monitoring (7 monitors) NocoDB Self-hosted Airtable — CRM & leads n8n Workflow automation Open WebUI Local AI chat interface Ollama Local LLM inference (GPU) AFF!NE Collaborative docs & whiteboards Plane Project management (roadmaps, sprints) Duplicati Encrypted daily backups Cloudflare Tunnel Zero Trust secure access — no open router ports All external-facing services sit behind Cloudflare Zero Trust with email OTP. No passwords to manage, no VPN clients — Cloudflare handles authentication at the edge. Architecture ┌──────────────────────────────────┐ │ Cloudflare Edge (Zero Trust) │ │ *.yourdomain.com — email OTP │ └──────────────┬───────────────────┘ │ HTTPS ┌──────────────▼───────────────────┐ │ Ubuntu Machine │ │ │ │ cloudflared (outbound tunnel) │ │ │ │ │ ┌─────▼────────────────────┐ │ │ │ homelab-net (bridge) │ │ │ │ │ │ │ │ portainer uptime-kuma │ │ │ │ nocodb n8n │ │ │ │ open-webui affine │ │ │ │ plane-* duplicati │ │ │ │ ollama (GPU passthrough) │ │ │ └───────────────────────────┘ │ └───────────────────────────────────┘ Everything runs on a shared Docker bridge network ( homelab-net ). The cloudflared container maintains an outbound-only encrypted tunnel — no inbound ports open on the router at all. Ollama runs in Docker with NVIDIA GPU passthrough. The AI model inference happens on the GPU, leaving CPU headroom for all other services. Prerequisites Ubuntu 24.04 LTS Docker Engine + Compose v2 NVIDIA GPU with driver 535+ NVIDIA Container Too

NousResearch Agent, Open-Source Notebook LM, & Local Multimodal OCR for Consumer GPUs Today's Highlights Today's highlights feature new open-source tools empowering local AI inference and deployment, including an adaptive agent from NousResearch, a self-hostable AI-powered notebook, and a lightweight multimodal OCR solution. These practical GitHub trending projects enable developers to build and run advanced AI applications directly on consumer hardware. NousResearch Unveils Hermes Agent for Adaptive Local AI (GitHub Trending) Source: https://github.com/NousResearch/hermes-agent NousResearch, a prominent contributor to the open-weight LLM ecosystem with models like the Hermes series, has unveiled hermes-agent , a new GitHub trending project described as "The agent that grows with you." This initiative represents a significant step towards practical, adaptive AI agents designed for local execution. While specific architectural details are awaiting a deeper dive into the repository, the "grows with you" philosophy strongly implies advanced capabilities for personalized learning, continuous adaptation, and long-term memory integration—features crucial for self-hosted AI applications. Such an agent is highly relevant for developers focused on local inference, as it provides an open-source framework to build sophisticated agentic workflows, potentially integrating seamlessly with local LLM runtimes such as llama.cpp or vLLM . This allows users to leverage powerful open-weight models directly on their consumer GPUs, enhancing privacy and reducing reliance on cloud services. The project's emergence from NousResearch solidifies its potential as a robust foundation for next-generation local AI applications. Comment: A NousResearch agent is exciting; it implies strong open-source model compatibility and local deployment. I'm keen to see its learning mechanisms and integration potential with local LLM runtimes. PaddlePaddle's Lightweight OCR Toolkit Bridges Images to Local LLM

This article was originally published on aifoss.dev --- title: 'Chroma vs Qdrant vs Weaviate 2026: RAG Database Compared' description: 'Compare Chroma, Qdrant, and Weaviate for local RAG in 2026: version snapshots, filtering tradeoffs, hybrid search, quantization, and a clear pick by use case.' pubDate: 'May 27 2026' tags: ["vectordb", "ai", "rag", "python", "opensource"] The three most commonly recommended open-source vector databases for RAG — Chroma, Qdrant, and Weaviate — are not interchangeable. Chroma is a prototyping tool that grew into a real product. Qdrant is a production workhorse written in Rust with the best filtering performance of the three. Weaviate is an enterprise-grade platform with hybrid search and the most built-in integrations. Using Weaviate when you need Chroma adds unnecessary ops overhead. Using Chroma when you need Qdrant means migrating under pressure when your collection outgrows it. Versions covered: ChromaDB v1.5.9 (May 2026), Qdrant v1.17.1 (March 2026), Weaviate v1.37 (May 2026). The quick answer Situation Best choice Local prototyping, notebooks, under 100K vectors Chroma Embedded in a Python process — no separate service Chroma Production RAG with filtering-heavy queries Qdrant Multi-user deployment, concurrent queries Qdrant Memory-constrained deployment at millions of vectors Qdrant Hybrid search (BM25 + vector in one query) Weaviate Multi-modal retrieval (text + images + audio) Weaviate Built-in re-ranking or generative AI modules Weaviate Kubernetes, team-operated, agentic MCP workflows Weaviate Getting from zero to working RAG in 10 minutes Chroma What each tool actually is ChromaDB (Apache 2.0, chroma-core/chroma ) started as a pure-Python embedded database and was rebuilt in Rust for the v1.0 release. The Rust core eliminates Python's GIL bottlenecks and delivers roughly 4× faster writes and queries compared to the pre-1.0 implementation — write throughput went from ~10K to ~40K+ vectors/second in server mode. Chroma's des

A homelab box that never sleeps runs 8,760 hours a year. So the spec that decides what it costs you is not the one on the box. It is the one nobody prints: how many watts it pulls sitting at the login prompt doing nothing. I kept hitting this while shopping for a Proxmox node, so I put the measured numbers in one place. More on that at the end. First, why the spec sheet lies to you. TDP is a thermal budget, not a power reading TDP is the heat the cooler has to handle at full tilt. It is a design target for the heatsink, not a measurement of what the chip draws, and it says almost nothing about idle. Your homelab box spends 95%+ of its life idle, so the number that runs up the meter is idle wall power, and that number is never on the product page. The arithmetic is unforgiving. One watt running continuously is 8.76 kWh a year. So the gap between a 7 W box and a 35 W box is not 28 watts, it is about 245 kWh a year, every year, for as long as the box is on. Plug in your own rate to get the dollars; the point is the gap compounds. Where TDP actively misleads you A few measured results from the dataset I'll link below, all from third-party wall-meter readings, not vendor claims: The new N100 wave is genuinely low. A Minisforum UN100C measures 5 to 7 W at idle. Beelink, GMKtec and Trigkey N100 boxes land in the 6 to 10 W range. For a Pi-hole, a few containers and some light VMs, this tier is hard to beat on running cost. AMD mini PCs idle far higher than their marketing suggests. A Minisforum UM790 Pro measures 25 to 45 W at idle. A Beelink SER6 Pro lands at 20 to 35 W. These are fast little machines, but if you picked one expecting "small box, small draw," the meter disagrees, and over a year that delta is real money. Newer and higher-TDP is not lower-idle. A Dell OptiPlex 7060 Micro idles just over 18 W on its 65 W-TDP desktop chip. The older 7070 with a six-core part sits around 13 W, and the low-power "T" SKUs lower still. The CPU's TDP class predicted idle better tha

This is a complete guide with steps by step process to deploy the gRPC and how to use Azure API Management to import the gRPC API. It cover step‑by‑step guide to deploying a gRPC API on Azure API Management (APIM), grounded in the Microsoft documentation and a real-world deployment workflow. NOTE: This post is published already in GITHUB here. https://github.com/shailugit/apimGrpc/blob/main/README.md The API Management can expose gRPC services, but with important constraints: APIM supports gRPC by importing a .proto file and forwarding calls to a gRPC backend. gRPC requires HTTP/2 end‑to‑end. gRPC APIs are supported in Self-hosted gateway and not supported in APIM v2 tiers. You can't use the test console to test gRPC The major steps claissfied in two major steps Creating a gRPC server Calling the gPRC application using APIM 1. Creating gRPC Application Typical backend deployment steps include the following Create a .NET gRPC server application Create a .NET gRPC client application Test the setup locally Publish the .NET gRPC server to Azure WebApp and verify the service works directly over HTTPS Step-1 As a first step we will be building a .NET gRPC server application. You can skip this step in case you already have gRPC server application. If you would like to view .NET Core sample used for this sample project, please visit here . Step-2 As a second step we will be building a .NET gRPC client application. You can skip this step in case you already have gRPC client. If you would like to view .NET Core client used for this sample project, please visit the below here . Step-3 Once your client and server code is ready here are the steps to Test your application locally Step-4 Deploy the server to Azure WebApp To understand how-to deploy a .NET 6 gRPC app on App Service, please visit here . Please make sure to enable HTTP version, Enable HTTP 2.0 Proxy and add HTTP20_ONLY_PORT application setting as gRPC only work using http2.0 as shown below 2. Calling gRPC from APIM T

Did you know that a 35-billion-parameter model can generate tokens at the same compute cost as a 4B model? That single fact made me abandon a multi-model agent architecture I'd spent a weekend building. But I had to run the benchmarks first to understand why. Here's the full breakdown, with commands, numbers, and the architectural reason it all falls apart on shared-memory hardware. The Discovery That Changed Everything I'd been running qwen3.6:35b on my Minisforum UM790Pro for weeks as my daily coding assistant. 17.8 tokens/second -- genuinely usable for interactive work. But I kept wondering: could I run a lightweight sidecar model alongside it for quick classification and tool-calling in an agent pipeline? Before I even started benchmarking, I dug into what qwen3.6:35b actually is under the hood. It's a Mixture of Experts model: 256 total experts with only 8 activated per token. The architecture also incorporates SSM (State Space Model) components alongside traditional attention -- Mamba-style layers that handle certain sequence patterns more efficiently than pure transformers. The math hit me: 8 out of 256 experts means each token only touches roughly 4-5B parameters worth of compute. The model carries 36 billion parameters of knowledge , but its per-token cost is comparable to a small dense model. I was planning to run a separate 4B model for "fast tasks" next to a model that already operates at 4B-class speed. But I had to prove it with numbers. Hardware and Ollama Setup The UM790Pro specs that matter for this experiment: CPU: AMD Ryzen 9 7940HS (Zen 4, 8C/16T) iGPU: AMD Radeon 780M (12 RDNA 3 compute units) RAM: 96 GB DDR5-5600 (~80 GB/s bandwidth) GPU memory pool: 2 GB dedicated VRAM + 46 GB GTT = 48 GB GPU-accessible That 48 GB GPU pool sounds enormous until you realize it's carved from the same DDR5 that the CPU also uses. There is no separate GDDR6 bus. Everything -- CPU inference, GPU inference, KV caches, OS operations -- flows through one 80 GB/s pipe.