标签：#selfhost

Overleaf is an open-source, collaborative LaTeX editor that bundles MongoDB, Redis, and the ShareLaTeX application into a single self-hosted stack. This guide deploys Overleaf Community Edition using the official Toolkit plus a Traefik override that adds automatic HTTPS via Let's Encrypt. By the end, you'll have Overleaf serving collaborative LaTeX editing securely at your domain. Set Up the Project Directory 1. Clone the Overleaf Toolkit: $ git clone https://github.com/overleaf/toolkit.git ~/overleaf-toolkit $ cd ~/overleaf-toolkit 2. Initialize the configuration: $ bin/init This creates config/overleaf.rc , config/variables.env , and config/version . 3. Create a directory for Traefik file-provider routes: $ mkdir traefik-routes Configure Overleaf for a Reverse Proxy 1. Edit config/variables.env : $ nano config/variables.env OVERLEAF_APP_NAME = "My Overleaf Instance" OVERLEAF_SITE_URL = https://overleaf.example.com OVERLEAF_NAV_TITLE = "Overleaf CE" OVERLEAF_BEHIND_PROXY = true OVERLEAF_SECURE_COOKIE = true The last two flags are required when Overleaf is fronted by an HTTPS reverse proxy. 2. Edit config/overleaf.rc : $ nano config/overleaf.rc SIBLING_CONTAINERS_ENABLED = false 3. Create the project-level .env file used by the Traefik Compose file: $ nano .env DOMAIN = overleaf.example.com LETSENCRYPT_EMAIL = admin@example.com SERVER_IP = 192.0.2.1 SERVER_IP should be the server's public IP (Traefik binds 80/443 to it). Add the Traefik Route 1. Create the dynamic route: $ nano traefik-routes/overleaf.yml http : routers : overleaf : rule : " Host(`overleaf.example.com`)" service : overleaf entryPoints : - websecure tls : certResolver : le services : overleaf : loadBalancer : servers : - url : " http://sharelatex:80" 2. Create the Traefik Compose file: $ nano docker-compose.traefik.yml services : traefik : image : traefik:v3.6 container_name : traefik restart : unless-stopped command : - " --providers.docker=true" - " --providers.docker.exposedbydefault=false" - " --

NocoDB is an open-source no-code platform that puts a spreadsheet-style UI on top of a relational database, with grid, form, Kanban, and gallery views plus a REST API. This guide deploys NocoDB using Docker Compose with a PostgreSQL backend and Traefik handling automatic HTTPS, then exercises the API with a sample base. By the end, you'll have NocoDB serving a base over HTTPS with API access at your domain. Set Up the Directory Structure 1. Create the project directories: $ mkdir -p ~/nocodb/ { data,pgdata,letsencrypt } $ cd ~/nocodb 2. Create the environment file: $ nano .env DOMAIN = nocodb.example.com LETSENCRYPT_EMAIL = admin@example.com POSTGRES_DB = postgres POSTGRES_PASSWORD = strong_password POSTGRES_USER = postgres Deploy with Docker Compose 1. Create the Compose manifest: $ nano docker-compose.yaml services : traefik : image : traefik:v3.6 container_name : traefik command : - " --providers.docker=true" - " --providers.docker.exposedbydefault=false" - " --entrypoints.web.address=:80" - " --entrypoints.websecure.address=:443" - " --entrypoints.web.http.redirections.entrypoint.to=websecure" - " --entrypoints.web.http.redirections.entrypoint.scheme=https" - " --certificatesresolvers.letsencrypt.acme.httpchallenge=true" - " --certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web" - " --certificatesresolvers.letsencrypt.acme.email=${LETSENCRYPT_EMAIL}" - " --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json" ports : - " 80:80" - " 443:443" volumes : - " ./letsencrypt:/letsencrypt" - " /var/run/docker.sock:/var/run/docker.sock:ro" restart : unless-stopped db : image : postgres:16 container_name : nocodb-db hostname : root_db environment : POSTGRES_DB : ${POSTGRES_DB} POSTGRES_USER : ${POSTGRES_USER} POSTGRES_PASSWORD : ${POSTGRES_PASSWORD} volumes : - " ./pgdata:/var/lib/postgresql/data" healthcheck : test : [ " CMD" , " pg_isready" , " -U" , " ${POSTGRES_USER}" , " -d" , " ${POSTGRES_DB}" ] interval : 10s timeout : 5s retries :

LocalAI is an open-source platform for running Large Language Models locally with an OpenAI-compatible API, so you can swap it in behind existing OpenAI client code without paying per-token or sending data off-server. This guide deploys LocalAI using Docker Compose with Traefik handling automatic HTTPS, persistent model and cache directories, and a working chat-completion test. By the end, you'll have LocalAI serving an OpenAI-compatible API securely at your domain. Set Up the Directory Structure 1. Create the project directories: $ mkdir -p ~/localai/ { models,cache } $ cd ~/localai models/ holds downloaded model files; cache/ persists between restarts. 2. Create the environment file: $ nano .env DOMAIN = localai.example.com LETSENCRYPT_EMAIL = admin@example.com Deploy with Docker Compose 1. Add your user to the Docker group: $ sudo usermod -aG docker $USER $ newgrp docker 2. Create the Compose manifest: $ nano docker-compose.yaml services : traefik : image : traefik:v3.6 container_name : traefik restart : unless-stopped environment : DOCKER_API_VERSION : " 1.44" command : - " --providers.docker=true" - " --providers.docker.exposedbydefault=false" - " --entrypoints.web.address=:80" - " --entrypoints.websecure.address=:443" - " --entrypoints.web.http.redirections.entrypoint.to=websecure" - " --entrypoints.web.http.redirections.entrypoint.scheme=https" - " --certificatesresolvers.le.acme.httpchallenge=true" - " --certificatesresolvers.le.acme.httpchallenge.entrypoint=web" - " --certificatesresolvers.le.acme.email=${LETSENCRYPT_EMAIL}" - " --certificatesresolvers.le.acme.storage=/letsencrypt/acme.json" ports : - " 80:80" - " 443:443" volumes : - /var/run/docker.sock:/var/run/docker.sock:ro - ./letsencrypt:/letsencrypt localai : image : localai/localai:latest-aio-cpu container_name : localai restart : unless-stopped volumes : - ./models:/models:cached - ./cache:/cache:cached healthcheck : test : [ " CMD" , " curl" , " -f" , " http://localhost:8080/readyz" ] interval :

LibreChat is an open-source, ChatGPT-style web UI that supports OpenAI, Anthropic, Azure OpenAI, Gemini, OpenRouter, local OpenAI-compatible endpoints, and more — with MongoDB-backed conversation history and Meilisearch-powered search. This guide deploys LibreChat using its official Compose manifest plus a Traefik override for automatic HTTPS. By the end, you'll have LibreChat running with a registration page and multi-provider chat at your domain over HTTPS. Clone LibreChat and Prepare the Environment 1. Clone the LibreChat repository and check out a stable tag: $ git clone https://github.com/danny-avila/LibreChat.git $ cd LibreChat $ git checkout tags/v0.8.3 2. Find the Meilisearch data directory name pinned by this release: $ grep -o 'meili_data_v[0-9.]*' docker-compose.yml | head -1 3. Create the required data directories (replace meili_data_v1.35.1 if the previous command printed a different name): $ mkdir -p data-node images logs meili_data_v1.35.1 uploads $ sudo chown -R 1000:1000 meili_data_v1.35.1 4. Copy the env template and uncomment the UID/GID lines: $ cp .env.example .env $ nano .env UID = 1000 GID = 1000 Override the Compose Stack with Traefik 1. Create a Compose override that adds Traefik and wires the API to it: $ nano docker-compose.override.yml services : api : labels : - " traefik.enable=true" - " traefik.http.routers.librechat.rule=Host(`librechat.example.com`)" - " traefik.http.routers.librechat.entrypoints=websecure" - " traefik.http.routers.librechat.tls.certresolver=leresolver" - " traefik.http.services.librechat.loadbalancer.server.port=3080" volumes : - ./librechat.yaml:/app/librechat.yaml traefik : image : traefik:v3.6.10 ports : - " 80:80" - " 443:443" volumes : - " /var/run/docker.sock:/var/run/docker.sock:ro" - " ./letsencrypt:/letsencrypt" command : - " --providers.docker=true" - " --providers.docker.exposedbydefault=false" - " --entrypoints.web.address=:80" - " --entrypoints.websecure.address=:443" - " --entrypoints.web.http.redirect

If you've ever tried to set up Prometheus by following the official getting-started path, you're likely to find a path that does not follow your infrastructure model. Out of the gate, page one mentions kube-prometheus-stack. Page two wants you to install a Helm chart, and page three assumes you already have a cluster running. The documentation for monitoring plain Linux servers is in there somewhere, but you have to dig for it. When you do find it, the tone suggests you are doing something slightly old-fashioned. If that sounds like your setup, the tooling is making this harder than it actually is. Monitoring a fleet of Linux VMs is fairly simple and has been for years. It is just obscured behind documentation that would prefer to sell you something bigger. Modern infrastructure tooling has quietly decided everyone runs Kubernetes. If you don't, the assumption is that you eventually will. Meanwhile, most real-world infrastructure still runs on VMs. TL;DR: Modern observability documentation often assumes you're running Kubernetes. Most small teams aren't. If you're managing a fleet of Linux VMs, node_exporter plus Prometheus gives you everything you need for infrastructure monitoring with a single lightweight agent and a straightforward deployment model. No cluster required. VMs are often the answer For most small businesses, running VMs instead of Kubernetes does not mean you failed to evolve. Most workloads under a certain scale perform better on VMs: One process per box, predictable resource limits, and the ability to ssh in and look at what's happening, which makes it easier to keep track of the infrastructure as a whole. They're cheaper, both financially and in the mental overhead of running them. Backups and snapshots are straightforward in a way stateful Kubernetes still isn't. There's no control plane that itself needs monitoring and upgrades and care. Kubernetes solves problems that mostly pertain to companies with dozens of engineers and hundreds of service

У большинства self-hosted VPN-панелей одна и та же боль: Docker-стек, внешняя БД, реверс-прокси и куча конфигов, которые надо связать между собой, прежде чем хоть что-то заработает. Мне хотелось наоборот — что-то, что можно закинуть на свежий VPS и поднять меньше чем за минуту. Так появилась РосПанель : self-hosted панель для администрирования личного VPN-сервера на Xray-core , который поставляется одним статическим бинарником . React-фронтенд вшит через go:embed , база — встроенный SQLite, отдельного веб-сервера нет. Поставил, открыл, добавил юзера. Главная идея: один бинарник, ничего лишнего Цель, которая определила всё остальное, — радикальная простота. В отличие от Marzban и 3x-ui, у РосПанели нет Docker-обвязки, нет внешней БД и нет отдельного веб-сервера, который надо настраивать. Всё живёт в одном исполняемом файле: Веб-интерфейс собирается в web/dist и вшивается в Go-бинарник на этапе сборки. Состояние хранится в SQLite (чистый Go-драйвер modernc , то есть без CGO ). Конфиг Xray всегда генерируется из базы и применяется супервизором — SQLite это единственный источник правды, а не JSON, который правят руками. В итоге деплой — это просто положить бинарник и systemd-юнит. Никакой оркестрации, ничего не надо держать в синхроне. Что она на самом деле делает РосПанель — это панель управления (control plane), а не VPN-клиент. Она настраивает и обслуживает ваш собственный сервер: генерирует конфиг Xray, выдаёт ссылки на подписки и показывает статистику. Протоколы из коробки — один конфиг Xray, один набор учёток: VLESS-Vision (TCP/443 + uTLS-fingerprint) Trojan-WS (через fallback на 443) Hysteria2 (UDP с port-hopping) VLESS-gRPC-REALITY (отдельный порт, маскировка под чужой TLS) Маскировка — панель спрятана за секретным путём. Любой другой путь отдаёт сайт-заглушку (11 готовых шаблонов), так что сервер неотличим от обычного хостинга. Без знания /<secret>/ форму логина не найти. TLS, который просто работает — ACME через Let's Encrypt или ZeroSSL, авто-продление и self

Build-in-public. This is the real architecture behind running ~50 local AI agents on 6GB of VRAM — one GPU lock, an eviction watchdog, a resource governor, and a model router. Originally posted on my blog. The question I get most often is some version of "there's no way you run that many agents on a 6GB laptop GPU." The honest answer: not the way you're picturing it. I don't run 50 models at once. I run one model at a time, very deliberately — and most of the engineering is about scheduling, not inference. Here's the actual architecture. The hard constraint: 6GB of VRAM A single consumer GPU with 6GB of VRAM holds roughly one 7B-parameter model at a usable quantization. Two at once? It thrashes — the GPU starts swapping, latency explodes, and eventually a driver out-of-memory can take the whole machine down. I've had the desktop freeze from exactly that. So the first design rule wrote itself: only one heavy model is allowed on the GPU at any moment. That sounds limiting. It isn't — because almost nothing I run is latency-sensitive. A blog post that publishes at 7am doesn't care if it was generated at 6:52 or 6:58. Once you accept that your AI workforce is a batch system, not a chat window, the whole problem changes shape. A lock, not a crowd Every agent that needs the GPU has to take a lock first. It's a simple file-based queue with: FIFO ordering PID-based ownership Stale-lock detection, so a crashed job can't wedge the line forever If an agent can't get the lock within its timeout, it skips gracefully and tries again on its next scheduled run instead of piling up. So at 50 agents, what's really happening is: dozens of cron-scheduled Python workers wake up throughout the day, and the ones that need the model form an orderly line for it. The fleet is huge; the GPU contention is always exactly one. That's the trick. It's less "50 models" and more "50 employees sharing one very busy workstation, politely." Eviction and a VRAM watchdog Even with the lock, idle models l

How I replaced Google Photos, Google Drive, and Google Home with a self-hosted Raspberry Pi 5 setup - and what the full technical stack looks like. There's a quiet moment every tech-literate person eventually hits. You open your cloud storage dashboard, see the number creeping up, and think: I'm paying a subscription fee, every month, forever, just to store my own photos and files on someone else's computer. That was me, but with a bit of added weight. It's not just my data I'm responsible for. Over the years, I've quietly become the unofficial digital curator for my entire family . The person everyone sends photos to after a birthday party. The one who backs up the wedding videos. The one who scans and stores the important documents, passports, contracts, and sentimental things so they don't get lost. My parents, siblings, extended family: if something matters and it's digital, there's a good chance it eventually lands with me. That's a responsibility I take seriously. And for a long time, Google was my answer - until the storage bill started quietly creeping past 2TB, and I started thinking more carefully about what it actually means to hand all of that data over to "big tech" . So I built my own home server. A tiny, almost silent box sitting in my house that now handles everything Google Drive and Google Photos did, plus more, for a one-time hardware cost of £340. This is the story of how I did it, why I did it, and exactly how you can too. Why I Did It The cost was the trigger, but it wasn't the only reason. When you use Google Photos, Google Drive, or any cloud storage service, your files live on their servers. You're trusting a corporation to keep them safe, not snoop through them, not change their pricing, and not shut down the product one day. Google has a long history of killing beloved products. Google Photos itself famously ended its unlimited free tier in 2021. As well as this, recent AI trends and auto opt-in data processing had me thinking there had to

Last year my side project was running on AWS. A t3.small EC2 instance, an RDS PostgreSQL db.t3.micro, an S3 bucket, and a CloudFront distribution. Total bill: $47/month for an app with 200 daily users. Then someone on Reddit told me to look at Hetzner. I now run the same stack on a single CAX21 (4 vCPU ARM, 8GB RAM, 80GB SSD) for €5.49/month. Here's exactly what happened. The Migration What I was running on AWS: Node.js API (Express) PostgreSQL database Redis for sessions Nginx reverse proxy Static files on S3 + CloudFront What I moved to Hetzner: Same Node.js API PostgreSQL installed directly on the server Redis installed directly on the server Nginx + Certbot for SSL Static files served by Nginx Total migration time: one Saturday afternoon. The hardest part was setting up automated backups (solved with a cron job + Hetzner's snapshot API). What Broke Nothing critical, but: No managed database failover. On RDS, if the database crashes, AWS restarts it automatically. On Hetzner, if PostgreSQL crashes at 3 AM, I'm the one fixing it. In 8 months, this has happened zero times. But it could. No CDN by default. My static assets now serve from a single Hetzner datacenter in Germany. For my EU-heavy userbase, this is actually faster than CloudFront. For US users, it's about 50ms slower. I added Cloudflare (free tier) in front and the problem disappeared. Deployment changed. No more eb deploy or push-to-deploy. I wrote a 12-line bash script that SSHs in, pulls from git, runs migrations, and restarts PM2. Takes 8 seconds. Honestly prefer it — I know exactly what's happening. The Cost Comparison at Every Scale This is what surprised me most. The gap isn't just at my small scale — it gets wider as you grow: SpecAWSDigitalOceanVultrHetzner2 vCPU, 4GB$30/mo$24/mo$24/mo€4.50/mo4 vCPU, 8GB$61/mo$48/mo$48/mo€8.50/mo8 vCPU, 16GB$122/mo$96/mo$96/mo€16/mo Hetzner is roughly 5-7x cheaper than AWS at every tier. DigitalOcean and Vultr sit in the middle. 👉 Calculate your exact costs When

I open-sourced MarketEye today. For anyone who missed the first post: MarketEye is a self-hosted competitor price monitor I built because I didn't want to pay $99/month for Prisync. The code is now up on GitHub under MIT license. GitHub: github.com/dachengzi065-gif/marketeye Why open source? Three reasons: 1. People actually asked for it. After my first post here, a few people DM'd me asking to see the code. They're developers too — they want to modify it, extend it, make it their own. That's fair. Selling source code to devs is like selling ice to eskimos. 2. Trust. A closed-source price tracker that "runs on your machine" — you either trust the author or you don't. Open source removes that doubt. You can read every line, check what data leaves your machine (nothing), and build it yourself if you want. 3. Longevity. Self-hosted tools have a dirty secret: if the developer disappears, you're stuck with a broken tool. Open source changes that. Even if I get hit by a bus tomorrow, you can fork the repo and keep going. What this means for the $49 version The Gumroad package still exists. It includes: The same code, pre-packaged Email support (I'll help you set it up) A clear conscience subscription (you're paying for convenience, not software) But honestly? If you can run pip install , just clone the repo. It's free. What's next I'm actively working on: Docker image (one-command deploy) More scrapers (plugins for different sites) Discord/Telegram bot alerts (requested by several people) PRs welcome. Issues welcome. Feedback welcome. 👉 github.com/dachengzi065-gif/marketeye

Local Inference Powers Browser Sign Language, Open-Source Agent Infra, & AI Engineering Guides Today's Highlights This week highlights practical advancements in local AI, featuring a browser-based sign language reader running entirely on-device, new open-source infrastructure for building and evaluating AI agents, and a comprehensive guide to AI engineering from scratch, focusing on building and shipping models efficiently. I Built a Webcam Sign-Language Reader in the Browser (No Cloud) (Dev.to Top) Source: https://dev.to/dev48v/i-built-a-webcam-sign-language-reader-in-the-browser-no-cloud-11hg This article details the creation of a real-time sign language reader that operates entirely within a web browser, without relying on cloud services or model uploads. The developer showcases how to achieve genuinely useful AI functionality, traditionally associated with heavy research labs and GPU clusters, using client-side processing. This approach emphasizes privacy, reduced latency, and accessibility by making advanced AI applications runnable on consumer hardware, specifically within the browser environment. The implementation leverages lightweight models optimized for on-device inference, demonstrating the power of WebAssembly or WebGPU for local execution of machine learning. Such a system offers significant advantages for applications requiring immediate feedback or handling sensitive user data, aligning perfectly with the principles of local AI and empowering developers to deploy sophisticated multimodal solutions without external dependencies. This project serves as an excellent example of practical, self-hosted AI and multimodal processing on consumer hardware. Comment: Running a vision model this complex purely client-side with decent performance is impressive. It really pushes the boundaries of what's feasible for local, privacy-preserving multimodal AI in the browser. trycua/cua — Open-source infrastructure for Computer-Use Agents (GitHub Trending) Source: https

Six months ago I moved my RAG pipeline from Pinecone to self-hosted Qdrant. My vector search bill went from $210/month to $6.50/month. Same latency. Same recall. Here's exactly how. The Setup My app does document Q&A for legal contracts. The numbers: 5.2 million vectors (1536-dim, OpenAI embeddings) ~800K queries/month P99 latency requirement: < 50ms On Pinecone Serverless, this cost me roughly $210/month — storage plus read units plus write units for daily ingestion of new documents. What I Moved To A single Hetzner CX32 server: 4 vCPU, 8 GB RAM, 80 GB SSD €8.50/month (about $9.20) Qdrant running in Docker Automated daily backups to S3-compatible storage ($0.50/month) Total: ~$10/month. That's a 95% cost reduction. The Migration Was Easier Than Expected bash# Export from Pinecone (I used their scroll API) python export_pinecone.py --index legal-docs --output vectors.jsonl Start Qdrant docker run -d -p 6333:6333 -v ./storage:/qdrant/storage qdrant/qdrant Import python import_qdrant.py --input vectors.jsonl --collection legal-docs The whole migration took an afternoon. The Qdrant Python client is straightforward, and the API is surprisingly similar to Pinecone's. Performance Comparison I ran the same 10,000 test queries against both setups: MetricPinecone ServerlessQdrant Self-HostedP50 latency23ms4msP99 latency89ms12msRecall@100.970.97Monthly cost$210$10 The self-hosted Qdrant is actually faster because the data sits in memory on the same machine. Pinecone Serverless loads data from object storage on demand, which adds cold-start latency. When Self-Hosting Is a Bad Idea I want to be honest about the trade-offs: Don't self-host if: You have zero DevOps experience and no one on the team does You need 99.99% uptime SLA for enterprise customers Your vector count is growing unpredictably (10M one month, 100M the next) You're a team of 1-2 and every hour on infra is an hour not building product Do self-host if: Your scale is predictable (you know roughly how many vectors

Last week I couldn't shake an idea: what if I had an AI that knew everything I know ? Not ChatGPT — something on my hardware, holding my knowledge, answering to no one's API bill. Yesterday I built it. Here's the honest breakdown. What it does NEXUS runs on a regular Windows laptop — aging i7, 16GB RAM, no GPU. It: Remembers everything. Drop any file in a folder; 60 seconds later it's searchable memory. Answers from MY knowledge. "Which of my projects were formally closed and why?" — it answers from my actual records. Watches the live web. Every 2 hours it pulls Hacker News and news feeds, learns what's trending, pings my Telegram. Reports to my phone. 7 AM daily briefing: what it learned, what's running, what needs me. The stack — all free, all open source Ollama runs the models (Llama 3.2, Mistral 7B). Open WebUI is my private ChatGPT. Qdrant stores memory. n8n automates. SearXNG searches privately. PostgreSQL, Redis, and MinIO handle data. Commercial equivalent: $300–500/month . My cost: electricity. The memory trick nobody explains simply Parse — extract text from any file Chunk — split into ~300-word pieces Embed — each chunk becomes 768 numbers representing its meaning Store — a database that searches by similarity Your question becomes 768 numbers too, and the database finds memories with similar meaning — not matching keywords. I asked "how do I get clients cheaper" and it found my notes on "reducing customer acquisition cost." Different words. Same meaning. That's the magic. What surprised me A 2GB model is genuinely useful. Llama 3.2 3B answers from my knowledge in seconds, on CPU. The automation matters more than the AI. The watched folder + Telegram bot turned a cool demo into a system I actually use. Windows is fine. Docker Desktop + WSL2 ran all nine services without drama. The bill, honestly Hardware: $0 (laptop I own) Software: $0 (open source) APIs: $0 (all local) Time: one focused day The only future cost is a cloud GPU server (~$65/mo) when I outg

I have been building Lunarr , a self-hosted web media server for people who want to scan, organize, and watch their own movie and TV libraries in the browser. It is still early, but the core idea is simple: Add local or SFTP media libraries, scan them, match metadata, and play them through a clean web UI. Lunarr is not trying to replace Plex or Jellyfin overnight. Those projects are mature and cover a huge surface area. Lunarr is currently focused on being small, direct, and practical for self-hosted setups where media may live on the same machine or on remote SFTP storage. What Lunarr does today Lunarr currently supports: Local movie and TV libraries SFTP movie and TV libraries TMDb metadata matching Movie, show, season, and episode organization Browser playback Direct streaming when the browser can play the file Temporary HLS remux/transcode when needed Seekable request-driven HLS playback Sidecar .vtt subtitle detection Admin/user accounts Library sharing controls Manual scans, scheduled scans, and local file watching Docker deployment The SFTP support is one of the important parts for me. A lot of self-hosted setups do not keep media on the same machine as the web app. Lunarr can scan remote folders and, when possible, play seekable remote media without first copying the whole file locally. Playback model Lunarr tries direct playback first when the browser can handle the file. When direct playback is not suitable, Lunarr uses temporary HLS playback. Instead of transcoding an entire movie up front, it generates segments around what the player is actually requesting. For example, if you jump from 55 minutes to 13 minutes and then to 80 minutes, Lunarr does not transcode everything between those points. It repositions FFmpeg, generates the requested segment, and prepares a small lookahead window so playback can continue. That keeps CPU and disk usage more proportional to what the viewer is actually watching. Quick start with Docker docker run -d \ --name lunarr \ -

Overnight agents do good work, then dump it in a log file or a noisy Slack channel. Here's a pattern for delivering their output to a private, end-to-end encrypted inbox you read with your coffee. You point an agent at a nightly job — audit the dependencies, summarize yesterday's support tickets, check the infra, scan the repo for regressions. It runs at 3 a.m. and does good work. Then the work goes... where? Usually one of three bad places: A log file you'll never open. A Slack channel that's already 200 messages deep by the time you wake up. A plaintext file on a server , which is fine until the report contains a leaked key, a customer name, or a security finding — and now it's sitting in cleartext on a box you don't fully trust. And the fix you'd reach for first — "just email the report to me" — is the one that bites hardest. You can do it cleanly: a locked-down, send-only API key sends mail and nothing else. But the path of least resistance is "connect your email account," and that grant is far wider than the job needs — now the agent can read and send your mail, not just hand you a file. I learned this the hard way. I once connected an agent to my email so it could send me updates — and it took that as license to start replying to my incoming messages on its own, without my ever asking. Mail went out under my name that I never wrote. The job was "send me a file." The access I'd handed over was "run my inbox." The work is good. The delivery is the broken part. Here's a pattern that fixes it: your overnight agent delivers its report to a private, end-to-end encrypted inbox, and you read it with your coffee — decrypted in your browser, with a passkey. What we're building cron, 3 a.m. ↓ agent does the work ↓ encrypted delivery ↓ your inbox (read at 8 a.m.) The agent produces a report (Markdown, PDF, a CSV, whatever), hands it to the Agent Relay CLI, and the CLI encrypts it locally before it ever leaves the machine. The server stores only ciphertext. When you open t

LLM KV Cache Optimization, Open Model Evaluation, & Agent Engineering Skills for Local Deployment Today's Highlights This week, a groundbreaking KV cache layer promises to supercharge local LLM inference, alongside a new workbench for evaluating open language models. Additionally, a trending repository provides production-grade engineering skills for building robust AI agents, crucial for self-hosted deployments. LMCache: Supercharge Your LLM with the Fastest KV Cache Layer (GitHub Trending) Source: https://github.com/LMCache/LMCache LMCache introduces a novel KV cache optimization layer designed to significantly accelerate Large Language Model (LLM) inference. The KV cache (Key-Value cache) is a critical component in LLM decoding, storing previously computed keys and values for attention layers to avoid redundant calculations. Optimizing this cache is paramount for achieving high throughput and low latency, especially when running large models on consumer-grade hardware or self-hosted servers. This project aims to provide the fastest KV cache solution, directly addressing a key bottleneck in local LLM deployment and performance. By improving KV cache efficiency, LMCache enables developers and researchers to run more complex models or serve more users with existing hardware, making advanced LLMs more accessible for local inference scenarios. Details on its architecture and comparative benchmarks against existing solutions will be critical for understanding its impact on various open-weight models and frameworks like vLLM or llama.cpp. Comment: Faster KV cache is a game-changer for anyone running LLMs locally. This project could unlock new performance levels for open models on consumer GPUs. olmo-eval: An evaluation workbench for the model development loop (Hugging Face Blog) Source: https://huggingface.co/blog/allenai/olmo-eval The olmo-eval workbench from AllenAI provides a comprehensive system for evaluating language models throughout their development lifecycle.

The box that ran everything started dying in April. Not dramatically. Machines almost never die dramatically. It started with instability... the kind you explain away once, side-eye twice, and start losing sleep over the third time production goes down while you're in the middle of something else. The host under my entire stack... public site, analytics, security tooling, the AI crew's memory layer... was getting flaky. And flaky hardware only trends one direction. Here's the thing about a homelab that lives in a 40ft fifth wheel: there is no second team. No vendor escalation. No change advisory board. No maintenance window negotiated three weeks out. There's me, a crew of governed AI instances, and a reclaimed Dell T3600 about to get the biggest promotion of its second life. So we didn't try to heal the sick box. We built a new hull alongside it and started moving the ship... plank by plank... while it was still sailing. One ground rule, set day one: the old host stays untouched and keeps serving production until the new hull is proven. Not "mostly proven." Proven. Hold that thought, it matters at the end. Moving containers is the easy part. Docker made that boring years ago, and boring is a compliment in infrastructure. What's never boring is the inventory of everything you assumed and never wrote down. A migration doesn't test your stack. It tests your assumptions. Here's what mine were hiding. 01: The umbilical nobody documented Security stack went first. Suricata, Zeek, Wazuh, CrowdSec, Falco, the whole alphabet, up clean on the new hull. Then the MCP server, the piece that gives the AI crew its hands, refused to come up right. It was hard-wired over HTTP to the crew's memory backend. A live dependency, in production for months, documented exactly nowhere. The crew that documents every f*cking thing had never documented its own umbilical cord. Fix was trivial once we could see it: deploy the brain before the hands. Reorder, redeploy, done. But the lesson isn't

Six months ago I got tired of switching between apps to talk to AI. ChatGPT in the browser. Claude in another tab. Local models in a terminal. It was like having five friends who all live in different cities and refuse to visit each other. So I did what any developer with too many GPUs and too little patience would do: I built my own assistant and put it where I already spend my day — Telegram. It's not a chatbot for customers. It's not a business automation tool. It's just... my assistant. It lives in a private chat on my phone and handles the stuff I used to do manually. Here's what six months of actually using it has looked like. What I Actually Built (And Why Telegram) I already had three machines running Ollama at home — a Mac Mini M4, a Windows PC with an RTX 3060, and an Ubuntu box. Three endpoints, eight models, and me constantly forgetting which model was good for what. Telegram was the obvious choice because: I'm already there all day (friends, family, a few dev groups) It works on my phone, my Mac, and my watch The Bot API is dead simple I can send voice messages, photos, documents — and the bot can handle all of them The setup: a Python bot running on the Mac Mini, connected to all three Ollama endpoints. When I message it, the bot classifies what I want, routes to the right model on the right machine, and replies in the same chat thread. Sounds simple. Took three evenings to get right. Took six months to make actually useful. The Things I Actually Use It For Here's the honest list. Not the marketing pitch — the real daily usage: 1. Quick questions without context switching "Summarize this article" (I paste a link). "Explain this error" (I paste a stack trace). "Rewrite this email less formally." These used to mean opening a browser tab, logging in, maybe hitting a rate limit. Now I just... send a message. The reply comes back in 2-8 seconds depending on which model handles it. The routing is simple but effective: quick chat → small model on the Mac. Cod

I Tried PewDiePie's Open-Source AI Workspace. It's Actually Good. Yes, that PewDiePie. Felix Kjellberg (110M YouTube subscribers) spent late 2025 building a home AI lab — 8 modified RTX 4090s, 256GB of VRAM, running on Arch Linux. He called it "The Swarm." He crashed it running 64 models in parallel. The web frontend he built for it? He open-sourced it. Called it Odysseus . It hit 59,000 GitHub stars fast. I dug into the code expecting a glorified Ollama wrapper. It's not. What it actually is Odysseus isn't just another chat UI. It bundles things no other self-hosted tool does in one place: Chat — local or cloud models (Ollama, vLLM, llama.cpp, OpenAI, OpenRouter, GitHub Copilot) Agent mode — shell, files, web, MCP tools, per-tool toggles Cookbook — scans your GPU, recommends models that actually fit, downloads and serves them in one click Deep Research — multi-step web research that writes you a cited report Email — IMAP/SMTP with AI triage, auto-tagging, draft replies Calendar — CalDAV sync with Radicale, Nextcloud, Apple, Fastmail Memory — persistent, evolving across all your conversations No cloud account. No telemetry. MIT license. Everything lives in your data/ folder. The Cookbook is the standout feature Every other self-hosted UI assumes you already know what model to run. Odysseus doesn't. It scans your hardware, scores 270+ models against your actual VRAM, and gives you a one-click download-and-serve. It understands GGUF vs FP8 vs AWQ. It picks the right backend (vLLM, llama.cpp, Metal on Apple Silicon). Downloaded models persist in a volume — no re-downloading after container restarts. For someone who wants local AI but finds the ecosystem confusing, this is the most accessible on-ramp that currently exists. The code is better than the meme suggests The README has a little ASCII bear face. Don't let it fool you. The entry point app.py is 1,092 lines of real production thinking. A few things that stood out: The .env loader handles Windows BOM silently: loa

Every time I started a side project, I rebuilt the same five things before I wrote a single line of the actual idea: auth, a database, file uploads, a deploy pipeline, and TLS. Different domain, same plumbing. By the third project I was copy-pasting my own docker-compose.yml from a folder two repos over and renaming things until they stopped erroring. So I stopped. I froze that plumbing into one boilerplate — PocketBase + Next.js + Caddy on a single cheap VPS — and now I ship every side project on it. Same shape every time: clone, rename, write the part that's actually new, push. The thing I care about most isn't the speed, though. It's that I stopped paying for it. A handful of real projects now run on this setup for a few euros a month, total — not a stack of per-service SaaS bills that each want $25 here and $20 there before you've shipped anything. No Vercel seat, no managed Postgres, no Auth-as-a-Service, no object-storage line item. Here's the whole thing. Why this stack The trick that makes the cost collapse is PocketBase . It's a single Go binary that gives you, in one process: Auth — email/password, OAuth, the works, with a real users collection A database — SQLite, with a schema you manage from an admin UI Realtime — subscribe to collection changes over SSE File storage — uploads handled, with on-the-fly thumbnails An admin dashboard — at /_/ , for free That's four or five separate SaaS products collapsed into one binary that runs anywhere and stores everything in a folder. Compared to wiring up Supabase or Firebase, the mental model is tiny: it's one process and one data directory. Back up the directory and you've backed up the entire app — database, uploaded files, auth tokens, all of it. For the front I use Next.js (App Router, React Server Components) because that's where I'm fastest, and Caddy as the reverse proxy because it gets you automatic HTTPS with zero config — it provisions and renews Let's Encrypt certificates on its own. And it all lives on