On-premise RAG without GPU, cloud, or Docker: five lessons that cost me a week each

Every RAG tutorial I've read makes the same two assumptions: you have a GPU, and you can call a cloud API. For the environments I build for, both assumptions are wrong. I work on health information systems in the public sector. The stack has to run inside institutional infrastructure — no data leaves the network — and the hardware I get is whatever the procurement cycle produced two years ago. In practice that means Windows Server, CPU only, and open-weight models running locally. So I built a RAG stack that runs entirely on-premise, no GPU, no cloud, no Docker. It's open source at github.com/psychohub/rag-onpremise : ASP.NET Core 9 for orchestration, Ollama for local inference, Qdrant for vectors, Python for the ingest pipeline, Mistral 7B as the LLM, nomic-embed-text for embeddings. Getting it into production took longer than the design did, because five things broke that no tutorial had warned me about. This is the field report. The environment, and why it matters Before the lessons, it's worth being precise about the constraint, because it changes what "good" looks like. The stack has to run on a Windows Server, not a Linux workstation. Docker is not available on many of the target machines — either because it wasn't approved, because GPO policies restrict it, or because ops teams already run everything as Windows services and adding a container runtime is a new operational surface nobody wants to own. GPUs are aspirational. In the meantime, you have CPU inference and you have to make it work. None of this is exotic. It's the default reality in a lot of public sector, healthcare, and legacy enterprise environments. It's also the reality most RAG content on the internet quietly assumes away. The overall shape of the system: Documents (PDF / Word / Excel) │ ▼ [ Python ingest ] ├─ Text extraction (pdfplumber, python-docx, openpyxl) ├─ Chunking (500 tokens, 50 overlap) ├─ Embeddings (nomic-embed-text via Ollama) └─ Store (Qdrant, cosine similarity) │ User query │ │