标签：#ha

This is a submission for the June Solstice Game Jam What I Built Turing's Mirror is a...

Disclaimer: This was made with LLMs. I made this tool to help understand large business processes that can be modelled as a single state machine. The core loop of this is to enable to walk stakeholders through discussing each step, adding comments, and reiterating with an LLM of their choice to generate the TLA+ syntax on the left. Users can click through the green state nodes to see how things work visually. You can see some sample state machines in the dropdown in the top left. The power would

I run a homelab with four RTX 3090s — 96 GB of VRAM, 44 CPU cores. For two weeks I tried to make it my daily driver for local LLM inference instead of paying for cloud APIs. I got it working. Then I looked at the numbers and subscribed to a paid API anyway. Here's the uncomfortable part, and the optimizations that still made it worth doing. ## The setup 4× RTX 3090 (Ampere — no native BF16), 96 GB VRAM total, 44 cores Models: Qwen3.6-35B-A3B (Q8_0, MoE) and Qwen3-Coder-Next (Q6_K, hybrid) llama.cpp in router mode + OpenWebUI Ceiling I hit: ~105 tokens/second ## The 6% problem The wall wasn't compute. GPU utilization sat at 6%. The bottleneck was CPU orchestration — llama.cpp dispatches across multiple GPUs sequentially, so the cards spent 94% of the time idle waiting on each other. Throwing more VRAM at it does nothing for this. ## What actually moved the needle | Change | Effect | |---|---| | --ubatch-size 512 | +40% throughput | | KV cache quantization (Q4_0) | 4× VRAM savings | | Speculative decoding (n-gram) | 2.5× speedup on repetitive tasks | | YaRN rope scaling | context extended to 1M tokens | Two things surprised me: MoE models tolerate aggressive quantization far better than dense ones — inactive experts don't eat bandwidth, so the quant hit lands softer. The 3B active -parameter model was great at local decisions but fell apart on coherence past ~300–400 lines of code — fine for a function, not for cross-file consistency. ## The conclusion I didn't want At ~11 kWh/day, plus hardware depreciation, against current API pricing, the math doesn't favor local for interactive work. The single biggest improvement to my daily AI workflow was paying for an API. Local still wins for privacy, high-volume batch jobs, or uncensored experimentation — but not as a general cloud replacement. It's an economics problem, not a capability one. I wrote up the full cost breakdown and the exact llama.cpp router configs on aipster.com . If you're weighing a local rig, I also benc

Hi everyone, I'm a student from Flanders and I like to use Claude Code for my purposes, ideas and also just for fun and also make solutions for problems in our world!) So that's why I built "Pulse", it's an local application that you can easily install to your device and easily follow what your claude agent is doing right now in your terminal session with an ambiance design and easily give permissions for your agent. For those who wants to see directly how much tokens you spent, and how much the