Mana: 2-3 Seconds to Feeling Human

so I shipped a voice AI assistant that runs entirely on my machine. no cloud, no APIs, no latency nightmares. the original idea came from Alice in Sword Art Online — an AI that feels like an actual person, not a chatbot. mixed with JARVIS's anticipation and Neuro-sama's quirky personality. here's what actually went into getting from "wouldn't it be cool" to "this runs 24/7 without issues." the problem with voice AI most voice assistants are cloud-first: you speak → sent to server → processed → response → back to you. each hop adds latency. you're looking at 3-6 seconds before you hear anything. for a voice interaction, that's dead. it kills the feeling of talking to something intelligent. I wanted something faster. something that responds . the constraint: do it locally. use an 8GB VRAM GPU, run everything on-device, no external APIs except for the live2d avatar bits (because that's hard to render locally and still look good). the latency wall here's the reality: I have a GPU with 8GB VRAM. no budget to experiment with better cards or more models. so every architecture decision was forced by what actually fits. naive approach: chain multiple specialized models. User speaks → Transcription model (Whisper) → Planning model (3B: what should I do?) → Coding model (7B: generate implementation) → Verification model (4B: is this correct?) → TTS (speak the answer) math: 1s + 2s + 3s + 1.5s = 7.5s of latency before the user hears anything. nope. the problem isn't just that each model is slow. it's model loading overhead . every time you swap from one model to another, you: unload model A from VRAM load model B into VRAM stall while the GPU rearranges memory with only 8GB, this gets gnarly fast. the decision: one unified model the constraint was hardware. 8GB VRAM. no more, no less. that forced clarity: pick one model that does everything, or pick nothing. so I went with a single model (4B by default, with 7B/8B quality modes available) that does reasoning + code generation +