今日已更新 327 条资讯 | 累计 23910 条内容
关于我们

标签：#rust

找到 310 篇相关文章

AI 资讯

The 4-layer voice-agent latency stack, traced with OTel spans

** How I instrument ASR, LLM, TTS, and the client with OpenTelemetry, and which number in each layer I actually look at ** TL;DR. A voice agent is four moving parts stuck together: speech to text, the model that writes the reply, text to speech, and the client that plays the audio back. End to end latency hides which of those four is slow on any given turn, so I stopped tracking it as one number and started tracing each stage as its own OTel span with a shared session id. The number I watch hardest is barge-in: when the user starts talking over the agent, how many milliseconds until the agent actually stops sending audio. In our setup we want that under 200ms, and when p95 barge-in creeps past that, the agent feels like it is talking at you instead of with you. Everything below is how I wire the spans, what attributes go on each one, and the p95 I page on per layer. The thing I keep saying, and the thing that keeps being true: voice agents fail in production not because of raw latency but because nobody simulated the audio and LLM pipeline together. You can have a fast ASR, a fast model, a fast TTS, and a voice agent that still feels broken, because the failure lives in the seams between them and in the parts (barge-in, jitter) that no single-stage benchmark touches. Tracing is how I get the seams to show up. A note before the layers. This is just the setup we run, the spans we emit, and the mistakes that made us add each attribute. Some of it is probably specific to our stack and will not transfer. I will flag that where I can. The shape of a turn, and why one span is not enough One turn is: user says a thing, agent says a thing back. Underneath that is roughly: audio frames come in, ASR turns them into text (streaming partials as it goes); the text plus history goes to the LLM, which streams tokens back; as text comes out, TTS turns it into audio, also streaming; the client receives audio frames and plays them, with some buffering to smooth out jitter. If you wrap

2026-06-09 原文 →
← 上一页 下一页 →