I Couldn’t Fix My LLM Costs Until I Measured Tokens Per Feature

My LLM bill kept growing, so I did what seemed obvious: I looked for a cheaper model. That helped a little, but it didn't explain why the bill was growing. The dashboard could tell me how many tokens the application used. It couldn't tell me what those tokens were doing. Were they coming from chat? Document summaries? Background classification? An agent retrying the same tool call? I was trying to optimize a total without knowing which product feature created it. The useful unit wasn't tokens per model . It was tokens per feature . Model-level totals hid the real problem A provider dashboard usually groups usage by model, API key, project, or time period. That is useful for billing, but not always for product decisions. Imagine an application with four LLM-powered features: interactive chat document summarization support-ticket classification an agent that prepares weekly reports If the bill increases by 30%, the model name doesn't explain which feature changed. Maybe chat traffic grew. Maybe summarization started sending entire documents instead of selected sections. Maybe the classifier received a much larger system prompt. Maybe the report agent retried after tool failures and generated the same plan several times. Those problems require completely different fixes. Switching every request to a cheaper model would reduce the bill, but it could also hide the engineering mistake. Tag every request with a feature I started giving every LLM call a small amount of application context: const context = { feature : " document_summary " , operation : " initial_summary " , customer_tier : " pro " }; The model provider doesn't need these fields. They belong in the application's usage record. I avoid using individual user IDs as the primary grouping dimension. For cost analysis, a product feature, workflow, or operation is normally more useful and creates fewer privacy problems. A practical record looks like this: { "timestamp" : "2026-07-22T03:12:48.201Z" , "feature" : "docu