Batch LLM Jobs vs Realtime APIs — Bulk Summarization Cost Attribution

Short answer: move marketplace review summarization, tagging, and extraction to batch LLM jobs when no customer is waiting, but keep realtime calls for interactive work and attribute every job to a tenant before it enters the queue. This is a deadline decision before it is a vendor decision. A nightly policy scan can wait; a seller asking why a listing was rejected cannot. Batch processing removes peak-time synchronous handling from the first case and gives the team a status-and-results workflow for backfills. It does not make latency disappear. The other constraint is accounting. A marketplace that pools every review into one opaque job may lower operational friction while making chargeback, abuse investigation, and budget alerts much harder. The useful unit is therefore a tenant-scoped batch with an internal ledger entry, not merely a large file of prompts. Treat every finding as governed evidence Create the ledger record before dispatch. It should connect an immutable internal job ID to the tenant, workload kind, input count, model choice, submission time, deadline, and estimated token total. Keep the provider job ID as a later mapping rather than using it as your primary key. That leaves audit history and cost attribution intact if the team changes providers. For review-code analysis, require structured findings such as severity, file, line, rule, and explanation. Summarization can tolerate some prose variation; compliance tagging and extraction usually cannot. Validate the result schema before marking a job complete, and quarantine individual invalid items instead of silently accepting a partially malformed export. The same instinct that keeps an OTP system from treating "accepted" as "delivered" applies here: provider acceptance, job completion, export retrieval, schema validation, and downstream application are separate states. Keep it boring. Really. A practical ledger can have one parent row per tenant batch and one child row per review. The parent holds fo