标签：#Observability

TL;DR Many LLM observability setups capture prompts, outputs, tokens, and latency while leaving retrieval failures hidden. A single search call may conceal query rewriting, filtering, fetching, deduplication, reranking, and evidence selection. A useful trace connects the original question to the effective query, returned sources, selected passages, and final claims. Retrieval tracing helps distinguish missing, stale, or ignored evidence from a genuine generation failure. Production teams should measure freshness, duplicate evidence, citation coverage, and cost per grounded answer. A user asks your AI assistant whether a product still supports a particular feature. The assistant responds confidently and links to the company’s documentation. The model request succeeded. Latency was normal. Token usage stayed within budget. No tool call failed. Every indicator on the dashboard is green. The answer is also six months out of date. The model trace cannot tell you whether the system searched for the wrong phrase, preferred an old page, discarded a better result, or ignored the correct evidence. It only shows the context that eventually reached the model. That is the blind spot in model-centred observability. For RAG applications and web-connected agents, the useful unit of observation is not the model call. It is the complete evidence path. A Successful Model Call Can Still Be a Failed Request A typical LLM trace records the prompt, response, model name, token consumption, latency, errors, and perhaps a tool invocation. That is useful for diagnosing slow requests, malformed inputs, and unexpectedly expensive generations. It does not tell you whether the model received the right facts. In a retrieval application, the final prompt is assembled by an upstream system. That system may rewrite the query, choose a search provider, apply time or domain filters, fetch pages, extract text, remove duplicates, rerank candidates, and select passages for the context window. The model ca

We shipped a filter that threw away bad GPS readings. Months later somebody asked whether it was working, and I could not answer. The evidence was gone. That question changed how I build anything that rejects data. The obvious version, and why it rots Mileage tracking depends on trustworthy distance, and GPS lies constantly. So the first version of our cleanup did what everyone's first version does: if (! fix . isPlausible ( previous )) return // drop it, move on accumulateDistance ( fix ) Clean data comes out the other end. It feels responsible. It is also a trap, because that return destroys the only record that could ever tell you whether the rejection was correct. Six months in, someone asked the reasonable question: is the filter right? I could not say how many readings we had dropped, on which journeys, or whether any of them had been a genuine drive through a tunnel rather than a glitch. We had built a thing that made a judgement call thousands of times a day and kept no record of any of it. Persist, then classify The rebuild flipped the default. Rejection stopped being a return and became a label. Only two cases are still deleted, because they cannot physically be real: // impossible coordinates if ( fix . lat ! in - 90.0 .. 90.0 || fix . lng ! in - 180.0 .. 180.0 ) return null // impossible accuracy: too precise to be true, or useless if ( fix . accuracyM <= 0.1f || fix . accuracyM >= 250f ) return null That is the entire delete list. Everything else is persisted and sorted into named accumulators: originalDistanceM += displacement // every metre we ever saw when { fix . isMock -> mockDistanceM += displacement abnormal -> { abnormalDistanceM += displacement if ( isHardSpike ) spikeDistanceM += displacement } accuracyGated -> { /* recorded, deliberately not counted */ } else -> cleanedDistanceM += displacement } Five numbers instead of one. The UI shows cleaned . The rest live beside it. And the row itself keeps its provenance: accuracy, provider, bearing, a

Six coding agents were running. I could not tell you what any of them had done. Not roughly. Not approximately. The output was there, the files had changed, and the honest answer to "which one did that" was a shrug. Three questions in particular had no answer: which run burned the tokens, whether they genuinely ran at the same time or merely started together, and whether two of them had quietly edited the same file. That last one is the expensive question. An agent working on the wrong file looks exactly like an agent working on the right one, right up until you read the diff. The thing that was already true Every one of those runners writes a transcript to disk while it works. Claude Code does. So do Cursor, Codex, Gemini CLI, Copilot CLI and Kiro. The record of what happened was sitting in my home directory the entire time, in six different formats, none of which I had ever looked at. So runlanes does not wrap anything. There is no SDK, no instrumentation step, no account, and nothing to start before the run starts. It reads what the runner already wrote. The consequence is the part I did not expect to matter as much as it does: it works on runs that already finished. Most tools in this space need you to have decided, in advance, that this particular run was worth watching. This one can answer a question you only thought to ask afterwards. npx runlanes That opens a console on 127.0.0.1:4180 for whatever project you are standing in. There is no configuration file to write first. What it actually shows Now is every live session, across every runner it found, with what the main conversation spent against what it handed to subagents. On the session that motivated the whole thing, that split was 8.3 million tokens of conversation against 2.1 million delegated, which was not the ratio I would have guessed. The parallelism figure is the one I keep coming back to. Peak concurrency was four agents. The share of elapsed time where anything genuinely overlapped was 9% . Four

Book: AI That Ships The series: AI in TypeScript — 5 books, from your first LLM call to agents in production — all five here My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub Name the tool call your agent made yesterday that wrote to the database twice. A few hundred went out. You cannot point at that one. You have the provider dashboard, so you know how many tokens went out. You have request logs, so you know a session opened at 09:14 and closed at 09:21. You have Sentry, so you know nothing threw. What you do not have is the ordered list of what the model asked for and what came back. That gap was survivable while agents mostly summarised text. The model OpenAI released on 3 September 2026 is a reasonable argument for why it is less survivable now, and the reason sits in everything OpenAI bolted on around the model before letting anyone touch it rather than in the benchmark table. OpenAI did not just ship a model GPT-6 Astra landed on 3 September 2026, described by OpenAI as the most capable model it has broadly deployed. The agentic scores it reported at launch are 74.1% on DeepSWE v1.1 and 72.6% on the OSWorld 2.0 offline subset. Vendor figures, and they say the thing is meant to be pointed at tools. The part worth your attention is in the system card , and none of it is a property of the weights. OpenAI deployed misalignment monitoring across tool-using inference in external deployment, using the same monitoring structure it runs internally, and added alignment evaluations that block a response rather than write it up afterwards. The card describes a layer of watching and gating built around the model, not only the model itself. OpenAI did that work at its own boundary, which is the inference call. Your boundary is one hop further down, where a string in a response becomes a DELETE or an outbound email. Nothing OpenAI monitors can see that hop, because it never leaves your proce

Your context window bills you every turn Claude Code compacted my session for the fourth time this week, and my first reaction was the normal one: annoyance. It just erased everything and I have to re-explain half of it. Then I pointed Claude Code at its own transcript files and did the arithmetic instead of the complaining. The transcripts are just JSONL on disk — every request, every token count, timestamped. Three sessions, 5,288 requests, a few minutes of parsing. The reframe that came out of it: compaction isn't the tax. It's the tax getting paid off. The tax is every turn before that. TL;DR Every turn re-sends the entire conversation so far. Nothing is "remembered" for free — a 900K-token context gets re-read, in some discounted form, on turn 901. Across three real sessions: 1.99 billion tokens read from cache, 62 million tokens written to it. A 32:1 ratio — each token you put in context gets paid for roughly thirty-two more times before it leaves. Four auto-compactions fired at 968K, 996K, 999K, and 771K tokens. Each one took 108–140 seconds of wall-clock time doing nothing but summarizing. Immediately after, cache-read dropped from ~990K to 0. At list-price API rates, cache discounting saved an estimated 86% versus paying full input price every turn — which is the whole mechanism working as intended, and also the reason a 1M-token context doesn't bankrupt anyone by turn 50. None of this is "Claude Code is expensive." It's "the meter is per-turn, not per-token-ever-seen," and almost nobody reasons about a session that way while they're in it. What actually happens on turn 500 There's no persistent working memory across a conversation. Each API call is stateless — the model sees whatever text is in the request, and nothing else. So a coding session's "memory" is an illusion built entirely out of re-sending: every prior file read, every tool result, every message, concatenated and shipped again, every single turn. Prompt caching is the thing that makes this sur

In meinem ersten größeren Projekt waren wir stolz auf unsere Alarme. Wir hatten Hunderte davon. Jede Metrik, die uns eingefallen ist, bekam einen Schwellwert und eine Benachrichtigung. Wir dachten, das sei gründlich. In Wahrheit hatten wir uns eine Umgebung gebaut, in der niemand mehr hinschaute. Der Effekt kam schleichend. Erst überflog man die Alarme noch. Dann klickte man sie weg. Am Ende gab es einen eigenen Ordner, in den die ganzen Mails wanderten, ungelesen. Als dann ein echter Ausfall kam, ging seine Meldung im Rauschen unter, genau wie alle anderen. Wir hatten Alarme, aber wir hatten keine Alarmierung mehr. Das ist ein Unterschied. Seitdem behandle ich Alarme mit einer fast strengen Sparsamkeit. Meine Leitfrage lautet: Wenn dieser Alarm nachts losgeht, würde ich wollen, dass jemand aufsteht? Wenn die Antwort nein ist, dann ist es kein Alarm. Dann ist es höchstens ein Eintrag in einem Dashboard, das man morgens anschaut. Diese Trennung zwischen „jemand muss jetzt handeln" und „das ist zur Kenntnis" hat unsere Bereitschaft gerettet. Der zweite Grundsatz ist, auf Symptome zu alarmieren, nicht auf Ursachen. Nutzer merken nicht, dass eine CPU bei achtzig Prozent liegt. Sie merken, dass die Seite langsam ist oder Fehler zurückkommt. Also alarmiere ich auf das, was der Nutzer spürt: Latenz, Fehlerrate, gescheiterte Anfragen. Die hohe CPU ist dann eine Information für die Diagnose, kein Grund, jemanden zu wecken. Oft löst sich eine hohe Auslastung von selbst, ohne dass ein Mensch je etwas tun müsste. Am meisten hat mir geholfen, jeden Alarm mit einer ehrlichen Verfallsprüfung zu versehen. Alle paar Wochen gehe ich die Liste durch und frage bei jedem einzelnen: Ist der schon mal losgegangen? Hat daraufhin jemand etwas getan? Wenn ein Alarm seit Monaten nur Lärm macht und nie zu einer Handlung geführt hat, dann löse ich ihn auf. Das fühlt sich mutig an, ist aber das Gegenteil. Ein Alarm, dem niemand traut, schützt niemanden. Gutes Monitoring erkennt man nicht an der

Short answer: For a Node.js SaaS app, start with a hosted metrics path only when the dashboard needs aggregate trends, bounded dimensions, and operational alerts. Use a detailed event store when individual customer actions must remain searchable or auditable. Define the KPI before comparing APIs. Pick this path Pick it when Main limitation Direct hosted metrics API The service is small, dimensions are controlled, and the team wants minimal infrastructure Application code owns credential, retry, buffering, and delivery decisions Collector in front of hosted metrics Several workloads need one controlled telemetry exit The collector becomes production infrastructure with its own deployment and telemetry Scraped application metrics Long-running services expose stable targets Short-lived jobs and some autoscaled runtimes need extra lifecycle planning Detailed business events Per-tenant investigation, audit, or record reconstruction matters Event search is a different job from low-friction aggregate KPI queries The least complex option is the one whose whole data path the team can test and explain. A clean dashboard is not evidence that the underlying business definition, delivery behavior, or missing-data policy is correct. How should a Node.js SaaS app choose a simple hosted metrics dashboard API? Start with the decision the metric must support. “Are completed trials falling?” is useful. “Can we put trials on a chart?” isn't. Product, engineering, and incident owners need one definition for the event, unit, time window, and exclusions. If a trial completion is retried, should it count once or twice? Settle that before sending a sample. Draw the system in words: business action -> typed measurement -> delivery path -> time-series store -> query -> dashboard -> owner . Every arrow can lose either data or meaning. This tiny diagram changes the evaluation from a screenshot contest into an engineering review of ingestion behavior, query portability, dimension limits, retenti

A sensor on my machine returned nothing at all — empty stdout, empty stderr, exit code 2 — on every invocation for 36 days. It was not crashed. It was not misconfigured. It was doing exactly what one line of well-intentioned code told it to do: announce a condition once . The line looked like this, and I suspect you have written it: if [ ! -f " $OFFLINEFILE " ] ; then echo "body context n/a — phone unreachable" > &2 touch " $OFFLINEFILE " fi exit 2 Read it as a rate limiter and it is obviously fine: don't spam the log with the same message every five minutes. Read it as what it actually is and it is a bug, because the guard does not limit a rate. It changes the tense of the sentence. Every number, code listing, and command output below was re-measured on the machine while writing this, not quoted from the commit that fixed it. Two of the things I expected to find turned out to be false; both are in section 6, and one of them is the most interesting part. 1. Present tense, past tense phone unreachable is a claim in the present tense . It is a statement about the world right now, and it is what a reader of this tool wants: is the body sensor readable at this moment? Wrapping it in [ ! -f "$SENTINEL" ] silently rewrites it into the past tense : the phone became unreachable, at some earlier point, at least once. That is a different proposition. It is true exactly once per transition and false forever after, which is why the guard can never fire twice, and why the sentinel's own mtime is the only surviving record of when the sentence was last true. The two propositions coincide on the first run. That is the whole trap. A first-time-only notice is indistinguishable from a live one for the length of one invocation, which is exactly the length of the test you will write for it. 2. What the reader got instead Here is the tool, before the fix, run twice in a row against a phone that is genuinely away. I pulled the pre-fix version straight out of git into a scratch path and ra

Most gateway tutorials stop at "here's how you route a request." That's the easy 20%. The hard part is what happens when a client hammers you with requests, a downstream service falls over mid-traffic, or you're staring at a 500 trying to figure out which of your four services actually caused it. I wanted to build something that hits those problems on purpose, so I put together spring-gateway-sample : a public gateway , an api-server that fans out to two downstream services, and a full observability stack sitting behind all of it. It's not a real product and never will be. But I tried to make it behave like one — including the annoying bits, like config tradeoffs and races that most demos just quietly ignore. Stack, for context: Spring Boot 4.1, Spring Cloud Gateway on WebFlux, Resilience4j, Redis, Postgres, Keycloak, Prometheus/Grafana/Tempo/Loki, and a small Vue 3 app for throwing traffic at it from a browser. The system, in one request Browser (Vue traffic simulator) │ Keycloak PKCE login + API key ▼ Gateway ── JWT + API-key auth, Redis rate limiting ──▶ routes to │ ▼ api-server ── WebClient delegation, circuit breakers, Caffeine cache ──▶ │ │ ▼ ▼ product-service pricing-service (JPA / Postgres) (JPA / Postgres) Every hop re-validates the JWT on its own — defense in depth, so the gateway isn't the single thing standing between the internet and the data. The gateway also checks an API key on top, because a JWT tells you who the user is, not which client application is calling on their behalf. You need that second identity if you want per-client rate limits or the ability to revoke one app's access without touching anyone else's. Two checks, one specific order Every request needs a Keycloak JWT and an API key, and the order they're checked in isn't an accident: Missing or expired JWT → 401 , before the API key is even looked at. Valid JWT, bad API key → 401 , but a different error code. Both valid, wrong role → 403 . Why bother with the ordering? Because "you're no

Free Tokens Are Not an SLO: An Ops Cost Drill for AI Batch Queues This week, two numbers trended: a harness at 100%, a model at 30%. For platform teams, a better pair is queue age and deadline slack. This article is a cost drill for the simplest AI batch path: free tokens, free server, non-negotiable deadline. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. That capacity is real. It is not an SLO. The tokens cost nothing. The queue is patient. Your deadline is not. The missing variable Token cost is easy to measure. Operations cost is easy to ignore. A free endpoint converts a per-token bill into a per-hour bill. The bill becomes your time, your retries, and your queue age. This drill keeps the ledger honest. It answers one question: what does a completed request cost when the token price is zero? Topology # worker.py (minimal, single-threaded) import queue import time import csv work = queue . Queue () for i in range ( 1000 ): work . put ({ " id " : i , " prompt_tokens " : 512 , " max_tokens " : 256 }) def call_model ( payload ): # replace with your free model endpoint return { " ok " : True , " in_tokens " : 512 , " out_tokens " : 180 } completed = 0 retries = 0 started_at = time . time () while not work . empty (): item = work . get () attempt = 0 while attempt < 4 : try : call_model ( item ) completed += 1 break except Exception : retries += 1 attempt += 1 time . sleep ( 2 ** attempt ) The worker is deliberately single-threaded. Free capacity often serializes. Serialization turns a token problem into a time problem. Declared test conditions 1,000 requests. One worker process. One free model endpoint. No client-side rate limiting. Deadline: 30 minutes. Ledger: one CSV row per request. Ledger and report # cost_ledger.py import csv import time HOURLY_OPS_COST = 50.0 # loaded engineering rate, adjust def record ( item , elapsed , retries ): with open ( " ledger.csv " , " a

Short answer: record one structured completion event at the request boundary, emit separate events for every asynchronous notification attempt, and join them with a stable notification ID; middleware latency and status code alone cannot reconstruct a delivery failure. For a gaming notification service, the deciding constraint is time. An API response may say that a guild invite was accepted while the actual push attempt occurs seconds later, perhaps on another process. Treating those two facts as one log event produces a comforting dashboard and a weak incident record. The architecture decision is to preserve both boundaries, give each event a precise meaning, and ship them outside the request's success path. This is deliberately an evidence design, not a logging-library choice. Express and Pino can implement the request-side contract in Node.js, but changing a serializer does not repair a missing correlation key or an ambiguous definition of completion. Decision, invariants, and failure boundaries The request completion event answers a narrow question: what did this process observe at its HTTP boundary? It should carry a timestamp, severity, service and environment, request ID, normalized route, method, response status code, and elapsed duration. If the request creates or addresses a notification, add a notification ID that remains stable across the queue and delivery worker. Do not make raw request or response bodies part of the default schema; tokens, chat text, player identifiers, and device data have different retention and access requirements from operational metadata. The delivery attempt event answers a different question: what happened when a worker tried to deliver that notification? Its useful fields include the same notification ID, an attempt number, channel, destination class rather than raw destination, outcome, and a bounded error category. A retry is another attempt event, not an edit to an old record. That append-only shape matters because the inte

Choice Setup burden Incident evidence Best fit Hosted metrics API Low Good if event context is preserved Small teams with an on-call rotation Postgres plus a custom dashboard Medium Excellent for joining metrics to business records Low-volume systems with strong SQL skills Self-hosted metrics stack High Configurable, but operationally demanding Teams that already run observability infrastructure Short answer: start with a hosted metrics dashboard API, send a small set of custom application metrics from Node.js, and retain reconstruction fields in Postgres. Choose the custom Postgres path when joins are the investigation, or self-hosting when data control outweighs maintenance. That recommendation has a catch. A chart can show when enrollment failures rose, but it cannot explain which course, release, region, or feature state produced them unless those dimensions were recorded at write time. For an edtech SaaS, the real deliverable isn't a pretty dashboard. It is enough evidence to replay the story of a customer incident without guessing. How can Node.js send custom app metrics to a hosted dashboard API? Capture the dimensions an investigator can act on: metric name, timestamp, deployment identifier, region, tenant or school identifier, operation, outcome, and a bounded error class. Keep direct student data out of labels. A useful event might say that lesson_publish failed validation in the EU region on deployment 7f3c2a1 ; it should not contain a learner's name, email, answer, or free-form support message. Small is good. Stop there. Start with service-level signals tied to customer work: request count, failure count, latency distribution, queue depth, and the age of the oldest queued job. Add business-flow counters such as course publication attempts only when they answer a concrete incident question. Don't export every database column as a label. High-cardinality dimensions make charts harder to read, alerts harder to tune, and the ingestion boundary harder to reas

Short answer: give each browser fetch a request ID, carry it to the backend in a standard HTTP header, and emit that same ID in structured logs on both sides. Keep the pricing decision itself behind a flag with an explicit evaluation ID, so a rollback can be verified instead of guessed. The browser is the first audit surface Rolling out a new pricing rule in an edtech app sounds like a feature-flag task. Operationally, it is a tracing problem with money attached. A student sees a price in the browser, the frontend calls the checkout backend, and the backend evaluates a flag before writing an order. When those events cannot be joined, a rollback turns into a debate about which request produced which price. I've been paged for missed jobs and duplicate deliveries. The same failure pattern appears here: a dashboard says the system is healthy, but the individual request that matters is hard to reconstruct. A request ID doesn't prove that a price was correct. It makes the evidence joinable. The smallest useful contract is straightforward: The browser creates a non-secret request ID for each outbound fetch. The ID travels in X-Request-ID (or the equivalent header chosen by the team). The server validates or replaces malformed values, then logs the accepted value. Every log record for the request includes the ID, route, outcome, and duration. A separate flag-evaluation ID identifies the pricing decision and its rule version. Don't put a user email, token, or price in the request ID. It's a correlation key, not an authorization mechanism or a business record. How should frontend and backend logs correlate a browser fetch request ID? The browser and server need a shared boundary, not a shared logging library. For a JavaScript or Node.js application, the fetch wrapper should generate an ID before sending the request and attach it to the headers. The Node.js service should read that header at the HTTP edge, bind it to request context, and include it in every subsequent log eve

A solid observability setup usually comes down to three pieces working together: something that collects metrics, something that exposes system-level metrics, and something that visualizes it all. Here's what each one does and how to install them. The Theory: How This All Fits Together Before installing anything, it helps to understand the model, because it's a bit different from how logging or alerting tools usually work. Pull, not push. Most people's first instinct is "the app should send its metrics somewhere." Prometheus flips that around — it pulls metrics on a timer instead. Every target (a machine, a service, an app) exposes a simple HTTP endpoint, usually /metrics , that just returns plain text numbers. Prometheus visits that endpoint every N seconds (the "scrape interval") and saves whatever it finds, with a timestamp attached. Nothing gets pushed to Prometheus — Prometheus goes and asks. This means for anything to show up in Prometheus, it has to satisfy one requirement: something has to expose a /metrics endpoint Prometheus can reach. That's the whole game. Everything else in this stack exists to satisfy that one requirement or to make the data useful afterward. Why Node Exporter exists. Your operating system doesn't naturally speak Prometheus's language — it doesn't expose CPU/memory/disk stats as a /metrics endpoint by default. Node Exporter's only job is to read stats the OS already tracks (via /proc and /sys on Linux) and republish them in the text format Prometheus expects, on port 9100. It's a translator, not a monitoring tool by itself — it collects nothing, decides nothing, alerts on nothing. It just answers "what does this machine look like right now?" whenever asked. Why Prometheus itself is separate. Prometheus doesn't know anything about CPUs or memory — it has no idea what it's scraping. It just knows: "go hit this list of URLs on a schedule, and remember what comes back." The intelligence is in the config (which targets to scrape, how often)

Short answer: compare a hosted app log search service, self-hosted Loki, and Elastic Cloud by the operational boundary each one creates. Low effort, data control, and search depth are different decision axes; the cheapest choice is the one that produces a trustworthy signal without making a small team operate a second product. That last sentence is the decision rule. A low invoice is not a useful bargain if the first incident reveals missing logs, duplicate alerts, or an index that nobody knows how to restore. The incident lesson: a log is not a health signal I've been paged for two different failures: a scheduled import that stopped producing results, and a job that delivered the same result twice. Both incidents had logs. Neither incident was solved by collecting more text. The invariant is simple: observability has to describe both activity and the absence of expected activity. An app log search system can help investigate an import after an alert fires. It cannot, by itself, prove that an import that should have run did not run. That missing event needs a heartbeat, a durable job record, or a metric with an explicit freshness deadline. For an edtech application importing course data, I would record the import name, run identifier, start and finish timestamps, outcome, item count, and an idempotency key. The alert should fire when the expected completion window passes, not whenever somebody happens to search a log stream. Duplicate deliveries should be visible as a repeated idempotency key, not mistaken for two successful business operations. Keep the signal narrow. The log search layer then answers the next question: what happened around the missed or duplicated run? That division keeps noisy search data from becoming the only source of truth for scheduled work. How should a small business compare self-hosted and hosted app log search? Compare the complete operating boundary, not the storage line item. A self-hosted Loki deployment gives the team direct control

Short answer: for a Node.js app in Docker or Kubernetes, give startup, readiness, and liveness probes separate meanings, keep routine health traffic out of application logging, and measure state transitions instead of counting every successful check. For a property-management API rolling out a new pricing rule, this preserves useful metrics: whether an instance can calculate rent correctly and accept traffic, without turning each kubelet poll into noise. Which health signal should control each container decision? Start with the decision, not the endpoint name. Signal Question it answers Include Exclude Action Startup Has initialization completed? Configuration parsing, pricing-rule compilation, required local warm-up Long-term dependency health Allow the process more time before other probes apply Readiness Can this instance safely receive a new pricing request now? Ability to serve the active rule version and any required dependency state Optional analytics and background exports Remove the pod from Service endpoints Liveness Is the process stuck beyond local recovery? Event-loop progress or another narrow process invariant Database, cache, and third-party availability Restart the container This split is the main noise filter. A downstream dependency becoming unavailable can make a pod unready, but restarting the same healthy process usually doesn't repair that dependency. If the dependency is placed in liveness anyway, every pod can restart together. The health response has then amplified one problem into two: lost capacity plus a restart storm. The pricing rollout makes readiness more demanding than “the port is open.” Imagine rule version rent-2026-08 is enabled for one building cohort. A newly started instance has loaded configuration but hasn't compiled that version yet. It is alive. It isn't ready. Its startup check should hold back liveness and readiness until initialization finishes; afterward, readiness should stay false until the active rule can be evalua

A failure message can be technically correct and still be frustratingly incomplete. Consider a timeout. It tells us something important about the failure mechanism, but not which operation encountered it. Adding the complete request target might answer that question, yet it can also expose identifiers, query parameters, access material, or other data that never belonged in a broadly visible diagnostic record. A safer middle ground is to give failures two separate coordinates: a reason code that explains how the operation failed, and a bounded operation label that explains where it failed. That distinction makes diagnostics more useful without turning failure handling into an accidental data-exposure channel. A reason code is not a location Reason codes describe failure mechanics. Generic examples might include deadline , cancelled , unauthorised , or invalid_response . These codes are valuable because they let systems group similar outcomes. A dashboard can count deadline failures across operations, while application logic can decide whether a particular reason is retryable. What a reason code cannot reliably explain is the operation being attempted. A deadline during a summary read may require a different investigation from a deadline while assembling a detailed response. Combining both meanings into one free-form message makes failures harder to query and encourages presentation text to become an informal data model. Model the two coordinates separately A deliberately generic, invented C# model might look like this: public enum OperationArea { Summary , Detail , Archive } public sealed record FailureDetail ( string ReasonCode , OperationArea ? Area = null ); The reason remains suitable for classification. The operation label adds location without carrying an unrestricted request value. An enum is not the only option. A validated value object or centrally managed set of constants can work too. The important constraint is that labels come from a small, reviewed voca

TL;DR For marketplace sales calls, use an async job when several documents must become one reviewed set of CRM actions; use an inline request only when one short document can finish inside the caller's latency budget. Preserve one result per input, expose partial progress, and export only records that carry their source ID, outcome, and schema version. Start with this decision table: Pick Use it when Quality and latency consequence Operational burden Inline request One short transcript produces one independent summary Fast feedback, but the request deadline limits retries and review stages Low until traffic spikes or callers retry Bounded parallel calls A small set of independent transcripts can finish separately Lower wall time, with variable completion order The caller owns concurrency, backoff, and reconciliation Durable async job Multiple documents feed one CRM export or need validation More queue latency, but enough room for retries and quality checks Requires job state, idempotency, metrics, and retention rules The important boundary is not "batch or no batch." It is ownership. If the API accepts a collection, the service should own that collection through terminal results and a verifiable export. Don't make a client reconstruct truth from whichever promises happened to resolve. What should a Node.js batch summarization API do with multiple documents? It should turn an admission request into a stable job record, process every document under a declared concurrency limit, and publish an item-level outcome before it declares the job complete. The result model needs at least four identities: job, input document, processing attempt, and export. Without them, a duplicate submission can look like new work, a retry can overwrite useful evidence, and an export can silently omit a failed call. For the marketplace example, imagine that a seller has three calls about the same account: discovery, pricing, and legal review. The desired CRM update is not merely three paragra

Grafana Labs has announced general availability for two tools that let AI coding agents query live observability data during development: the gcx CLI and the Grafana MCP server. Both allow agents to pull metrics, logs, traces, SLOs, and Synthetic Monitoring results from Grafana Cloud or a self-hosted stack By Claudio Masolo

The retry trap A free model CI job fails on a timeout. You click retry. The whole pipeline starts over: checkout, build, dependencies, model call. That is the trap. Why re-run the world for one timeout? Retrying the pipeline does not isolate the flaky step. It makes a small problem expensive. I wanted a workflow that replays just the model call, not the whole pipeline. So I made every free model call leave behind a tiny reproducible record. A record has two halves: the input envelope and the output hash. If the job fails, I can replay the input against the same model and compare the output hash. No full pipeline re-run. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I use MonkeyCode's free model access for the model step and its free server option as a small replay store. I do not assume exact quotas, model names, or availability windows here. The pattern works with any free HTTP model endpoint and any tiny key-value store or CI artifact. Why a hash and not the full prompt Full prompt logs are useful until they are not. A free model job may receive a snippet of a merge request, an error message, or an environment variable. Store the raw text in CI logs and you can accidentally leak source or secrets. Store a hash and the replay input in a locked artifact, and the risk drops. A hash also gives me one cheap comparison target. I do not need to reason about the entire response to see that an endpoint changed. I only need byte-level equality. The record shape For every model call, I save the fields below. request_id: a hash derived from model, prompt hash, and a timestamp. prompt_hash: the hash of the normalized prompt. response_hash: the hash of the raw response. status: the HTTP status of the original call. bytes: the length of the response. The exact hash algorithm matters less than using the same one on both sides. I use SHA-256 because it is available everywhere. GitLab CI wiring I run two jobs. The first job calls the model and post