标签：#node

It's easy to reach for the popular tool before asking what your system actually needs. For job queueing, the usual advice is to use RabbitMQ or Kafka. The underlying burden of using these tools will be additional processes to deploy, monitor and reason about. But what if using your existing database is possible? I built a job queueing system with a PostgreSQL database that cleared the bar without adding infrastructure. The next question will be, does this solution meet the criteria of a job queue? A job needs a few things to execute properly within a system. It needs to be persistent, surviving unforeseen crashes. Jobs must not be processed more than once by workers; one job should be processed once by one worker. Also, a job's state has to be tracked through every step. A job state must show when it's pending, completed or failed. That's the bar any solution must clear. With the Postgres approach, persistence comes free. Jobs live in a table so when a worker dies mid-job, the job still exists in a row in the db. Whereas with in-memory queues, a crash loses everything still in memory. A broker like RabbitMQ has to be configured for persistence and if configured wrongly, jobs get lost. A database however is fundamentally built for durability. Now, let's say three workers poll the queue at the same instant and run the same query. They'll see the same pending job at the top and nothing stops them all from grabbing it. If that job is a payment, the customer gets charged three times for one service. All the workers successfully process the job with no indication of an error or alerts. This is the requirement that seems to demand a real message broker, and it's exactly where people assume a database can't compete. It can. Postgres has a specific tool for exactly this. The SQL clause FOR UPDATE is used to lock rows. This can be called on a job when a worker picks it up to process. By default other workers will get blocked during this process, they'll wait for the lock to r

# Module Resolution Algorithm (Part 1): How Node.js Finds the Right Module In the previous article, we explored one of the most fascinating parts of Node.js—the hidden Module Wrapper Function. We learned that every CommonJS module is wrapped inside a function before execution, and we also discovered that require() is not a JavaScript feature. It is provided by the Node.js runtime. But a very important mystery still remains. When we write: const fs = require ( " fs " ); or const math = require ( " ./math " ); how does Node.js know where these modules are located? How does it decide whether "fs" is a built-in module or a file inside your project? Why does require("./math") work even if you don't write .js ? And what happens internally before your code starts executing? The answer lies inside one of Node.js's most important systems: The Module Resolution Algorithm Understanding this algorithm is essential because every Node.js application uses it hundreds or even thousands of times while starting. What is Module Resolution? The word resolution simply means: Finding the actual file represented by the string passed to require() . Suppose you write: require ( " ./math " ); To you, "./math" looks like a file. But for Node.js, it is initially nothing more than a string. "./math" Node cannot execute a string. It needs the real file. So its first job is to answer one question: "Which exact file should I load?" The complete process of converting the string inside require() into an actual file on disk is called Module Resolution . Why Does Node Need a Resolution Algorithm? Imagine a project like this: project/ ├── app.js ├── math.js ├── database.js ├── auth.js └── utils/ ├── logger.js └── helper.js Now look at these statements. require ( " ./math " ); require ( " ./database " ); require ( " ./utils/logger " ); require ( " fs " ); require ( " express " ); All of them look similar. But internally they are completely different. Some point to your own files. Some point to Node's bu

When you're building a multi-tenant SaaS, the first architectural question is usually: how do you keep tenant data isolated? The options range from separate databases per tenant (maximum isolation, maximum cost) to a shared database with row-level filtering (minimum cost, more careful coding required). But there's an equally important question that gets less attention: how does your API know which tenant context a request belongs to? This post covers a pattern we've used in production: a custom request header for tenant scoping, combined with JWT authentication. Simple to implement, easy to audit, and flexible enough to support multi-tenant access from a single user account. The Three Common Approaches 1. Subdomain-based ( tenant.yourdomain.com ) The tenant is encoded in the hostname. Each subdomain routes to the same backend, which extracts the tenant from the Host header. Good: Intuitive, visible in the URL. Bad: Requires wildcard TLS certs, more complex DNS setup, awkward in development, doesn't work for mobile API clients the same way. 2. URL path-based ( /api/tenants/{tenantId}/... ) The tenant identifier is part of every route path. Good: RESTful, self-documenting. Bad: Bloats all route definitions, requires every endpoint to include the tenant segment, makes API versioning messier. 3. Header-based ( x-tenant-id: <id> ) A custom header carries the tenant context. Routes stay clean. The tenant scope is resolved in middleware before the handler runs. Good: Routes stay simple, middleware handles scoping uniformly, works well with JWT auth, easy to test. Bad: Less visible (the tenant isn't in the URL), requires clients to always include the header. We use the header approach. The Implementation The API accepts two forms of auth: A JWT token in the Authorization header — identifies who is making the request A tenant ID in the x-tenant-id header — identifies on behalf of which tenant POST /api/v1/members Authorization: Bearer eyJhbGciOiJIUzI1NiIs... x-tenant-id: ten

The problem Most resilience libraries in the TypeScript ecosystem are tied to HTTP clients (axios-retry, p-retry, Polly.js) or require wrapping your function in a class with a .execute() ceremony. What if you just want to wrap any async function — a database query, an internal service call, a file operation — with retry logic, a timeout, and a circuit breaker, without pulling in heavy dependencies? Enter houhou . What is houhou? Houhou is a zero-dependency TypeScript library (~500 LOC) that wraps any async function with composable resilience policies. The wrapped function keeps the exact same signature — you call it like the original. import { task } from ' houhou ' const charge = task ( chargeCard ) . retry ( 3 ) . timeout ( 10 _000 ) . fallback (() => ({ status : ' pending ' })) await charge ( account , amount ) Policies at a glance Retry Re-execute on failure with fixed or exponential backoff: task ( fetchUser ). retry ( 3 ) // shorthand task ( fetchUser ). retry ({ attempts : 5 , backoff : ' exponential ' , jitter : true , delay : 500 }) Timeout Reject if the function doesn't complete in time: task ( fetchUser ). timeout ( 5000 ) Fallback Run an alternative function on failure: task ( fetchUser ). fallback (() => loadFromCache ( id )) Circuit Breaker Prevent repeated calls to an unhealthy service: task ( queryDb ). circuitBreaker ({ failureThreshold : 5 , successThreshold : 2 , resetTimeout : 30 _000 }) Delay Wait before execution: task ( syncData ). delay ( 1000 ) Policy ordering matters Policies are nested : the last method called wraps the previous ones. Execution order is reverse of declaration order. task ( fn ). retry ( 3 ). timeout ( 1000 ) // → timeout wraps retry // → function runs → retry on failure (up to 3 times) → 1s total timeout // → if the timeout fires, there are no more retries task ( fn ). timeout ( 1000 ). retry ( 3 ) // → retry wraps timeout // → function runs → 1s timeout → if timeout fires, retry catches it // → the whole cycle repeats up

You've probably seen this: Kubernetes probes are green, /health returns 200, CPU isn't on fire - and users are timing out. Process is up. Just not doing anything useful. Classic event loop freeze. Something sync and expensive (or a dumb busy loop) grabs the thread, and suddenly timers, HTTP, DB callbacks - all of it waits. Including your health endpoint, because that also needs the loop. monitorEventLoopDelay() is fine for dashboards. It won't yell at you while the process is stuck, and it definitely won't write a log line from outside the frozen loop. That's the part that annoyed me enough to build this. I made @js-ak/watchdog - tiny N-API addon. The monitor runs in C++ on its own thread and writes JSON lines to stderr/file even while the JS event loop is stuck. Your freeze / recovered handlers only run after the loop unblocks. Optional RSS/CPU, optional stack sample. npm install @js-ak/watchdog const watchdog = require("@js-ak/watchdog"); watchdog.on("freeze", (e) => console.log(e.event, e.duration_ms)); watchdog.on("recovered", (e) => console.log("back after", e.duration_ms, "ms")); watchdog.start({ freezeThresholdMs: 1000, logTarget: "stderr", // or "file" | "both" source: "payments-api", }); Real freezes usually aren't while (true) . More like giant JSON.parse, a cursed regex, sync crypto/compress, or some dependency doing sync I/O on a hot path. Stuff you only notice after people complain. Prebuilds for the usual platforms (Node 22+). MIT. Repo: https://github.com/JS-AK/watchdog Curious how other people catch loop stalls in prod - and what you'd actually want in the freeze payload. Roast the API if it's wrong.

A background worker sits quietly for weeks, chewing through a job queue without incident, and then one Tuesday afternoon it disappears mid-batch, taking forty in-flight jobs down with it. No stack trace pointing at the offending line, no alert until a customer notices their export never finished. Nine times out of ten, the culprit is one of the most under-discussed failure modes in server-side JavaScript: unhandled promise rejections. They are easy to introduce, easy to miss in code review, and in modern Node.js they no longer just print a warning — they can end your process outright. This post walks through what unhandled promise rejections actually are at the engine level, why they are far more dangerous in long-running Node.js services than in typical web requests, how Node's default behavior around them has shifted over the years, and the concrete patterns you can use to stop them from quietly killing your jobs. What an Unhandled Promise Rejection Actually Is At the JavaScript engine level, a promise is a state machine with three possible states: pending, fulfilled, or rejected. When an async operation fails — a network call times out, a database query throws, a JSON parse blows up — the promise representing that operation transitions to the rejected state and carries a reason, usually an Error object. That's all a rejection is: a value flowing down the promise's error channel instead of its success channel. The engine only considers a rejection "handled" if, by the time it does its bookkeeping (which happens on a microtask checkpoint, not synchronously), there is a .catch() handler, a second argument to .then() , or a surrounding try/catch around an await somewhere in that promise's chain. If none of those exist anywhere in the chain, the rejection is classified as unhandled. This is the precise, narrow definition of unhandled promise rejections: a promise that reaches the rejected state with zero handlers attached anywhere along its chain of consumers. It's wo

For quite a while now, I've had the feeling that AI agents are surrounded by this mystical aura....

Express does not validate request input for you. Without a check at the edge, handlers get raw req.body , req.query , and req.params - strings where you expected numbers, missing fields, and shapes that only blow up deep in business logic. Zod is a TypeScript-first schema library. You declare the shape once, infer types with z.infer , and parse at the HTTP boundary so route handlers only see valid data. Invalid input becomes HTTP 400 before your code runs. This post covers Zod 4 schemas ( z.email() , z.uuid() , z.coerce ), Express validation middleware, error formatting and pitfalls. Prerequisites Node.js version 26 Zod 4: npm i zod Express: npm i express and npm i -D @types/express Zod 3 method forms like z.string().email() still work but are deprecated in v4. Prefer the top-level APIs below. Schemas // schemas.ts import { z } from ' zod ' ; export const createUserSchema = z . object ({ email : z . email (), name : z . string (). min ( 1 ). max ( 100 ), age : z . number (). int (). min ( 0 ). max ( 150 ). optional () }); export type CreateUserInput = z . infer < typeof createUserSchema > ; export const userIdParamSchema = z . object ({ id : z . uuid () }); export const listUsersQuerySchema = z . object ({ limit : z . coerce . number (). int (). min ( 1 ). max ( 100 ). default ( 10 ), q : z . string (). trim (). min ( 1 ). optional () }); z.coerce.number() is useful for query strings - HTTP query values arrive as strings. Prefer safeParse over parse at the edge so you control the HTTP status and response body. Format errors once Map ZodError.issues into a stable JSON body, or use Zod 4 helpers z.flattenError() / z.treeifyError() when you want field-keyed or nested shapes. // format-zod-error.ts import { ZodError } from ' zod ' ; export function formatZodError ( error : ZodError ) { return { message : ' Validation failed ' , issues : error . issues . map (( issue ) => ({ path : issue . path . join ( ' . ' ) || ' (root) ' , message : issue . message , code : issue . c

Documentation is often the last thing developers think about—until a broken link frustrates users or a README sends someone to a 404 page. I wanted a simple way to automatically verify links in Markdown documentation during CI, so I built urldn-link-check. It's an open-source GitHub Action and CLI that scans your documentation and reports issues before they're merged. Features ✅ Detect broken links (404/500) ↪️ Detect redirect chains 🔒 Detect insecure HTTP links 📏 Find overly long URLs 📄 Scan Markdown & MDX files ⚡ Fast concurrent scanning 💬 GitHub PR summaries 📊 JSON & Markdown reports Installation npm install -D urldn-link-check or npx urldn-link-check . GitHub Action uses: urldn/link-check@v1 That's it. Every push or pull request can automatically verify your documentation. Why I Built It While maintaining documentation, I noticed that broken links often go unnoticed until someone reports them. Instead of checking them manually, I wanted a lightweight tool that integrates directly into GitHub Actions and fits naturally into a CI workflow. Open Source The project is MIT licensed and contributions are welcome. ⭐ GitHub: https://github.com/urldn/link-check 📦 npm: https://www.npmjs.com/package/urldn-link-check https://www.npmjs.com/package/urldn-link-check This is the first developer tool in the URLDN ecosystem, with more open-source projects planned in the future. If you have ideas or feedback, I'd love to hear them.

It started with the dumbest problem in computing: I'm in bed, a movie is playing on my laptop across the room, and pausing it requires physically getting up . Every existing fix annoyed me in some way. Remote desktop apps are overkill and route through someone's cloud. Remote-control apps want accounts, subscriptions, or a native app install. I just wanted my phone to poke my laptop over my own Wi-Fi. So I built LapDeck : one Node.js process on the laptop, a PWA on the phone. Scan a QR code once and your phone becomes an app launcher, touchpad, keyboard, live screen viewer, and media/power remote. MIT licensed, plain JavaScript, no build step. This post is about the four problems that turned out to be more interesting than I expected. The architecture in one line Phone (PWA) ── WebSocket + MJPEG over Wi-Fi ──► Node.js agent (Windows) The agent serves the PWA, exposes a WebSocket for commands, and streams the screen as MJPEG. The protocol is deliberately dumb JSON envelopes — no protobuf, no RPC framework — so a native Android client or a CLI script can speak it in an afternoon. Windows-specific glue (volume, brightness, power, capture) is isolated in src/win/ , so a macOS/Linux port only has to reimplement that layer. Problem 1: mobile keyboards lie to you The obvious way to build a remote keyboard is to listen for keydown and forward key codes. On mobile, this collapses immediately: swipe typing, autocorrect, and IME composition don't emit per-key events. Android will happily tell you every key is keyCode 229 . The fix: stop listening to keys entirely. I keep a hidden input, and on every input event I diff the field's value against the last known state — compute the common prefix, then emit "delete N chars, type this string" to the laptop. Swipe-type a whole word and the laptop receives one clean text insertion. IME composition, autocorrect rewrites, emoji — all just become diffs. The lesson generalizes: on mobile web, treat the text field as the source of truth, n

A customer clicks the Pay button, the server sends the request to a payment provider, and the charge succeeds. The trouble begins when the response never reaches the browser because the connection drops. The frontend sees a timeout and retries the same request, while the backend treats it as a brand-new payment. One customer action has now created two valid charges. This kind of failure is easy to miss because every part of the system appears to behave correctly on its own. The browser retries because it never received a response, the API processes a valid POST request, and the payment provider accepts the instruction it was given. The defect sits between those systems, where no component has enough context to know that the second request is a repeat. The same pattern can create duplicate orders, emails, subscriptions, shipments, support tickets, or background jobs. The common advice is to retry failed requests, but that advice is incomplete. A retry is safe only when the server can distinguish a repeated business operation from a new one. For payment APIs, that usually means introducing an idempotency key, storing the original result, and protecting the write path against race conditions. The rest of this article walks through that flow using Node.js, Express, and PostgreSQL. The bug starts with a normal-looking endpoint Consider a small Express endpoint that creates a payment by calling an external provider. The code receives the request body, sends the amount and customer data to the provider, and returns the resulting payment object. Nothing about this route looks obviously unsafe during a basic code review, and the happy-path test is likely to pass without trouble. The risk appears only when the request succeeds remotely but the response is lost before the client receives it. `app.post("/payments", async (req, res, next) => { try { const payment = await paymentProvider.charge({ customerId: req.body.customerId, amount: req.body.amount, currency: req.body.currenc

Headline: Zod 4 is a rewrite of the TypeScript-first schema validation library, released as the stable major in 2025. Four changes hit my code directly: string formats moved to top-level functions ( z.email() instead of z.string().email() ), the four error options collapsed into one error parameter, error formatting moved to standalone helpers ( z.flattenError , z.treeifyError , z.prettifyError ), and .strict() / .passthrough() became z.strictObject() / z.looseObject() . The deprecated Zod 3 APIs still work with warnings, so I migrated incrementally. Key takeaways Zod 4 is the stable major of the TypeScript-first schema validator, released in 2025; it requires TypeScript 5.5 or newer. String formats are now top-level tree-shakeable functions — z.email() , z.uuid() , z.url() — and z.string().email() is deprecated but still works. A single error parameter replaces Zod 3's message , invalid_type_error , required_error , and errorMap . Error formatting moved to z.flattenError (form fields), z.treeifyError (nested), and z.prettifyError (human-readable string). The zod/mini build exposes the same validators through a functional, tree-shakeable API; z.infer , .parse() , and .safeParse() did not change. I reach for Zod on almost every project to validate untrusted input at the boundary — request bodies, form data, environment variables, API responses. Zod 4 changed enough of the surface that a mechanical upgrade tripped a handful of files, so I mapped exactly what moved. What actually changed in Zod 4? Zod 4 is a ground-up rewrite of the TypeScript-first schema validation library, released as the stable major in 2025. The headline is performance: the Zod team's release notes report large reductions in TypeScript compiler instantiations and faster runtime parsing, which matters most in large codebases where schema types dominate type-check time. Four API changes touched my code directly — string formats moved to top-level functions, the four error options collapsed into one,

My LLM bill kept growing, so I did what seemed obvious: I looked for a cheaper model. That helped a little, but it didn't explain why the bill was growing. The dashboard could tell me how many tokens the application used. It couldn't tell me what those tokens were doing. Were they coming from chat? Document summaries? Background classification? An agent retrying the same tool call? I was trying to optimize a total without knowing which product feature created it. The useful unit wasn't tokens per model . It was tokens per feature . Model-level totals hid the real problem A provider dashboard usually groups usage by model, API key, project, or time period. That is useful for billing, but not always for product decisions. Imagine an application with four LLM-powered features: interactive chat document summarization support-ticket classification an agent that prepares weekly reports If the bill increases by 30%, the model name doesn't explain which feature changed. Maybe chat traffic grew. Maybe summarization started sending entire documents instead of selected sections. Maybe the classifier received a much larger system prompt. Maybe the report agent retried after tool failures and generated the same plan several times. Those problems require completely different fixes. Switching every request to a cheaper model would reduce the bill, but it could also hide the engineering mistake. Tag every request with a feature I started giving every LLM call a small amount of application context: const context = { feature : " document_summary " , operation : " initial_summary " , customer_tier : " pro " }; The model provider doesn't need these fields. They belong in the application's usage record. I avoid using individual user IDs as the primary grouping dimension. For cost analysis, a product feature, workflow, or operation is normally more useful and creates fewer privacy problems. A practical record looks like this: { "timestamp" : "2026-07-22T03:12:48.201Z" , "feature" : "docu

alright let me be real with you for a second i've been building production systems for a few years now, mostly backend with node and nestjs, react and next on top, and the one thing that always drove me absolutely crazy was how performance content online just doesn't respect your time you google "how to optimize react rendering" and you get a medium article with 47 claps that shows you useMemo on a counter app you google "how to optimize node performance" and you get a 2019 blog post that tells you to use async await neither of them has real numbers, neither of them shows you how to actually FIND the problem before you start fixing things, and absolutely none of them connect the frontend and backend story together so i spent the last few months building it myself what i built it's called frontend-backend-performance-mastery and it's structured across 6 levels, each level has a frontend folder (react, next.js, typescript) and a backend folder (node, express, nestjs), and every single folder follows the exact same 3-file structure detect.md — how to find the problem, what to look for, what tools to use, before you even open devtools fix.md — the actual fix, with a before and after, when to apply it, and just as important when NOT to apply it project/ — a fully runnable code example, npm install and npm run dev and you're looking at real numbers on your screen, not a screenshot from someone's laptop from 2021 the 6 levels level 01 — fundamentals: web vitals, profiling, baselines, benchmarking with clinic.js and autocannon level 02 — rendering: ssr vs csr vs ssg, react fiber internals, hydration, response streaming, fast-json-stringify level 03 — caching: react query, swr, service workers, redis, cache-aside pattern, cdn and http headers level 04 — database and api: n+1 queries, cursor pagination that stays fast at 10 million rows, dataloader, query optimization, indexes level 05 — advanced: wasm in next.js, worker threads, piscina, bull queue, grpc, node streams, code

After five years of shipping features at Tittri, the team gave me a piece of feedback that stung a little: I was good, but I only ever built what I was asked to build. And we delivered, end to end, every week. From dashboards to multi-step workflows to third party integrations, all of it to make dense workflows understandable. Somewhere along the way I'd become all about business critical web apps, where the frontend is not just visual, it's how people execute operations reliably. But it just wasn't enough for me. So in January 2026 I stopped waiting to be asked, and pitched an AI integration into a UK-based fintech client's product, one that handles real business loans. When I finally presented the idea to the team and the client, after enough brainstorming and planning and identifying the best way for the brokers to benefit from it, expectations rose. No prior AI integration experience, no matching tutorials, real users, real data, real money. Then I had to actually build it. What is the client and why AI The client is one of the UK's pre-eminent business finance brokers and comparison services that combines advanced technology with a team of finance experts to help small and medium-sized enterprises (SMEs) and their advisors find, compare, and apply for the most appropriate and affordable funding options from across the entire market. The client's B2B product is a commercial finance brokerage SaaS, where brokers manage deals, calls, emails, documents, lenders, AIPs (Agreement In Principle), credit/KYC, corporate structures, properties...etc. The brokers use the SaaS to run the whole lifecycle of a deal, and before AI every single step of it was manual. A typical deal goes like this: An enquiry comes in by call or email, something like "my client needs £400k to buy out a GP partner". The broker gets on a discovery call that can run 30–90 minutes, writes up the notes afterwards, creates the deal, and then starts gathering everything a lender will want to see. Finan

Event-Driven Architecture (EDA) is one of the most popular approaches for building scalable distributed systems. Instead of tightly coupling services through synchronous APIs, services communicate by publishing and consuming events. It sounds perfect. Until your consumers start making API calls for every event they receive. At that point, you've reintroduced the very coupling Event-Driven Architecture was supposed to eliminate. This is where Event-Carried State Transfer (ECST) comes in. The Hidden Problem in Event-Driven Architecture Imagine an e-commerce platform. A customer places an order. The Order Service publishes an event: { "event" : "OrderCreated" , "orderId" : "ORD-10234" } Several services subscribe to this event: Inventory Service Notification Service Analytics Service Shipping Service Billing Service Everything looks asynchronous. But here's what actually happens. Order Created Event │ ▼ Inventory Service │ GET /orders/ORD-10234 │ Order Service Notification Service does the same. Analytics Service does the same. Shipping Service does the same. Suddenly, one event generates dozens of synchronous API calls. Congratulations, You've Recreated a Monolith Your architecture now looks like this: Order Service ▲ ┌───────────┼───────────┐ │ │ │ Inventory Shipping Notification │ │ │ └───────────┼───────────┘ API Requests Although events are being used, every consumer still depends on the Order Service. If the Order Service is unavailable: Notifications fail Inventory updates fail Analytics stop processing Shipping cannot continue The event broker isn't the bottleneck anymore. The originating service is. Event-Carried State Transfer Solves This Instead of publishing only an identifier, publish the data consumers actually need. Instead of this: { "event" : "OrderCreated" , "orderId" : "ORD-10234" } Publish: { "event" : "OrderCreated" , "orderId" : "ORD-10234" , "customerId" : "USR-1001" , "customerName" : "John Doe" , "totalAmount" : 249.99 , "currency" : "USD" , "i

Error handling has a quiet problem. You catch an error deep in a call stack, wrap it in something...

I've been working in software for more than 15 years. This is the first piece I've decided to write about the job. I picked this topic because it was a small, real problem that cost me time more than once, until I finally stopped and fixed it properly. Quick disclaimer: this isn't distributed tracing or multi-service observability. It's a targeted fix for one specific problem: correlating logs within a single Step Function execution. If your scenario is different, the problem takes a different shape. What matters here is the reasoning, not the recipe. Quick context for anyone who doesn't work with serverless: AWS Step Functions is an orchestration service. You design a workflow as a state machine (JSON), and each state usually triggers an AWS Lambda (a function that runs on demand, no server to manage). In my case, a request kicks off a state machine execution that passes through several Lambdas in sequence: some do validation, others call external services, and one waits on a callback response before moving on. The fix turned out to be simpler than it sounds: a field Step Functions already offered for free, and I just wasn't using it. A request was processed 120 days ago. The customer calls: "Did you receive my request? I never heard back." You open the AWS Console. The Step Functions execution is already gone. Execution history only sticks around for 90 days, and even within that window, finding one specific execution among thousands is already a hassle. That leaves CloudWatch, with logs from every Lambda in the pipeline. You have one piece of data: the case ID. But every Lambda logged its own way: no standard, no correlation between them. To reconstruct what happened, you have to open each function's logs in the order the workflow probably ran, and piece it together by hand. It wasn't the error itself that hurt. It was the time lost just organizing the logs before you could actually start investigating. The Chaos The pipeline wasn't huge, but it wasn't trivial ei

My upload CLI looked restart-safe until I tested two failures together: the process crashed at 63%, and the signed resume URL expired before restart. The checkpoint needs identity, not credentials: { "file" : "release.tar.gz" , "fingerprint" : "sha256:..." , "uploadId" : "up_123" , "localOffset" : 66060288 , "updatedAt" : "2026-07-19T12:00:00Z" } Do not persist the signed URL. On restart, exchange the durable upload ID for fresh authorization, query the server offset, then reconcile: const remote = await headUpload ( freshUrl ); if ( remote > file . size ) throw new Error ( " invalid remote offset " ); if ( remote !== checkpoint . localOffset ) { await saveCheckpoint ({ ... checkpoint , localOffset : remote }); } await sendFrom ( file , remote , freshUrl ); The server offset wins because a crash can occur after bytes are accepted but before the local checkpoint is renamed. Save checkpoints through write-to-temp plus atomic rename so a partial JSON file cannot destroy recovery. My fixture matrix includes: Failure Expected behavior crash after remote commit rewind/advance to remote offset expired URL refresh without creating a second upload changed local file stop on fingerprint mismatch missing remote upload ask before starting over checkpoint write interrupted retain previous valid checkpoint The tus resumable upload protocol specifies offset discovery and conflict handling that are useful even if the service uses a smaller custom protocol. The key idea is explicit reconciliation, not assuming client memory is authoritative. I also shipped upload status , upload forget , and an exportable checkpoint directory. Recovery is a user-facing feature; if users cannot inspect or remove state, “resumable” becomes hidden lock-in.

How machine learning supports aircraft maintenance using operational data. Key Takeaways Predictive maintenance estimates component health before failure. Data quality determines model performance. Explainable models support maintenance decisions. Human review remains part of every maintenance action. Model performance requires continuous validation. Introduction Aircraft produce large volumes of operational data. Machine learning converts this data into maintenance support inspection planning and fault detection. What Is Predictive Maintenance? Predictive maintenance estimates the condition of aircraft components using historical and real-time data. The goal is to identify early signs of degradation before a failure affects operations. Traditional maintenance often follows fixed inspection intervals. Data-driven maintenance adds condition-based recommendations using operational evidence. Data Sources Model quality depends on reliable data. Common sources include: Engine sensor readings Flight data recorder information Maintenance records Aircraft utilization history Environmental conditions Component replacement history Incomplete or inaccurate data reduces prediction accuracy. Machine Learning Workflow A typical workflow includes: Collect operational and maintenance data. Remove errors and missing values. Create features from sensor measurements. Train the prediction model. Validate performance using unseen data. Monitor prediction accuracy after deployment. Retrain the model as new data becomes available. Model Selection Different problems require different algorithms. Common choices include: Random Forest XGBoost LightGBM Support Vector Machine Long Short-Term Memory (LSTM) Transformer-based time-series models Model selection depends on the prediction task, dataset size, and operational requirements. Engineering Challenges Data Quality Sensor failures, missing records, and inconsistent maintenance logs reduce model reliability. Class Imbalance Aircraft failures