AI 资讯
Agent Accounts Quickstart in Node.js
Provisioning a working email mailbox from Node.js takes less code than the average OAuth callback handler. No consent screen, no token refresh job, no provider SDK — one fetch call returns a grant ID, and from there the mailbox sends, receives, and RSVPs to calendar invites. That's the pitch for Nylas Agent Accounts , hosted email-and-calendar identities you control entirely through the API. They're in beta, and the official quickstart promises a working account in under 5 minutes. The docs show it in curl; here's the same flow in JavaScript. What you need Two things: an API key, and a registered domain for the mailbox to live on. For testing, the zero-DNS path is a *.nylas.email trial subdomain registered from the Dashboard — addresses like test@your-application.nylas.email work immediately. For production you'd register your own domain (the Dashboard generates the MX and TXT records to publish, and verification is automatic once they propagate), but the trial domain is fine for this walkthrough. export NYLAS_API_KEY = "nyk_..." Create the mailbox The endpoint is POST /v3/connect/custom — the same Bring Your Own Auth route used for other providers — with "provider": "nylas" . Unlike OAuth providers, there's no refresh token in the body; just the address: const BASE = " https://api.us.nylas.com " ; const headers = { Authorization : `Bearer ${ process . env . NYLAS_API_KEY } ` , " Content-Type " : " application/json " , }; const res = await fetch ( ` ${ BASE } /v3/connect/custom` , { method : " POST " , headers , body : JSON . stringify ({ provider : " nylas " , settings : { email : " test@your-application.nylas.email " }, }), }); const { data } = await res . json (); const grantId = data . id ; // save this — every later call needs it That grantId is the whole handle. The mailbox behind it is live as soon as the response comes back, and it works with every existing endpoint — messages, drafts, folders, calendars, events, webhooks. One optional field deserves a menti
AI 资讯
Translating 'I missed you' so it doesn't land like a form letter
I was trying to tell someone something real in her first language — not "I missed you" from a dropdown, but the version that sounds like a person said it. Google Translate gave me one answer. No indication whether it was what you'd text at midnight or what you'd write in a letter to someone's grandmother. That's the failure mode of literal translators: one output, no register, no sense of what you're actually choosing between. konid returns 3 options per query, ordered casual to formal, with the register explained and a cultural note on the difference between them. For Mandarin or Japanese, audio plays through your speakers via node-edge-tts — no API key, no browser tab — because reading a pinyin romanization and actually hearing the tone contour are two different things. The vowel length in Korean, the pitch drop in Japanese, the stress pattern in Arabic: you don't internalize those from text. You internalize them from hearing them repeated back while you're still in the context of trying to say something. The setup for Claude Code is one line: claude mcp add konid-ai -- npx -y konid-ai It runs as an MCP server, so it works in Cursor, VS Code Copilot, Windsurf, Zed, JetBrains, and Claude Cowork. Also installs as a ChatGPT app via Developer mode using the endpoint https://konid.fly.dev/mcp . Supports 13+ languages: Mandarin, Japanese, Korean, Spanish, French, German, Portuguese, Italian, Russian, Arabic, Hindi, and more. The name is Farsi — konid (کنید) means "do." MIT licensed. https://github.com/robertnowell/konid-language-learning
AI 资讯
Async APIs: The 202 Accepted + Polling Pattern for Long-Running Operations
Some API requests can't finish in time for a single HTTP response. Generating a report, transcoding a video, running a batch import — these take seconds or minutes, far longer than any client should hold a connection open for. If you try to do this work inside a normal request, you'll hit gateway timeouts, frustrated clients retrying half-finished jobs, and load balancers killing connections at 30 or 60 seconds. The fix is a well-established HTTP pattern: accept the work, hand back a receipt, and let the client poll for the result. Here's how to build it properly. The shape of the pattern The client POST s the job. The server validates it, enqueues it, and immediately returns 202 Accepted with a URL where the status lives. The client polls that status URL until the job is done (or failed ). When complete, the status response points to the finished resource. The key detail most implementations get wrong: 202 does not mean "success." It means "I accepted this and will work on it." The actual outcome arrives later. Step 1: Accept the job import express from " express " ; import { randomUUID } from " crypto " ; const app = express (); app . use ( express . json ()); const jobs = new Map (); // use Redis or a DB in production app . post ( " /v1/reports " , ( req , res ) => { const id = randomUUID (); jobs . set ( id , { status : " pending " , createdAt : Date . now (), result : null }); // Kick off work without blocking the response processReport ( id , req . body ). catch (( err ) => { jobs . set ( id , { status : " failed " , error : err . message }); }); res . status ( 202 ) . location ( `/v1/reports/ ${ id } ` ) . json ({ id , status : " pending " }); }); Notice the Location header. It tells the client exactly where to look — no need to construct the URL itself. Step 2: Expose a status endpoint app . get ( " /v1/reports/:id " , ( req , res ) => { const job = jobs . get ( req . params . id ); if ( ! job ) return res . status ( 404 ). json ({ error : " unknown job " })
开发者
I Spent 30 Days Building a Complete Node.js Learning Path (Free for Everyone)
What This Repository Is A complete, structured, beginner-friendly Node.js learning path. 30 sessions. Each session has Clear learning objectives Step-by-step explanations Working code examples Practice exercises Interview questions Summary of key points No fluff. No assumptions. Just code. The Complete Curriculum Phase 1 - Node.js Fundamentals (Sessions 1-5) Session Topic What You Will Build 01 Introduction to Node.js Your first Node.js program 02 Project Setup and npm package.json, node_modules 03 How Node.js Works Event loop, blocking vs non-blocking 04 Modules and Imports Your first custom module 05 File System Module Read, write, update, delete files Sample code from Session 05 const fs = require ( " fs " ); // Create a file fs . writeFileSync ( " student.txt " , " Welcome To Node.js " ); // Read the file const data = fs . readFileSync ( " student.txt " , " utf8 " ); console . log ( data ); // Welcome To Node.js // Append to file fs . appendFileSync ( " student.txt " , " \n New line added " ); // Delete file fs . unlinkSync ( " student.txt " ); Phase 2 - Core Modules (Sessions 6-10) Session Topic What You Will Build 06 Path Module Cross-platform file paths 07 OS Module System information 08 Events and EventEmitter Custom event handling 09 HTTP Module Create a server 10 Multi-Route Server Multiple routes, JSON responses Sample code from Session 10 const http = require ( " http " ); const server = http . createServer (( req , res ) => { if ( req . url === " / " ) { res . end ( " Home Page " ); } else if ( req . url === " /about " ) { res . end ( " About Page " ); } else if ( req . url === " /products " ) { res . setHeader ( " Content-Type " , " application/json " ); res . end ( JSON . stringify ([{ id : 1 , name : " Laptop " }])); } else { res . statusCode = 404 ; res . end ( " Page Not Found " ); } }); server . listen ( 3000 ); Phase 3 - Building REST APIs (Sessions 11-15) Session Topic What You Will Build 11 CRUD with Dummy Data Complete REST API using array 12
AI 资讯
Reading a Paginated API Without Holding the Whole Thing in Memory
Your API hands out 50 records at a time across 400 pages. You need all of them. You do not need them all at once. Here's a very familiar situation that shows up constantly on the backend. Some API returns data in pages, 50 or 100 records at a time, and you need to walk every page: sync them to your database, export them to a file, run a report. The endpoint gives you a cursor or a page number and you keep asking until there's nothing left. The way most of us write it the first time looks like this: async function getAllRecords () { const all = []; let cursor = 0 ; while ( cursor !== null ) { const { records , nextCursor } = await fetchPage ( cursor ); all . push (... records ); cursor = nextCursor ; } return all ; } const everything = await getAllRecords (); for ( const record of everything ) { process ( record ); } It works. At four hundred records it's fine. The trouble starts when the dataset grows, and it has three separate problems hiding in it. It holds the entire dataset in memory before you touch a single record. It's all or nothing: if page 380 fails, you've thrown away the 19,000 records you already fetched . And it's eager. You can't start processing record one until the very last page has landed , even if all you wanted was the first ten. There's a shape in JavaScript built for exactly this, and if you read the first two posts in this series you already have both halves of it. Two ideas you've already seen In the CSV post , we pulled rows out of a huge file one at a time with a generator, so the file never fully loaded into memory. Lazy. Pull-based. You ask for the next row, you get the next row, nothing more. In the async/await post , we saw that a generator can pause at a yield and resume later.A generator can hold its place across an asynchronous gap. Put those together. A generator that pulls data lazily, and can pause to await something between pulls. That's an async generator, and it's the natural tool for walking a paginated API. You pull records
开发者
Fable 5 Pwned: Inside the First Mythos-Class Leak
The post hit X at some point on June 10, the morning after Anthropic's biggest launch in years. I...
AI 资讯
Validate and Mask Environment Variables in Node.js and TypeScript
Configuring environment variables seems simple, but it frequently leads to two common issues in production-grade applications: Missing Variables: A developer adds a new variable locally but forgets to update the .env.example file. Other team members pull the changes, and their local environments crash. Leaked Credentials: A debug log like console.log(process.env) prints database connection strings or API tokens to the terminal, leaving them visible in plaintext logs. To solve this, we created @novaedgedigitallabs/envkit . It is a zero-dependency (other than Zod) utility that validates, loads, and masks environment variables, and keeps your example configurations updated automatically. Key Features Schema Validation: Define your environment variables with Zod to enforce types and formats. Log Masking: Automatically redact sensitive credentials in console output. Example Syncing: Automatically append missing variables to your .env.example file without overwriting comments or values. Module Support: Runs in ESM and CommonJS. Getting Started Install the package and Zod: npm install @novaedgedigitallabs/envkit zod Initialize your configuration in a file like env.ts : import { createEnv } from ' @novaedgedigitallabs/envkit ' ; import { z } from ' zod ' ; export const env = createEnv ({ schema : { DATABASE_URL : z . string (). url (), JWT_SECRET : z . string (). min ( 32 ), PORT : z . coerce . number (). default ( 3000 ), NODE_ENV : z . enum ([ ' development ' , ' production ' , ' test ' ]). default ( ' development ' ), }, secrets : [ ' JWT_SECRET ' , ' DATABASE_URL ' ], generateExample : true , // Appends new keys to .env.example at runtime }); Because the variables are validated using Zod, your editor will provide full TypeScript autocomplete and type safety: env . PORT ; // Resolved as a number env . JWT_SECRET ; // Resolved as a string Preventing Secret Leaks in Logs Logging environment configs is a common debugging habit. With envkit, any variables listed in the secre
AI 资讯
3 Gotchas I Hit Deploying the Claude Agent SDK to Railway
I deployed a Slack bot app built on the Claude Agent SDK to Railway, and immediately hit a string of landmines around the SDK itself. Every one of them was the "the logs don't tell you what's wrong" kind, and the second one in particular ate a lot of my time. Since other people are likely to get stuck in the same spots, I'm writing it down. This is aimed at junior-to-mid-level devs using @anthropic-ai/claude-agent-sdk ( query() ) in Node.js. TL;DR Gotcha 1 : In a root container, bypassPermissions isn't allowed, and the child process dies with code 1 . Worse, stderr is swallowed, so you can't see why. Gotcha 2 : stdio MCP servers don't wait for connection by default, so on turn 1 the tool list is empty — and the model "acts out" tool calls and fabricates the results. Gotcha 3 : haiku shows up in your API logs, but that's not the model degrading — it's by design. It's used for internal chores. Gotcha 1: bypassPermissions doesn't work in a root container What happened Code that ran fine locally started dying with code 1 the moment I deployed it to Railway — the agent did nothing and just exited. The entire error message was essentially this: Error: Claude Code process exited with code 1 That tells you nothing. The only stack trace was from my app; what the child process (the claude binary) actually said before dying was a complete black box. The cause query() spawns a claude binary internally. That binary refuses --dangerously-skip-permissions (which the SDK calls permissionMode: "bypassPermissions" ) when running as root or under sudo . It's a safety measure — skipping all permission checks as root is far too dangerous. Railway, like many container environments, runs as root by default, so if you've set bypassPermissions you will always hit this. You can't catch it locally if you're running as a normal user. Why there are no logs This is the nasty part. Unless you pass an options.stderr callback, the SDK discards the child process's stderr with "ignore" . In other wor
AI 资讯
Nestjs — Stop burning AI credits to write Swagger docs, let the CLI do it!
Last Sunday I shared nestjs-docfy, a small library to move Swagger decorators out of NestJS controllers into companion *.controller.docs.ts files. The reception was better than I expected, and a lot of the feedback pointed in the same direction: the separation is nice, but writing those docs files by hand is still tedious. So I spent some time on that, and there's quite a bit new in this release. A CLI that writes the boilerplate for you The biggest addition is a generate command that reads your project with static analysis (no code execution, no ts-node overhead) and produces a pre-filled docs file for every controller: npx nestjs-docfy generate The generated file comes with inferred summaries, response types, and common error responses already in place. You edit from there instead of starting from scratch. It's idempotent by default, running it again won't touch files that already exist. When you add a new endpoint and want to merge only the new method block without losing your existing edits: npx nestjs-docfy generate --force The CLI auto-detects your project layout, so monorepos (Nx, Nest CLI, generic packages/ or apps/ structures) work without any configuration. There's also a --dry-run flag if you want to preview output before writing anything to disk. A check command for CI The other side of the workflow is keeping docs in sync as the codebase evolves. The check command exits with code 1 if any controller has undocumented methods or no companion file at all: npx nestjs-docfy check Output looks like this when something is out of sync: ✖ UsersController, undocumented methods: updateProfile, deleteAccount → run nestjs-docfy generate --force to merge new methods ✖ 2 controller(s) out of sync. Drop it into your pipeline and docs drift gets caught before it reaches main. Type-safe method keys The docs() function now enforces that every key in config.methods actually exists on the controller class. Typos are a compile error, not a silent runtime warning: docs ( User
AI 资讯
Diagnose Node.js CommonJS vs ESM Errors with Claude: A Copy-Paste Prompt Kit (ERR_REQUIRE_ESM, ERR_MODULE_NOT_FOUND)
By the end of this article you'll have a small Node.js script that pipes a module-resolution error ( ERR_REQUIRE_ESM , ERR_MODULE_NOT_FOUND , Cannot use import statement outside a module ) plus the surrounding config into Claude and gets back a specific fix — not a Stack Overflow lecture. You'll also have four hardened prompts you can paste straight into claude.ai, and a script that auto-detects whether your project is CJS or ESM before you even ask. Everything below runs on Node 18+. Why "just use ESM" doesn't fix the CommonJS/ESM ERR_REQUIRE_ESM error The reason these errors waste so much time is that the failing line is almost never where the problem lives. You see this: Error [ERR_REQUIRE_ESM]: require() of ES Module /app/node_modules/node-fetch/src/index.js from /app/server.js not supported. and your instinct is to edit server.js . But the actual decision is made by four things you can't see from the traceback: the "type" field in your package.json , the "type" (or "exports" map) in the dependency's package.json , your file extension ( .js vs .mjs vs .cjs ), and — if you use TypeScript — the module and moduleResolution fields in tsconfig.json . node-fetch v3 went ESM-only; that's why require('node-fetch') blows up while v2 was fine. The traceback tells you none of that. This is exactly the shape of problem an LLM is good at: lots of small context scattered across files, one correct answer, and a human who keeps pattern-matching on the wrong line. The trick is to feed Claude the config alongside the error, not the error alone. A prompt that only gets the stack trace will confidently tell you to "convert your project to ESM," which is often the most destructive possible fix. Prompt 1 for Claude: force a root-cause classification before any code The failure mode of asking an AI to "fix my module error" is that it jumps to a rewrite. The fix is to make it classify first. Paste this into claude.ai, filling the three blocks: You are debugging a Node.js module resolut
AI 资讯
Architecting a Production-Ready Express + TypeScript Backend: Type Augmentation, Global Errors, and Middleware Factories
When building a personal finance tracker, data integrity and system reliability are non-negotiable. One missing try/catch block can crash your whole server, and weak types can let invalid financial payloads corrupt your database. While building the backend for my personal finance tracker, I decided to move past generic tutorials and build a bulletproof, production-grade API core using Express, TypeScript, and Zod. In this post, I’ll show you how I implemented a type-safe middleware ecosystem, leveraged TypeScript declaration merging to extend the native Request object, and eliminated repetitive try/catch boilerplate across the entire codebase. 1. The Weapon Against Boilerplate: The asyncHandler HOC Writing try/catch blocks in every single controller handler clutters code and introduces human error—it’s easy to forget to pass an error to next() . To solve this, I engineered a Higher-Order Function (HOC) factory that wraps asynchronous request handlers and automatically catches rejected promises, safely routing them into the global error handler. import { Request , Response , NextFunction , RequestHandler } from ' express ' ; export const asyncHandler = ( fn : RequestHandler ): RequestHandler => { return ( req : Request , res : Response , next : NextFunction ) => { Promise . resolve ( fn ( req , res , next )). catch ( next ); }; }; Why this matters: Reliability: Async errors always reach the centralized error middleware. Readability: Route controllers stay beautifully clean, focusing only on business logic rather than async control flow wiring. 2. TypeScript Magic: Declaration Merging & Type Augmentation When dealing with authentication tokens, request tracing ( requestId ), or custom validated payloads, developers frequently resort to casting the request as any (e.g., (req as any).userId ). This completely destroys Type Safety. Instead of fighting the compiler, I leveraged TypeScript Declaration Merging to reopen Express's internal Request interface and merge my cust
AI 资讯
I built a JS image compressor that actually handles iPhone HEIC photos
If you've ever built a file upload feature, you've probably hit this: a user uploads a photo from their iPhone, and your app breaks. No preview, no compression, just a silent failure — because the file is .heic and browsers can't read it. HEIC is Apple's default photo format since iOS 11. Every iPhone photo taken today is HEIC. And virtually every JS image compression library just ignores the problem. I spent a weekend building PixSqueeze to fix that. The problem with HEIC in the browser Browsers use the <canvas> API to compress images. You draw the image onto a canvas, call canvas.toBlob() , and get a compressed file back. Clean, client-side, zero server cost. The problem: canvas.drawImage() only works if the browser can decode the image first. And no major browser can decode HEIC natively — not Chrome, not Firefox, not even Safari on macOS (Safari on iOS can, but that doesn't help your web app). So when a user picks a HEIC file, your image element fires onerror , your canvas stays blank, and your compression pipeline silently does nothing. The solution: server-side conversion, then client-side compression PixSqueeze handles this in two stages: Stage 1 — Server converts the format HEIC (and TIFF, camera RAW) files get sent to a small Express server. The server uses heic-convert running in a dedicated worker thread to convert to JPEG. Worker threads matter here — HEIC decoding is CPU-intensive WASM work, and running it on the main event loop would block every other request. Stage 2 — Client compresses the result The converted JPEG comes back to the browser, where the normal canvas-based compression pipeline takes over. Quality, resize, watermark hooks — all the usual options. The detection is important too. You can't just check file.type — iOS often sets HEIC files with an empty MIME type. PixSqueeze checks the ISO Base Media File Format magic bytes directly: async function isHeicFile ( file ) { const buffer = await file . slice ( 0 , 12 ). arrayBuffer (); const byt
AI 资讯
I Got Tired of Repeating Validation Logic in Every Node.js Project — So I Built Zero Validation
How I Published My Own Validation Package on npm As developers, we've all done this: if ( ! email ) { throw new Error ( " Email is required " ); } if ( typeof email !== " string " ) { throw new Error ( " Email must be a string " ); } if ( ! email . includes ( " @ " )) { throw new Error ( " Invalid email " ); } Now imagine doing this for: User Registration Login APIs Product Creation Payment Requests Admin Panels Microservices The validation code starts growing faster than the actual business logic. The Problem In many Node.js projects, validation ends up being: Repetitive Hard to maintain Inconsistent across APIs Difficult to scale Every endpoint contains similar checks: if ( ! name ) ... if ( ! email ) ... if ( ! password ) ... if ( password . length < 8 ) ... As projects grow, these validations become scattered throughout the codebase. Existing Solutions There are already some excellent validation libraries available: Zod Joi Yup Express Validator I've used many of them and they're great. But for some smaller projects and APIs, I wanted something: Lightweight Easy to understand Minimal setup Zero configuration TypeScript friendly That's what led me to build Zero Validation . Introducing Zero Validation Zero Validation is a lightweight schema validation package for Node.js and TypeScript applications. The goal is simple: Define your validation schema once and validate data consistently everywhere. Installation npm install zero-validation Basic Example import { z } from " zero-validation " ; const userSchema = z . object ({ name : z . string (), email : z . email (), age : z . number (), }); const result = userSchema . parse ({ name : " John " , email : " john@example.com " , age : 25 , }); console . log ( result ); Handling Validation Errors const result = userSchema . safeParse ( data ); if ( ! result . success ) { console . log ( result . errors ); } Instead of crashing your application, you can safely inspect validation errors and return meaningful API responses
AI 资讯
Building a production TypeScript CLI in 2026: oclif vs commander vs custom.
Building a production TypeScript CLI in 2026: oclif vs commander vs custom. I shipped my first Node CLI in 2019 with a 12-line arg slicer and process.argv . It worked until it needed a second command and then collapsed into spaghetti. The other extreme is grabbing a full framework for a tool that runs one command. In 2026 there are three reasonable paths between those extremes, and each one wins on a specific slice of the problem. This post covers @oclif/core v4, commander v14, and a zero-dependency parser that fits in 30 lines. Same "greet" command in all three. Same distribution steps at the end. Honest tradeoffs throughout. TL;DR oclif v4 commander v14 zero-dep npm install size ~8 MB ~220 kB 0 B Type inference on flags Full, generated Good, manual Manual Plugin ecosystem Yes (Heroku, Salesforce) No No Learning curve High (day 1) Low (hour 1) None Best for Multi-team, multi-command CLIs Most real-world tools One-shot scripts 1. The decision: framework vs no framework Reach for a framework when the tool needs subcommands, a plugin system, or auto-generated help text. The second engineer who touches the CLI should be able to find where things live without reading your code twice. Build your own when the tool does one thing, ships as a one-file script, or lives inside a monorepo where pulling in 8 MB of transitive deps is not welcome. A zero-dep parser also removes the surface area for supply-chain incidents, a real concern on tools that run in CI. Commander sits in the middle: a 220 kB install that covers most real tools without the scaffolding overhead of oclif. 2. Project skeleton Every path shares the same bin setup. Start with a package.json that declares the executable: { "name" : "greet-cli" , "version" : "1.0.0" , "bin" : { "greet" : "./dist/cli.js" }, "scripts" : { "build" : "tsc" , "dev" : "tsx src/cli.ts" }, "type" : "module" } The tsconfig.json for a CLI targets the Node release line you plan to support. Node 24 LTS handles ESM natively, so use "module":
AI 资讯
LLM integration with OpenRouter
OpenRouter is a unified API gateway to hundreds of language models from providers such as OpenAI, Anthropic, Google, and Meta. You use one API key and one billing surface, and swap models by changing a provider/model slug. OpenRouter exposes a Chat Completions -compatible HTTP API. This post shows three Node.js integration paths: the official @openrouter/sdk , the openai package with baseURL , and the Vercel AI SDK with @openrouter/ai-sdk-provider . For deeper patterns on each stack, see the Chat Completions API , OpenAI Responses API (OpenAI direct only), and Vercel AI SDK posts. Prerequisites OpenRouter account API key Credits or billing enabled as needed Node.js version 26 Install packages for the path you use: @openrouter/sdk ( npm i @openrouter/sdk ) openai ( npm i openai ) ai and @openrouter/ai-sdk-provider ( npm i ai @openrouter/ai-sdk-provider ) Configuration Read credentials from the environment in production. Variable Purpose OPENROUTER_API_KEY Bearer token from OpenRouter settings OPENROUTER_MODEL Default model slug, for example openai/gpt-5.5 OPENROUTER_SITE_URL Optional site URL sent as HTTP-Referer for rankings on openrouter.ai OPENROUTER_SITE_TITLE Optional app name sent as X-OpenRouter-Title Model IDs use the provider/model format, for example openai/gpt-5.5 , anthropic/claude-opus-4.8 , or google/gemini-3.1-flash-lite . Browse the full catalog at openrouter.ai/models . The examples below use openai/gpt-5.5 , matching the model in the other LLM posts in this series. Override it with OPENROUTER_MODEL when you want a different model. @openrouter/sdk OpenRouter's official TypeScript SDK is type-safe and generated from the OpenAPI spec. Client setup import { OpenRouter } from ' @openrouter/sdk ' ; const client = new OpenRouter ({ apiKey : process . env . OPENROUTER_API_KEY , httpReferer : process . env . OPENROUTER_SITE_URL , appTitle : process . env . OPENROUTER_SITE_TITLE , }); Basic integration const response = await client . chat . send ({ chatReques
AI 资讯
The Anti-Bot Detection Checklist I Use Before Every Scraping Project
The Anti-Bot Detection Checklist I Use Before Every Scraping Project Every scraping project I take on starts with this checklist. Not because I'm paranoid — but because I've learned the hard way that production scrapers fail silently. They return 200 OK with garbage data, or they get rate-limited so gradually you don't notice for days. This is the systematic approach I've refined over 50+ scraping projects. Pre-Scraping: Know Your Target 1. Identify the CDN and Protection Stack Before writing a single line of code, check what you're up against: # Check CDN and headers curl -I https://target-site.com # Look for these common protection headers: # X-Engine: akamai-html-protection # X-Served-By: DataDome # cf-ray: Cloudflare # X-Bot-Status: blocked Common protection platforms: Cloudflare → Look for cf-ray and __cfduid cookies DataDome → Look for datadome in headers or scripts PerimeterX → Look for _pxff cookies Akamai → Look for akamai-html-protection headers 2. Check Robots.txt Respectfully curl https://target-site.com/robots.txt | grep -v "^#" Don't take this as gospel — but it's a good signal. If they explicitly disallow your use case, that's a flag. 3. Map the Site's JavaScript Rendering Some sites are fully static (fast, easy). Others render everything with JavaScript (need Playwright/Puppeteer). Check: // Quick check - fetch raw HTML vs rendered content // If they differ significantly, you need JS rendering const https = require ( ' https ' ); const html = await fetch ( ' https://target.com ' ). then ( r => r . text ()); const hasAngularVueReact = /ng-app|vue|react|__NEXT_DATA__/i . test ( html ); console . log ( ' Needs JS rendering: ' , hasAngularVueReact ); Code-Time: Defensive Patterns 4. Rotate User Agents const USER_AGENTS = [ ' Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 Chrome/120 Safari ' , ' Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 Chrome/120 Edge/120 ' , ' Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 Chro
AI 资讯
Fix: babel-plugin-transform-flow-strip-types broken in Babel 7 and 8
The original babel-plugin-transform-flow-strip-types hasn't been updated in 9 years and breaks silently in Babel 7 and 8 environments. The fix I published a maintained fork that works as a drop-in replacement: npm install --save-dev babel-plugin-transform-flow-strip-types-maintained Then update your .babelrc: { "plugins": ["transform-flow-strip-types-maintained"] } That's it. No other changes needed. What's fixed Babel 7 and 8 peer dependency conflicts Missing syntax plugin declaration Deprecated visitor patterns allowDeclareFields support Automated migration If you want to update your entire project automatically: npx flow-strip-migrate . This updates your package.json and babel config in one command. More info: https://flowstrip.netlify.app npm: https://www.npmjs.com/package/babel-plugin-transform-flow-strip-types-maintained
AI 资讯
Why DaloyJS Is the Best Backend (or BFF) for Your Electron App
If you've ever built an Electron app that needs a backend, you know the problem. You want something lightweight, something that runs on Node, something where you don't spend three hours configuring Swagger, and something that doesn't make you feel like you're setting up a microservices architecture just to expose two endpoints to your own desktop UI. I've shipped Electron apps with Express. I've tried Fastify. Both work, but they leave you doing a lot of plumbing yourself. Then I found DaloyJS , and honestly, it clicked. The Electron Backend Problem Here's the thing about Electron: your renderer process is basically a browser, and your main process is a Node.js server. When you need data from external APIs, or you need a clean layer between your UI and your business logic, you want a BFF, a Backend for Frontend. A thin server that composes upstream calls, holds the session, and returns exactly the shape your UI needs. DaloyJS was built for this role. The docs even say so plainly: typed upstream client, fetchGuard for safe egress, session handling, and edge runtime support. That combination is exactly the BFF toolkit. Contract-First Means Less Glue The killer feature for desktop apps is contract-first routing. You define a route once, and DaloyJS gives you validation, OpenAPI 3.1 docs, and a typed in-process client all from the same source. No stale spec files. No writing types by hand. Here's what a basic Electron BFF route looks like: import { z } from " zod " ; import { App , requestId , secureHeaders , rateLimit } from " @daloyjs/core " ; import { serve } from " @daloyjs/core/node " ; const app = new App ({ bodyLimitBytes : 1 << 20 , requestTimeoutMs : 5 _000 , docs : true , // auto-mounts /docs and /openapi.json }); app . use ( requestId ()); app . use ( secureHeaders ()); app . use ( rateLimit ({ windowMs : 60 _000 , max : 120 })); app . route ({ method : " GET " , path : " /settings/:userId " , operationId : " getUserSettings " , request : { params : z . objec
AI 资讯
Stop Hardcoding 301s: How I Built a Redirect Engine That Doesn't Break at 2 A.M.
Marketing wants an A/B landing page by Friday. Product wants to gracefully deprecate a legacy API without breaking old mobile clients. Growth wants ten thousand short links, and Ops does not want ten thousand Nginx edits. At some point, a single return 301 in your CDN stops being a configuration problem and becomes a routing product . Someone has to answer, on every HTTP request: Given this host, path, query, and method—where does this visitor go, and with which status code? I built that answer as a pipeline at LinkShift . Not a pile of special cases, but a fixed sequence of steps that runs the exact same way for real visitors and for the tools you use to test rules before rollout. Here is how I designed a deterministic redirect engine, the pipeline that powers it, and the edge cases that kept me up at night so they don't have to keep you up. Why "Usually Works" Is Not Enough A redirect engine fails quietly. The browser follows a broken 302 and nobody files a ticket. The damage shows up days later in analytics: wrong campaign, wrong locale, or an infinite loop that only appears when two rules on the same host point at each other. What I wanted early on was boring, bulletproof reliability: Same inputs → same decision. Two engineers simulating the same request against the same rule set should get the same target URL. Same resolution logic everywhere. Matching and destination resolution must not diverge between the "Test Rule" button in the dashboard and an actual click on a custom domain. Guards before cleverness. Rate limits and access checks run before anyone evaluates a ternary conditional in a destination string. Expressiveness is easy. Ordering is what saves you in production. The Pipeline, Told as a Story Picture a request hitting a hostname. Before the engine asks "which rule wins?", the request walks through a strict corridor of gates. Only then does it enter the rule loop. Gate 1: The Host Has to Exist If the hostname does not resolve to a domain or LinkShift
AI 资讯
Serverless Framework Deployment: Unleash the Power of AWS Lambda
Let me tell you exactly what happened the first time I tried to set up Lambda manually. Four hours. IAM trust policies I didn't fully understand, ARNs copy-pasted into the wrong fields, an API Gateway that was technically configured but somehow not routing anything correctly, and a deploy that failed with an error message pointing me nowhere useful. I hadn't written a single line of actual business logic yet. That's when someone on my team mentioned the Serverless Framework. My first reaction was honestly skepticism — another abstraction layer sounded like another thing to learn and eventually fight with. I was wrong about that. This isn't a "look how clean this tool is" post. It's more like: here's what I actually did to get a Postgres-backed CRUD API running on Lambda, step by step, including the parts that tripped me up. What the Framework Is Actually Doing Under the Hood Worth knowing before you start: the Serverless Framework isn't magic. It's generating CloudFormation templates and submitting them to AWS on your behalf. Your Lambda functions, API Gateway routes, CloudWatch log groups — all of it gets provisioned from a single config file. It works with other providers too, but the AWS integration is where it really earns its keep. The console clicking and manual ARN-wiring that burns time at the start of every serverless project? Gone. Same deploy workflow whether you're building a REST API, an event processor, or a cron job. Once you've done it once, the second project takes a fraction of the time. What You're Building Four live endpoints backed by PostgreSQL. A Users table. Create, read, update, delete — nothing exotic, but a real enough foundation that you can extend it into something actual once this guide is done. You'll need an AWS account, the AWS CLI installed, and the Serverless Framework installed before starting. That's it. Step 1: Sort Out Your AWS Credentials Run this to create both config files in one go: bash cat << EOF > ~/.aws/credentials [def