标签：#ast

Searching billions of documents for a phrase and getting ranked results in tens of milliseconds looks like magic. It is not. It comes down to two ideas working together: an index that maps words to documents instead of scanning documents for words, and a way to spread that index across machines so each holds only a slice. Understand both and full-text search stops being mysterious. The core problem A database scans rows. If you ask a plain database to find every document containing a word, it reads documents and checks them, which is linear in the amount of data. That is fine for exact key lookups and hopeless for free-text search across huge corpora. You need the opposite mapping. Instead of "given a document, what words does it have", you want "given a word, which documents have it". That inversion is the whole trick. The second problem is size. One machine cannot hold the index for billions of documents, and one machine cannot serve the query load. So the index has to be split across nodes, and a query has to find the right nodes and combine their answers. Key design decisions Build an inverted index. At index time, each document is broken into tokens by an analyzer that lowercases, splits on word boundaries, and often strips or stems words. For every token, the engine keeps a posting list: the set of document ids that contain it, often with positions for phrase matching. A query for a word becomes a direct lookup of its posting list, not a scan. A multi-word query intersects or unions posting lists, which is fast because the lists are sorted. Store the index in immutable segments. New documents go into small new segments rather than editing existing ones. Segments are immutable, which makes them cache-friendly and safe to read without locks. A background process merges small segments into larger ones over time. A delete is just a marker; the document is removed for real during a later merge. Split an index into shards. An index is divided into shards, each a sel

In this podcast Shane Hastie, Lead Editor for Culture & Methods spoke to Gabriela Moreira about making formal methods accessible through the Quint specification language, how AI is dramatically lowering the barrier to entry for formal specification and model-based testing, and why defining correct system behaviour remains essential human work in an AI-driven world. By Gabriela Moreira

TL;DR Retailers process thousands of inventory transactions every second across physical stores, eCommerce platforms, warehouses, suppliers, and fulfillment centers. Yet many inventory systems still rely on scheduled synchronization, causing stock levels to become outdated within minutes. The result is overselling, delayed replenishment, inaccurate inventory visibility, and avoidable stockouts. Apache Kafka enables real-time inventory management by treating every inventory movement as an event that is streamed the moment it occurs. Sales, returns, warehouse transfers, supplier deliveries, and IoT sensor updates are continuously processed to maintain a consistent inventory view across all retail systems. This event-driven approach helps retailers improve inventory accuracy, automate replenishment, detect stockouts before they occur, and respond to changing demand in near real time. In this guide, you'll learn how Apache Kafka powers real-time inventory management, explore a production-ready reference architecture, understand how inventory events are processed across retail systems, and discover implementation best practices for building scalable, resilient inventory streaming applications. Introduction Retail inventory management has evolved far beyond tracking products on warehouse shelves. Today's retailers operate across physical stores, eCommerce platforms, online marketplaces, distribution centers, and supplier networks, where inventory levels change continuously throughout the day. Every sale, return, warehouse transfer, supplier delivery, and inventory adjustment impacts product availability, making accurate inventory visibility essential for delivering a seamless customer experience. However, many retailers still rely on scheduled synchronization between Point-of-Sale (POS) systems, Warehouse Management Systems (WMS), Enterprise Resource Planning (ERP) platforms, and online storefronts. While these systems perform different functions, they all depend on accur

Today on Uncanny Valley, we unpack the political debacle unfolding in Maine surrounding the campaign of Democratic candidate Graham Platner.

The company is taking a modular approach to designing these chips, anticipating that their needs will change as AI evolves rapidly by the time the chips are in production.

Today on Decoder, I’m talking with Peter Kafka, who is chief correspondent at Business Insider and host of Channels, a podcast about the media industry. And it’s a big week for the media industry — Comcast just announced that it’s splitting itself up, into the Comcast broadband company and the NBCUniversal entertainment company. That’s after […]

In version 1.11, HashiCorp introduced Terraform Ephemeral resources and write-only attributes to allow for root configs that do not store secrets in the Terraform statefile. But many users ask about how they can adopt ephemerals. This blog attempts to lay out the ways secrets can be stored in state and how you should update your configurations to remove those secrets. Note: For a primer on ephemerals ( see this blog post ). Scenarios to consider: Data sources that fetch a static secret Resources that receive a secret Resources that generate a dynamic a secret Resources that fetch generated secrets to store in another 3rd party system Scenario 1: Data sources with static secrets Ephemeral resources can often be a drop-in replacement for data sources pulling static values: data "vault_kv_secret_v2" "static_kv" { mount = "kvv2" name = "my_password" } ephemeral "vault_kv_secret_v2" "static_kv" { mount = "kvv2" name = "my_password" } However, using these values has 1 specific difference. The attributes on a ephemeral resource are considered ephemeral and can only be used as ephemeral arguments. That means 2 places: Provider blocks Provider blocks are considered ephemeral, so ephemeral resources may populate arguments: provider "example" { password = tostring ( ephemeral . vault_kv_secret_v2 . static_kv . data . password ) } Write-only arguments Write-only arguments are special arguments that require the ephemeral taint for values: resource "aws_db_instance" "example" { ... password_wo = tostring ( ephemeral . vault_kv_secret_v2 . static_kv . data . password ) } If the resource you wish to pass a value to does not have an available ephemeral, open an issue with that provider. You can reference: this blog post this agent skill Scenario 2: Resources that receive a static secret Without duplicating to the section above, write-only arguments are a way to get secrets out of state. Above has guidance if the secret value comes from a data source, but what if its from a variable?

AWS has outlined how ProGlove, an industrial-wearables manufacturer, was able to scale its SaaS platform to run more than one million AWS Lambda functions spread across thousands of dedicated customer accounts. By Matt Foster

It was 2 AM on a Tuesday, and I was staring at my CourseSpeed dashboard looking at a graph that claimed I had just finished a 14-hour AWS certification course in 47 minutes. I hadn't. I was just testing the 16x speed toggle. But my analytics engine thought I was a god. When I started building CourseSpeed—a browser extension to inject custom playback speeds and track learning analytics across Udemy, Coursera, LinkedIn Learning, and Skillshare—I thought the hard part would be the UI. It wasn't. Injecting a floating control panel and setting document.querySelector('video').playbackRate = 2.5 takes about ten lines of JavaScript. The actual nightmare was the learning analytics. Specifically, accurately tracking effective watch time versus wall-clock time across wildly different Single Page Applications (SPAs). The naive approach that burned me My first pass at the analytics tracker was straight out of MDN. I listened to the standard HTML5 video events. // The approach that worked perfectly in my head const video = document . querySelector ( ' video ' ); video . addEventListener ( ' ratechange ' , ( e ) => { sendAnalytics ({ type : ' speed_change ' , rate : e . target . playbackRate }); }); video . addEventListener ( ' timeupdate ' , () => { logWatchTime ( video . currentTime , video . playbackRate ); }); This worked flawlessly on Udemy. Then I opened LinkedIn Learning. The dashboard flatlined. Then I tried Coursera. The time spent was wildly inaccurate, drifting by minutes over an hour. I spent three days debugging this, tearing my hair out over console logs. Here is what I missed: modern learning platforms don't just drop a raw <video> tag on the page and leave it alone. They wrap it in custom players, throttle events to save CPU, and dynamically destroy and recreate the DOM node when you skip chapters or when the SPA router transitions. My event listeners were getting orphaned. Or worse, they were firing with stale data because the platform's custom wrapper was dispatc

It is been while I am learning and build around FastAPI. So there is a project where I was thinking how to add this new feature over exiting one. Like what changes I need to make in database which need to be reflected in my backend and frontend. I already lunched the web locally. Problem started When I when back to the web and reload it it shows this error: ERROR: ConnectTimeout: Unauthorized 401. I was like what? Why? I thougth there is some issue with login endpoint or refresh token function. When i did some debugging and found some new information which is: "Either Supabase's edge/pooler (or OS, or an intermediate proxy/NAT) silently kills those idle connections server-side after some timeout but client-side pool doesn't know that." As I was doing nothing in become idle state so to save the resources server side silently close that particular connection. So I came back and try to connect it give this error. First thought come it my mind after this was there should be a way to automatically check this idle state and if user was in ideal state then create a new connection. Proposed Solutions After a while I come up with these solution: Calculate the Idle time: if it is more then server connection timeout then establish new connection. Retry logic: retry once on the specific connection errors. I thought this will work but This again give me error then this new issue I faced. Cold-start connection problem There is something call dual-stack (IPv4 and IPv6) networks and Happy Eyeballs is a network mechanism which automatically move to IPv4 connection if IPv6 fails. But supabase-py uses httpx and it doesn't support Happy Eyeballs. So in first try after the connection time out it try to establish IPv6 connection which is not routeable in most Pakistani ISPs and ultimately it fails and wait for timeout. There is no way to try it again for IPv4. So we have to do it manually. So this error help me to learn many thing in process. Share your thoughts.

Static analysis isn't just for application source code. Terraform, Pulumi, OpenTofu, and CloudFormation files are code too — and they get misconfigured just as often as a backend service. A public S3 bucket, a security group open to 0.0.0.0/0 , or an unencrypted RDS instance are all bugs you can catch before apply ever runs. TFSec is the tool most people reach for first, but it's not the only option on the OWASP Source Code Analysis Tools list . In this article I'll use Checkov , a free, open-source policy-as-code scanner built by Bridgecrew (now part of Palo Alto Networks), to scan a Terraform project end to end — from a local scan to a GitHub Actions gate that blocks merges on critical misconfigurations. The same approach works with OpenTofu and Pulumi projects too, since Checkov understands HCL directly and also has native support for Pulumi's rendered plan output and CloudFormation/ARM/Kubernetes manifests. Why Checkov? 100% open source (Apache 2.0), actively maintained, thousands of built-in policies. Understands Terraform, OpenTofu, CloudFormation, Kubernetes, Helm, Dockerfile, ARM, Serverless Framework, and Pulumi (via cdktf /synthesized plans) — one tool across most of your IaC surface. No account or API key required to run locally or in CI. Supports custom policies written in Python or YAML if the built-in rule set doesn't cover something specific to your org. ## 1. The sample infrastructure A small AWS setup with a few intentionally introduced misconfigurations — the kind that get merged during a rushed sprint: # main.tf provider "aws" { region = "us-east-1" } resource "aws_s3_bucket" "data" { bucket = "company-app-data-bucket" } # Vulnerable: bucket has no encryption, no versioning, and is publicly readable resource "aws_s3_bucket_acl" "data_acl" { bucket = aws_s3_bucket . data . id acl = "public-read" } resource "aws_security_group" "web" { name = "web-sg" description = "Allow web traffic" # Vulnerable: SSH open to the entire internet ingress { from_port

Vermilion Cliffs Ventures announced Wednesday the close of a $25 million Fund II.

Why Every Modern SaaS Needs a Billing Engine (Not Just a Payment Gateway) When building a SaaS product, API platform, AI application, or cloud service, most teams start by integrating a payment gateway like Stripe or Paystack. That solves one important problem: Collecting payments. But collecting payments and managing billing are two completely different challenges. As products grow, pricing becomes more sophisticated. Customers expect usage-based billing, flexible subscriptions, prepaid credits, seat-based pricing, tiered plans, and transparent invoices. Suddenly, the billing logic inside the application becomes one of the most complex parts of the entire system. This is why more engineering teams are separating payment processing from billing infrastructure . Payment Processing vs Billing Infrastructure A payment gateway is responsible for moving money. A billing platform is responsible for determining what should be charged, when it should be charged, and why it should be charged. Although these responsibilities often get grouped together, they solve very different problems. Payment providers typically handle: Payment authorization Card processing Refunds Payment retries Bank settlements Customer payment methods A billing engine manages: Usage metering Subscription billing Pricing rules Billing cycles Invoice generation Revenue analytics Customer usage tracking Payment orchestration Multi-gateway billing Proration Credits and quotas Keeping these responsibilities separate makes systems easier to maintain, easier to scale, and much more flexible when pricing changes. Why Billing Logic Doesn't Belong Inside Your Application Many engineering teams start with a simple subscription model. Then the business evolves. Suddenly the product needs to support: API usage billing AI token billing Storage billing Compute billing Per-seat pricing Hybrid subscriptions Enterprise contracts Prepaid credits The application slowly becomes filled with billing-specific code. Developers

Something on Pluto and one of Saturn’s moons, Titan, absorbs light in a way unexplained by anything in spectroscopic databases.

Decentralized Autonomous Organizations are often presented as simple governance systems: token holders create proposals, vote, and execute decisions on-chain. In practice, building a production-grade DAO is far more difficult. A DAO is not only a smart contract. It is a distributed coordination system that combines governance logic, treasury security, token economics, identity, off-chain infrastructure, and human decision-making. A failure in any one of these layers can compromise the entire organization. Below are some of the most important technical problems DAO developers must solve. 1. Governance Attacks Through Borrowed Voting Power Many DAOs calculate voting power based on the number of governance tokens held at a specific moment. This creates a serious attack surface when tokens can be borrowed through lending protocols or flash loans. An attacker may temporarily acquire a large amount of voting power, submit or approve a malicious proposal, and return the borrowed assets shortly afterward. The standard defense is snapshot-based voting power. Instead of checking a user’s current balance, the governance contract reads historical balances from a previous block. function getVotes( address account, uint256 blockNumber ) public view returns (uint256) { return token.getPastVotes(account, blockNumber); } However, snapshots alone do not solve every problem. Developers should also consider proposal delays, minimum token-holding periods, quorum requirements, and vote-delegation risks. 2. Dangerous Proposal Execution The most sensitive part of a DAO is usually the executor. A successful proposal may call arbitrary contracts, transfer treasury assets, upgrade protocols, or change governance parameters. If proposal calldata is incorrectly validated, a governance action can execute unintended operations. A DAO should clearly separate: Proposal creation Voting Proposal queuing Timelock execution Emergency cancellation Using a timelock gives token holders and security teams

Introduction AI governance is often framed around risk, ethics, safety, and international cooperation. These are essential, but they are not sufficient. Governance only becomes real when countries have the computing infrastructure required to run, monitor, and maintain modern AI systems. Without compute, governance is theory. With compute, governance becomes capability. This article explores the missing execution layer in global AI governance — and why bridging the AI divide requires far more than policy alignment. The Hidden Dependency: Governance Assumes Infrastructure Most governance frameworks implicitly assume that nations already have: access to high performance compute reliable data pipelines secure storage operational tooling energy capacity connectivity skilled operators But this assumption is false for the majority of the world. The global AI divide is not primarily about access to models. It is about access to the infrastructure required to run them. Governance frameworks that ignore this reality risk becoming aspirational rather than actionable. The Execution Layer: Where Policy Meets Reality The execution layer is the part of AI governance that turns policy into practice. It includes: compute infrastructure data ingestion and processing pipelines monitoring and evaluation tooling human in the loop operational workflows maintenance and lifecycle management energy and cooling requirements secure deployment environments This layer is rarely discussed in governance conversations, yet it is the foundation upon which all responsible AI depends. Without an execution layer, governance collapses into paperwork. The Real Global Divide Isn’t About Models — It’s About Compute There is a persistent misconception that the AI divide is about access to large models. In reality, the divide is driven by: insufficient compute unreliable infrastructure lack of operational capacity limited data availability absence of secure environments dependency on external providers A c

I built a web front end for an Nmap-based port scanner: a FastAPI backend, a React dashboard, background scan jobs, a plugin system. It worked. Then I sat down and audited it like an attacker would — and found a stack of real weaknesses, plus a lesson in why you verify an exploit before you call it one. This is the honest version: the holes I found, the unauthenticated-RCE chain I thought I had, why it didn't actually fire, and the hardening I shipped anyway. Repo: https://github.com/DipesThapa/PortScanner This is my own project, audited and fixed by me. No third-party systems were touched. Scanners are dual-use — only ever point one at hosts you own or are authorised to test. Hole 1: no authentication, anywhere The foundation: every API route and the /ws/status WebSocket were open. No API key, no session. The Dockerfile bound 0.0.0.0:8000 and ran as root. Anyone who could reach the port could drive scans, hit the upload endpoint, and read every job's logs. api_router = APIRouter () # no dependencies — fully open This is the real, unambiguous problem. Everything below is only interesting because it sat behind no auth. Hole 2: an upload endpoint that allowlisted its own files Deep-dive follow-up commands ran against an allowlist — good instinct. But an upload endpoint wrote a file, chmod +x 'd it, and then added it to that same allowlist: for item in scripts_dir . glob ( " * " ): if item . is_file (): allowed . add ( str ( item . absolute ())) # upload authorises itself An allowlist any input can extend isn't an allowlist. This is a genuine design footgun. Hole 3: the RCE I thought I had — and why it didn't fire Here's the chain I got excited about: the scan target flows toward Nmap's argv, and it's subprocess.run(..., shell=False) . No shell injection — but you don't need a shell to abuse Nmap. If a target became --script=/uploaded.nse , Nmap would load and run that NSE (Lua) script, and NSE can call os.execute . Upload a malicious .nse (Hole 2), get Nmap to load it

The Torifune asteroid turns out to be shaped like a peanut.

We’ve got another special episode of Decoder today, recorded at the Cannes Lions advertising festival in the South of France. I’m talking with Ali Berman and Raina Penchansky, who run the Creators division at United Talent Agency. UTA is an enormous talent agency. Half the people you’ve ever heard speak or perform or who show […]

In this episode, Alex Zenla (CTO/Co-founder, Edera) challenges the "laissez-faire" attitude toward modern infrastructure. She promotes "spite-driven development", building software to solve genuine technical pain points rather than passively accepting flawed abstractions, as a philosophy of improving the world of software. By Alex Zenla