标签：#python

(Translation of my Japanese article on Zenn.) This is part 4 of a series where I keep delegating implementation to AI without being able to read the code, building a vulnerability triage CLI called triage-lens. This installment is about distribution rather than the tool's internals: the tool had been sitting on GitHub, and I published it to PyPI so a single pip install triage-lens brings it in. A confession first. Shipping took more nerve than any of the feature work did. And three "leaks" actually turned up right before release. From the installing side to the installed side I can't read code, but I have typed pip install before. Years ago I dabbled in Python out of curiosity, and the one thing that stuck was the experience of a useful tool arriving in one line. Now that I'm the one publishing, the other side of that one line finally became concrete. Someone builds a thing, shapes it into a package, and puts it on the public shelf called PyPI. That's why it installs in one line anywhere in the world. My turn to put something on the shelf. I delegated the release work to AI too: package metadata, the release workflow, and one thing I insisted on. Instead of an API token, authentication to PyPI uses Trusted Publishing (OIDC). Nothing like a long-lived password gets stored anywhere; you declare "trust publishes from this workflow in this GitHub repository" and that's it. A secret you never hold is a secret that can't leak. The pre-release check caught three real ones In this project, nothing goes out to a public repository without passing a mechanical check. Procedures and tests, not eyeballs, verify that no personal or development-only information is mixed in. For three releases it came up empty. That's what insurance looks like. On the fourth run it caught something real. Three somethings. First, test code had slipped into the distribution. The packaging tool's default behavior had a path where the whole development test suite gets bundled along. I was about to scat

If you’ve ever tried to open an Apple Health export.xml file in VS Code, you’ve probably watched your RAM melt into a puddle of sadness. 🫠 Apple’s HealthKit data is a treasure trove of biological insights, but at the scale of 5GB+ of "dirty" XML, it’s a Data Engineering nightmare. In this tutorial, we are building a high-concurrency Apple Health ETL Engine . We’ll be leveraging Rust for blazing-fast parsing, Apache Arrow for memory-efficient data transport, and ClickHouse for lightning-fast analytical queries. Whether you are building a personal bio-hacking dashboard or a population health platform, this architecture is designed to handle "Big Data" on "Small Hardware." The Problem: Why XML is Killing Your Pipeline Apple Health exports everything as a single, massive XML file. A typical 3-year history contains millions of <Record> tags with inconsistent attributes. Standard DOM parsers (like Python’s ElementTree ) will crash your system because they try to load the entire tree into memory. To solve this, we need a Streaming ETL approach. The Architecture 🏗️ Our pipeline follows a "Performance-First" philosophy: we parse in a low-level language, pass data through a zero-copy memory format, and sink it into a columnar database. graph TD A[Apple Health export.xml] -->|Streaming I/O| B(Rust XML Parser) B -->|Schema Mapping| C{Apache Arrow Batches} C -->|Zero-copy| D[Python/Polars Wrapper] D -->|Bulk Insert| E[(ClickHouse OLAP)] E -->|SQL/Grafana| F[Health Insights] style B fill:#f96,stroke:#333,stroke-width:2px style E fill:#00f,stroke:#fff,stroke-width:2px Prerequisites 🛠️ Before we dive in, ensure you have the following installed: Rust (Latest stable) Python 3.10+ ClickHouse (Local or Cloud) Tech Stack : quick-xml , arrow-rs , polars , clickhouse-connect . Step 1: The High-Speed Rust Parser 🦀 We use the quick-xml crate because it provides a "pull-based" API. This allows us to read the file byte-by-byte without ever loading more than a few KB into memory. // src/parser

Short answer: for a small edtech SaaS sending a weekly digest in Europe and the US, persist one idempotent delivery job per customer and week, then start with a polling worker; adopt queue push or subscription delivery only when measured queue delay, regional isolation, or worker operations justify a public HTTPS receiver. The transport is not the guarantee. A public webhook can be retried, a subscriber can redeliver, and a polling loop can crash after sending but before recording success. In all three designs, the hard boundary is the same: a durable job identity, an atomic claim, an expiring lease, and a delivery operation that tolerates repetition. Get those right first. The easiest setup is then the one with the fewest independently failing parts your team must operate, not the one with the shortest quick-start page. This matters for a weekly digest because duplicates damage trust while an omitted message is difficult to notice. A customer who was active at the cutoff must map to a stable key such as customer_id + digest_week ; changing from polling to push must not change that identity. What delivery guarantee does the weekly digest actually need? “Exactly once” is an application outcome, not a useful promise to infer from a queue label. There are at least four moments to distinguish: eligibility is calculated, a job is committed, a worker claims it, and the downstream delivery system accepts it. A process can stop between any two writes. If it stops after acceptance but before the job is marked complete, retrying is the conservative action, and that retry can duplicate the digest unless the downstream operation accepts the same idempotency key. Write the contract before choosing a transport: Every active customer at the weekly cutoff gets one durable job. A job may be attempted more than once. The same digest_key is used on every attempt and is unique in the ledger. A claim expires, so a stopped worker cannot own work forever. Operators can distinguish pending

Every lottery player knows the saying: "One line is no line, you have to play a few more to boost your chances!" That's why the average player often fills out a complete ticket with 8 lines. Sounds like a solid strategy, right? Wrong. It’s actually the fastest way to systematically burn through your cash. To prove it, we wrote a Python simulation that uses historical payout data and cold, hard combinatorics to see who actually has any money left in their account at the end. What exactly is a Monte Carlo simulation? Named after the famous casino in Monaco, the Monte Carlo simulation is basically the brute-force approach to probability theory. Usually, mathematicians use a single, elegant formula to calculate the expected value. That formula will dryly inform you: "You lose an average of 1 Euro per Eurojackpot line." That might be mathematically correct, but emotionally, it's a bit of a snooze. It doesn't capture the true pain of slowly bleeding out financially. The Monte Carlo simulation throws that elegant formula right out the window. Its core concept is pure, raw computing power. Instead of just calculating the theoretical outcome, we let the computer simply play through reality thousands of times. It’s not an equation; it’s a simulation and iteration of real events. The computer spawns 1,000 fictional players. For every player and every draw over the last 10 years, it generates a random number based on the actual Eurojackpot probabilities. It simulates the real-world winning and (mostly) losing, step by step. At the end, we aren't looking at some abstract theoretical number, but at the very real, blood-red bank accounts of 1,000 ruined clones. The Setup We're using Polars for lightning-fast data processing, NumPy to simulate millions of random draws, and Matplotlib to visualize our financial doom. First, grab our historical Eurojackpot database and drop it into the same folder. The Script Here is the complete Python code. It calculates the exact mathematical odds

Last week I wrote about the French voiceover API that only accepts payment from robots . Today: the part people actually asked me about — how do you verify a $0.05 payment on-chain with zero payment processor, zero SDK, and zero KYC? The answer: one Python function, ~40 lines, stdlib only. Here's the real production code. The setup My endpoint sells French neural TTS voiceovers for $0.03–0.05 USDC. At that price, Stripe is a non-starter (their floor is ~$0.50 per charge) and any processor's KYC kills the "robots welcome" model. So payments go through the x402 pattern: client pays USDC on Base, sends me the transaction hash, I verify it myself against a public RPC before delivering. The verification function import json , os , urllib . request WALLET_BASE = " 0x3f97...D074 " # where I receive USDC_BASE = " 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913 " # USDC on Base BASE_RPC = " https://mainnet.base.org " TRANSFER_TOPIC = " 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef " def rpc ( method , params ): req = urllib . request . Request ( BASE_RPC , data = json . dumps ({ " jsonrpc " : " 2.0 " , " id " : 1 , " method " : method , " params " : params }). encode (), headers = { " Content-Type " : " application/json " }) with urllib . request . urlopen ( req , timeout = 30 ) as r : return json . load ( r ). get ( " result " ) def verify_payment ( tx_hash , min_usdc ): # 1. format sanity if not tx_hash . startswith ( " 0x " ) or len ( tx_hash ) != 66 : return False , " bad hash format " # 2. anti-replay: one hash = one delivery if tx_hash . lower () in load_used_txs (): return False , " tx already used (replay) " # 3. fetch the receipt receipt = rpc ( " eth_getTransactionReceipt " , [ tx_hash ]) if not receipt : return False , " tx not found on Base " if receipt . get ( " status " ) != " 0x1 " : return False , " tx failed on-chain " # 4. scan logs for a USDC Transfer TO my wallet want_to = WALLET_BASE . lower (). replace ( " 0x " , "" ) for log in receipt

GitHub热门项目 | 中国专利.skill：专利点挖掘与交底书（发明/实用/外观）编写，通俗解读专利，嗅探政策动向，辅助审查答复。 | Stars: 5,527 | 38 stars today | 语言: Python

GitHub热门项目 | RL training environments for Microduck (mjlab) | Stars: 654 | 147 stars today | 语言: Python

AI agents are good at interpreting goals, but prose instructions are a weak place to enforce exact rules. If a skill says "keep the commit subject short" or "never commit without approval," an agent can still misunderstand the boundary. The open-source how-to-create-a-skill-tutorial shows a practical split: let the agent make judgments, and let small local scripts validate repeatable rules. This tutorial builds the smallest useful version of that pattern: a commit-crafter skill with a SKILL.md file, a Python validator, and tests that run with the Python standard library. TL;DR An Agent Skill is a directory containing at least SKILL.md . Put the workflow and safety boundaries in that file. Put exact validation in a script. Keep the script deterministic, return meaningful exit codes, and run it before presenting the result to a user. The finished repository's example skill validates Conventional Commit messages. You can copy the same structure for release notes, config generation, research reports, or any other workflow with rules that can be checked mechanically. Prerequisites You need: Python 3.12 or newer for the repository's CI example. Git if you want the skill to inspect staged changes. An agent that supports the Agent Skills directory convention. A shell. The commands below use POSIX syntax; the files themselves are also designed for Windows. The project has no stable release tag at the time of writing. The examples and commands below are checked against the current main branch. Read the Agent Skills specification if your client uses a different discovery directory. 1. Create the skill directory The repository documents two useful scopes. A personal skill belongs in your user skills directory. A project skill belongs in the repository so a team can review and install it with the project. mkdir -p .agents/skills/commit-crafter/scripts mkdir -p .agents/skills/commit-crafter/references The required layout is simple: commit-crafter/ |-- SKILL.md |-- scripts/ | `--

Wrap the payment. It runs once across retries, crashes, resumes, and replays. exactly-once is a Python library that makes a side effect run a single time. Wrap the function that pays an invoice or sends an email, or submits a transaction and it executes once per key, then replays its stored result on every later call. Here is the whole integration: from exactly_once import once , Store , current_key store = Store . sqlite ( " effects.db " ) @once ( store , key = lambda inv , ** _ : f " pay: { inv . id } " ) def pay_invoice ( inv ): return payments . transfer ( inv . vendor , inv . amount , idempotency_key = current_key ()) Call pay_invoice(invoice) and it pays the vendor. Call it again from a retry, a resumed run, a replay, or a second worker and it returns the recorded result. The vendor is paid once. The crash it's built for An agent pays an invoice. The transfer reaches the provider and succeeds. The process dies in the moment between the provider's 200 OK and the line that records the result. The agent restarts and reaches the same step again. exactly-once writes a record the instant the agent enters the call. pay_invoice claims the key pay:{invoice.id} , and the store marks it IN_FLIGHT . When the result returns, the store marks it COMMITTED and saves that result. After the crash the record reads IN_FLIGHT with an empty result the library knows a payment started and holds no proof it finished. So it quarantines the key. The agent leaves that payment for a decision and moves on. You give @once a prober that asks the payments API whether a transfer with that idempotency key exists: the library commits the key when the provider confirms the payment, and releases it when the provider confirms none. Until an answer arrives, the held payment stays in the ledger where you can see it: store . list ( state = " in_flight " ) # every payment awaiting a verdict How the guarantee holds Three states, one atomic operation: FRESH ──claim──▶ IN_FLIGHT ──commit──▶ COMMITTED clai

Astrology apps are one of India's quietest huge markets — panchang widgets, kundli generators, matrimonial matching, muhurta pickers. Under every one of them sits the same unforgiving requirement: the astronomy has to be exactly right , because your user's grandmother has a printed panchang on her wall and she will check. I spent the last few months building GrahaAPI — 237 REST endpoints across 23 modules of Vedic astrology, Hindi + English in every response. This post isn't a feature tour. It's the four engineering problems I didn't expect, because I think they're interesting even if you never touch astrology. First, 60 seconds of domain: what the computer actually calculates Strip away the mysticism and Vedic astrology is a coordinate system plus 1,500 years of lookup tables: Tithi (the "lunar date"): the Moon-Sun angular separation, divided into 12° slices. 30 per lunar month. Nakshatra : which of 27 equal 13°20′ segments of the ecliptic the Moon occupies. Dasha : a 120-year planetary period cycle, seeded entirely by the Moon's exact position at birth — a birth-time error of minutes shifts period boundaries by months . The whole thing runs on the sidereal zodiac, offset from the tropical zodiac by ~24° (the ayanamsa — we use Lahiri, the Indian government standard). So: an ephemeris gives you planetary longitudes, and everything else is careful classical bookkeeping. Which brings me to the first bug. Bug #1: the thread-local zodiac Our ephemeris core is a C library with Python bindings, and it holds "which zodiac mode are you in" as global state — per thread . FastAPI runs sync endpoints on a threadpool. First request warms up thread A: sidereal mode set, positions correct. Then a request lands on freshly-spawned thread B: mode silently defaults to tropical , every longitude comes back ~24° off, and — because 24° is almost exactly one nakshatra-and-a-bit — the Moon lands in a plausible but wrong nakshatra. Which seeds the dasha. Which means the API happily returne

Um script meu corrige acentuação automática em português usando um dicionário de mais de noventa mil palavras. Ele existe justamente para consertar texto gerado por IA, que às vezes esquece acento. Só que hoje descobri que o próprio dicionário estava trocando "pele" por "Pelé" e "teve" por "tevê". O corretor não tinha bug de lógica nenhum. O dado dentro dele é que estava errado. Como um dicionário de correção vira fonte de erro O dicionário foi gerado a partir de uma lista ampla de palavras em português, mapeando a forma sem acento pra forma com acento correspondente. Para a maioria das palavras isso funciona bem: uma palavra comum sem acento tem sempre a mesma forma acentuada certa, sem exceção. O problema aparece em palavra que também é nome próprio. "Pele" sem acento é ambíguo: pode ser a palavra comum (pele, órgão do corpo) ou o apelido do jogador, que leva acento (Pelé). Na hora de montar o dicionário, alguma etapa do processo escolheu a forma acentuada como "a certa" para aquela entrada, sem checar que a forma comum, sem acento, é muito mais frequente no uso real. Mesma história com "teve" (passado do verbo ter) virando "tevê" (forma informal de televisão). Por que isso é mais perigoso que parecer Um erro de dicionário desse tipo não quebra nada visivelmente. O texto sai fluente, gramaticalmente correto, sem nenhum sinal de que uma palavra foi trocada por engano. Quem lê rápido não percebe. Isso é o oposto de um erro de sintaxe, que pelo menos avisa que algo está errado. A única forma de achar foi ler o texto gerado com atenção, palavra por palavra, depois de já ter rodado o corretor automático. O corretor não se autodenuncia. Onde traçar a linha entre corrigir e não mexer Nem toda ambiguidade é bug. Uma palavra genuinamente ambígua, tipo "e" (conjunção "e" vs verbo "é"), "esta" (demonstrativo vs verbo "está") ou "pais" (progenitores vs "país"), não tem solução de dicionário. Só quem lê a frase inteira sabe qual acento é o certo. Forçar uma escolha automática

A área de Cibersegurança atrai profissionais devido à complexidade das ameaças e à necessidade de proteção de infraestruturas críticas. No entanto, iniciantes costumam se perguntar por onde começar. A resposta estratégica envolve dominar a lógica de programação, o funcionamento de redes de computadores e os fundamentos dos sistemas operacionais. Compreender linguagens de programação, especialmente Python, permite que um analista de segurança compreenda a mecânica dos sistemas em vez de apenas operar ferramentas prontas. A lógica de programação desenvolve o raciocínio estruturado para a resolução de problemas. Na prática do dia a dia, a automação via scripts é fundamental para criar rotinas de verificação, tratar grandes volumes de dados e analisar eventos de segurança com rapidez. Além da programação, a navegação em ambientes Linux via terminal e o domínio dos protocolos de rede (como TCP/IP e o modelo OSI) formam a base necessária para a triagem de incidentes. Compreender como os dados trafegam e como as permissões do sistema operacional funcionam permite ao estudante visualizar o caminho que um ataque cibernético pode percorrer. A combinação entre a teoria de defesa cibernética (como os conceitos transmitidos pelo curso da Cisco Networking Academy) e o raciocínio lógico é o diferencial para quem busca ingressar em um Centro de Operações de Segurança (SOC). O mercado de tecnologia exige profissionais que saibam interpretar relatórios de segurança, analisar logs de eventos e propor medidas efetivas de mitigação. O aprendizado contínuo e a prática em laboratórios virtuais são essenciais nesse processo. Construir uma base sólida em algoritmos e redes transforma o estudo teórico em uma carreira sólida e preparada para os desafios reais da proteção de dados e da infraestrutura corporativa.

GitHub热门项目 | | Stars: 346 | 71 stars today | 语言: Python

GitHub热门项目 | 🕵️‍♂️ (2-in-1) Email & Username OSINT suite for deep data extraction just from a single Email/Username. Analyzes 455+ actively maintained scan vectors (175+ email / 280+ username) for security research, investigations, and digital footprinting. | Stars: 3,481 | 39 stars today | 语言: Python

A few weekends ago I got annoyed at typing prompts into a terminal and decided the fix was, obviously, to control my AI agent with hand gestures instead. This is the story of building that, and the two hours I lost fighting a GPU crash that had nothing to do with my code. The idea: a webcam watches your hand, MediaPipe tracks the landmarks, and three gestures map to three actions on an Anthropic-powered coding agent. Pinch (thumb and index touching) - the agent writes code Spinning your index finger in a circle - the agent brainstorms an idea Two fingers "running" up and down - it runs whatever code it just wrote No keyboard. No prompt box. Just your hand in front of a webcam, like you're a conductor telling an orchestra what to play. The MediaPipe detour I started with MediaPipe's newer Tasks API (HandLandmarker), because it's the one all the docs point you to now. It crashed immediately on my Mac with a Metal/GPU service error, even when I forced it onto the CPU delegate. Spent way too long assuming it was my setup before realizing the new API just doesn't play nice with this machine. Switched to the legacy mp.solutions.hands API, pinned to mediapipe==0.10.21, and the problem vanished. Sometimes the fix for a shiny new API is to not use it yet. Gestures are messier than they sound Detecting "pinch" is easy: measure the distance between thumb and index tip, threshold it, done. The other two took more work. "Running" fingers needed the vertical oscillation of the index and middle fingertips, counted by sign crossings, so it doesn't false trigger on a hand that's just drifting. "Spinning" tracks the index fingertip's trajectory and accumulates the signed angle around a center point, so a real circle reads differently than a shaky hand. Both run on a rolling 1.5 second buffer of landmarks, edge triggered so a gesture fires once, not once per frame. Letting the agent run its own code, unsandboxed, on purpose The runner executes whatever the agent wrote as a subprocess

I created this article for the purpose of entering the All Things Agentic Hackathon. TL;DR — An ADK output_key writes into the session of the agent that declares it. In-process that session is shared, so it looks like state flows. Across a RemoteA2aAgent hop it is the worker's session, and it never comes back. Nothing raises. Nothing warns. Every local run and every CI job exercises the working topology, so the failure is invisible to an offline test suite by construction — including at 100% coverage. The system that passed Bastion is a three-agent access-governance fleet built with Google ADK and A2A. An Orchestrator owns investigation state, an Access Auditor reads production IAM through a read-only identity, and a model-free Escalation Agent delivers validated count-only reviews. The local graph passed its configured core statement and branch coverage gate. Every branch, every seam. Then the same graph was split across deployed A2A workers, and an assumption that looked natural in-process became false. The boundary we had not modeled In-process, the previous step's result is simply there : # The Auditor declares output_key; the Orchestrator reads it back. report = ctx . session . state . get ( AUDIT_FINDINGS_KEY ) Deploy the same sequence and only the construction changes. The graph is identical: RemoteA2aAgent ( name = " access_auditor " , agent_card = card_url ( auditor , " access_auditor " ), description = " Reads the live IAM policy and flags anomalies. Read-only. " , httpx_client = private_a2a_client ( auditor ), a2a_request_meta_provider = _forward_investigation , ) output_key still writes. It writes into the worker's session, which never crosses back. The deployed Orchestrator saw an empty state key while every local run and every test saw a populated one. Observed 2026-08-22: the Auditor completed a full sub-trail, and the next step then refused with "returned no structured report." No exception at the boundary. No warning at construction. The run still r

The AI Wrote the Diff. The Tests Wrote the Verdict. AI refactor suggestions are hypotheses. Not facts. A free coding model rewrites your messy legacy function. The diff looks clean. CI stays green. Then a customer hits an edge case you forgot. This article shows a small workflow. Characterize legacy behavior first. Let the model propose a refactor. Run the same tests against both versions. The verdict: safe or not safe. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Why Characterization Comes First Legacy code has no spec. The only reliable spec is current behavior. Even bugs are behavior. If your refactor changes a bug, you need to know. A characterization test records inputs and outputs. It does not judge right or wrong. It freezes the current contract. After freezing, every difference becomes visible. Step 1: Capture Real Inputs and Outputs Pick one messy function. I used a shipping calculator. Nested conditionals, magic numbers, zero tests. Write a probe script. Call the function with realistic cases. Save outputs as JSON. import json from legacy import calculate_shipping cases = [ { ' items ' : [{ ' weight ' : 2.0 , ' qty ' : 3 }], ' region ' : ' US ' }, { ' items ' : [{ ' weight ' : 0.5 , ' qty ' : 10 }], ' region ' : ' EU ' }, { ' items ' : [{ ' weight ' : 0.2 , ' qty ' : 1 }], ' region ' : ' US ' }, { ' items ' : [{ ' weight ' : 5.0 , ' qty ' : 2 }], ' region ' : ' JP ' }, ] for c in cases : result = calculate_shipping ( c [ ' items ' ], c [ ' region ' ]) print ( json . dumps ({ ' input ' : c , ' output ' : result })) Save output to captured.json . That becomes ground truth. Step 2: Ask the Model for a Refactor MonkeyCode's free model access lets me prompt from the CLI. I gave the model one strict instruction: keep behavior identical. Refactor calculate_shipping into smaller functions. Do NOT change edge cases. Do NOT change rounding. Extract private helpers only. The model returned a diff. It split the function into three he

By Michael Nocito , data analyst · Published August 8, 2026 By the end of this page you can load a CSV into pandas, find out in twenty seconds what type every column became, stop the identifier columns losing their leading zeros, get dates read the way they were written, and turn a money column that arrived as text into numbers. It is about twenty-five minutes, and every output below was produced by running the code. Here is what to do today, the moment after you first load a file. Run df.dtypes . Not df.head() , which shows you what the values look like, but dtypes , which shows you what they are. A column of identifiers that says int64 has already lost its leading zeros, and a money column that says object or str is text that will refuse to add up. The short version: read_csv reads characters and guesses a type per column. The guess is usually right, it is silent when it is wrong, and four arguments replace guessing with instruction. The same characters becoming two different values is the idea, so it gets the picture. The original carries a diagram here. In words: On the left, a strip of five small square boxes holds one character each, reading zero, eight, zero, five, three, as the characters appear in the file. Two arrows branch out from that strip. The upper arrow leads to a strip of five boxes in which the first box is empty, crossed through and outlined in amber, while the remaining four hold eight, zero, five and three; the leading character has been discarded. The lower arrow leads to a strip of five boxes holding zero, eight, zero, five and three, identical to the original, outlined in blue. Both destinations came from the same source strip, and only one of them still contains everything the file did. Every output on this page is real. Run on pandas 3.0.2 against a small CSV built to contain the four problems every real export has: an identifier with leading zeros, ambiguous dates, a text marker for missing values, and money with a thousands separator. If

About a year ago, I was building a terminal-based workflow manager called Glyph.Flow. It was mostly a learning project. I wanted to understand Python better, experiment with Textual, think about commands, state, configuration, logging, and all the small architectural decisions that suddenly appear when a script stops being a script. Somewhere between then and now, the workflows became a little more real. For my Master's thesis, I built a data pipeline to construct and process a cross-national research database from multiple sources. It had a clear purpose: take heterogeneous input data, transform it consistently, validate important assumptions, and produce the dataset I needed for the analysis. And it worked. But this is no longer enough. I am not rebuilding it because the original system failed. I am rebuilding it because the question changed: My Master's thesis needed a pipeline. My PhD will need research infrastructure. And I am slowly discovering that these are not the same thing. A pipeline can be finished There is something comfortable about building software for a well-defined research project. You know the research question. You know most of the variables you need. You know which datasets are involved. You can define the transformations, produce the outputs, validate them, run the analysis, and eventually say: Done. Of course, research is never really that clean. Data sources change. Weird edge cases appear. A country disappears from one dataset. Another source changes a variable name. An indicator turns out to mean something slightly different than you thought. But there is still a boundary around the problem. A PhD changes that boundary. Now I have to think about a system that may need to survive several years of research, new questions I have not formulated yet, datasets I have not discovered yet, and methodological decisions I will probably reconsider more than once. Suddenly, "Does it work?" becomes a surprisingly weak design criterion. The more useful

By Michael Nocito , data analyst · Published August 8, 2026 By the end of this page you can turn transactions into a monthly series, add period-on-period change and a cumulative total, get a share-of-total column, smooth a noisy line, and run all of it separately for every group. It is about twenty-five minutes, and every number below came out of running the code. Here is what to do today, on the series you already have. Count its rows against the number of periods in your date range. If your data covers January to May and the series has four rows, a period produced nothing, it never became a row, and every change figure after the gap is comparing the wrong pair. The short version: pct_change() divides each value by the one in the row above; cumsum() adds everything up to and including the current row. Both trust the rows you gave them to be the periods you meant. What happens when the previous period is zero is the idea, so it gets the picture. The original carries a diagram here. In words: Three bar positions stand on a baseline, labelled Mar, Apr and May. The March position holds a tall bar and the May position holds a slightly shorter tall bar. The April position holds no bar at all; there is only a short flat mark sitting on the baseline where a bar would start, drawn in amber to show a value of zero. An arc runs from the top of the March bar down to the April mark, and the figure minus one hundred percent is printed on it, which is a perfectly ordinary answer. A second arc runs from the April mark up to the top of the May bar, and the symbol printed on that one is not a percentage at all but the sideways figure eight that means infinity. The picture shows that a fall to nothing has an answer and a rise from nothing does not. Every number on this page is real. The sixteen-row orders table used across this whole set of guides, run in pandas 3.0.2. It runs from 5 January to 25 May 2026 and contains no April orders at all, which is not staged for this page; it is