标签：#pens

Are you a "data hoarder" when it comes to your health? Between Apple Health exports, Oura Ring logs, and Garmin CSVs, I found myself sitting on nearly 10 million rows of biometric data. Trying to analyze a multi-year trend of Heart Rate Variability (HRV) or Resting Heart Rate (RHR) in Excel is a one-way ticket to "Application Not Responding" hell. 📉 In this tutorial, we are diving into the world of Quantified Self data engineering. We will leverage DuckDB —the Swiss Army knife of OLAP—and Apache Superset to build a lightning-fast, local-first biometric dashboard. We'll explore how to turn messy JSON/CSV exports into high-performance insights using dbt for modeling and DuckDB for compute. If you've been looking for a way to master Data Engineering for personal use, this is the ultimate "learn in public" project! 🚀 The Architecture: From Raw Export to Real Insights Before we write a single line of SQL, let’s look at how the data flows. We want a system that is modular, fast, and stays entirely on our local machine (privacy first, right? 🥑). graph TD A[Raw Data: Apple Health / Oura / Garmin] -->|CSV/JSON| B(DuckDB Storage) B --> C{dbt Transformation} C -->|Cleaned Views| D[DuckDB Analytical Layer] D --> E[Apache Superset / Grafana] E -->|Visualization| F[Personal Biometric Dashboard] style B fill:#fff,stroke:#333,stroke-width:2px style D fill:#fbbf24,stroke:#333,stroke-width:2px Prerequisites 🛠️ To follow along, ensure you have the following in your tech stack: DuckDB : Our ultra-fast in-process analytical database. dbt-duckdb : For data modeling and transformations. Apache Superset : For the "wow factor" visualizations. Python 3.10+ : To glue it all together. Step 1: Ingesting the "Mess" with DuckDB DuckDB is incredible because it can query CSV and JSON files directly without an ingestion step. Let’s say you have a massive heart_rate.csv from an Apple Health export. Instead of waiting for a traditional DB to "load" the data, we can create a view instantly: -- Create a

TL;DR I designed a proposed AWS Transform Custom community skill that prepares Glue 2.0, 3.0, and 4.0 repositories for Glue 5.0. It separates safe mechanical transformations from changes that require human evidence, generates a migration report, and preserves already-compatible files unchanged. Because I didn't have live atx access, the benchmarks in this post are explicitly labeled manually simulated, not agent-executed. The proposal is open as issue #75 — not yet merged, not yet a pull request. The missing data-engineering transformation AWS Transform Custom can apply agent-driven code transformations across a single repository — or thousands of them at once, via AWS Batch and Fargate. As of July 30, 2026, its public sample repository, aws-samples/aws-transform-custom-samples , contained three community-contributed transformations: an EKS version-upgrade-readiness skill, a JBoss-to-Spring-Boot migration, and a Kubernetes readiness migration. None of them touched data engineering. Given that most of my day-to-day work sits across AWS data engineering, Databricks, and Delta Lake, that gap was the obvious thing to fill. What an AWS Transform Custom "skill" looks like Before writing anything, I studied the deepest existing example, jboss-to-springboot , since the pattern it establishes is effectively an unwritten spec for the other two skills as well: README.md — the problem, what the skill does, and how to invoke it via the atx CLI. This is also where the repo draws a clear line: these are readiness transformations. They modify repository artifacts — code and infrastructure-as-code — but they don't deploy jobs, call AWS APIs to change running resources, or claim data-level equivalence. That distinction matters throughout everything below. SKILL.md — the agent-facing definition: YAML frontmatter with trigger keywords, an Objective, explicit Non-Goals , Constraints, worked before/after examples, a "signal in source code → reference file" routing table, and a numbered V

I went looking for a small improvement to an open-source tool. I found a number that pointed the wrong way, and then I found out why it had to. Live demo — paste your own file and watch it happen: https://pin-on-expand.onrender.com The setup Paritok is a 4B model that compresses AI coding-agent context. It sits between your agent and Anthropic or OpenAI, squeezes the file reads and tool output, and tells you what it saved. It's genuinely good work. Trained on 45,000 real agent trajectories, so it knows a function signature matters more than a debug line. Apache 2.0. Runs on a consumer GPU. Their benchmark numbers hold up. I wanted to build a policy improvement on top of it. To prove my improvement helped, I first had to measure what stock Paritok cost. That measurement is the whole story. Two numbers that disagree One coding-agent session. One 20,005-token file in context. Paritok's own /stats endpoint: 64.0% of input tokens saved. What the provider was actually POSTed: 69.2% more than sending the file with no compression at all. Same session. Same file. Both numbers correct. Where the missing tokens went Paritok is non-destructive by design, which is the good part. Compressed content gets tagged [REF:id] , and when the model needs the exact original it calls an injected expand_context tool to pull it back. Lossy on the wire, recoverable when it counts. The proxy answers that call itself . It appends the full original to a proxy-local thread and POSTs that thread upstream a second time. And stats is computed once, in process_request — before that loop runs. post 0: 6,919 tokens compressed request ← counted by /stats post 1: 26,924 tokens carries the full original ← never counted ───────── billed: 33,843 Then it compounds. The proxy conceals the virtual exchange from the client, so your agent never sees it. Next turn the agent re-sends the original file, Paritok re-compresses it to the same reference, and the model expands it again. Every turn. Forever. In fairness:

There is a number going around that roughly half of all remote MCP servers are dead. I had repeated it myself, in the README of a tool I published. I could not find where it came from, so I measured it. The answer is that about one in ten is actually broken. The "half" figure appears to come from counting servers that require an API key as if they were down. Here is the method and the full breakdown. What I measured On 29 July 2026 I pulled every entry from the official MCP registry — 1,200 servers. Of those, 297 had status: active and advertised a remote endpoint URL (the rest are stdio/local packages with nothing to probe over the network). Each got one anonymous JSON-RPC initialize over streamable HTTP, with a 10 second timeout: { "jsonrpc" : "2.0" , "id" : 1 , "method" : "initialize" , "params" : { "protocolVersion" : "2025-06-18" , "capabilities" : {}, "clientInfo" : { "name" : "mcp-uptime" , "version" : "0.1.0" } } } Then I classified the response: a valid result containing protocolVersion or serverInfo is up, 401/403 is auth-gated, and everything else got bucketed by its actual failure. Results (n = 297) Result Count Share Completed an MCP handshake 133 44.8% Auth-gated (401/403) 134 45.1% DNS failure 8 2.7% Server error (5xx) 6 2.0% Not found (404/410) 5 1.7% Redirect (307/308) 4 1.3% Timeout 2 0.7% Non-MCP response 2 0.7% Other (400, 405, connection) 3 1.0% Reachable: 267 (89.9%). Genuinely broken: 30 (10.1%). Where "half are dead" comes from Look at the first two rows. 55.2% of these endpoints will not complete an anonymous handshake — and that is suspiciously close to the number people quote. But 134 of those 164 are returning a clean 401 or 403. They are running. They are answering. They want an API key, which is a completely reasonable thing for a hosted service to want. Counting those as dead inflates the failure rate by roughly five times. This matters beyond pedantry: if you believe half the ecosystem is rubble, you build defensively against the wron

GitHub热门项目 | Development at the speed of light | Stars: 24,917 | 60 stars today | 语言: Rust

GitHub热门项目 | free online AI resume editor，the only official website is https://magicv.art | Stars: 9,264 | 107 stars today | 语言: TypeScript

GitHub热门项目 | Fast, expressive styling for React. Server components, client components, streaming SSR, React Native—one API. | Stars: 41,126 | 1 star today | 语言: TypeScript

GitHub热门项目 | Blazing fast, instant realtime GraphQL APIs on all your data with fine grained access control, also trigger webhooks on database events. | Stars: 32,058 | 2 stars today | 语言: TypeScript

GitHub热门项目 | The Monorepo Platform that amplifies both developers and AI agents. Nx optimizes your builds, scales your CI, and fixes failed PRs automatically. Ship in half the time. | Stars: 29,157 | 5 stars today | 语言: TypeScript

GitHub热门项目 | Python scraper based on AI | Stars: 28,792 | 44 stars today | 语言: Python

GitHub热门项目 | Native and Compact Structured Latents for 3D Generation | Stars: 9,511 | 412 stars today | 语言: Python

GitHub热门项目 | ASP.NET Core is a cross-platform .NET framework for building modern cloud-based web applications on Windows, Mac, or Linux. | Stars: 38,249 | 5 stars today | 语言: C#

GitHub热门项目 | 12 Weeks, 24 Lessons, AI for All! | Stars: 53,343 | 115 stars today | 语言: Jupyter Notebook

Modern web development has become incredibly powerful. But also increasingly complicated. Many projects begin by installing hundreds of megabytes of dependencies before writing a single page. Frameworks, bundlers, routers, templating systems, CSS tooling, and runtime libraries all solve important problems, but they also introduce additional complexity. I wanted something different. I wanted to build websites that start with plain HTML, while still providing the features developers expect today: Reusable components Markdown support TypeScript CSS and JavaScript bundling Internationalization (i18n) Content collections RSS/Atom feeds Sitemap generation Fast development builds That idea became Rino.js. What is Rino.js? Rino.js is an HTML-first website compiler for building static websites, documentation, blogs, portfolios, company websites, and other content driven projects. Instead of introducing a custom templating language or requiring a frontend framework, Rino.js treats HTML as the primary language. Pages remain valid HTML while additional functionality is added through a small set of build-time conventions. The goal is simple: Write HTML. Generate optimized static websites. Why HTML First? HTML has existed for decades, yet modern web development often treats it as something generated by another language. Rino.js takes the opposite approach. Instead of writing components in JSX or another template language, components are simply HTML files. <component rino-import= "header" ></component> That's all it takes. The compiler replaces the component during the build, producing plain static HTML with no runtime dependency. Starting Rino.js Rino.js has a command that is designed to provide default project. npm create rino@latest Project Shape A Rino.js project usually looks like this: my-site/ rino-config.js dev.js generate.js feed.js sitemap.js backoffice.js pages/ index.html about.html components/ header.html footer.html public/ images/ photo.webp scripts/ export/ app.js

Every recorded mobile test I have ever inherited died the same way: someone moved a button. The recording said "tap at (340, 712)". The redesign moved that button up by one row, and the test kept tapping — now on empty space, or whatever happened to land there instead. It didn't fail right away. Three sprints later, it started failing in confusing ways, and by then nobody trusted the suite anymore. The fix isn't a better recorder. It's recording a different thing: not where you tapped, but what you tapped. That needs an element tree, and for a while we didn't have one. tapflow is an open-source, self-hosted tool that streams iOS simulators and Android emulators into a browser, so a whole team can test builds without installing anything. Until now, everything it moved was pixels in one direction and taps in the other. This post is about getting an element tree out of a simulator with no window, on both platforms. What we do with that tree — replaying flows that survive a redesign — is the next post in this series. The automation axis this feeds — the flow runner and the MCP server — is experimental . The manual browser QA path is the mature one. The constraint: no WebDriverAgent, and no simulator window tapflow already injects touches into the iOS simulator without WebDriverAgent — it loads CoreSimulator.framework and pushes HID events through SimDeviceLegacyHIDClient (that story is ep.1 ). Streaming reads the framebuffer IOSurface directly. Neither path needs Simulator.app on screen, and that's deliberate: an agent Mac in a closet running four simulators shouldn't be babysitting four windows. So whatever we used for the tree had to follow the same rule. No WDA to install and keep in sync with Xcode. No simulator window on screen. Our first attempt ran into exactly that limitation. macOS exposes an accessibility API ( AXUIElement ), and Simulator.app publishes its content through it. We wrote a helper around it, and it worked perfectly on a developer's laptop. On the

I think many of us have already noticed that a growing number of open-source projects and libraries are moving towards commercial or dual-licensing models. In the .NET ecosystem, several widely used libraries have taken this path over the past year or so. AutoMapper and MediatR introduced commercial editions under a dual-licensing model, Fluent Assertions began requiring a paid licence for commercial use with version 8, and MassTransit 9 became a commercial product. These libraries were widely used in .NET applications and I mean widely used. Many projects treated them almost as a standard part of the ecosystem. Now, the same change is reaching the frontend world. PrimeTek recently announced that future major versions of PrimeNG, PrimeReact and PrimeVue will no longer be released as open source. All these projects were widely adopted, and many commercial applications depended heavily on them. Their licensing changes were primarily driven by the cost of long-term maintenance, but this raises a broader question: Is AI also changing the world of open source? You have probably already read many articles about code inflation. With AI, we can generate a huge amount of code in a very short time, even if the quality is sometimes questionable. The same thing is happening in open source. Maintainers can now receive more AI-generated issues, pull requests and feature requests than they can realistically review. Producing code has become cheaper, but understanding, testing and maintaining that code still requires significant human effort. Maintainers can become overwhelmed very quickly. AI may also discourage some developers from publishing their work publicly. Even small experiments, educational repositories and proof-of-concept projects can become training material for large language models. Some authors may therefore decide to keep their repositories private because they do not want AI companies learning from their work without permission, attribution or compensation. Licens

Hey everyone, For the past few months I've been working on a side project called MeshDrop. The idea started because I wanted a simple way to send files and folders directly between my own devices (and with friends) without uploading everything to cloud storage or relying on third-party servers. MeshDrop is built on the Holepunch ecosystem using Pear Runtime, Bare JS, and Hyperswarm. It supports direct transfers over LAN and can also connect over the internet using DHT hole punching with end-to-end encryption. I've also been experimenting with a few extra features like short 8-character pairing codes, cross-device clipboard sharing, and a remote drive feature that's still a work in progress. I'm still learning as I build this project, and I've been using AI coding tools alongside documentation, testing, and a lot of trial and error to help me move faster. I'm trying to understand the code and improve with every feature instead of just generating code and hoping it works. This is very early beta, so please expect bugs, rough edges, missing features, and probably a few questionable UX decisions. I'm sharing it now because I'd rather get feedback early than spend months building something people don't actually enjoy using. If you decide to give it a try, I'd really love honest feedback on things like: Does the overall workflow feel simple or confusing? Is the UI easy to understand? Did you run into any bugs or connection issues? Are there features you'd expect from a P2P file sharing app that are missing? Is there anything that feels unnecessary or poorly designed? Please don't hold back. Constructive criticism is exactly what I'm looking for. If something feels wrong, confusing, or badly designed, I'd much rather hear about it now so I can improve it. GitHub: https://github.com/aamirali51/MeshDrop Latest Beta: https://github.com/aamirali51/MeshDrop/releases/tag/v1.0.0-beta.1 Thanks for taking the time to read this. Whether you try it, report a bug, suggest a feature, o

On May 20, 2026, AWS ended support for DMS Fleet Advisor. Fleet Advisor answered a question every migration team asks first: what is actually in my database estate, and how hard will it be to move? It was free. It was fully managed. It was backed by the largest cloud provider on earth. It still lost. AWS's official notice says only: "After careful consideration, we decided to end support for AWS DMS Fleet Advisor." No reason given. But you don't need one — the documentation tells you. Here is what Fleet Advisor required before it would tell you a single thing about your databases: Install a standalone data collector in your local environment Create an Amazon S3 bucket Create IAM policies, roles, and users — via CloudFormation, which was the recommended path Create database users with the minimum required permissions on every source Establish network access from the collector to each database server Then you'd meet the ceilings: recommendations for up to 100 databases at a time, one-to-one target mapping only, no multitenant server support. Now picture running that gauntlet inside a bank. You are a Business Solution Architect. You have been asked to scope a migration. You do not yet have approval for the migration — that approval is what the assessment is for . And to produce the assessment, you must first request production database credentials, get an agent binary through software approval, provision an S3 bucket, and get an IAM stack past a security review. That is a six-week procurement conversation to answer a question you were hoping to answer this week. AWS's replacement recommendation is Migration Evaluator — a consulting-led engagement. Read that as the finding it is: AWS looked at self-serve migration assessment, and concluded that humans and services do it better than a product. I think they were half right. And the half they got wrong is the interesting part. The lesson: friction is a competitor, and it usually wins We talk about developer tools as if the

AI agents often begin as application code: a prompt, a model call, a few tools, and enough control flow to make the first example run. As the agent grows, the definition and the environment tend to collapse into each other. Model configuration lives beside credentials. Tool access is mixed with prompts. Persistence assumes a particular process. Deployment choices become part of the agent itself. That coupling makes an agent harder to inspect, test, move, and recover. We built Clear Ideas Agent Runtime around a different boundary: the agent definition should be portable, while the infrastructure that executes it should remain under host control. The Agent Manifest is the portable contract An Agent Manifest is a versioned YAML or TypeScript definition. It can describe: prompts and structured outputs; typed variables; conditions and loops; tools and MCP connections; approvals and webhooks; sandboxed code steps; sub-runs; limits and final outputs. A separate Agent Run Manifest supplies the values and execution choices for one invocation. That keeps the reusable agent definition distinct from the inputs and operational decisions associated with a particular run. Here is a small example: schemaVersion : " 1.0" name : research-brief variables : topic : type : string researchNotes : type : string briefDraft : type : string steps : - id : research type : prompt prompt : | Research {{ topic }} and return concise notes. outputVariable : researchNotes - id : draft type : prompt prompt : | Draft a brief using these notes: {{ researchNotes }} outputVariable : briefDraft The manifest describes the agent. It does not contain the credentials, infrastructure account, or persistence implementation that happens to run it. The host controls the operational boundary The host application supplies and controls: models and provider credentials; connections and tool authorization; persistence and artifact stores; local or remote compute; sandbox providers; telemetry; concurrency and resource

Missing data is not the same thing as a convenient default. That sounds obvious, yet temporal software regularly converts an empty time field into midnight, noon, the current time, or the start of a day. The interface may look complete after that conversion, but the program has silently changed an unknown fact into a known one. This matters anywhere an hour can change the result: medical timelines, transport schedules, legal deadlines, astronomical calculations, historical records, and calendrical systems. I encountered the problem while working with a BaZi calculation pipeline. A BaZi chart can use year, month, day, and hour components. If the birth time is absent, the honest result is a three-component analysis with hour-dependent conclusions withheld. Inserting noon would make the output look richer while making its provenance weaker. The useful engineering question is not “Which fallback time should we choose?” It is “How do we keep uncertainty visible through every layer of the system?” The public calculation evidence repository provides the concrete calendar-domain fixtures referenced below. The rest of this article focuses on the reusable software boundary behind them. Model knowledge, not just a string A common input model makes absence too easy to erase: const birthTime = form . time || " 12:00 " ; After this line runs, downstream code cannot tell whether noon came from the user or the fallback. Validation, analytics, caching, and the result renderer all see the same string. The information loss happens before the calculation begins. A small discriminated union keeps the two states separate: /** * @typedef {{ kind: "known", localTime: string, source: "user" }} * KnownTime * @typedef {{ kind: "unknown" }} UnknownTime * @typedef {KnownTime | UnknownTime} BirthTime */ function parseBirthTime ( value ) { const normalized = value ?. trim (); return normalized ? { kind : " known " , localTime : normalized , source : " user " } : { kind : " unknown " }; } This typ