标签：#Python

Have you ever tried to track your calories by manually searching for "half-eaten avocado toast" in a database? It’s a nightmare. While basic AI Computer Vision can identify an "apple," traditional models often fail at the granular level—distinguishing between 100g and 250g of pasta or identifying hidden toppings in a complex salad. In this tutorial, we are building a high-precision food nutrition AI engine. By combining the Segment Anything Model (SAM) for pixel-perfect object isolation and GPT-4o Vision for multi-modal reasoning and volume estimation, we can transform a simple smartphone photo into a detailed nutritional report. If you’re looking to dive deeper into production-grade AI patterns, I highly recommend checking out the advanced engineering guides at WellAlly Blog , which served as a major inspiration for this architecture. 🏗️ The Architecture: A Hybrid Vision Pipeline To achieve high accuracy, we don't just throw an image at an LLM. We use a "Segment-then-Analyze" pipeline. This ensures the LLM focuses on specific regions of interest (ROIs) rather than getting distracted by the background. graph TD A[User Uploads Food Image] --> B[Pre-processing with OpenCV] B --> C[SAM: Segment Anything Model] C --> D{Multi-Object Masking} D -->|Mask 1: Protein| E[GPT-4o Vision Reasoning] D -->|Mask 2: Carbs| E D -->|Mask 3: Veggies| E E --> F[Nutrient Mapping & Volume Estimation] F --> G[FastAPI Response: JSON Schema] G --> H[Final Dashboard] 🛠️ Prerequisites Before we start, ensure you have your environment ready: Python 3.10+ GPT-4o API Key (OpenAI) SAM Weights ( sam_vit_h_4b8939.pth ) Tech Stack : FastAPI , OpenCV , PyTorch , segment-anything 🚀 Step-by-Step Implementation 1. Object Segmentation with SAM First, we use Meta’s SAM to generate masks. This allows us to "cut out" each individual food item. import numpy as np import cv2 from segment_anything import sam_model_registry , SamPredictor # Initialize SAM sam_checkpoint = " sam_vit_h_4b8939.pth " model_type = "

I recently calibrated a recovery-rate model that had only two weak features. Its point accuracy was almost nothing — R² basically zero. I expected its uncertainty estimates to be junk too. They weren't: the 90% conformal prediction intervals covered ~89% of held-out outcomes. Valid, just wide . That surprised me enough to nail it down, because it contradicts a belief a lot of us carry around: "my model isn't accurate, so I can't trust its uncertainty." For split conformal prediction, that's backwards. Here's the precise statement, a runnable demo, and the one caveat that actually bites. Coverage is a property of the procedure, not the model Split conformal prediction gives a distribution-free, finite-sample marginal coverage guarantee : P( Y ∈ Ĉ(X) ) ≥ 1 − α and it holds for any point model, as long as the calibration and test data are exchangeable. The model is a black box. You fit it however you like, then on a held-out calibration set you take the (1−α) quantile of the absolute residuals, and that quantile becomes the half-width of your intervals. Nowhere does that construction require the model to be good. A bad model just has large residuals, so the calibration quantile is large, so the intervals are wide — wide enough to still cover at the stated rate. Accuracy doesn't buy you validity ; it buys you efficiency (narrower intervals at the same coverage). The demo (numbers are reproducible, seed fixed) Same dataset and target, three models from strong to useless, target coverage 90%: model R² marginal coverage mean interval width gradient boosting 0.741 0.895 5.39 weak linear (1 noisy feature) 0.061 0.905 10.39 predict-the-mean −0.000 0.907 10.83 All three land at ~90% coverage. The only thing that changes is width: the good model's intervals are half as wide . That's the whole story in one table — validity is constant, efficiency tracks accuracy. import numpy as np from sklearn.linear_model import LinearRegression from sklearn.ensemble import GradientBoostingReg

By Sergio Colque Ponce — Software Engineering, Universidad Privada de Tacna. Full source code: github.com/srg-cp/spec-driven-development If you have used an AI coding agent — Copilot, Claude Code, Gemini CLI — you have probably lived this moment: you describe a feature, the agent produces code that compiles and looks right, and then it quietly does the wrong thing. The agent is not weak; the input was ambiguous. We have been treating coding agents like search engines when they behave more like very literal pair programmers. Spec-Driven Development (SDD) is the answer to that problem: instead of jumping straight to code, you write down what you want and why , refine it, and only then let the implementation follow. The specification — not the code — becomes the center of the project. What Spec-Driven Development actually is The idea is old (anyone who has written a Product Requirements Document will recognize it), but it has become practical again thanks to tools like GitHub's open-source Spec Kit . Spec Kit organizes the work into a small set of Markdown artifacts, each feeding the next: Constitution — the non-negotiable principles of the project (security rules, coding standards, architectural constraints). Spec — what you are building and why , with no implementation detail. Plan — the technical blueprint derived from the spec (stack, structure, decisions). Tasks — the plan broken into small, ordered, verifiable steps. Implement — the agent (or you) builds the tasks, with the previous artifacts as structured context. The workflow is usually summarized as Spec → Plan → Tasks → Implement , and the same process is meant to work regardless of language, framework, or which of the 30+ supported agents you use. The real shift is not "more documents." It is this: when requirements change, you update the spec, regenerate the plan, and let the implementation follow — instead of patching code and hoping the intent survives. The spec is a living artifact, not a dusty Word file

Rate Limiting and Circuit Breakers in Distributed AI Systems Distributed AI systems are inherently complex, handling massive volumes of requests, variable latency from model inference, and dependencies on external services like GPU clusters, databases, or third-party APIs. Without proper safeguards, a single misbehaving component or a sudden traffic surge can cascade into system-wide failure. Two fundamental patterns— rate limiting and circuit breakers —provide essential protection. This post explores their roles, implementation strategies, and practical Python examples tailored for AI workloads. Why Distributed AI Systems Need These Patterns Consider a typical AI pipeline: a user sends a prompt, which hits a load balancer, then an API gateway, then an inference service (e.g., a large language model), which may call a vector database or a fine-tuning API. Each component has capacity limits: GPU inference servers can handle limited concurrent requests. External APIs (e.g., OpenAI, HuggingFace) impose rate limits. Database connections are finite. Without rate limiting, a single abusive client can exhaust resources. Without circuit breakers, a failing downstream service can cause cascading timeouts and resource exhaustion across the entire system. Rate Limiting: Controlling Request Flow Rate limiting restricts how many requests a client, user, or service can make in a given time window. It prevents resource starvation and ensures fair access. Common Algorithms Algorithm Pros Cons Token Bucket Smooth burst handling, easy to implement Memory per bucket Leaky Bucket Constant outflow rate, simple Less flexible for bursts Fixed Window Simple, low overhead Boundary spikes (reset issues) Sliding Window Smoother than fixed, accurate Slightly more complex For AI systems, token bucket is often preferred because it allows short bursts (e.g., a user sending a batch of prompts) while maintaining a long-term average. Python Implementation: Token Bucket Rate Limiter import time impor

GitHub热门项目 | Always know what to expect from your data. | Stars: 11,565 | 2 stars today | 语言: Python

GitHub热门项目 | 分享 GitHub 上有趣、入门级的开源项目。Share interesting, entry-level open source projects on GitHub. | Stars: 161,720 | 167 stars today | 语言: Python

GitHub热门项目 | RF-DETR is a real-time object detection and segmentation model architecture developed by Roboflow, SOTA on COCO, designed for fine-tuning. [ICLR 2026] | Stars: 7,853 | 65 stars today | 语言: Python

GitHub热门项目 | General plug-and-play inference library for Recursive Language Models (RLMs), supporting various sandboxes. | Stars: 4,816 | 37 stars today | 语言: Python

GitHub热门项目 | World's first open-source, agentic video production system. 12 pipelines, 52 tools, 500+ agent skills. Turn your AI coding assistant into a full video production studio. | Stars: 4,959 | 71 stars today | 语言: Python

GitHub热门项目 | TimesFM (Time Series Foundation Model) is a pretrained time-series foundation model developed by Google Research for time-series forecasting. | Stars: 21,526 | 84 stars today | 语言: Python

As a backend developer, I sometimes help companies evaluate candidates by reviewing their recorded technical interviews. However, over time, I’ve noticed a deeply ingrained misconception. When discussing memory management or data streaming, many developers explicitly state: "Iterators in Python are inherently eager. If you want true lazy loading or lazy evaluation, you have to use generators and the yield keyword." This misconception is common. Many popular bootcamps and online courses introduce lazy evaluation exclusively through generators . Custom class-based iterators are usually skipped or dismissed as boilerplate-heavy OOP theory rarely used in production Python. This confusion is further reinforced by two common educational simplifications : The List vs. Generator Expression Analogy: Beginners are taught that square brackets [...] (list comprehensions) are eager and take up memory, while parentheses (...) (generator expressions) are lazy. This often creates a false binary mental model: "generators = lazy, everything else = eager." Standard "Textbook" Examples: When courses demonstrate a custom iterator, they usually write a basic class that accepts an already fully loaded list in its __init__ and simply increments an index in __next__ . While this is valid for in-memory data, it leads developers to assume that custom iterators inherently require loading all data upfront. In reality, generators are a specialized language feature designed to implement the iterator protocol automatically . They comply with the exact same interface ( __iter__ and __next__ ). A generator is lazy not because of some magical property of the yield keyword, but simply because it adheres to this underlying contract. To show that custom iterators can be lazy without using any generators or yield keywords, I’ve put together a lightweight and reproducible benchmark. 🧪 The Experiment: Proving Lazy Loading with Custom Iterators Suppose we need to read a database export file ( test_users_db.

json.loads(response) fails at a certain point. You told the model "return JSON only," but it added a ```json markdown code fence around everything. A quick regex strips it — until that regex hits an edge case, and that edge case blows up in production. Since Ollama 0.3.0, passing a JSON schema to the format parameter eliminates this problem at the root. The model's inference itself is constrained by the schema, so no code fences, no explanatory text, no mid-thought artifacts. Just parseable JSON. I ran these tests locally with Gemma4 and Ollama 0.30.7 to see how well it holds up in practice. Why LLM Response Parsing Is Tricky The most common problem when running Ollama locally — without a cloud LLM API — is JSON parsing. Two reasons. First, text generation models are trained toward "natural text." Even if you ask for JSON only, they'll often wrap it in json ... blocks or prepend "Of course! Here is the JSON you requested:" style text. Here's what I reproduced directly: json Input: 'Give me 3 Python tips as JSON with keys: tips (array), difficulty (1-5)' Model output (no format parameter): ```json { "tips": [ "Master the fundamentals first...", ... ] } JSON parse: FAILED Python ' s `json.loads()` can ' t handle the markdown wrapper . The " JSON only " instruction is unreliable in production . Second , speed . I measured the same query both ways : 32 seconds without structured output , 5 seconds with it . More on why below . ## How the Ollama format Parameter Works Ollama ' s `/api/generate` endpoint has a `format` field. Pass a JSON schema object and Ollama applies **constrained decoding** during inference. python import json import urllib.request def ollama_structured(prompt, schema, model="gemma4:e4b"): payload = { "model": model, "prompt": prompt, "format": schema, # ← pass JSON schema object directly "stream": False, "options": {"temperature": 0} } data = json.dumps(payload).encode() req = urllib.request.Request( " http://localhost:11434/api/generate ", data=data

A pip-installable CLI tool that auto-centers off-center page number annotations created by macOS Preview, or batch-adds new ones — with smart content avoidance and landscape support. The Problem If you've ever used macOS Preview to add page numbers to a PDF (via the text annotation tool), you know the pain: numbers land wherever you drop them, never centered, and manually positioning dozens or hundreds of them is soul-crushing. Especially when the PDF has mixed portrait and landscape pages. I ran into this preparing a thesis — 200+ pages of final manuscript, page numbers visibly off-center on every single page. Editing each one by hand wasn't an option. The Solution pdf-pagenum is a single CLI command that reads a folder of PDFs and centers every page number annotation to the bottom of its page. It works by: Detecting FreeText annotations that look like page numbers Measuring body content boundaries on each page Repositioning the annotation to a clean, centered position below the content — with proper margins Preserving the original page dimensions (no resizing, ever) If your PDF has pages with no annotations at all, it can generate new page numbers from scratch in the correct position. Install pip install pdf-pagenum That's it. PyMuPDF and natsort come along as dependencies. Usage Fix Mode (default) Reposition existing page number annotations so they're centered at the bottom: pdf-pagenum ./scans/ ./output/ This is the mode you'll use 90% of the time — it takes whatever rough page numbers Preview gave you and snaps them to the mathematically correct center. Add Mode Generate brand-new page numbers on pages that lack them: # Number all pages starting from 1 pdf-pagenum ./scans/ ./output/ --add all # Number pages 3 through 7 only pdf-pagenum ./scans/ ./output/ --add 3-7 # Number specific pages, starting count from 10 pdf-pagenum ./scans/ ./output/ --add 1,3,5-7 --start 10 Ranges and comma-separated lists can be mixed freely. Start Offset The --start N flag works in b

How I Cut Costs 65% Migrating LangChain to DeepSeek I want to tell you about a switch I made recently that genuinely surprised me. If you're running LangChain in production and haven't explored the DeepSeek models yet, this one's for you. Let me show you what I learned, what broke, and what I'll never go back to. The short version? I was burning cash on a generic LLM setup. I migrated to DeepSeek through Global API's unified interface, and my monthly inference bill dropped by over 60%. Setup took me less time than brewing coffee. Let me walk you through it. Why I Even Looked at This in the First Place Here's the thing about working in AI engineering: the model landscape moves so fast that whatever you chose six months ago is probably overpriced now. That's been my experience, anyway. When I first built my LangChain pipeline, I defaulted to a popular name-brand model because, well, that's what everyone was using. It worked. It was fine. Then I looked at my AWS bill. That's when I started digging into alternatives. And let me tell you, the rabbit hole is deep. Global API alone exposes 184 AI models at prices ranging from $0.01 to $3.50 per million tokens. That's a wild spread. The trick is finding the sweet spot where cost meets quality, and for migration workloads (think: code translation, schema conversion, content rewrites), I found it with DeepSeek. Let me show you the numbers that actually mattered to me. The Pricing Reality Nobody Talks About I built a comparison table when I was making this decision, and I want to share it because staring at these numbers side by side is what convinced me. Here's the lineup I evaluated through Global API: DeepSeek V4 Flash sits at $0.27 per million input tokens and $1.10 per million output tokens, with a 128K context window. That's my default for most production traffic now. Fast, cheap, and smart enough for almost everything. DeepSeek V4 Pro comes in at $0.55 input and $2.20 output with a beefier 200K context. I use this when

WP Maintenance Manager ships from a single Python codebase to both Windows and macOS. "Python is cross-platform — write once, run anywhere," the saying goes. The reality is that the distribution pipeline is completely separate per OS , each with its own pitfalls. PyInstaller / Inno Setup / Apple Notarization / eSigner — the release cycle is a combination of OS-specific toolchains. Here's the full picture, plus what to watch out for at each step. (The choice of internal architecture, Flask + browser UI, is covered separately in why we built a desktop app on local Flask + browser UI ; this post is about distributing that architecture across two operating systems.) The per-OS pipeline at a glance Step Mac Windows Build PyInstaller ( --target-arch x86_64 ) PyInstaller Distribution format .app bundle → .dmg folder → .exe installer Installer creation hdiutil / create_dmg.sh Inno Setup ( .iss script) Code signing codesign + Developer ID certificate eSigner CSC (cloud signing) Pre-distribution validation Apple Notarization SmartScreen reputation buildup Final artifact WP_Maintenance_Pro_X.X.X.dmg WP_Maintenance_Pro_Setup_X.X.X.exe Both OSes share PyInstaller, but the path diverges from there. Mac sits inside Apple's review process; Windows runs through Microsoft's reputation system. They're fundamentally different ecosystems. Mac — PyInstaller → sign → Notarization → DMG The Intel / Apple Silicon trap The first trap in Mac PyInstaller builds is architecture . Running pip install + python build_app.py on an Apple Silicon Mac without thinking produces native binaries (like cffi ) for arm64 only — which then don't run on Intel Macs at all. The fix is to run the entire build through arch -x86_64 : arch -x86_64 pip3 install -r requirements.txt arch -x86_64 python3 build_app.py That produces an .app containing only x86_64 binaries, which runs natively on Intel Macs and through Rosetta 2 on Apple Silicon — a unified distribution. Sign inside-out The .app PyInstaller produces conta

I built a tool called "kaggle-dingdong" that automatically fetches Kaggle competition information and sends notifications to Email, Slack, and Discord. It runs daily on a schedule via GitHub Actions, and you get notified whenever a new competition is published. https://github.com/asherish/kaggle-dingdong Why I Built This Checking the Kaggle competitions page every day is tedious. Featured competitions in particular have entry deadlines, so missing them means losing the opportunity. While RSS feeds and official notification features exist, I wanted notifications delivered directly to the channels I actually use (Discord and Slack), so I built my own. Tech Stack Python 3.13 uv — Package manager and build tool (by Astral) Kaggle Python SDK v2.0.0 — Fetching competition info GitHub Actions — Automated daily execution at 09:00 UTC pytest — Testing Three notification channels are supported: Channel Method Format Email SMTP HTML (card layout) Slack Incoming Webhook Block Kit Discord Webhook Rich Embed Architecture GitHub Actions (cron: daily at 09:00 UTC) ↓ Fetch competition list via Kaggle API ↓ Filter by conditions in config.json ↓ Compare with sent history to extract unnotified competitions ↓ Send notifications to configured channels ↓ Update sent history (max 200 entries) The project structure is as follows: kaggle-dingdong/ ├── src/kaggle_dingdong/ │ ├── __main__.py # Entry point │ ├── config.py # Configuration loading │ ├── competitions.py # Fetch & filter competitions from Kaggle API │ ├── email_sender.py # Email notifications │ ├── slack_sender.py # Slack notifications │ ├── discord_sender.py # Discord notifications │ └── history.py # Sent history management ├── tests/ # pytest tests ├── config.json # Filter configuration └── .github/workflows/ └── notify.yml # GitHub Actions workflow Implementation Highlights Fetching and Filtering Competitions The Kaggle SDK is used to fetch the competition list. In addition to the default sort order, it also fetches with recentl

This article presents a straightforward approach to automatically and efficiently tune hyperparameters for machine learning models using Optuna as the optimisation framework. We explore how to use both Optuna’s native storage options and InterSystems IRIS as a database backend to track the progress of hyperparameter searches. We also show how MLflow can be used to monitor experiments and manage models through its tracking and model registry UI. This article is based on this Kaggle Notebook , which you can run and directly edit yourself. When training ML models, the choice of hyperparameters can strongly influence performance. They are not the only factor, but they can significantly affect both convergence and generalisation. Tuning hyperparameters manually takes a lot of effort. This is especially true because hyperparameters interact with each other, so tuning them independently is usually not enough. For example, higher regularisation may require a lower learning rate for more stable optimization. A more complex model may require stronger regularization to avoid overfitting, but at the same time, a very small learning rate on a complex model can make learning too slow. Optuna is an MIT-licensed open source library, which allows commercial use, that automates hyperparameter search for ML models developed with the most popular frameworks such as scikit-learn, PyTorch, TensorFlow, and LightGBM. It works by defining a search space and an objective metric to either minimize or maximize. Optuna then explores the search space efficiently to find well-performing configurations. Here we use Optuna to tune a LightGBM model on a dummy dataset and show how to scale the search using shared database storage. We will also use MLflow for experiment tracking and model registry, and IRIS DB as a possible Optuna storage backend for concurrent studies. We will use the California Housing dataset, commonly used in ML examples, to populate IRIS tables and run the tuning workflow. Note:

When I started building , I needed reliable access to Instagram data. Like many developers, my first instinct was to use a self-hosted solution such as instagrapi. It worked for experimenting, but once I started depending on it for production workflows, I spent more time maintaining the scraper than building features. Eventually I switched to HikerAPI, a hosted REST API for Instagram. This post isn't about saying one approach is universally better—it's about why it ended up being the better fit for my project. My use case I needed to fetch Instagram profile data for . The requirements were fairly simple: Look up public profiles Process structured JSON Integrate the results into my backend Avoid spending time maintaining login sessions I wasn't interested in reverse engineering Instagram every time something changed. Getting started One thing I liked was that it behaves like a normal REST API. Authentication is done through an x-access-key header, so integrating it into an existing Python backend took only a few minutes. import requests headers = {"x-access-key": "YOUR_KEY"} r = requests.get( " https://api.hikerapi.com/v2/user/by/username?username=instagram ", headers=headers, ) print(r.json()) That's enough to start requesting data and integrating it into your own application. If you want to explore the API, you can find it at HikerAPI. Why I moved away from self-hosted scraping I originally tried , including instagrapi. There wasn't a single issue that made me switch—it was the accumulation of small operational problems: Login sessions expiring Accounts getting challenged Temporary bans Instagram changing internal behavior Regular maintenance after updates None of those problems are impossible to solve. The question became whether solving them was the best use of my time. For my project, the answer was no. I'd rather focus on shipping features than maintaining scraping infrastructure. Tradeoffs Using a hosted API isn't free. Pricing starts at $0.001 per request, wi

Data privacy regulations such as GDPR, LGPD, and HIPAA demand that organizations know exactly where Personally Identifiable Information (PII) lives inside their databases. Yet in practice, most teams rely on manual inventories, tribal knowledge, or external scanning tools that require data to leave the database engine — a process that itself creates privacy and security risks. This article presents an MVP that takes a different approach: it runs PII detection inside InterSystems IRIS using Embedded Python, analyzing data where it lives and never exporting it to an external process. The result is a lightweight, non-intrusive utility that scans your tables, identifies PII using AI, and produces a structured CSV report — all without data ever leaving the IRIS process. The Problem: PII You Don't Know You Have Organizations today face a painful blind spot. A typical IRIS instance may contain hundreds of tables across dozens of schemas, some holding decades of accumulated data. Columns named ContactInfo , Notes , or Description might silently contain social security numbers, email addresses, or government IDs — sometimes intentionally, sometimes as a side effect of free-text fields that capture whatever users type in. Traditional approaches to PII discovery share a common flaw: they require data extraction. You export samples, send them to an external service, or pipe them through a standalone tool. Every step in that pipeline is an additional attack surface and a potential compliance violation. The principle of data sovereignty — keeping data within its jurisdiction and under controlled access — suggests a better path: bring the analysis to the data, not the data to the analysis. This is not just a technical preference; it is a governance requirement: GDPR (EU) — Article 28 requires that any processing of personal data by a third-party processor be governed by a binding contract covering subject-matter, duration, purpose, data types, and obligations [ Art. 28 GDPR ]. Art

An AI agent has two kinds of memory: conversation (semantic) and context (exact reference). Keep them separate with Strands and AgentCore.