标签：#Python

Building a real-time crypto trading bot sounds like a weekend project — until exchange APIs return cryptic errors, WebSocket connections drop mid-trade, and rate limits turn your strategy into a debugging nightmare. After building my own bot from scratch, I learned that reliability is what separates a hobby script from a system that actually survives in production. This guide walks through the entire process: modular bot architecture, a real-time trading loop, plug-in strategies, backtesting, paper trading, and deployment to a $5 VPS — with production reliability patterns baked in from day one. Full source code included — the free AlgoTrak Backtest Lab on GitHub has 5 classic strategies, a complete backtesting engine, and Jupyter notebooks to get started immediately. Architecture Overview Before writing code, here's the modular structure we'll build: crypto_bot/ ├── strategies/ │ ├── rsi_strategy.py │ ├── macd_strategy.py │ └── ... # Plug in your own ├── core/ │ ├── trader.py # Data fetching + order execution │ └── logger.py # File + DB logging ├── config/ │ └── settings.json ├── cli.py # Entry point ├── bot.py # Main loop └── logs/ Each strategy is a standalone Python class. The trader handles exchange communication. The CLI lets you switch between strategies, symbols, and modes (paper vs live) without touching code. Real-Time Trading Loop Here's the core loop that runs every candle interval: while True : df = fetch_ohlcv ( symbol , interval ) signal = strategy . evaluate ( df ) if signal == " BUY " : trader . buy ( symbol , quantity ) elif signal == " SELL " : trader . sell ( symbol , quantity ) sleep ( next_candle_time ()) Three key points: fetch_ohlcv() pulls the latest OHLCV candle data from the exchange Your strategy evaluates the last N candles and returns a signal Orders execute only on valid signals — no guesswork Modular Strategy Example (RSI) Strategies follow a simple class interface. Here's a complete RSI strategy: import pandas as pd import pandas_ta a

Apache Iceberg looked like the answer to everything when we first adopted it. Open format, ACID transactions, time travel, schema evolution. We migrated our Hive tables, ran a few queries, and felt good about life. Three months later, our S3 costs doubled. Queries that used to take 10 seconds were taking 4 minutes. Metadata operations were timing out. Nobody on the team could explain why. That was the beginning of a real education in how Iceberg actually behaves in production. This post covers what I wish someone had told us before we went all-in. The Small Files Problem Is Not Optional Iceberg is append-friendly by design. Every micro-batch write, every streaming insert, every incremental load creates new Parquet files. Each file also gets its own metadata entry. After a week of hourly loads, you might have 10,000 files in a single partition where you wanted 20. The result: Iceberg's metadata layer has to plan queries across thousands of file manifests. Planning takes longer than execution. Your 10-second query becomes a 4-minute query, and your users start filing tickets. Fix: automate compaction from day one. In Spark, compaction is called rewrite_data_files . The basic call looks like this: -- Run this on a schedule, not on-demand CALL iceberg_catalog . system . rewrite_data_files ( table => 'analytics.events' , strategy => 'binpack' , options => map ( 'target-file-size-bytes' , '134217728' , -- 128MB target per file 'min-input-files' , '5' -- only compact if 5+ small files exist ) ) Target file size of 128MB to 512MB is the practical sweet spot. Smaller than that, you still have too many files. Larger, and your query engines cannot parallelize reads efficiently. If you are not using Spark, PyIceberg exposes compaction through the table maintenance API (as of 0.7.x). For Flink or Trino-only shops, schedule compaction as a separate Spark job. Yes, it is annoying, but it is the right call. Hidden Partitioning Is the Feature You Are Probably Ignoring Old Hive parti

Introduction Most developers today are learning how to “use AI APIs.” But that’s not enough anymore. The real shift happening in software engineering is this: We are moving from building APIs → to building AI-powered systems. And that requires a completely different mindset. The Problem with Most AI Tutorials Most tutorials show this: Call OpenAI API Get response Print output That’s it. But in production systems, this approach fails because it ignores: Context management State handling Reliability Tool integration System design In real applications, AI is not a function call — it is an orchestrated system. What an AI System Actually Looks Like A production AI system usually includes: 1. Input Layer Validation Preprocessing Safety checks 2. Reasoning Layer (LLM) Prompt engineering Context injection Model selection 3. Tool Layer APIs Databases Search engines Internal services 4. Memory Layer Conversation history Vector DB / embeddings User context 5. Output Layer Formatting Validation Response filtering Simple Example: From API Call → AI Agent Thinking Instead of this: response = client . chat . completions . create (...) We design something like this: class AIAgent : def __init__ ( self , llm , tools ): self . llm = llm self . tools = tools def run ( self , user_input : str ): context = self . build_context ( user_input ) response = self . llm . chat . completions . create ( model = " gpt-4o-mini " , messages = context , temperature = 0.2 ) return self . post_process ( response ) Now AI becomes: ✔ structured ✔ extendable ✔ production-ready Key Shift in Thinking Old mindset: “How do I call the model?” New mindset: “How do I design the system around the model?” That’s the difference between: ❌ AI script ✅ AI product system Why Tools Matter More Than Prompts Modern AI systems are not just text generators. They are tool-using systems . Examples: Search APIs (RAG systems) Databases (SQL, NoSQL) External APIs Internal business logic This turns AI from “chatbot” into “agent

Hook: Why Your Next Sports Analytics Project Shouldn't Cost a Fortune Last summer, a college student in Ohio built a machine learning model that predicted NBA player performance with 87% accuracy—without spending a single dollar on data. Meanwhile, a startup in London created a real-time football analytics dashboard that rivaled paid enterprise solutions. The secret? Free sports data APIs. The sports data landscape has transformed dramatically. Where teams once paid six figures for proprietary datasets, developers and data scientists now have access to institutional-quality information at zero cost. Whether you're building a fantasy sports optimizer, analyzing player statistics, or creating predictive models, the barrier to entry has never been lower. But not all free APIs are created equal. Some offer comprehensive historical datasets spanning decades. Others provide real-time updates but limited depth. This guide cuts through the noise and delivers a practical, hands-on review of the best free sports data tools available in 2025. Why Free Sports Data Matters Now More Than Ever The democratization of sports data represents a fundamental shift in the industry. Five years ago, accessing granular sports statistics required partnerships with ESPN, official league APIs, or expensive data brokers. Today's ecosystem has flipped that model. The practical advantages: Lower barriers to entry : Students, hobbyists, and early-stage startups can build sophisticated analytics projects without capital constraints Rapid prototyping : Test hypotheses and validate ideas before investing in premium data services Educational access : Learn data engineering, machine learning, and API integration with real-world sports datasets Competitive alternatives : Many free APIs now compete directly with paid solutions in specific domains The catch? Free doesn't mean unrestricted. Rate limits, update frequencies, and feature sets vary dramatically. Understanding what each tool offers—and its limi

If you've ever tried to pull live options data into a Python script, you've probably hit the same wall I did: the cheapest real-time providers start at $99/mo. Here's how to do it for $20/mo — or free if you stay within 1,000 credits/day. What You'll Need Python 3.8+ requests library ( pip install requests ) An API key from market-option.com (free tier available, no card required) Fetching a Full Options Chain import os import requests API_KEY = os . environ [ " MARKET_OPTIONS_KEY " ] BASE_URL = " https://market-option.com/api/v1 " def get_chain ( ticker : str ) -> list [ dict ]: res = requests . get ( f " { BASE_URL } /options/chain/ { ticker } " , params = { " apiKey " : API_KEY }, ) res . raise_for_status () return res . json ()[ " results " ] contracts = get_chain ( " SPY " ) print ( f " { len ( contracts ) } contracts returned " ) print ( contracts [ 0 ]) Each contract in results looks like this: { "details" : { "contract_type" : "call" , "strike_price" : 530 , "expiration_date" : "2026-01-17" , "ticker" : "O:SPY260117C00530000" }, "last_quote" : { "bid" : 3.45 , "ask" : 3.50 , "midpoint" : 3.475 }, "greeks" : { "delta" : 0.42 , "gamma" : 0.031 , "theta" : -0.18 , "vega" : 0.29 }, "implied_volatility" : 0.182 , "open_interest" : 12418 } Filtering by Expiration and Strike def get_near_the_money ( ticker : str , expiration : str , spot : float , width : float = 0.05 ): """ Return contracts within ±width% of spot price. """ contracts = get_chain ( ticker ) low = spot * ( 1 - width ) high = spot * ( 1 + width ) return [ c for c in contracts if c [ " details " ][ " expiration_date " ] == expiration and low <= c [ " details " ][ " strike_price " ] <= high ] atm = get_near_the_money ( " SPY " , " 2026-01-17 " , spot = 530 ) for c in atm : print ( c [ " details " ][ " strike_price " ], c [ " details " ][ " contract_type " ], c [ " last_quote " ][ " bid " ], c [ " greeks " ][ " delta " ], ) Scanning for High IV Contracts def high_iv_scan ( ticker : str , iv_threshold :

A couple weeks ago I posted about ImageX - a dumb little CLI that lets you edit images without googling "resize image online" for the hundredth time. If you haven't seen it, the tldr is: pip install imagex && imagex , a menu pops up, pick what you want, done. No flags to memorize, no syntax to look up - just arrow keys and enter. Everything runs locally on your machine - no uploads, no server ever sees your images, no browser,no ads. Honestly? Didn't expect much. But: 10 stars on GitHub 1 contributor already dropped a PR (Flip feature - thanks! :) A bunch of feature requests in issues So I kept building. What's new in v0.3.0 Three features, one fix: Grayscale / B&W - luminosity conversion or threshold-based true black & white. Slider for the threshold so you can dial in exactly how much black you want. Invert Colors - instant negative. Handles RGBA without destroying alpha. Flip - mirror horizontally, vertically, or both. (PR from a contributor!) Plus: version check on startup (tells you when to upgrade), and all operations now preserve EXIF/ICC metadata instead of silently dropping it. Yeah, that was a bug. Fixed. The code is still stupid simple NAME = " Invert Colors " DESCRIPTION = " Invert image colors (negative effect) " def run ( file , output_path , args ): img = Image . open ( file ) # ... invert logic ... img . save ( output_path ) return True Links GitHub: github.com/kushal1o1/ImageX PyPI: pip install -U imagex PRs welcome. You could write a feature in the time it took to read this.

GitHub热门项目 | Model Context Protocol (MCP) to enable AI LLMs to trade using MetaTrader platform | Stars: 561 | 8 stars today | 语言: Python

GitHub热门项目 | Multi-harness agentic plugin marketplace for Claude Code, Codex CLI, Cursor, OpenCode, GitHub Copilot, and Gemini CLI | Stars: 37,124 | 43 stars today | 语言: Python

GitHub热门项目 | Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM] | Stars: 51,382 | 152 stars today | 语言: Python

Why I Stopped Picking AI Models by Hype and Started Picking by Speed Three months ago I almost lost a $14,000 retainer because my chatbot felt sluggish. The client didn't say "your TTFT is too high." They said "it feels dumb." That's freelancer code for "users are bouncing and I'm about to find someone else." I rebuilt that bot in a weekend using a model I'd never even heard of six weeks earlier, dropped average response time from 1.4 seconds to under 300ms, and the client renewed for another six months. That single pivot paid for my rent. So I went down a rabbit hole. I ran the same speed test on every model I could get my hands on through Global API's unified endpoint. Fifteen models. Same prompt. Same regions. Ten iterations each. I'm writing this up because if you're billing by the hour or running a side hustle on a shoestring, speed isn't a vanity metric — it's a profit metric. Let me show you what I found. The Setup (How I Actually Ran the Tests) I'm not a researcher with a rack of GPUs. I'm a guy with a M2 MacBook, a $19/mo Hetzner box, and a stopwatch in the form of Python's time.perf_counter() . Here's how I kept it honest. Date window: All tests run on May 20, 2026 Regions tested: US East (Ohio) and Asia (Singapore) Prompt used: "Explain recursion in 200 words" — boring on purpose, because boring prompts are where most apps actually live Output length: Roughly 150 tokens per run Iterations: 10 runs per model per region, average recorded Streaming: Yes, SSE throughout Endpoint: Global API at https://global-apis.com/v1 I measured two things: TTFT (time to first token — the lag before the user sees anything move) and sustained tokens per second (how fast the words actually arrive after that). Both matter. TTFT is the "is this thing broken?" feeling. Tokens per second is the "is this thing fast?" feeling. Here's the script I used, stripped down to the essentials: import time import requests from statistics import mean API_KEY = " your-global-api-key " BASE_URL

Building point-in-time correct, production-grade feature pipelines — from raw Kafka events to online feature serving in milliseconds, using Spark Structured Streaming and the Databricks Feature Store. Table of Contents The Feature Engineering Problem Architecture Overview Feature Store Concepts: ERD Environment Setup Streaming Feature Pipeline Point-in-Time Correct Training Dataset Generation Writing Features to the Online Store Serving Features at Inference Time Feature Table Reference References The Feature Engineering Problem Feature engineering is where most ML projects silently fail in production. Not because the model is wrong — but because the features the model sees at training time are different from the features it sees at inference time . This is called training-serving skew , and it's the #1 silent killer of ML systems. Three specific failure modes cause it: Online/offline inconsistency — the batch pipeline that computes training features uses different logic than the real-time service that computes inference features Data leakage — training features accidentally include information from the future (e.g. joining on a label that was created after the event) Feature staleness — a model trained on 30-day rolling averages is served features that are 6 hours stale because the pipeline backfills are slow The Databricks Feature Store — now part of Unity Catalog as Feature Engineering in Unity Catalog — solves all three by: Storing feature computation logic alongside the data (no drift between training and serving) Enforcing point-in-time lookups during training dataset creation Providing a unified API for both batch offline reads and low-latency online reads Architecture Overview Feature Store Concepts: ERD Understanding the data model behind the Feature Store is essential for designing correct pipelines. Here's how the entities relate: The critical relationship: a Model Version is bound to a Training Set , which records exactly which feature tables and which p

A deep dive into how Spark transforms your SQL into a physical execution plan — and how Databricks layers Adaptive Query Execution and the Photon vectorized engine on top to squeeze out maximum performance. Table of Contents Why Query Optimization Matters The Catalyst Optimizer Pipeline Stage 1: Parsing — From SQL to Unresolved Logical Plan Stage 2: Analysis — Binding to the Catalog Stage 3: Logical Optimization — Rule-Based Rewrites Stage 4: Physical Planning — Strategies and Cost Models Adaptive Query Execution (AQE) The Photon Engine Reading Explain Plans Tuning Reference Table References Why Query Optimization Matters A Spark query written by a human and a Spark query executed by the engine are often very different things. The gap between them — the optimization — is what separates a job that runs in 3 minutes from one that runs in 3 hours on identical hardware. Databricks compounds Spark's native Catalyst optimizer with two additional layers: Adaptive Query Execution (AQE) — re-optimizes the query at runtime using actual statistics collected mid-job Photon — a C++ vectorized execution engine that replaces the JVM-based Spark executor for eligible operators Understanding all three lets you write queries that cooperate with the engine rather than fight it. The Catalyst Optimizer Pipeline Catalyst is Spark's rule-based and cost-based query optimizer. Every query — whether written in SQL, DataFrame API, or Dataset API — passes through the same four-stage pipeline before a single byte of data is read. Stage 1: Parsing — From SQL to Unresolved Logical Plan # ── Catalyst Stage 1: Parsing ───────────────────────────────────────────────── # Spark uses ANTLR4 to parse SQL into an Abstract Syntax Tree (AST). # At this point column names are NOT validated — the plan is "unresolved". from pyspark.sql import SparkSession spark = SparkSession . builder . appName ( " catalyst-demo " ). getOrCreate () # Both of these produce identical internal representations df_api = ( spark .

An AI agent that's flawless in the demo can still fall apart the first time a tool fails in production: a timeout, a network error, a response that comes back corrupted. The fix isn't a smarter model. It's testing the agent against those failures on purpose, before your users do, and hardening it one failure type at a time.

Production vLLM is 100,000+ lines of C++, CUDA, and Python. It powers most of the industry's LLM serving — but reading it cold is brutal. So I built a study series around nano-vLLM , an open-source reimplementation of vLLM's core ideas in ~1,200 lines of pure Python. Every algorithm is visible. Every design decision is legible. It turned out to be the perfect lens for actually understanding how LLMs generate text. The result is an 11-chapter interactive guide. No ML background required — every piece of jargon is explained from scratch with analogies, diagrams, annotated source code, interactive simulators, and quizzes. What it covers: What Is LLM Inference? — tokens, autoregressive generation, Q/K/V attention, HBM vs SRAM Architecture — how 1,200 lines are organised; CPU control plane vs GPU data plane KV Cache — why storing Keys and Values turns O(N²) recomputation into O(1) lookup PagedAttention — virtual memory for the KV cache; how fragmentation wastes 60–80% of GPU memory The Scheduler — continuous batching; keeping the GPU at 95% utilisation instead of 12% Prefill vs Decode — same model, two completely different bottlenecks (compute-bound vs memory-bound) Prefix Caching — skip prefill for shared tokens; ~700ms → ~90ms TTFT Sampling Strategies — greedy, temperature, top-k, top-p, and what each does to the distribution Tensor Parallelism — splitting a model across GPUs; column/row parallel and all-reduce The Optimization Stack — FlashAttention, kernel fusion, CUDA Graphs, torch.compile Benchmarks — measuring honestly; why nano-vLLM matches vLLM on core throughput Each chapter is fully self-contained and interactive. A few of the simulators I'm most happy with: a PagedAttention block allocator you can fill up and watch fragment, a live scheduler you step through token by token, and a sampling playground where you reshape the probability distribution with sliders and sample from it. 🔗 Read the full series: https://ashwing.github.io/vllm-guide/ It's free and open.

Introduction Every serious backend developer eventually faces the same problem: you need to make multiple changes to a database as part of a single business operation, and you need all of them to succeed or none of them to go through. Partial updates are worse than no updates at all - they leave your data in an inconsistent state that can be nearly impossible to debug in production. This is not a new problem. Enterprise developers have been solving it for decades, and Martin Fowler documented the canonical solution in his 2002 book Patterns of Enterprise Application Architecture : the Unit of Work pattern. In this article we are going to go deep on what Unit of Work is, why it exists, how it works internally, and how to build a clean, production-quality implementation from scratch in Python using only the standard library. By the end you will have a working implementation you can adapt to any project, and a solid understanding of how popular frameworks like SQLAlchemy and Django ORM implement this pattern under the hood. The full source code is available on GitHub: 👉 github.com/diegocastillo12/unit-of-work-python - ## Background: What is the Unit of Work Pattern? The Unit of Work pattern is part of Martin Fowler's catalog of Patterns of Enterprise Application Architecture (PoEAA), a collection of battle-tested solutions for common problems in enterprise software design. Fowler defines it as follows: > "A Unit of Work maintains a list of objects affected by a business transaction and coordinates the writing out of changes and the resolution of concurrency problems." Let's unpack that definition carefully. "Maintains a list of objects affected by a business transaction" - this means the Unit of Work acts as a tracker. When your business logic creates a new object, modifies an existing one, or marks one for deletion, it does not immediately write to the database. Instead, it registers the change with the Unit of Work, which keeps an in-memory list of everything that ne

Line AI Chatbot In Production: A CTO's Honest Breakdown Three months ago I was staring at our infrastructure bill wondering where the hell our runway went. We'd been running a customer-facing chatbot powered by a popular "enterprise" AI provider, and the cost curve looked like a hockey stick in the wrong direction. Every new sign-up bled money. I knew we had to make a change before our next board meeting, but I also couldn't afford a six-week migration that would tank our product velocity. What I found surprised me. After running the numbers, testing 184 models through Global API, and stress-testing everything at scale, I cut our inference costs by more than half without touching quality. This isn't a theoretical comparison from a vendor whitepaper. These are the real numbers from my production stack, with my actual users, in my actual platform. If you're a CTO weighing your options for 2026, here's everything I wish someone had told me before I started. Why The Line AI Chatbot Approach Matters Now Most chatbot guides treat AI integration like a toy problem. Send a prompt, get a response, ship the demo. That's fine for a hackathon, but it's not how you run a production system. The questions I care about are different: What's my cost per active user? How do I avoid vendor lock-in? Where's the single point of failure? How fast can I iterate on model choice when something better drops next Tuesday? The Line AI Chatbot framework flips the typical approach. Instead of treating the model as a black box you can't replace, you build a thin abstraction layer over a model-agnostic API. That single architectural decision is what unlocked every other win I describe below. If you're not thinking about model portability on day one, you're going to pay for it later. I learned this the hard way. In 2026, the market has matured to a point where you genuinely have 184 models to choose from, with input prices ranging from $0.01 to $3.50 per million tokens. That's not a marketing line.

I'll be honest — I didn't set out to build a developer tool. I'm an engineer by trade. I build structural and forensic engineering software. C++, WinUI 3, heavy desktop apps. But a big chunk of my prototyping and internal tooling happens in Python — and every time I sat down to spin up a quick Python desktop app, I hit the same wall. Every launcher, every hot-reload tool, every dev cockpit I found wanted something from me. Install this. License that. Set up a virtual environment. Add five dependencies just to watch a file change. I just wanted to run my app, see it update when I changed something, and get back to work. So I built ILX Launcher. The rule I gave myself was simple: pure Python stdlib and tkinter. Nothing else. If it couldn't be done with what Python already ships with, I didn't need it. What came out of that constraint surprised me. No pip install. No virtual environment required. No licensing headaches. You clone it, you run it, it works. That's it. It's a developer cockpit for Python desktop apps — run, hot-reload, test, profile, and ship, all from one place. The kind of tool I wished existed six months ago. It's early. It's rough around the edges. But it works, and it's already saving me time every single day. If you've ever felt like your dev tooling was getting in the way of actually building — I'd love for you to try it and tell me what you think. 👉 github.com/ilxstudio/ILX-Launcher And if it saves you even five minutes — drop a ⭐ on the repo. It genuinely helps others find it.

About A simple calculator built in PyCharm at the age of 14 ! I created this learning solely through YouTube tutorials which allowed me to become an intermediate Python programmer . It helped me learn and understand core programming concepts . This project is part of my journey toward becoming a stronger Python programmer GitHub & Code - https://github.com/muhammadkharwa/Calc.v2 I’m always open to feedback, so feel free to let me know if there’s anything I could improve or any mistakes I may have made.

There's a moment in every ambitious AI engineering project where you convince yourself that more agents means more power. I hit that moment early in building my Python orchestration framework — and I spent several painful weeks learning exactly why that intuition is wrong. The seductive pitch: decompose complex tasks into specialized sub-agents, run them in parallel, let them coordinate. What actually happened was a reliability nightmare that taught me more about agentic architecture than any framework documentation ever could. The Problem I Actually Built My Python orchestration system was designed to automate complex, multi-step workflows — the kind that require planning, research, code generation, and validation to happen in a coherent sequence. Early on, I structured it as a web of parallel agents: a planner, several workers, a validator, and a synthesizer, all exchanging structured messages. On paper it was elegant. In practice, it had three failure modes I couldn't engineer away: Context drift. Each agent only saw the slice of information it was handed. The worker writing one module couldn't see what the worker writing another module had decided. By the time the synthesizer tried to combine outputs, I had conflicting assumptions baked into the results — variable names that clashed, patterns that contradicted each other, interfaces that didn't align. Cascading partial failures. When one agent produced ambiguous output, every downstream agent amplified the ambiguity. A planner that returned a slightly underspecified task description produced workers that each interpreted it differently. Nothing failed loudly. Everything just drifted, quietly, until the final output was incoherent. Debugging opacity. When something went wrong in a parallel multi-agent system, tracing the failure was miserable. Was it the planner? One of the workers? The message-passing layer? I'd rebuilt the worst parts of distributed systems debugging inside a single Python process. The Architec

GitHub热门项目 | Official, AWS-supported MCP servers, skills, and plugins to help AI agents build on AWS | Stars: 958 | 13 stars today | 语言: Python