标签：#python

If your app sends email — transactional or marketing — three DNS records decide whether it lands in the inbox or the spam folder: SPF , DKIM , and DMARC . Here's how to look them up and sanity-check them in Python, no third-party API required. Install the one dependency: pip install dnspython SPF: who is allowed to send SPF lives in a TXT record on the domain itself and starts with v=spf1 . import dns.resolver def get_spf ( domain ): for rec in dns . resolver . resolve ( domain , " TXT " ): txt = b "" . join ( rec . strings ). decode () if txt . startswith ( " v=spf1 " ): return txt return None print ( get_spf ( " github.com " )) # v=spf1 ip4:... include:_spf.google.com ~all A quick gotcha worth checking: SPF allows at most 10 DNS-querying mechanisms ( include , a , mx , ptr , exists , redirect ). Go over and receivers return permerror , which quietly breaks authentication: def spf_lookup_count ( spf ): return sum ( spf . count ( m ) for m in ( " include: " , " a: " , " mx: " , " ptr " , " exists: " , " redirect= " )) spf = get_spf ( " example.com " ) if spf and spf_lookup_count ( spf ) > 10 : print ( " ⚠️ SPF exceeds the 10-lookup limit " ) DMARC: the policy that ties it together DMARC is a TXT record on the _dmarc. subdomain and starts with v=DMARC1 . def get_dmarc ( domain ): try : for rec in dns . resolver . resolve ( f " _dmarc. { domain } " , " TXT " ): txt = b "" . join ( rec . strings ). decode () if txt . startswith ( " v=DMARC1 " ): return dict ( kv . strip (). split ( " = " , 1 ) for kv in txt . split ( " ; " ) if " = " in kv ) except dns . resolver . NXDOMAIN : return None print ( get_dmarc ( " github.com " )) # {'v': 'DMARC1', 'p': 'reject', 'rua': 'mailto:...'} The key field is p : none (monitor only), quarantine (spam folder), or reject (bounce). If a domain sends real mail but has p=none , it's not protected against spoofing yet. DKIM: the signature key DKIM is trickier because you need the selector — a label chosen by the sender that lives at SELECT

When you scrape at any real volume, the bottleneck is rarely your code — it's the target site's rate limiting and IP bans. Rotating residential proxies solve this by routing each request through a different real-user IP. Here's how to wire them into requests and Scrapy in Python, including the sticky-session trick most tutorials skip. The proxy URL format A residential proxy is just an authenticated HTTP/SOCKS endpoint. With a pool gateway, you target a country and control session behavior through the username , not separate endpoints: http://USERNAME_country-us_session-a1b2c3_lifetime-30:PASSWORD@proxy.gproxy.net:1000 country-us — exit country (ISO code) session-a1b2c3 — a sticky-session id; reuse it to keep the same IP , change it to rotate lifetime-30 — how many minutes that session's IP stays fixed Basic request through a rotating proxy import requests USER = " USERNAME " PWD = " PASSWORD " def proxy ( country = " us " , session = None , lifetime = 30 ): tag = f " _country- { country } " if session : tag += f " _session- { session } _lifetime- { lifetime } " url = f " http:// { USER }{ tag } : { PWD } @proxy.gproxy.net:1000 " return { " http " : url , " https " : url } # New IP on every call (no session id): r = requests . get ( " https://api.ipify.org?format=json " , proxies = proxy (), timeout = 30 ) print ( r . json ()[ " ip " ]) Run it in a loop and you'll see a different IP each time — the gateway rotates automatically when no session id is present. Sticky sessions: keep one IP across requests Some flows (login, multi-step checkouts, paginated results behind a cookie) break if your IP changes mid-session. Pin the IP by passing a stable session id: import uuid sess = uuid . uuid4 (). hex [: 8 ] # one id for the whole flow p = proxy ( country = " de " , session = sess , lifetime = 30 ) s = requests . Session () s . proxies . update ( p ) s . get ( " https://example.com/login " ) s . post ( " https://example.com/login " , data = {...}) # same exit IP When you

Agents that call tools over a network eventually retry a call they shouldn't have. If the tool isn't idempotent, that retry becomes a duplicate charge, a duplicate email, a duplicate order — quietly, with nothing in the logs to flag it. It's a decades-old distributed-systems problem wearing a new agent costume, and as far as I could tell, nobody had shipped a small, pip-installable fix for the agent-shaped version of it. So I built one. It's called latch . Worth saying up front: this is not a novel idea, and I'd rather say so myself than have someone point it out in the comments. The bug, live Here's the whole thing in one file, no library involved: def charge_card ( order_id , amount ): time . sleep ( 0.4 ) # the payment API is a little slow today ledger [ order_id ] += 1 return { " order_id " : order_id , " status " : " charged " } def agent_charge_with_naive_retry ( order_id , amount , max_retries = 2 ): for attempt in range ( max_retries ): thread = threading . Thread ( target = lambda : charge_card ( order_id , amount )) thread . start () thread . join ( timeout = 0.2 ) # the agent's own client-side timeout if thread . is_alive (): continue # "no response, let's retry" return # got a response The payment call takes 0.4s. The agent gives up waiting after 0.2s and retries. The first attempt is still running in the background — it wasn't cancelled, it can't be safely cancelled, Python threads don't work that way. So it finishes on its own time and charges the card. Then the retry charges it again. The agent's own view of the world is "everything's fine, got a response eventually." The ledger says otherwise. This is examples/naive_agent_example.py in the repo, if you want to run it and watch it happen rather than take my word for it. None of this is exotic. It's the same class of problem as "what happens when a payment gateway's webhook fires twice," which every backend engineer who's touched Stripe has dealt with. The fix has a name — idempotency keys — and it's b

Every "run this model locally" guide tells you to grab a Q4 GGUF and move on. That advice is fine right up until you try a long-context run and your machine starts swapping. The weights are the part everyone budgets for Quantization maths is straightforward. A model's weight footprint is roughly params x bits / 8 : Quant Bits/param 12B model Quality note Q8_0 ~8.5 ~12.8 GB Near-lossless, rarely worth it Q6_K ~6.6 ~9.9 GB Very close to Q8 Q4_K_M ~4.8 ~7.2 GB The usual sweet spot Q3_K_M ~3.9 ~5.9 GB Noticeable degradation Below Q4 the loss stops being subtle. Instruction-following degrades before raw perplexity does, which is why benchmark numbers can look fine while the model quietly stops respecting your system prompt. The KV cache is the part that bites Here is what the guides skip. The KV cache scales with context length , and it is not quantized by default: kv_bytes ~= 2 (K and V) x layers x kv_heads x head_dim x seq_len x dtype_bytes The practical consequence: a model that loads in 7 GB can need well over twice that at long context. Grouped-query attention helps a lot — kv_heads is much smaller than attention heads — but the term still grows linearly with sequence length while your weights stay fixed. Two knobs matter more than picking a fancier quant: --ctx-size : do not allocate 128K if your prompts are 8K. You are reserving memory you will never touch. KV cache quantization ( q8_0 for K/V): roughly halves cache memory for a quality hit most workloads never notice. Underused. A decision order that works Start at Q4_K_M Set context to what you actually use, not the model maximum If you are still tight, quantize the KV cache before dropping to Q3 Only move up to Q6/Q8 if you have headroom left over That ordering matters: dropping to Q3 to buy context is the most common mistake, and it trades a permanent quality loss for memory you could have gotten from the cache instead. Per-quantization benchmarks and deployment notes for the Gemma 4 family are collected at ge

A few weeks ago, I set out to build a small portfolio project: a Streamlit app that could take any tabular dataset, understand something about its structure, and give honest guidance on how to model it. I called it ContextLens . I expected it to be a practical exercise in Python, machine learning, and deployment. What I didn't expect was how closely it would connect with the same questions I work with every day in my PhD research on context-aware intelligent systems. The problem I started with Most introductory machine-learning tutorials follow a familiar sequence: Load a CSV. Choose a model. Train it. Check the accuracy. What often gets skipped is the layer of judgment that should come before any of that: Is this actually a classification problem or a regression problem? Is the target so imbalanced that accuracy becomes misleading? Is that "ID" column secretly leaking the answer into your model? Are there duplicate rows, missing values, high-cardinality categories, or too many features for the number of available observations? Experienced practitioners make these judgments almost automatically. But that reasoning usually remains invisible—it sits in someone's head rather than inside the system, where another person can inspect it. ContextLens is my attempt to make that layer visible. Upload a dataset, and it profiles the data, flags structural risks—missingness, duplicate rows, likely identifier columns, class imbalance, and high-dimensional settings—and adapts its evaluation guidance to what it finds before training a single model. The point is not simply to train a model. The point is to ask whether the modelling process makes sense in the first place. Why I call it "context-aware" rather than "AI-powered" I was deliberate about this distinction, just as I have been throughout my PhD work, and it turned out to be the most important design decision in the whole project. ContextLens does not claim to be intelligent in the way a human expert is. It does not hide its

GitHub热门项目 | 🌐 Make websites accessible for AI agents. Automate tasks online with ease. | Stars: 106,530 | 270 stars today | 语言: Python

GitHub热门项目 | FlashInfer: Kernel Library for LLM Serving | Stars: 6,017 | 6 stars today | 语言: Python

As part of my learning journey with Django, I wanted to build a project that would challenge me beyond the basics. I decided to create RecipeHub, a web application where users can create, manage, and share recipes while exploring recipes from other users. The project started from a Django starter template, but I customized it by adding new features, redesigning the interface, and deploying it online. Features RecipeHub allows users to: Register and log in Create, edit, and delete recipes Browse recipes by category Save favourite recipes Upload recipe images Access a personal dashboard Use the application in both light and dark mode The application is fully responsive, making it easy to use on both desktop and mobile devices. Technologies Used I built the project using: Python Django Django Allauth PostgreSQL Tailwind CSS DaisyUI HTMX Vite Gunicorn Render GitHub was used for version control throughout the project. Challenges One of the biggest challenges was deployment. While everything worked locally, deploying to Render required configuring PostgreSQL, environment variables, and static files correctly. I also encountered an issue with uploaded recipe images. Since the application is hosted on Render's free tier, uploaded media is stored on an ephemeral filesystem, meaning uploaded images are lost after redeployment. Learning why this happens gave me a better understanding of the difference between development and production environments. Another challenge was redesigning the dashboards. I wanted them to feel clean and modern instead of looking like a default Django application, so I spent time improving the layout, spacing, and responsiveness. What I Learned This project helped me improve my understanding of: Django project structure Authentication and user management CRUD operations Database relationships Responsive UI design Git and GitHub workflows Deploying Django applications Debugging real-world issues More importantly, it taught me how to troubleshoot proble

Hey everyone, my name is AbdulRahman Elzahaby ( @egyjs ), a software engineer from Egypt who’s recently fallen down the rabbit hole of LLMs. Like so many of us, you’ll find me in the thick of automation, bots, and making AI work for fast-tracking features and workflows. As I started diving into complex, multi-agent workflow automation, I experimented with… everything. I fiddled with n8n, played with OpenClaw, tinkered with Hermes Agent, andspent what felt like ages manually chaining together Python scripts with LangChain and LangGraph. Every tool showed promise, but my work became increasingly complex and every single workflow somehow felt... Unfinished. The way the industry seems to approach building these kinds of agents revealed a massive, structural gap in tool design. On one hand, we have intuitive, visual workflow tools like n8n. The drag-and-drop interface is great for a bird’s eye view of higher-level logic. But when you need more advanced concepts, like complex looping with conditional logic, custom state reduction, or a system that integrates properly with code review and versioning, these low-code boxes quickly hit limitations. On the other hand, we have powerful code-first frameworks like LangGraph. They provide incredible flexibility and raw execution power, but as soon as you start to build even a simple three-agent triage workflow, the boilerplate code starts to pile up. You have to write custom state schemas (TypedDict), initialize every node function individually, define custom logic for how to route between agents, set up the graph checkpointer, and grapple with environment and dependency management. What seemed missing was a sweet spot - a seamless bridge between the design intuition of visual workflows and the production-ready execution power of code-based frameworks. I wanted a tool that allowed me to simply design a workflow, and then run it efficiently without getting tangled in repetitive boilerplate code. Ultimately, I concluded that what we

I've been building AI agents lately, and honestly, their security model terrifies me. We give LLMs access to powerful tools like shell.run , http.post , and fs.read . But if an agent reads a malicious email or a poisoned webpage, it can be tricked by prompt injection into using those tools to exfiltrate data. Hoping the LLM refuses the attack isn't a real security strategy. So, I built ModelFuzz. It's an open-source Python library that intercepts the tool call at the execution layer. The defense: @shield_tool Instead of trying to filter prompts, ModelFuzz checks the arguments before the tool runs. If it detects a policy violation, like a stolen API key or an unsafe URL, the tool simply doesn't execute. from modelfuzz import shield_tool @shield_tool def send_email ( to_address : str , subject : str , body : str ) -> None : smtp . send ( to_address , subject , body ) Even if the LLM is completely tricked by a prompt injection, the tool never fires. The offense: modelfuzz scan I also built a CLI scanner that red-teams your agent. It fires deceptive prompt injection attacks at your local model to see if it can be tricked into calling a tool. modelfuzz scan --endpoint http://localhost:11434/v1 --model qwen2.5:1.5b I tested it against a local qwen2.5:1.5b model, and it got breached 4 out of 5 times. Try it out It's 100% open source and live on PyPI. pip install "modelfuzz[scan]" GitHub: higagan/modelfuzz Website: modelfuzz.com I'd love to know what security policies you think are missing, or what agent frameworks you want supported next!

Sometimes you export a spreadsheet and the next tool in your pipeline wants JSON instead. Pulling in pandas for that feels like overkill — Python's standard library already has everything needed. Here's a small converter that does it in a handful of lines, no third-party packages required. 1. Reading the CSV csv.DictReader turns each row into a dictionary keyed by the header row automatically — no manual header parsing needed. import csv def read_csv ( csv_path ): with open ( csv_path , newline = "" , encoding = " utf-8 " ) as f : reader = csv . DictReader ( f ) return list ( reader ) Wrapping the result in list() consumes the whole reader up front, which is fine for small-to-medium files and much simpler to reason about than lazily iterating later. 2. Writing the JSON import json def write_json ( rows , json_path ): with open ( json_path , " w " , encoding = " utf-8 " ) as f : json . dump ( rows , f , indent = 2 ) indent=2 keeps the output human-readable, which matters if anyone will actually open the file to sanity-check it. 3. Wiring it together def csv_to_json ( csv_path , json_path ): rows = read_csv ( csv_path ) write_json ( rows , json_path ) return len ( rows ) Returning the row count gives a quick sanity check — if you expected 500 contacts and it says 3, something upstream went wrong before you even open the output file. 4. The full script, start to end import csv import json def read_csv ( csv_path ): with open ( csv_path , newline = "" , encoding = " utf-8 " ) as f : reader = csv . DictReader ( f ) return list ( reader ) def write_json ( rows , json_path ): with open ( json_path , " w " , encoding = " utf-8 " ) as f : json . dump ( rows , f , indent = 2 ) def csv_to_json ( csv_path , json_path ): rows = read_csv ( csv_path ) write_json ( rows , json_path ) return len ( rows ) if __name__ == " __main__ " : count = csv_to_json ( " contacts.csv " , " contacts.json " ) print ( f " Converted { count } rows -> contacts.json " ) 5. Trying it out Given a contact

GitHub热门项目 | 基于Python的开源量化交易平台开发框架 | Stars: 43,785 | 137 stars today | 语言: Python

GitHub热门项目 | The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider. | Stars: 3,351 | 18 stars today | 语言: Python

GitHub热门项目 | A powerful MCP toolkit for coding, providing semantic retrieval and editing capabilities - the IDE for your agent | Stars: 26,786 | 85 stars today | 语言: Python

GitHub热门项目 | Awesome list of 300+ agentic AI resources | Stars: 2,029 | 67 stars today | 语言: Python

Most bibliography failures show up the night before arXiv or the journal deadline: placeholder DOIs, year pasted into volume= , inverted page ranges, invented case reporters. The fix is a paper repo that fails closed from day one . One command pip install https://github.com/SybilGambleyyu/citesure/releases/download/v0.5.68/citesure-0.5.68-py3-none-any.whl citesure init my-paper cd my-paper citesure gate . --preset ci citesure gate . --preset arxiv citesure init writes refs.bib , pre-commit hooks ( gate --preset ci + soft-lint), .github/workflows/citesure.yml , and a short CITESURE.md for coauthors. Empty bibliographies skip hard-ID floors until entries appear. What the gate checks Soft-lint — placeholder number/issue, inverted pages, year-like volume/month/edition, unsafe keys, all-caps titles, missing venues, duplicate DOIs/titles Health — hard-ID coverage floors Promote dry-run — DOIs still buried in url= Live verify — Crossref, doi.org, arXiv, PubMed, Europe PMC, DataCite, OpenAlex, CourtListener Domain packs Fifty-five live-clean packs (demography, sociology, political science, anthropology, ML, law, ecology, …): citesure packs --gate-all citesure packs --run anthropology-classics Evidence: 256/256 integrity · 209/209 claim pairs · 55 packs. Source: github.com/SybilGambleyyu/citesure · Demo: citesure.sybilgambleyyu.workers.dev

I'll guide you through Python fundamentals by comparing concepts with JavaScript. Let's start! 1. Hello World & Basic Syntax JavaScript console . log ( " Hello World " ); let x = 5 ; Python print ( " Hello World " ) x = 5 # No semicolon, no let/const Key Differences: No semicolons in Python Indentation matters (replaces curly braces) Comments use # instead of // 2. Variables & Data Types JavaScript let name = " Alice " ; // string let age = 30 ; // number let isStudent = true ; // boolean let scores = [ 95 , 87 , 91 ]; // array let person = { // object name : " Bob " , age : 25 }; let nothing = null ; let notDefined = undefined ; Python name = " Alice " # str age = 30 # int (or float for decimals) is_student = True # bool (capital T/F) scores = [ 95 , 87 , 91 ] # list (mutable) person = { # dict (dictionary) " name " : " Bob " , " age " : 25 } nothing = None # Python's null/undefined Key Differences: Python uses snake_case (not camelCase) True / False capitalized None instead of null / undefined Lists ≈ Arrays, Dicts ≈ Objects 3. Control Flow JavaScript // If-else if ( age >= 18 ) { console . log ( " Adult " ); } else if ( age >= 13 ) { console . log ( " Teen " ); } else { console . log ( " Child " ); } // For loop for ( let i = 0 ; i < 5 ; i ++ ) { console . log ( i ); } // While loop let count = 0 ; while ( count < 5 ) { console . log ( count ); count ++ ; } Python # If-else (indentation instead of braces) if age >= 18 : print ( " Adult " ) elif age >= 13 : # NOT else if print ( " Teen " ) else : print ( " Child " ) # For loop (more like for...of in JS) for i in range ( 5 ): # range(5) = [0, 1, 2, 3, 4] print ( i ) # Iterate over list (like for...of) for score in scores : print ( score ) # While loop count = 0 while count < 5 : print ( count ) count += 1 # No ++ operator in Python 4. Functions JavaScript // Function declaration function add ( a , b ) { return a + b ; } // Arrow function const multiply = ( a , b ) => a * b ; // Default parameters function greet ( n

Deploy Python Bots on Render Without Sleep Using StayPresent Render is one of the most popular free hosting platforms for small Python projects, but it comes with a well-known catch: free-tier web services spin down after a period of inactivity and require a fresh incoming request to wake back up. For a render python bot — a Discord bot, Telegram bot, or scraper — that "wake up" delay can mean minutes of downtime every time the service goes idle. This guide covers exactly how Render's sleep behavior works, and how to eliminate it using StayPresent . Table of Contents Understanding Render's Free Tier Why HTTP Port Requirements Matter Setting Up StayPresent for Render Health Checks Explained Self-Ping to Avoid Idle Sleep Crash Recovery on Render Full Working Example Best Practices Common Mistakes FAQs Conclusion Understanding Render's Free Tier Render's free web services sleep after roughly 15 minutes without incoming HTTP traffic. Once asleep, the next request has to spin the container back up before it responds, so real users (or a bot's own polling loop) experience a delay. Render also expects your web service to bind to a port it provides via the PORT environment variable — if nothing is listening there, Render's own health checks will consider the deploy unhealthy. [Render Health Check] --HTTP GET--> [Your Service on $PORT] | No response = unhealthy Why HTTP Port Requirements Matter A typical Telegram or Discord bot doesn't open an HTTP port — it just connects outward to Telegram's or Discord's API and waits for events. That's perfectly normal bot behavior, but it fails Render's expectations for a web service. The fix isn't to change how your bot works; it's to run a small HTTP server next to it, purely so Render has something to check. Setting Up StayPresent for Render Install the production extra so Waitress serves the app instead of Flask's development server: pip install staypresent[prod] Then in your entry point (commonly main.py ): import os import staypres

Most code that calls an AI model works like a conversation: ask, wait a second, get a reply. Fine-tuning doesn't. You hand off a job and walk away, checking back every few seconds to see if it's finished. DSPy is a framework for building programs that call language models. Instead of hand-writing and endlessly tweaking prompt strings, you declare what you want in terms of inputs and outputs, and DSPy turns that into the actual prompt. It can even optimize those prompts for you automatically, so getting a better result doesn't mean rewording things by hand. Here's something I didn't know starting out: DSPy already knows how to talk to almost any AI model, Together AI included. Asking a question and getting an answer back is handled by a shared layer that works for everyone, so no new code is needed there. Fine-tuning (the "hand off a job and walk away" thing from the top) is the exception. Every company does fine-tuning its own way, so DSPy needs a small custom piece, called a Provider, to handle each one. Building the Provider for Together AI is what my PR does. Why does this matter? Together AI is one of the cheaper, more popular places to fine-tune open-source models like Llama, so a lot of people building with DSPy end up wanting to use it. Before this, they had to step outside the framework: fine-tune on Together by hand, then wire the finished model back into their DSPy program themselves. With the provider in place, fine-tuning becomes a first-class option. You point DSPy at your training data, and it handles the upload, the job, the waiting, and hands back a model you can drop straight into the rest of your pipeline. That is the whole point of a framework, taking a fiddly manual process and making it one clean step, and adding a provider is how that gets extended to one more company. The Provider does one job from start to finish: take your training examples and hand back a fine-tuned model. Under the hood, that's five steps: Check your training data is in a

If you've ever tried converting a bank statement PDF to CSV and ended up with a jumbled mess of merged cells, missing rows, or split transaction descriptions — you're not alone. This is one of the most common data pain points for accountants, bookkeepers, and anyone who does their own finances. In this post, I'll walk through why this happens, what the right approach looks like, and how to get clean, analysis-ready CSV output from any bank statement. Why Bank Statement PDFs Are So Hard to Parse Bank statements aren't structured documents — they're designed for printing, not data extraction. Here's what generic converters run into: Merged cells : PDF renderers often group date + description + amount into a single visual block. Naive converters pick one cell boundary and split it wrong. Multi-line transactions : A single transaction entry (especially with memos) can span 2–3 lines in the PDF, but gets split into separate rows in the spreadsheet. Negative vs. positive amounts : Debit/credit columns vary by bank.Chase uses a single "Amount" column with negatives for debits. BoA uses two separate columns. A generic converter treats them identically and produces wrong signs. Running balance drift : If even one row is misaligned, every balance figure below it is off. Method 1: Manual Copy-Paste (What You're Probably Doing Now) The baseline. Open the PDF, select all, paste into Excel, then spend 30 minutes fixing column alignment. Pros: Free, no tools required. Cons: Slow (20–40 minutes per statement), error-prone, completely unscalable if you have multiple accounts or months to process. Method 2: Python + pdfplumber For developers who want a scriptable solution: import pdfplumber import csv with pdfplumber . open ( " statement.pdf " ) as pdf : rows = [] for page in pdf . pages : table = page . extract_table () if table : rows . extend ( table ) with open ( " output.csv " , " w " , newline = "" ) as f : writer = csv . writer ( f ) writer . writerows ( rows ) This works reas