🔥 kvcache-ai / ktransformers - A Flexible Framework for Experiencing Heterogeneous LLM Infe
GitHub热门项目 | A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations | Stars: 18,166 | 328 stars today | 语言: Python
找到 12137 篇相关文章
GitHub热门项目 | A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations | Stars: 18,166 | 328 stars today | 语言: Python
Jurig (Sundanese: ghost ) — an autonomous AI agent that haunts your binaries. .-. ██ ██ ██ ██████ ██ ██████ (o o) ██ ██ ██ ██ ██ ██ ██ | u | ██ ██ ██ ██████ ██ ██ ███ | | ██ ██ ██ ██ ██ ██ ██ ██ ██ '~-~' █████ ██████ ██ ██ ██ ██████ autonomous reverse-engineering agent · android · binary · frida Point it at an APK, XAPK, or native binary and it plans, decompiles, searches, hooks, captures traffic, and writes you a report — by itself. In one live run it took a real Android loan app, auto-extracted the XAPK, decompiled 13,367 classes , grepped the sources, and surfaced a hardcoded AES key with a zero IV plus the full API endpoint map — then asked me whether it should go dynamic with Frida. This post is the build story: the architecture, the design bets, and the three bugs that genuinely fought back. Repo: https://github.com/ReverserID/JURIG Why build another agent? Existing "AI reverse engineering" is mostly a pile of MCP servers you wire into a chat client. That's fine, but I wanted something opinionated: Autonomous , not chat — it drives a real toolchain end to end. A single portable binary — no Python venv soup, no MCP daemons. Multi-model — my Claude subscription, OpenRouter, local Ollama, Kimi, Qwen. A TUI that feels like a hacker tool , not a log dump. So: Go. Charmbracelet for the TUI (Bubble Tea + Lipgloss + Glamour). And a hard rule — no MCP . Every capability is a native Go function that shells out to a portable RE binary, or does the work in pure Go. Architecture ┌─ agent loop ─┐ plan → ask scope → recon → locate → dynamic → report │ │ │ LLM router │ anthropic · openai-compat (openrouter/ollama/kimi/qwen) · claude-cli │ │ │ 25+ tools │ jadx · apktool · radare2 · ghidra · frida · adb · proxy │ │ + secret_scan · url_extract · manifest · elf/pe_info · search_code │ │ │ TUI │ animated ghost header · code cards · model picker · NET panel └──────────────┘ One wire format, many providers The whole thing speaks the Anthropic Messages protocol internally. A router a
Python quickstart: nutrition data in 10 lines You can search Dietly's public nutrition catalog with one GET request to https://api.getdietly.com/search . The response is a JSON array of foods, not an object with a results property. Install requests , run the example below, and you have a working nutrition lookup. The rest of this page turns that single call into something dependable: a client class, correct field handling, and retries that respect the rate limit. Make your first request The only required parameter is q (minimum two characters). Use limit to cap results between 1 and 50. Ten is plenty while you explore. import requests r = requests . get ( " https://api.getdietly.com/search " , params = { " q " : " greek yogurt " , " limit " : 5 }, timeout = 10 ) r . raise_for_status () foods = r . json () for food in foods : print ( food [ " name " ], food . get ( " protein_g " )) Understand the fields you get back Each result is a flat object. Nutrient values are per 100 g of the product unless noted, and any nutrient can be null when the source label did not list it. Treat null as unknown, never as zero. Field Meaning id Stable Dietly food ID, used for direct lookups name , brand Product name and brand when known calories_kcal Energy per 100 g protein_g , fat_g , carbs_g Macronutrients per 100 g fiber_g , sugar_g , saturated_fat_g , sodium_mg Detail nutrients per 100 g, often null serving_size_g , serving_desc One serving in grams and its label wording, when provided source , confidence Where the record came from and how sure Dietly is static_url Path to the food's page on getdietly.com, when one exists To convert a per-100 g value to a portion, multiply by grams and divide by 100. For a 150 g serving of a food with 10 g protein per 100 g, that is 10 * 150 / 100 = 15 g. Read the result defensively An empty search is [] , which is a normal outcome rather than an error. Guard for it, and keep null nutrients as null in your own model. food = foods [ 0 ] if foods else
Introduction Linux is the backbone of modern infrastructure. From cloud servers and firewalls to SIEM platforms and security tools, Linux runs silently behind most enterprise environments. For a Security Operations Center (SOC) analyst, understanding Linux is not optional — it is a core skill. One of the most critical security mechanisms in Linux is its file permission and ownership model. Attackers abuse permissions to execute malware, hide persistence, escalate privileges, and erase evidence. SOC analysts rely on permission analysis to detect anomalies, investigate incidents, and build accurate timelines. Become a Medium member This article covers Linux File Permissions and Ownership in deep detail from a SOC analyst’s perspective. It is designed to take you from absolute beginner to security-aware professional, with real-world examples, attack scenarios, and investigation insights. Why Linux File Permissions Matter in SOC In SOC operations, analysts constantly deal with: Authentication logs System logs Application logs Scripts and binaries Configuration files Evidence files during incident response Every one of these objects is protected by Linux permissions. From a SOC perspective: Incorrect permissions = security risk Permission changes = potential indicator of compromise Executable permissions = possible malware Ownership changes = possible log tampering Understanding permissions allows SOC analysts to: Detect unauthorized access Identify privilege escalation Spot malware execution Preserve forensic evidence Reconstruct attacker activity Understanding Linux File Permission Basics Linux follows a Discretionary Access Control (DAC) model. This means: The owner of a file controls who can access it Permissions define what actions are allowed Every file and directory in Linux has: A type Permissions An owner (user) A group These attributes decide: Who can read the file Who can modify it Who can execute it Viewing Permissions Using ls -l The most common command to i
I don’t log in to my LinkedIn account anymore, haven’t done so in years. “AI first this, AI driven that,” alright, alright, I get it! You guys absolutely love AI, and I can’t stand the fact that after almost 5(!!!) years, people still talk about it like it’s the best thing since chicken nuggets. Back when LLMs have started showing somewhat positive results when it came to generating less-than-average code, I came to a devastating conclusion: My friends and colleagues might not actually enjoy coding at all. Maybe they just didn’t enjoy coding the way I do—but either way, I was extremely sad and actively burnt out to find that everybody around me jumped on the prompting bandwagon without giving it much of a critical second thought. I could argue that it was partially because of their employers, but it was definitely because they chose and wanted to. They pressed me “It’s the future, aob2f, we don’t need to write code anymore.” “I just tell Claude to do it, and it’s done.” “I don’t write a single line of code anymore, don’t be left behind.” Some even outrageously claimed “Yeah I reviewed Claude’s 40k lines PR in two days, it was good.” I was baffled. Was I in this sick, absent-minded Truman show spinoff? Were these the same smart, even genius individuals that have built and driven the world’s innovation in the past 30 years with their own hands and minds? It got to a point where I started doubting myself. Maybe it’s really that good? I genuinely gave it a thought. Eventually, I reluctantly gave in and tried vibe coding for the first time… You see, up until then I exclusively used AI as a sophisticated search engine. I asked a technical question—got an answer. More than 50% of the times it was inaccurate or a full-on hallucination, and then I validated the result through trial and error. Admittedly it was better than blindly googling niche bugs and finding nothing but a lonely and vague question from 8 years ago on Stack Overflow. But matter of fact—vibe coding was a mi
A few weeks back I picked up google-gemini/gemini-cli issue #28090: the shell tool was forwarding a command's entire stdout/stderr straight into the model's context, with no cap unless you opted into an LLM-based summarization step. Run one noisy build command and you'd hand the model tens of thousands of tokens of log spam it never asked for. The fix sounded trivial: cap the output before it goes into llmContent . I had a one-liner in my head before I'd even opened the file. That one-liner is exactly the kind of "obviously correct" fix that ships bugs. The one-liner The naive version looks like this: const MAX = 32 * 1024 ; // 32 KiB function truncate ( output ) { if ( output . length <= MAX ) return output ; return output . slice ( 0 , MAX ) + ' \n ...[truncated]... \n ' + output . slice ( - MAX ); } It compiles. It passes a quick manual test with a big ASCII log file. It looks done. I almost committed it as-is before writing the actual test suite. The problem is what .slice() is slicing. JavaScript strings are sequences of UTF-16 code units, not bytes and not Unicode codepoints. Most characters in typical shell output (letters, digits, punctuation) are one code unit each, so .slice() looks safe in casual testing. But the moment real-world command output contains anything outside the Basic Multilingual Plane — an emoji in a commit message, certain box-drawing/progress-bar characters some CLIs use, non-Latin filenames — that character is represented as a surrogate pair : two 16-bit code units that only mean something together. Slice between them and you don't get an error. You get one dangling unpaired surrogate on each side of the cut, silently baked into the string that gets sent to the model. No exception. No lint warning. JSON.stringify on the payload can even throw later, in a completely unrelated part of the request pipeline, for a reason that has nothing to do with where the bug actually is. Or worse: it doesn't throw, and the model just receives a slightly
Most portfolios have a boring "About Me" paragraph. I replaced mine with something you can talk to — an AI terminal that answers questions about me, pulls my live GitHub activity, and remembers the conversation. You can try it right now on my portfolio: rehbarkhan.in . I'm Rehbar Khan , a Full Stack & Gen-AI developer, and in this post I'll break down exactly how it works — the RAG pipeline, the LangGraph agent, the memory layer, and how I wired in real GitHub data with MCP. No fluff, just the architecture. The problem I wanted a portfolio assistant that could: Answer questions about my background accurately — no hallucinated jobs or fake projects. Fetch live data (my latest GitHub activity), not a stale snapshot. Remember the conversation across messages. Stream responses token-by-token like a real terminal. That rules out "just prompt an LLM." You need retrieval for grounding, tools for live data, and state for memory. Here's the stack I landed on. Architecture at a glance Next.js 16 chat UI ──► FastAPI (streaming) ──► LangGraph agent ├── RAG retriever → pgvector (Neon Postgres) ├── GitHub MCP tool → live GitHub data └── Redis checkpointer → conversation memory Frontend: Next.js 16 (App Router, TypeScript) — a streaming terminal UI. Backend: FastAPI with streaming responses. Orchestration: LangGraph ( StateGraph , ToolNode , tools_condition ). LLM: OpenAI gpt-4o-mini . Retrieval: text-embedding-3-small → pgvector on Neon Postgres. Memory: Redis checkpointer for per-session history. Live data: GitHub via the Model Context Protocol (MCP) . 1. The RAG pipeline Everything the bot knows about me lives in a single reference.txt . I chunk it, embed it, and store it in Postgres with pgvector: from langchain_text_splitters import RecursiveCharacterTextSplitter from langchain_openai import OpenAIEmbeddings from langchain_postgres import PGVector splitter = RecursiveCharacterTextSplitter ( chunk_size = 500 , chunk_overlap = 80 ) chunks = splitter . split_text ( open ( " refe
I Built an Open-Source Lab to Learn the OWASP Top 10 for LLM Applications Over the past few months, I've been exploring the security challenges around Large Language Models. While there are plenty of articles explaining prompt injection, system prompt leakage, insecure tool usage, and other LLM vulnerabilities, I kept asking myself one question: Where can someone actually practice exploiting these vulnerabilities? That's what led me to build LLMVault . LLMVault is an open-source, intentionally vulnerable platform that helps developers and security professionals learn the OWASP Top 10 for LLM Applications (2025) through hands-on labs instead of theory. Each lab simulates a vulnerable AI application inspired by real-world LLM attack scenarios. Instead of reading about prompt injection, you'll exploit it yourself, capture flags, understand why it worked, and then review the recommended mitigation. The objective is to bridge the gap between theory and practical AI security. Why I built LLMVault When learning web security, platforms like DVWA, WebGoat, and Juice Shop made learning practical. For AI security, I couldn't find a similar project that was: Open source Self-hosted Free to use Designed around the OWASP LLM Top 10 Built as a hands-on learning environment So I decided to build one. What is LLMVault? LLMVault is a deliberately vulnerable AI application where every challenge demonstrates a real-world LLM security issue. Instead of simply reading about prompt injection or system prompt leakage, you exploit vulnerable AI assistants, capture flags, and learn why the attack works. Each challenge also includes defensive guidance so you understand how to prevent the same issue in production. Features 🛡️ OWASP Top 10 for LLM Applications (2025) 💥 CTF-style challenges 🔍 Realistic AI attack scenarios 📚 Defensive explanations 🐳 Docker support 🔑 No API keys required 💻 Fully offline 🧩 Extensible challenge framework Getting Started Clone the repository: git clone https://github
A salamander can lose a leg and grow a new one. Cut a zebrafish's fin and it simply builds another. Mammals, us included, got the consolation prize: a scar. For a century, biologists assumed that somewhere on the evolutionary road to becoming warm-blooded, fast-moving animals, we traded regeneration away for good. Two research teams working on opposite sides of the planet have just made that assumption look wrong. The headline is almost hard to believe: the ability to regrow lost body parts may not have been deleted from our biology at all. It may simply have been switched off, and switches can be flipped back on. The genetic "remote control" that stopped working The first clue comes from a team at the National Institute of Biological Sciences in Beijing, working with genomics powerhouse BGI-Research. Publishing in Science , they zeroed in on a gene called ALDH1A2 , the instruction sheet for an enzyme that turns vitamin A into retinoic acid, a molecule that acts like a foreman on a construction site, telling cells where to go and what to build during tissue repair. Animals that regenerate freely crank this gene up at the wound site. Mice, it turns out, still carry the same gene. They've just lost the genetic "remote controls," the regulatory DNA that tells the gene to fire after an injury. The hardware is intact; the software command was disconnected somewhere in evolution. So the researchers reconnected it. By reactivating that dormant switch and restoring the flow of retinoic acid, they got mice to regenerate damaged outer-ear tissue, something a normal mouse simply cannot do. In their own words, they had found "a genetic switch involved in the evolution of regeneration." Meanwhile, in Texas, they regrew a limb joint The second piece of evidence lands the point with force. At Texas A&M, a group led by Dr. Ken Muneoka took a different route to the same destination. Instead of editing a genetic switch, they used a precisely timed sequence of two signaling proteins.
Syncthing keeps files synchronized across devices you control, making it one of the most practical tools for a self-hosted knowledge infrastructure that avoids cloud lock-in. Unlike cloud storage platforms, Syncthing uses a peer-to-peer model where each device holds its own copy of synced folders and exchanges changes directly with trusted peers. There is no central server that owns your data, no subscription account, and no vendor lock-in. The project is open-source and community-driven, with more details at syncthing.net . This architecture makes Syncthing especially useful for knowledge workers who manage markdown notes, research documents, PDFs, and project files across a desktop, laptop, home server, and possibly a phone. The tool is simple in concept but requires careful setup to avoid common pitfalls like treating sync as backup or syncing folders that should remain isolated. What Syncthing Is and Is Not Syncthing synchronizes files between two or more devices. Each device maintains its own copy of a folder, and changes propagate between trusted peers. Discovery and relay services may help devices find each other across networks, but the storage model remains local-first. The Syncthing documentation covers installation and configuration in detail. The calm but important opinion is this: Syncthing is excellent when treated as sync infrastructure. It becomes dangerous when treated as backup. It is not a cloud drive. It is not a complete backup system. It is not a collaboration suite. It is a private, peer-to-peer file synchronization tool. Why Syncthing Matters for Knowledge Management Knowledge management is not only about note-taking. It is also about where knowledge lives, how it moves, and whether it remains accessible over time — see the knowledge management guide for the broader picture of tools, methods, and self-hosted platforms this fits into. A useful personal or team knowledge system often contains: markdown notes PDFs and papers diagrams and screens
This is The Stepback, a weekly newsletter breaking down one essential story from the tech world. For more on video games and physical media, follow Jay Peters. The Stepback arrives in our subscribers' inboxes on Sunday at 8AM ET. Opt in for The Stepback here. How it started As a kid, I relished trips to […]
Monitoring is expensive and labor intensive. But it helps public health officials stop outbreaks.
A recent update made video playback the default for songs and podcasts with a visual component. If you just want to listen to music—and save your data and battery—follow these tips.
Keep your phone, laptop, handheld gaming console, and other electronics running with these travel-friendly power banks.
On July 19th, dozens of teams of high school students will begin a five-day, 630-mile road race from Fort Worth to Fort Stockton in Texas. But this is not your typical contest. The students design and build the cars themselves, using off-the-shelf parts and 3D printed materials. The winner is the team that accumulates the […]
Get those lost minerals back with the help of our top electrolyte powders, tablets, drops, and chews for athletes, partiers, and everyone in between.
Google's AlphaEvolve reached general availability on the Gemini Enterprise Agent Platform, turning the DeepMind research project into an evolutionary code optimization service. Evaluators run client-side so code never leaves the customer's infrastructure. Klarna doubled ML training throughput; practitioners note it only works where a measurable evaluation function exists. By Steef-Jan Wiggers
The Great Migration: Moving Beyond the SPA If you have been building in the React ecosystem recently, you've likely started with Vite. It’s fast, the Developer Experience (DX) is unparalleled, and it just works. However, as projects scale, the requirements often evolve. You suddenly need better SEO, faster First Contentful Paint (FCP), or sophisticated server-side logic without managing a separate backend. This is usually when the conversation turns to Next.js. While the migration seems straightforward on paper—it's all just React, right?—the reality involves a fundamental shift in how you think about routing, data fetching, and the browser lifecycle. Here is everything I wish I knew before I made the jump from Vite to Next.js. 1. Routing: From Configuration to Convention In a Vite project, you probably used react-router-dom . You defined a <Routes> component, listed your paths, and mapped them to components. It was explicit and centralized. Next.js (specifically the App Router) uses file-system routing. Every folder in your app directory represents a route segment. The Shift in Thinking Vite: You decide where files live; the router links them. Next.js: The folder structure is the URL structure. You will spend your first few hours moving About.tsx to about/page.tsx . It feels tedious at first, but it eliminates a massive category of "broken link" bugs and makes code-splitting automatic. 2. The "use client" Directive This is perhaps the biggest stumbling block for Vite developers. In Vite, every component is a client component—it runs in the browser. In Next.js, components are Server Components by default. If you try to use useState , useEffect , or browser APIs like window or localStorage in a default Next.js component, your build will crash. You must add the 'use client' directive at the top of the file. Pro-Tip: Don't just add 'use client' to everything. The goal is to keep as much logic as possible on the server to reduce the JavaScript bundle sent to the client.
Astronomers have discovered a rocky exoplanet that has an atmosphere and sits within the habitable zone of a solar system.
This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry . There is a version of this story where I walk you through the technical challenges in a clean, detached voice. Structured. Professional. Composed. This is not that version. Seven months. A hundred pages of documentation written alongside the code, not after it. A live system serving rural clinics in Delta State, Nigeria. And three bugs so specific and so cruel that each one felt designed to find the exact gap between what I thought I knew and what I actually knew. The Delta Health Information and Appointment Booking System was my final year project at the University of Port Harcourt. That sentence makes it sound smaller than it was. This was a real system for real clinics, serving patients who travel for hours to reach a healthcare facility only to find the clinic is full, the doctor is not in, or nobody told them their appointment was cancelled. These are not hypothetical users. These are people whose access to healthcare depends on whether the software works. That is a different kind of pressure than getting a good grade. And it did not start at deployment. Before a single line of code was written, I had to pitch the entire concept to my university supervisor, the Head of Department, the Faculty Dean, and an external examiner. The system had to be defensible not just technically but practically. It had to demonstrate that it solved a real problem that real people in Delta State were actually facing. I was presenting to people with decades of experience who would ask questions I had not thought of yet. That kind of scrutiny either sharpens you or breaks you. I chose to let it sharpen me. The stack was React.js, Node.js, PHP with Laravel, MySQL, AWS, and Docker. Communication channels included SMS gateways, WhatsApp Business API, and USSD because not every patient in rural Delta State has a smartphone. The system had to run on 2G connections, on entry-level Android phones with 2GB RAM