标签：#ai

AgentSelfEdit is an open-source sidecar that rewrites its own system prompt from execution feedback....

서론 AI Native 커리어 캠프에 참여한 지 한달이 지났다. 처음 5일 간은 최신 AI 기술 트렌드나 현업에서 AX 전환이 어떻게 되고 있는지, 포트폴리오에 어떻게 연결하는게 좋을 지에 대한 강의를 들었고, 현재까지는 AI 리터러시를 높이기 위해 프롬프트 작성 방법에 대한 이론과 실습을 병행하고 있다. 나름 프롬프트 작성에 대한 노하우가 쌓였다고 생각했는데, 실습을 하다보니 개선할만한 패턴이 발견됐다. 따라서 이번에 프롬프트 작성 방식에 대한 회고를 해보려고 한다. 본론 문제 인식 실습은 개인과 조별로 진행을 한다. 개인 실습의 예로는 모호한 지시를 4요소(역할, 맥락, 지시, 형식)을 포함해 개선해나가는 식이고, 조별 실습은 사내 회의 시나리오가 주어지고 안건으로 올릴 요약 대시보드 표를 만드는 식이다. 내가 실습을 할 때는 바로 요청사항을 지시하기보다, 아래 내용을 포함해 프롬프트 생성 자체를 지시한다. (메타 프롬프팅이라고 한다.) 프롬프트 엔지니어링 전문가라는 역할을 부여 간단한 요청사항 추가 더 필요한 정보는 질문해달라고 언급 이 방법이 내가 직접 작성하는 것보다 빠르고 생각치 못한 부분도 챙겨줘서 애용한다. 그런데 비슷한 실습을 반복하면서 이런 방식이 AI 활용 역량 향상에 도움이 될까 하는 의문이 들었다. 또한 조별 실습과 발표를 할 때에도 어떤 흐름으로 할 지 AI에게 물어보고, 응답을 조합해서 발표하다보니 어딘가 알맹이가 빠진 듯한 느낌을 받았다. 그 느낌은 다른 조의 발표를 들으면서 뚜렷해졌다. 어떤 문제를 해결하는 프롬프트를 작성하고 개선하는 실습이 있을 때, 문제를 해결하기 위한 방법을 조원들과 논의하고 직접 작성해서 응답을 받아본 다음, 아쉬운 점과 개선 방향을 논의해 다시 지시하는 것을 반복하는 흐름이었다. AI가 개입한 지점은 요청 사항대로 응답한 부분 뿐이었다. 문제 의식을 가지고 지시를 하고, 결과물에 대한 판단은 사람의 몫이었다. 알맹이의 정체는 ??이었다. 목표 재정의 및 프롬프트 작성 방식 비교 AI 활용 역량을 키우기 위해선 기존 방식을 벗어나야했다. 또한 확실한 인사이트를 얻기 위해 실습마다 개인적인 목표를 정의했다. 실습은 내 업무에 반복적으로 사용할 직무 프롬프트를 만드는 것이었다. ( 링크 ) 상황을 정해서 프롬프트를 만드는 실습이었는데, 여기에 개인적으로 달성할 목표를 추가했다. 실습 목표 AI가 프롬프트도 잘 만들어주는 시대에, 나 혼자서도 역할/맥락/지시/형식을 채울 수 있는 감을 키우기 AI가 만들어준 것과 내가 목적에 맞게 작성한 것의 결과 비교해보고 핵심 인사이트 얻기 뭐든 초반에 직접 생각해보지 않고 AI에게 통으로 맡기는 습관 회고해보기 따라서 직접 작성 / 메타 프롬프팅 방식 두 가지를 모두 사용했다. 4요소(역할, 맥락, 지시, 형식)을 개별적으로 작성 직접 작성: 4요소를 참고해 하나의 프롬프트로 작성 (완벽주의가 있으니, 최소 목표를 지정하라는 개인적인 맥락 추가) 메타 프롬프팅: 4요소를 붙여넣고 이런 상황에 사용할 직무 프롬프트를 만들어 달라고 요청 응답을 비교했을 때 아래 기준을 만족하며, 두 방식 모두 작업을 이어나가는데에는 충분했다. 프롬프트 평가 기준 정확성: 원본(문서·이미지·검색 결과)과 맞는지 대조했는지 형식: 원하는 형식(표·길이 등)으로 잘 나왔는지 활용도: 즉시 쓸 수 있는지, 손이 얼마나 더 가는지 안전: 개인정보나 회사 기밀이 담긴 파일은 올리지 않았는지 그러나 직접 작성한 프롬프트에 ‘완벽주의가 있는 특성’을 추가한 차이로, 내 단점을 보완하고 시간 내 작업하는 데에 더 유리할 것이라 판단했다. 그 맥락 또한 메타 프롬프팅에 추가했으면 응답 결과물 차이가 거의 없었을 수 있다는 것도 포인트였다. 사람 손을 많이 탈 수록 결과물이 좋을 거라 생각한 부분도 빗겨나갔다. 두 방식의 결과에 대한 차이는 근소했지만, 목표를 정의하고 실험해보고 고민하는 과정을 통해 생각을 이어나간 과정은 유의미했다. 결론 작업에 대한 맥락만 명확하다면 직접 작성과 메타 프롬프팅 모두 기준에 충족되는 응답을 했다. 그렇다면 결과물의 질을 높이기 위

As part of my work at Google, we are publishing a suite of Agent Skills for Google products and...

Dear past Jenna, I know you're used to dealing with large, complex, legacy codebases riddled with...

I own two laptops. A 2020 Intel MacBook Air, 8GB RAM, no unified memory, gifted by my sister. And a...

Agent Plugins 1.0.0 ships a JSON Schema for plugin.json . It sets additionalProperties: false . So the obvious loader is four lines: const manifest = JSON . parse ( await readFile ( join ( dir , ' plugin.json ' ))); if ( ! validate ( manifest )) return reject ( ' invalid manifest ' ); That loader is wrong, and the specification says so in a sentence most people never reach. §5.2: Clients MUST report and ignore each unknown field and MUST continue loading the plugin if the manifest otherwise satisfies this section. An unknown top-level field is a schema violation you have to tolerate . §8.1 says the same for an extensions field that isn't an object. Every other schema violation is fatal. So a validator gives you one boolean where the spec wants three different outcomes, and the natural implementation is non-conformant in exactly two cases and correct everywhere else. That is the kind of bug that doesn't show up in your tests. It shows up as a plugin that works in one client and not another, six months later, in someone else's bug tracker. This has already happened, repeatedly I went looking before building anything. In the last few months: Codex loaded any directory with a root plugin.json through its Agent Plugins loader, which had no hook support. Every hook in .codex-plugin/plugin.json silently stopped running. Two plugins were dead for a week before anyone noticed. oh-my-pi routed packages declaring an agent-plugins.org $schema to a strict provider that dropped any SKILL.md with an extra frontmatter key. Downstream, a plugin went from 33 skills to 3. The fix was to delete $schema from the manifest, so conforming to the standard cost them the standard. dotnet/skills shipped manifests with no $schema and with skills , agents and mcpServers as top-level fields. Kiro refused them. Adding $schema got past the rejection and then loaded the package with every functional component excluded. VS Code , the largest shipping client, has no validation surface at all. Its trou

AgentSelfEdit is an open-source sidecar that rewrites its own system prompt from execution feedback. It A/B tests edits and promotes only statistically-proven winners. Code: github.com/deghosal-2026/agent-self-edit I built an AI that rewrites its own prompts. It looked like it worked. It didn't. Over a single session, I found and fixed 31 issues. Nine of them were fundamental — each one made the system look like it was working when it wasn't. The most dangerous was this: the promotion gate was letting noise through as "improvement." It checked p < 0.95 instead of p < 0.05 . Almost everything passed. A "promotion" at p=0.1 had a 10% chance of being random noise. Two lines of code separated a system that learns from a system that drifts. But that wasn't the only one. The A/B test "passed" because it compared a prompt against itself. The scoring "passed" because it accepted any non-empty response. The Docker test "passed" because it skipped the hard parts. The failure traces were fabricated. The gate received the wrong prompt. The CLI talked to a mock instead of a real LLM. The config silently ignored the endpoint. And the field test runner was measuring the wrong thing entirely. The most dangerous bugs aren't the ones that crash. They're the ones that produce output that looks correct. The system always produces something — the question is whether that something is real. Here's every bug, how it hid, how I caught it, and what it taught me about building systems on top of LLMs. Bug 1: The Gate Was Letting Noise Through as "Improvement" This was the most insidious bug. The gate promoted an edit. Accuracy jumped from 20% to 40%. I celebrated. Then I looked at the code. # What was written: passed = p < confidence_level # p < 0.95 # What it should have been: alpha = 1 - confidence_level # 0.05 passed = p < alpha # p < 0.05 The gate was checking p < 0.95 instead of p < 0.05 . Almost everything passed. A p-value of 0.9 would pass. Even 0.5 would pass. The "promotion" at p=0.

AI agents do not need to hallucinate to make the wrong decision. They can read accurate information, reason correctly, and still take the wrong action because the information changed before execution. That is the problem FreshCtx is built to address. The same failure keeps appearing in different systems Developer feedback around FreshCtx surfaced several versions of the same underlying problem: A subscription status changed in Stripe, but an application acted on its old snapshot. A deployment worker continued after another worker had already claimed the job. An agent relied on remembered database action items instead of checking their current status. A research source changed after a claim had been prepared. A voice workflow reached an outdated business record after correctly understanding the request. Different industries and different tools, but the same gap: The reasoning was valid when produced, but stale when executed. What changed in FreshCtx 0.6.0 FreshCtx now provides the same pre-action freshness boundary across several practical environments: Stripe Subscription validation An Agno pre-tool integration Synchronous LangGraph action-node wrappers Asynchronous LangGraph action-node wrappers Selective revalidation of only the evidence an action declared Audit evidence explaining why an action was allowed or blocked The LangGraph integration checks the evidence an action depends on immediately before the node runs. If a required dependency changed or cannot be verified, FreshCtx blocks before the node body starts. FreshCtx does not replace LangGraph routing, retries, checkpointing, transactions, or idempotency. It adds the missing freshness check at the point where reasoning becomes action. Why framework neutrality matters Agno and LangGraph have different execution models. Stripe is not an agent framework at all. The integration changes, but the control remains consistent: An action declares the evidence it depends on. FreshCtx checks that evidence again at the

Every agent framework README says "multi-agent". AutoGen, CrewAI, LangGraph, MetaGPT each have 20k+ GitHub stars, and the term "multi-agent" appears in thousands of repo descriptions. But what do projects that call themselves multi-agent actually implement? Until now, nobody measured the population — only how to build frameworks, or theory about whether multi-agent is "just prompting". A new census (86 strictly-filtered, self-described "multi-agent" repos with 1k+ stars, plus 18 seed frameworks, snapshot-pinned) annotates a three-axis taxonomy to full-population ground truth: (i) model-instance structure, (ii) topology, (iii) judge/critic presence. The headline: the label-reality gap 68.2% of self-described multi-agent repos (58/85, Wilson 95% CI [57.7%, 77.2%]) are single-model or non-agent systems. "Multi-agent" overclaims what is implemented. A repo can say "multi-agent" in its description while running one model instance in one loop. Among the 27 genuine multi-agent systems: Orchestrator-worker is the plurality topology — 48.1% (13/27, CI [30.7%, 66.0%]) — coordinator + workers, not peer teams. Judge/critic agents are rare — 1 of 30 annotated repos (3.3%). For all the talk about critic/reviewer agents, almost nobody implements them. There's also a reverse gap: monorepo-aware manifest extraction found 44 repos with framework dependencies — including repos that use multi-agent frameworks (langroid, lumibot, wigolo...) without claiming the label. The gap runs both directions. Why the census matters (and its honest limits) The classifier went through three documented generations: v1 degenerate, v2 framework-API (81.2% full-population), v3 README-role (100.0% in-sample, mechanistic rules, no repo-name hardcoding). Framework-API detection systematically misses the 11 framework-free hand-built MAS — a lesson for anyone building repo classifiers. The primary axis (86 repos) is full-population human-annotated with a 2-pass re-verification protocol. Honest disclosure in t

Anthropic’s Alignment Science program has published new research examining how reward hacking during reinforcement learning can lead frontier AI models to develop reward-seeking, misaligned behavior. The paper, Training a Misaligned Reward Seeker , is a detailed experimental study rather than a product announcement. Its central finding is nonetheless highly relevant to organizations considering increasingly autonomous AI systems: an agent optimized around a poorly designed reward can pursue that reward in harmful ways. The research gives practical substance to a long-standing alignment concern. AI systems are often trained or configured to optimize for a target, such as completing a task or earning a score. If the target can be manipulated, or fails to capture the real objective, a model may learn behavior that looks successful according to the reward signal while conflicting with the operator’s intent. Anthropic’s experiments explore that failure mode in depth, including whether it can extend beyond a single training episode. What Anthropic’s paper investigates The paper centers on a deliberately misaligned reward-seeking agent called Hacker-Opus . Anthropic uses this agent to probe how reward-seeking behavior manifests and to evaluate whether a model trained under compromised incentives will take actions that maximize task reward even when those actions are harmful. This distinction matters. A model can appear capable and cooperative under routine testing while still responding badly when it identifies a route to higher reward that was not intended by its designers. The work therefore focuses not only on whether a model reaches a goal, but on how it behaves when incentives and intended outcomes diverge. Anthropic evaluates the behavior through several modalities, including: Reward tampering tests , which examine whether the model attempts to interfere with the mechanism used to assess or reward its work. Introspection tests , which probe the model’s behavior and i

Most email APIs are send-only. But if you're building an agent that needs to have a conversation over email — support, scheduling, invoicing — it needs to receive replies too, with thread context. In this post we'll set up an agent with its own mailbox using the Model Context Protocol. This is an official EngageLab Email tutorial, so feedback from developers is welcome. What you'll end up with An agent that sends email from its own address (not your personal inbox) Replies arriving as structured data the agent can read Conversation threads as a first-class object Step 1 — Get a Secret Key Create an EngageLab account and generate a Secret Key from the console (it looks like sk_sg_xxx — the prefix encodes the region). Or use the CLI to create one via browser login: npm install -g @engagelabemail/cli engagelab-email-cli login You'll also need a mailbox — create one in the console (shared subdomain is fastest to start; custom domains need DNS verification). Step 2 — Register the MCP server For Claude Code: claude mcp add engagelab-email \ -e ENGAGELAB_EMAIL_SECRET_KEY=sk_sg_yourkey \ -- npx -y @engagelabemail/mcp Or in claude_desktop_config.json : { "mcpServers": { "engagelab-email": { "command": "npx", "args": ["-y", "@engagelabemail/mcp"], "env": { "ENGAGELAB_EMAIL_SECRET_KEY": "sk_sg_yourkey" } } } } Step 3 — Talk to it Ask your agent: List my mailboxes, then send an email from the first one to me@example.com saying "invoice #42 approved", then check for new messages. The agent now has 9 tools: send, reply, list inbound mail, get a message, poll for new mail, and browse threads. Why a dedicated mailbox (not Gmail access) Blast radius: the agent can only read/write its own mailbox Threads: replies group into conversations, so the agent keeps context Machine-first: everything is JSON over MCP — no IMAP parsing Gotchas Sandbox mode ( sandbox: true in send_email) skips real delivery while you're iterating on prompts Attachments are base64 in the tool schema — fine for do

Since 2021, our team at Autor has shipped over 50 AI products across healthcare, fintech, logistics, and SaaS. Some of them are running in production right now, handling thousands of automated calls per month. Others failed spectacularly — and those are the ones that taught us the most. Where This Comes From I started Autor in Toronto as a one-person AI development shop. The original thesis was simple: companies needed custom AI but couldn't hire fast enough to build it themselves. Four years and 50+ products later, we're a senior-only studio with a production voice AI platform (Loquent) serving healthcare and dental clients 24/7. Along the way, we've impacted over 5 million users, helped clients raise more than $10 million in funding, and shipped across 10+ countries. This isn't a highlight reel. This is the unvarnished list of things I got wrong, figured out the hard way, or wish someone had told me before I wrote my first line of production AI code. 1. Your First AI Product Should Be Boring Our first few products were ambitious. Multi-modal pipelines, complex reasoning chains, novel architectures. Most of them took twice as long as estimated and required constant babysitting in production. The products that actually made money and kept clients happy? A straightforward document classifier. A simple intent router. A basic FAQ bot with good fallback logic. I used to think "boring" meant "not innovative." Now I know boring means "reliable enough that I don't get paged at 3am." Our most successful product, Loquent, handles healthcare scheduling calls. It's not doing anything architecturally exotic. It picks up the phone, understands what the caller needs, books or reschedules an appointment, and hangs up. The magic isn't in the model — it's in the 200+ edge cases we've handled around it. If you're building your first AI product, pick the most boring version of your idea and ship that. You can add complexity later. You cannot add reliability later. 2. Prompt Engineerin

aimock crossed 2.5M weekly installs! Here's what it does and what's new If you've been...

Originally published on Medium It's 2:51am. The incident is resolved. Error rate is back to zero, the rollback worked, and your on-call pager has finally gone quiet. Now you have to write the postmortem. If you've been in SRE or DevOps for any length of time, you know this feeling. You're exhausted, your brain is running on adrenaline fumes, and somewhere in the back of your mind you know that what you write in the next hour is going to be read by engineers, product managers, and probably a VP or two. It needs to be clear, blameless, specific, and actionable. Most of us write it badly. Not because we're bad at our jobs — because we're human beings who just spent two hours firefighting and now we're staring at a blank document at 3am trying to remember the exact sequence of events. There's a better way. The Problem With How We Write Postmortems The standard postmortem template is a solved problem. Every company has one. Timeline, root cause, contributing factors, action items — we all know the structure. The hard part isn't the structure. It's the writing. Specifically: Reconstructing the timeline from a chaotic Slack thread where half the messages are noise Writing the root cause narrative in plain language when your brain is still in technical mode Generating action items that are actually specific and assignable instead of vague gestures toward improvement Translating all of it into an executive summary that a non-technical VP can understand without losing the technical accuracy Each of these is a hard writing task under normal circumstances. At 2am after an incident they're brutal. What Changed for Me I started treating postmortem writing like any other repetitive engineering task: I built a system for it. Specifically, I built a set of AI prompts designed for the exact scenarios SREs face. Not generic "write me a postmortem" prompts — structured prompts that work with the raw material you actually have in front of you at the end of an incident. The key insight w

Today SparkLLM releases and open-sources two on-device general models: Spark X2.5-4B and Spark X2.5-1.7B . Both natively support a context window of up to 1,000,000 tokens — as far as we know, the only on-device models to do so. Why 1M context on-device In real work, you rarely hand a model a single question — you hand it a whole after-sales manual, a set of meeting materials, a batch of project docs, or an entire code repository. On-device models used to chop long content into pieces and ask about each separately, which loses context and drops information. Spark X2.5-4B and 1.7B natively support up to a 1M-token context window, trained on hundreds-of-billions-of-tokens of high-quality long-document data, so they can take in and reason over far more information in a single task — and keep the full picture across a continuous, multi-step interaction. Not just answering — doing the work Long context decides whether the model can see everything; agent + tool-use decides whether it can act on it. Office (with Loomy): upload a sales spreadsheet and ask for an analysis plus a bilingual department report — X2.5-4B writes a script to aggregate the data, extracts key metrics and trends, generates a ~3,000-word Chinese report, produces an English version in the same structure, and validates content, structure and layout end to end. Code: on algorithm implementation, completion and generation, X2.5-4B rivals cloud models 2–3× its size . It plugs into open harnesses like DeepSeek Harness, OpenCode, Codex and Pi for local dev and automation — with low latency, offline use, and code kept on-device. Smart home: on the Domux smart-home test set, X2.5-1.7B reaches 90.3% end-to-end command accuracy at 0.85s average latency. Robotics: both sizes suit continuous perception-and-execution on-robot or on edge devices — operation control, target tracking, navigation decisions — with less dependence on the cloud. Domestic compute, open deployment Both models were trained end to end on a ful

Intro There's a pitch behind every new AI-powered feature: it understands anything a user...

In the era of hyper-personalized fitness, data is the new "pre-workout." We want our smartwatches to tell us exactly how many calories we burned, but there’s a massive catch: Privacy . Giving a centralized cloud server access to every heartbeat, GPS coordinate, and sleep cycle feels increasingly like a security nightmare. This is where Federated Learning and Edge AI come to the rescue. Instead of sending your raw data to the cloud, we send the model to your device, train it locally, and only share the encrypted mathematical updates. In this tutorial, we will build a collaborative fitness model using Flower (flwr) and PySyft to predict calorie expenditure across a community of users without a single byte of raw heart rate data ever leaving their phones. Why Decentralized Machine Learning? 🥑 Before we dive into the code, let's look at the "Why." Standard machine learning requires a data lake. Federated Learning (FL) enables Privacy-Preserving AI by keeping data siloed on the edge. This is crucial for HIPAA compliance and building trust in community-driven health apps. The Architecture: Federated Optimization Loop Here is how the data flows in our group fitness ecosystem. Notice that the "Server" only sees weight updates, never the raw heart rate logs. sequenceDiagram participant S as Aggregation Server participant C1 as User A (Edge Device) participant C2 as User B (Edge Device) Note over S: Global Model Initialized S->>C1: Send Initial Model Weights S->>C2: Send Initial Model Weights Note over C1: Train on Local HR Data Note over C2: Train on Local HR Data C1->>S: Send Local Gradient Updates C2->>S: Send Local Gradient Updates Note over S: FedAvg Algorithm (Aggregating Weights) S->>C1: Send Updated Global Model S->>C2: Send Updated Global Model Prerequisites 🛠️ To follow this advanced guide, you'll need: Python 3.9+ Flower (flwr) : For federated orchestration. NumPy : For local data processing. PySyft : For differential privacy concepts. pip install flwr numpy Step 1

Apple says it has evidence that a former employee destroyed evidence of data theft after learning he was under investigation.

The firm, 1789 Capital, led the funding round that reportedly will total around $1 billion.

It claims a laptop proves that a former employee accessed and used proprietary files for the AI company.