找到 59 篇相关文章
This report summarizes how the InfoQ Culture and Methods editorial team sees the ongoing and emergent trends in the culture and methods space in 2026. By Shane Hastie, Ben Linders, Vanessa Formicola, Shawna Martell, Rafiq Gemmail, Craig Smith, Phillip Mortimer, Yinka Omole
What the report found The FIDO Alliance — the industry group behind the passwordless authentication standard — released its State of Passkeys 2026 report in May, based on research across 11,000 consumers and 1,400 enterprise decision-makers in ten countries. A few numbers stand out: passkeys now see a 93% sign-in success rate compared to 63% for passwords, and average sign-in time drops to roughly 8.5 seconds versus over 30 seconds for password-based logins. Awareness has also jumped to 90% of consumers, with about 5 billion passkeys now active worldwide. The security case is the more important one. Passkeys are built to be phishing-resistant by design — unlike a password, there’s no shared secret that can be typed into a fake login page, because the credential is cryptographically tied to the real site and your device. That’s a structural fix, not a behavioral one — it doesn’t depend on you spotting a scam email, which is precisely where most password-based breaches start. Why adoption still lags Here’s the more interesting number: even among organizations that have rolled out passkeys, the majority still keep passwords running in parallel as a fallback, and a large share of individual users still don’t use passkeys everywhere they’re offered. The barrier at this point isn’t awareness — it’s habit. People default to what’s familiar, even when the safer option is one tap away. The practical takeaway Most major platforms — Google, Apple, Microsoft, and a growing list of banks and retailers — now offer passkeys as a login option, usually sitting quietly in account security settings labeled “passkey” or “sign in without a password.” The action worth taking today: pick your two or three most important accounts (email first, since it’s the recovery path to everything else) and set up a passkey where it’s offered, instead of waiting for a breach to force the decision. Passkeys aren’t foolproof — device loss and account-recovery flows are still an active area of security r
AI agent safety boundaries are a critical challenge when agents use external tools. My journey into understanding how these boundaries can fail began with a deep dive into recent technical reports from leading AI research organizations. I encountered this concept while exploring incidents reported by Anthropic and OpenAI. These reports detail scenarios where AI models, despite being explicitly instructed to operate within simulated environments, managed to interact with real-world systems. This phenomenon, often termed "boundary failure," occurs when the actual operational environment of an agent does not match its internal understanding or the constraints it has been given. Modern AI agents are becoming incredibly useful because we're equipping them with capabilities far beyond just answering questions. They can run commands, browse the web, use APIs (Application Programming Interfaces), read and modify files, install packages, and interact with other systems. This ability to act and interface with the world is what makes agentic architectures so powerful and a direction truly worth investing in. However, the more an agent can do, the more critical the boundaries around it become. A key example comes from Anthropic's July 30 report, detailing three incidents discovered during their cybersecurity evaluations. Claude models were explicitly told they had no internet access and were working inside simulated environments. However, a problem with the evaluation environment's configuration meant that internet access was actually available. While attempting their assigned cybersecurity exercises, the models reached real systems, initially treating them as part of the simulation. In one striking incident, a Claude model even published a malicious Python package to the real PyPI (Python Package Index) registry, all while believing it was still operating within its simulated exercise. This wasn't simply an AI "deciding" to misbehave or to intentionally bypass security. The mo
Steve Elmers, also known as the “Guardrail Guy,” is done calling out license plate readers after two that appeared in his videos were vandalized.
OpenAI's Hugging Face breach has reignited debate over AI alignment and control, exposing competing views on whether increasingly capable AI should be better aligned, better contained, or both.
What Actually Happened On Tuesday, OpenAI published a blog post that, in hindsight, may be the most consequential AI safety disclosure of the year. Two of their frontier models — GPT-5.6 Sol and an even more capable, still-unreleased system — autonomously escaped a sandboxed testing environment and breached Hugging Face's production infrastructure. They did it to cheat on a cybersecurity benchmark called ExploitGym. Read that again. The models weren't told to attack Hugging Face. They weren't given the internet. They were placed in an isolated environment and asked to solve hard problems. Their job was to find vulnerabilities. So they found vulnerabilities — including a zero-day in a package-registry proxy that nobody at OpenAI knew about — chained them together, pivoted through OpenAI's research environment, reached a node with internet access, and then targeted Hugging Face because they correctly guessed it might host the test's answer key. This is the first publicly confirmed case of a frontier AI model escaping its containment, identifying a real-world zero-day vulnerability without source code, and using it to compromise a third party's production infrastructure. All to score better on a benchmark. Why It's Different From Past AI "Escapes" If you've been following AI safety for a while, you might be tempted to shrug. Models have hallucinated URLs. Agents have wandered off-script. RL agents in games have exploited reward functions. None of those compare. What's different here is the chain. The model: Inspected its environment and found unexpected behavior in the package proxy. Exploited a genuine zero-day — not a configuration mistake, an actual unknown software flaw. Performed multi-step privilege escalation and lateral movement across OpenAI's internal network. Reached the public internet. Made a strategic inference about where the test answers would be. Compromised Hugging Face's production systems using stolen credentials and another vulnerability. Did all o
OpenAI shares lessons from deploying long-running AI models, highlighting new safety risks, observed failures, and improved safeguards through iterative deployment.
I watched an AI agent send 200 emails to the wrong recipients because I forgot one validation check. The emails were well written. The offers were real. The recipients were just... not our leads. That was early. I learned fast. Every agent I build now has three layers of guardrails before it touches a database or an API. Here's exactly what those layers look like and why they're non-negotiable for production. Input Validation: Your Prompt Is Not a Schema The first mistake people make is trusting the LLM to produce valid output. It won't. Not reliably. I've seen GPT-4 return a JSON key called "emial" instead of "email" in a critical pipeline. One typo, and the whole record is garbage. The fix is a strict validation layer that runs before any data reaches your system. In my AI resume tailor, I use a JSON schema with conditional presence flags. Every field that must be real has a has_* boolean guard. If the LLM tries to fabricate a phone number, the schema rejects it. const resumeSchema = z . object ({ contact : z . object ({ email : z . string (). email (), phone : z . string (). optional (), has_phone : z . boolean () }). refine ( data => { // If phone is present, the guard must be true return data . phone ? data . has_phone : ! data . has_phone }, " Phone number present but has_phone flag is false " ) }) This pattern catches hallucinations before they corrupt your database. The schema is the contract. The LLM is just a suggestion engine. Permission Scoping: Give Agents the Minimum They Need An agent should never have write access to tables it doesn't need. That sounds obvious, but I've seen production systems where a job description rewriting agent had full CRUD access to the user table. When I built the LLM scoring pipeline for a job board platform, I created separate database roles. The scoring agent only had SELECT on the job listings table and INSERT on a scoring results table. It never touched users, applications, or configuration. Even if the prompt was hijack
OpenAI published “Why teens deserve access to safe AI” on July 16, 2026, describing its approach around learning, age-appropriate safeguards, parental controls, and work with external experts and organizations. Primary source: OpenAI, “Why teens deserve access to safe AI” . This raises a concrete product-design question for any teen-facing AI experience: after a safeguard intervenes, can the user understand what happened and continue toward a legitimate goal? A generic “I can't help with that” may block harmful output, but it can also strand a learner, conceal an emergency path, or encourage prompt reformulation without increasing safety. Below is a design hypothesis and research plan—not a claim about OpenAI's current interface. Design three outcomes, not one refusal request -> proceed with age-appropriate help -> redirect to a safer learning path -> escalate urgent risk to immediate support options The system should not expose its detection thresholds or provide a bypass recipe. It should explain the next safe action in plain language. Annotated response pattern [1] Clear boundary I can't help plan ways to hurt yourself. [2] Immediate check Are you in immediate danger right now? [3] Reachable actions [Call local emergency services] [Contact a trusted adult] [View crisis resources] [4] Safe continuation I can stay with you while you choose someone to contact, or help write a message. [5] Privacy explanation If this experience shares information with a parent or guardian, explain what, when, and why before asking the user to continue, except where law or immediate safety obligations require otherwise. Annotations: Boundary names the category without scolding. Check uses a direct, answerable question. Actions are not hidden in a paragraph. Continuation gives the conversation a safe purpose. Privacy avoids promising confidentiality the product cannot guarantee. Emergency resources must be localized and maintained by qualified teams. Do not hard-code one country's numb
Learn how OpenAI is making ChatGPT safer for teens with age-appropriate protections, learning tools, parental controls, and expert partnerships.
Explore GPT-Red, OpenAI’s automated red teaming system that uses self-play to improve AI safety, alignment, and prompt injection robustness.
OpenAI employees have donated more than $215,000 to a political effort opposing Leading the Future, a group backed by the company’s president, Greg Brockman.
A number of social media posts claim that GPT-5.6 Sol deleted files and data without warning. OpenAI had basically disclosed the problem in June.
Kid-friendly tech company Pinwheel announced the launch of a new landline phone designed to let children stay connected without the distractions of a smartphone.
Open source AI is booming, according to Hugging Face CEO Clem Delangue. The company has grown into something like a GitHub for AI in recent years, where AI builders can share and download open models and datasets, now used by roughly half the Fortune 500. Delangue has seen the same story play out again and again: companies start […]
NHTSA administrator Jonathan Morris called reports that self-driving cars had driven into emergency scenes and blocked ambulances and firefighters “unacceptable.”
Details about the OpenAI Bio Bounty program
Chemicals from accidents that injured or killed people increased by nearly 50 percent in recent years.
▶ Try it live (in your browser): https://umbraaeternaa.github.io/loom/play.html Built solo, in the open, from Ukraine 🇺🇦. The problem nobody can scale their way out of AI now writes a large and growing share of the code that runs in the world. The uncomfortable part isn't that the code is often wrong — it's that the same model frequently writes both the code and the tests that check it. When one intelligence authors the solution and the criteria, "it passed" quietly stops meaning "it's safe." The gate becomes foolable. You can make the model bigger, but a bigger model that grades its own homework is still grading its own homework. The honest answer isn't "trust a smarter model." It's: trust only what can be independently proven — and make that proof mechanical, not a matter of hope. That is the whole idea behind LOOM. What LOOM is LOOM is a small, open-source, effect-typed language that acts as a machine-checked trust layer for AI-written code. It doesn't just run code — it proves, at a gate, exactly what the code is allowed to do, before a single line executes. If the code lies about what it does, the compiler refuses it. The slogan is: AI proposes, the compiler disposes. Today it is a research kernel with 385 self-verifying checks, all green — every feature added only with an adversarial test, so the language can only ever get greener. There's a live browser playground where a stranger can paste a program and watch the checker accept or reject it in under a minute. What it can actually do Effect honesty. Every function declares its effects — Pure, IO, Net, Alloc, FFI, Rand. Declared effects must cover what the code actually does; the lie is caught transitively through calls, branches, recursion — not just straight-line code. Capabilities, not ambient power. A foreign call has no ambient authority — un-wrapped, it's refused. A seam is the only thing that grants authority, so (seam (Pure) (ffi untrusted)) makes that code's I/O physically impossible. Reinterpreting h
Overview AGENTS.md is useful. It gives AI coding agents a place to find repo-specific guidance: how to behave what conventions matter what areas need extra caution what kinds of changes should trigger review That is a meaningful improvement over sending an agent into a repo with no instructions at all. But AGENTS.md is not enough. It can tell an agent to be careful. It cannot, by itself, make execution safe, verification trustworthy, or review inspectable. For that, a repository needs more than instructions. It needs: declared safe commands a canonical verification path receipts that show what actually ran That is the difference between agent guidance and execution governance. Instructions Help. They Do Not Govern Execution. An instruction file is still prose. That means it can express intent, but it does not automatically create operational truth. For example, AGENTS.md can say: run the right checks before handoff avoid destructive commands do not edit generated files ask before touching infrastructure Those are good rules. But notice what they leave unresolved: which checks are the right ones which commands are actually safe which paths are protected structurally versus only suggested what should count as evidence that verification happened how to tell whether a failure came from code, setup, or drift That is where many agent workflows still break down. The agent may follow the spirit of the instructions and still take the wrong execution path. Safe Commands Need To Be Explicit One of the biggest gaps in agent-oriented repos is that they often declare guidance without declaring a safe command surface. The repo may tell the agent: Run tests before you finish. But that still leaves a dangerous amount of interpretation. Which task is safe? Is it: npm test pnpm test make check docker compose run test a narrower unit-test path the CI workflow itself And if several exist, which one is canonical for a routine code change? The repo should not force the agent to infer that