标签：#bots

Your robots.txt may express an AI policy you did not write. We checked the homepage and robots.txt of 9,037 live AI tools listed on directree on 6 and 7 September 2026. Of those, 945 explicitly disallow OpenAI’s GPTBot in its own user-agent group: 10.5% of the sample. Treat AI crawler rules as deployment configuration. Review them when you change hosting, enable a CDN feature, adopt a starter template, or hand site operations to someone else. Read the full research and methodology . GPTBot, search, and user browsing are separate A common configuration blocks model training while keeping a site available in AI-assisted search and browsing: User-agent: GPTBot Disallow: / User-agent: OAI-SearchBot Allow: / These are separate crawlers with separate purposes. In our sample, 839 of the 945 sites that block GPTBot, or 88.8%, still allow OAI-SearchBot. That is a deliberate and useful distinction if your goal is to opt out of training while remaining eligible to be cited in ChatGPT search. The same pattern appears across AI labs. ClaudeBot is explicitly blocked by 10.1% of the 9,037 tools, while Claude-SearchBot is blocked by just 0.1%. Google-Extended is blocked by 9.9%, but its purpose is also distinct from ordinary Google Search crawling. Do not assume a broad-looking rule has the result you want. Check the actual crawler names and decide which capabilities you want to permit. A safe way to review your file Start by opening the public URL: https://your-domain.example/robots.txt Then look for three things: A named crawler group, such as User-agent: GPTBot . A Disallow: / directly inside that group. A wildcard group, User-agent: * , that could affect all crawlers. Our measurement only counts a site as blocking GPTBot when the named GPTBot group itself contains Disallow: / . This matters because ordinary technical exclusions are widespread. Only 31 sites in the 9,037-site sample, or 0.3%, block every crawler outright. Meanwhile, 44% have a path-level Disallow rule in a wildc

Every small business has one person who is the office search engine. Where is the refund form. What goes in the Friday report. Which supplier do we use for rush jobs. The answers are written down somewhere, but asking that person is faster than finding them, so the questions keep coming and that person never gets a full hour of their own work. An SOP chatbot fixes exactly that. Staff type the question, and the bot answers with the steps from the procedures you already wrote, and shows which document it took them from. Nothing else. This article explains how that works using one picture, shows two bots we actually run, and is honest about what the bot will never do. The box Picture the AI as a very capable temp worker who shows up every morning with no memory of your business at all. Not the address, not the prices, not how you handle a late delivery. Smart, fast, and completely blank. Before you ask them anything, you hand them a box. In the box are your documents. The rule, taped to the lid, says: answer only from what is in the box, and if the answer is not in the box, say so. That box is what people in the AI world call the context. Everything the bot knows about you at the moment it answers is what you put in the box for that one question. It does not learn your business over time. It reads the box, answers, and forgets. Next question, new box. Two things follow from this picture, and they explain almost everything about SOP chatbots. The box has a size. Anthropic, the company behind the Claude models, says in its engineering write-up on contextual retrieval that a knowledge base under about 200,000 tokens, roughly 500 pages, can simply be included with every question, with no extra machinery. Most small businesses have far less than 500 pages of procedures. So for most of you, the whole manual fits in the box every time. If the manual is bigger than the box, someone has to pick. Then a librarian step runs first: it reads the question, pulls the few pages that m

Christopher was sick of being ghosted by AI recruiters. So he unleashed ChatGPT on his robot interviewer.

The company is testing robots on tasks that can performed by technicians.

Installing a large language model on your personal computer gives you a handy digital assistant that won’t compromise your data privacy.

The company is testing robots that can swap cables, reset servers, and take on other tasks performed by technicians, fueling concerns among some workers that their jobs could be at risk.

The potential for AI to automate scientific research and manufacturing must be balanced with new risks, Anthropic says.

Beijing’s endlessly delightful Robot Games featured tons of impressive stunts. But the most mind-blowing tricks challenged the humanoid’s brain, not its brawn.

Record-breaking robot races are less substantial than household chore challenges.

FCC ban on foreign-made robots accelerated RoboStore’s US manufacturing plans.

“We're not necessarily building in a dogmatic fashion towards full autonomy.”

The Unitree G1 has found online fame as a relatively affordable robot that can charm a crowd. But can it ever hold down a real job?

Joi AI hired 10 people to masturbate using AI companions as part of a monthlong “wellness” study. The company claims the practice could help “solve male loneliness.”

Human-AI marriages are not currently recognized by US law. Some Republican state policymakers are drafting legislation to keep it that way.

Bartesian's cocktail makers would be best described like a Keurig or Nespresso machine, but for alcoholic drinks.

Short answer: the least risky alternative to a single-provider OpenAI-compatible API is a thin routing layer with one internal contract, a small Python adapter, and an eval set that measures answer quality before price. Treat “cheapest” as a workload result, not a label. A US/EU chatbot also needs a deliberate data-residency decision before a key or SDK enters production. The attractive story is easy: one API key, one SDK, and a familiar chat-completions shape. Measure it. The production story has more edges. Provider-specific tool calls, token accounting, streaming events, retention settings, and regional routing can differ while the first text response still looks fine. That is how an in-app chatbot passes a demo and fails an eval. Consider a support bot that retrieves three passages, answers in a stream, and offers an escalation tool. A compatibility test that checks only the final sentence can miss an empty retrieval marker, a tool argument that is valid text but invalid JSON, a stream terminator that the client never handles, and a fallback that sends the same user request to a second region. The transcript still looks plausible in a screenshot. The trace tells a different story. I've learned to make those states explicit in the adapter before tuning a model. I build RAG and agent features in Python, so my first question is not “which model wins?” It is “which contract can I test?” The app should own that contract. A provider adapter should translate it at the boundary, and the rest of the application should never know whether the request went to an OpenAI-compatible endpoint, a Claude-style API, a Gemini-style API, or a local service. How can an app chatbot compare compatible API alternatives across US and EU? Start with the request that matters to the user: a message plus retrieved context, a latency budget, a maximum output, and a trace ID. Record the selected region and provider in server-side metadata, but don't send a secret to the browser. “One API key”

Cloudflare recently introduced Precursor, a client-side behavioral analysis engine that continuously evaluates session interactions, such as mouse movements and keyboard timing, to improve detection of sophisticated bots and AI agents without relying solely on one-time challenges like CAPTCHAs. By Renato Losio

WIRED spoke with Tucker Bryant, an artist and former Google employee who created ChatTJB to get people to reflect on the “strange moment” we’re in.

Ati Robotics assembles its robots in India and uses just a few Chinese parts—a strategy that could pay off as the Trump administration cracks down on Chinese humanoids.

And it’s personal information (alternate link ): The exposed data includes an AI-powered therapy app that someone appears to have vibe-coded, notes on meetings, and a dashboard someone made apparently to analyze medical billing data. Exposed chats reportedly include private cryptocurrency wallet keys and personal information like peoples’ addresses. What seems to be the issue is a user setting about data sharing. Anthropic’s position is that it’s not their problem : “We give people control over sharing their Claude conversations publicly, and in keeping with our privacy principles, we do not share chat directories or sitemaps with search engines like Google,” the company said in a statement. “These shareable links are not guessable or discoverable unless people choose to share them themselves. When someone shares a conversation, they are making that content publicly accessible, and like other public web content, it may be archived by third-party services.”...