AI 资讯

Last year my team had to pick a coding agent, and I volunteered to run the evaluation. I felt good about it. I pulled up the public benchmark scores, lined up the contenders, took the one at the top, and told everyone we had a winner. Then we actually pointed it at our repo. It did not blow up dramatically. It just kept being slightly wrong in ways that ate our time. It wrote diffs our reviewers would not approve. It renamed a function and broke three files it had never opened. The tests it ran passed, and the repo was still broken. I had confidently recommended a tool based on a number that turned out to say almost nothing about our situation. That was embarrassing enough that I went and figured out why. It took a few weeks of reading and a couple more bad calls before I landed on something that works. This is that, written plainly, and I hope it saves you the meeting where you have to walk your recommendation back. Why the benchmark score lied to me The score was not fake. It was just measuring somebody else's code. Once I looked properly, four gaps explained the whole thing: The agent might have already seen the answers. The problems in these public benchmarks are old. Models were very likely trained on the actual fixes used to grade them. So the score partly measures memory, not problem-solving. The setup is nothing like real work. A benchmark gives the agent a clean repo, one clear issue, and one command to run the tests. My engineers give it a half-open editor, a messy branch, a Slack thread, and a reviewer comment. Completely different job. Our codebase has its own habits. Our internal libraries, our wrappers, our test style, the imports we ban. No benchmark knows any of that, so an agent can write textbook-perfect code that our reviewers still reject on sight. The bar for passing is way lower. A benchmark passes a patch if the broken test now passes. My team passes a patch if it does that, and does not break unrelated tests, does not reformat the whole file,

One weekend I wrote an LLM eval framework in about two hundred lines of Python. It demoed beautifully. I felt clever. Six months later that same framework was a mess. Three different judge models with three different parsing hacks. A test dataset nobody had touched since November. A CI gate that kept failing because a vendor nudged their model, not because anyone broke a prompt. And the second engineer on rotation asking me, fairly, "how does this even work?" The framework did not fail. The eighty percent of the work the weekend tutorial skipped is what failed. That gap is the whole story, and this is what I would tell myself before starting again. The one line I wish someone had told me: build the rubric, buy the runner Here is the split that took me six months to see. Some parts of an eval setup are yours and only yours. The rubric that decides what "good" means for your product. The dataset built from your real failures. The rules for when a change is bad enough to block a release. Nobody else can write these, because they encode your domain. The rest is the same at every company. The thing that calls the judge model, parses its answer, retries, and caches. The machinery to run thousands of checks in parallel. The plumbing that scores live traffic. The system that groups failing calls together. Every team rebuilds these, hits the same bugs, and gains nothing by writing them twice. So build the first list. Do not hand-build the second. I rebuilt the second, and it cost me most of a year. Two questions I now ask about every single piece Before writing any part of this, I ask two things: Is it specific to me, or generic? A rubric for my domain is specific and worth owning. A retry-and-cache loop around a model call is generic. Everyone writes the same one. Does it compound, or does it rot? A dataset that grows from real production failures compounds. A year in, it is a regression suite no competitor can copy. A hand-built tracing layer rots. The moment a vendor chan

A number of social media posts claim that GPT-5.6 Sol deleted files and data without warning. OpenAI had basically disclosed the problem in June.

It seems some people take issue with one billionaire family overseeing a third of the US' entertainment media.

SPOILER: De 1 à 2000 Durante minha jornada como jogador , existiu uma coisa que me despertou muita curiosidade: os créditos. Quando eu jogava coisas como Final Fantasy, ficava completamente arrepiado quando via aquelas cenas antes do menu inicial, com CGIs bem bonitões e um monte de nomes em japonês — muitos deles que, honestamente, até hoje não sei quem são. Esse arrepio também acontecia quando eu chegava ao final de algum jogo e começavam a aparecer nomes e mais nomes. Quando eu entendi o que eram aqueles nomes, a primeira coisa que me veio à mente foi: peraí, precisa de tudo isso de gente pra fazer um jogo? Too long, didn't read : Sim e não. Skyrim levou aproximadamente 6 anos com uma equipe de 100 pessoas (e ainda é dito que é uma equipe enxuta), enquanto Stardew Valley levou 4,5 anos com uma "equipe" de apenas 1 pessoa. Eles têm quase o mesmo tamanho em horas jogadas na campanha principal. Not too long, I'll read it : É um pensamento lógico que trabalhos maiores no mundo da tecnologia envolvam uma quantidade maior de pessoas trabalhando E tempo. Dessa forma, seria correto assumir que um trabalho menor exige menos tempo e menos pessoas trabalhando. Mas o que os dados dizem? King of Fighters 94: 2 anos, começou com 6 pessoas, aumentou para 60. Clair Obscure - Expedition 33: aproximadamente 6 anos, 30 pessoas. TES V - Skyrim: 6 anos, 100 pessoas. Super Mario Bros: 2 anos, 7 pessoas na equipe principal. Stardew Valley: 4,5 anos, 1 pessoa. Kenshi: 12 anos, 1 pessoa por 6 anos (trabalhando meio período), aproximadamente 8 pessoas por mais 6 anos (tempo integral). Final Fantasy VII (PS1): aproximadamente 2 anos, de 100 a 150 pessoas (uma das maiores equipes da indústria na época). Final Fantasy VII Remake: Cerca de 5 anos, e não se tem números exatos, mas os créditos citam mais de 2000 pessoas, incluindo muitos -istas e -ores (vou tentar confirmar esse número depois). É claro que existem muitos fatores envolvidos aí, como época, demandas de mercado, prazos, tecnologia

Design + Product Thinking: NYC’s Path to Reliable AI AI delivers value when it’s useful, trusted, and operational. For city services that affect millions, those qualities don’t happen by accident — they come from applying design thinking (who the service is for, how it’s used) together with product thinking (what outcome we’re trying to achieve and how we operate over time). This article explains why hiring designers and product managers matters for NYC’s digital and AI initiatives, summarizes the city’s PIT Crew program, and outlines how Flamelit applies outcome-focused delivery in the public sector. Why design and product roles matter Designers and product managers have distinct but complementary responsibilities that reduce common AI delivery failures: Designers (Design Thinking): center human needs, prototype user flows, and validate that interfaces and decision workflows are understandable and accessible. They surface usability and trust issues early, preventing technically accurate models from becoming unusable in practice. Product managers (Product Thinking): define the measurable outcomes, prioritize use cases, align stakeholders, and manage the lifecycle from discovery to ongoing operations. They ensure work is evaluated against mission impact, not just technical metrics. Together they prevent common failures: building technically impressive models that nobody trusts, deploying brittle systems without human review, or shipping features with unclear ownership that decay in production. PIT Crew and NYC hiring context NYC’s PIT Crew program is a city initiative designed to attract and staff product, engineering, and design talent for public service projects. It’s a practical recognition that public-sector digital transformation needs people skilled in user research, product management, and delivery. Read more about the PIT Crew and how it works here: https://www.nyc.gov/content/pitcrew/pages/ (open in a new tab). Hiring programs like PIT Crew help create the c

Genkit Agents API, ORA, Python AI Explainer: New Tools for Workflow Automation Today's Highlights This week, Google's Genkit ships a powerful Agents API for TypeScript and Go, featuring human-in-the-loop capabilities for robust production deployments. Additionally, a new Go-based open-source task orchestrator, ORA, emerges for efficient model routing, alongside a practical Python tutorial for building an AI error explainer. Google's Genkit Ships Agents API with Detached Turns and Human-in-the-Loop for TypeScript and Go (InfoQ) Source: https://www.infoq.com/news/2026/07/genkit-agents-api-preview/?utm_campaign=infoq_content&utm_source=infoq&utm_medium=feed&utm_term=global Google has released a preview of its Genkit Agents API, significantly enhancing its open-source AI framework for building generative AI applications. This new API introduces features critical for deploying robust AI agents in production, specifically "Detached Turns" and "Human-in-the-Loop" functionalities. Detached Turns allow agents to operate asynchronously, handling long-running tasks or waiting for external events without blocking the main workflow, which is essential for complex, multi-step agentic processes. The Human-in-the-Loop feature provides crucial mechanisms for human oversight and intervention, ensuring reliability, safety, and compliance in critical applications where full automation is not yet feasible or desirable. The Genkit Agents API supports both TypeScript and Go, targeting a broad range of developers integrating AI into existing systems or building new agentic workflows. By offering structured patterns for agent interaction, state management, and human review, Genkit aims to streamline the development and deployment of intelligent agents, addressing key challenges in control and reliability for real-world AI applications. Comment: The introduction of Human-in-the-Loop into Genkit's Agents API is a game-changer for production-grade agent systems, offering the control and reliab

SQLite Internals: lcd-ex vs hctree; PostgreSQL 19 SQL/PGQ Rewrites & pg_timetable Migration Today's Highlights This week's highlights feature a deep dive into SQLite's internal data structures, offering insights for advanced optimization. Also, PostgreSQL users gain practical guidance on migrating to pg_timetable for robust job scheduling and understanding how SQL/PGQ translates to efficient joins in PostgreSQL 19. Replacing pgAgent with pg_timetable: Installing as a Linux Service (Planet PostgreSQL) Source: https://postgr.es/p/9pE Regina Obe presents a crucial guide for PostgreSQL administrators looking to modernize their task automation by replacing pgAgent with pg_timetable . This second part of the series focuses specifically on the practical steps of installing and configuring pg_timetable as a systemd service on Linux, ensuring it runs reliably in a production environment. The article details the process from downloading binaries and creating dedicated user accounts to setting up service files and enabling autostart, providing a comprehensive walkthrough for seamless integration. pg_timetable offers significant advantages over pgAgent , including advanced scheduling capabilities, event-driven task execution, parallel job processing, and improved logging. This migration strategy is vital for enhancing the robustness and efficiency of database maintenance, data synchronization, and complex ETL pipelines within the PostgreSQL ecosystem. By following this guide, developers and DBAs can transition to a more powerful and flexible job scheduler, leading to greater control and reliability over their automated PostgreSQL operations. Comment: Migrating to pg_timetable from pgAgent is a significant step forward for job scheduling in PostgreSQL. This guide provides the hands-on steps needed to get it running as a service, which is essential for any production deployment. SQLite Forum Discusses lcd-ex vs hctree (SQLite Forum) Source: https://sqlite.org/forum/info/3494bff42

Browser automation used to be easier to describe. A test opened a page, filled in a form, clicked a button, and checked the result. The hardest parts were usually selectors, waits, and browser compatibility. Those problems still exist, but the surface area has expanded. Today, browser tests may need to handle streaming interfaces, MFA, AI-generated content, multiple operating systems, preview deployments, canary releases, and code changes proposed by AI assistants. The challenge is no longer just writing a script that passes. The challenge is building a testing system that remains understandable and affordable after hundreds of tests and thousands of CI runs. Start by measuring instability instead of normalizing it Flaky tests often become accepted background noise. A test fails, CI retries it, and the second run passes. The pipeline turns green, so the team moves on. Over time, the retry count grows and nobody is sure which failures matter. The problem is that a passing retry does not erase the cost of the first failure. The article on calculating the real cost of flaky test retries in CI provides a useful framework for evaluating compute costs, developer interruptions, delayed feedback, and investigation time. A simple reliability metric can help: first-attempt pass rate = tests passing without retry / total test executions This is often more revealing than the final pipeline pass rate. A suite with a 99% final pass rate may still be deeply unstable if many tests require multiple attempts. Reproduce the environment before changing the test When a browser test fails only in CI, teams often edit the test before reproducing the environment. That can lead to unnecessary waits and conditionals. One of the most common variations is a test that passes in visible Chrome but fails in headless mode. The explanation is not always “headless Chrome is flaky.” Differences in viewport, rendering, animation, fonts, and resource timing can all change application behavior. This det

For a long time, teams treated browser test reliability as a framework problem. When tests failed, the usual response was to change selectors, add waits, increase retries, or replace one automation library with another. That approach made sense when the main challenge was simply controlling a browser. Modern applications are different. A single user journey may now include an identity provider, multi-factor authentication, a streaming AI response, a background API request, a feature flag, a canary deployment, and a frontend rendered differently across several operating systems. The test framework is still important, but it is only one part of the reliability problem. The bigger question is whether the entire testing system gives the team enough evidence to make a release decision. Headless failures are usually a symptom, not the real problem A common example is a test that passes locally but fails only in headless Chrome. It is tempting to assume that headless mode is simply unreliable. In practice, the difference is often caused by viewport size, rendering behavior, animation timing, fonts, resource loading, or elements being positioned differently when no visible browser window exists. This breakdown of why browser tests fail only in Chrome headless is useful because it separates several failure categories that are often grouped together as “timing issues.” That distinction matters. A test that fails because an element is outside the viewport needs a different fix from a test that fails because a network request completes later in CI. Adding a longer timeout may hide both problems temporarily, but it does not make the test more trustworthy. Retries can make a weak test suite look healthy Retries are one of the easiest ways to reduce visible failures in CI. They are also one of the easiest ways to hide instability. A flaky test that passes on its third attempt still consumed runner time, delayed feedback, created extra logs, and made it harder to determine whether

Hello Reddit, I'm 33, currently in school for my Bachelor's in IT (~2 years from graduation). I am taking Calculus (Math 110 at Penn State) for the 3rd time. I failed it twice before. I am currently a nurse of 10 years going back to school. I am taking a summer class version of it, which I did not realize is accelerated / asynchronous, and am slowly starting to do poorly in it. My end goal after school is to work remotely in a programming-related job. But math has always been hard for me. So I am asking: -Any tips / resources / insights for passing Calculus -Any way I can pursue a degree in programming / computers without having to take Calculus -what other degree options that aren't math heavy will allow me to pursue tech-related careers? Open to suggestions, but my necessity list is being able to work remotely and earn a comfortable living abroad. Thanks! submitted by /u/Born_Cod_8931 [link] [留言]

While building with OpenAI, Anthropic, and other AI providers, I realized something surprising. I monitored my servers, databases, and application performance—but I had almost no visibility into my AI API spending until I checked the provider dashboard or received the monthly invoice. That led me to build AICostPass . It helps developers, indie hackers, startups, and agencies: ⚡ Track AI API costs in near real time 📊 Monitor spending by project or client 🚨 Get budget threshold email alerts 📧 Receive weekly spending summaries 💰 Export billable CSVs for client invoicing The goal is simple: help developers understand and control AI costs before the invoice arrives. 👉 https://aicostpass.com I'd love to hear how you're currently tracking AI API costs. Are you using provider dashboards, spreadsheets, or another tool?

OpenAI's first device is set to be a smart speaker that lets you talk with ChatGPT, according to a report from Bloomberg. The device apparently won't have a screen, but will use a camera and additional sensors to "understand" your environment. The report comes just days after Apple filed a lawsuit against OpenAI that accused […]

Meta cut its workforce by 10 percent in May.

If Claude can already search the web, read a 10-K, and explain what a rate cut does to long-duration equities, the fair question is why you would connect anything to it at all. It is the right question, and the honest answer is that for a large class of questions you should not. Raw Claude is enough. The gap is narrower and sharper than "Claude does not know finance." Claude knows finance. What it does not know is you. What raw Claude already does well Be clear about this before the sales pitch, because pretending otherwise would insult anyone who has actually used it. Claude with web search will look up a current quote, summarize an earnings call, explain a valuation multiple, walk you through how a Monte Carlo simulation works, and reason about a macro scenario better than most of the commentary you would read instead. If your question is about the world, and not about your own balance sheet, a connector adds nothing. Ask Claude directly. The trouble starts the moment the answer depends on what you actually own. Four things that break when the question is about your money 1. It starts from zero every time A chat has no memory of your holdings. You can paste them in, and many people do, and it works for exactly one conversation. There is no cost basis, no purchase date, no daily snapshot series behind it. So "how concentrated am I really", "what is my realized gain this year", and "how correlated are my top five positions over the last 90 days" are not questions it can answer. It can only answer them about the numbers you re-typed, this once, from memory. 2. The same question gives a different answer twice LLM inference is not deterministic, and it is not deterministic even at temperature zero. Thinking Machines Lab traced the cause to batch-invariance in inference kernels: the batch your request lands in varies with server load, so the arithmetic varies with it. They fixed it in a research setting and got 1,000 bitwise-identical runs, which tells you how much engi

Look inside almost any modern connected device -- a smartphone, a smartwatch, a Wi-Fi thermostat, a battery-powered sensor node -- and you will find a processor core designed by ARM. It is one of the most successful engineering ideas in computing history. And here is the strange part: ARM has never manufactured a single one of those chips. It does not own a factory. It sells blueprints. A three-person project in Cambridge The story starts at Acorn Computers in Cambridge, England, in the early 1980s. Acorn had built the BBC Micro, a hugely popular educational computer in the UK, and it needed a faster processor for its next machine. The commercial chips available at the time were disappointing, so a tiny team decided to design their own. The acronym everyone knows today originally stood for Acorn RISC Machine . Sophie Wilson designed the instruction set and wrote the simulator; Steve Furber led the physical chip design. RISC -- Reduced Instruction Set Computing -- was the key bet. Instead of piling on complex instructions, they kept the instruction set small and simple, which made the chip easier to build, cheaper, and remarkably power-efficient. The first silicon, the ARM1, was fabricated by VLSI Technology and delivered to Acorn on 26 April 1985. When the team powered it on, it simply worked -- first try. For anyone who has designed hardware, that is almost unheard of; new processors normally need several rounds of revisions to shake out design errors. A famous piece of Acorn lore is that the early ARM chips drew so little current they could keep running on leakage alone after the power was disconnected. From a British computer to the whole world Acorn the computer company faded, but the processor design did not. In 1990 the ARM team was spun out into a separate joint venture, and the acronym was quietly re-expanded to Advanced RISC Machines . The new company made a decision that defined its future: it would not build chips. It would license the designs and let oth

Scope creep usually does not start with a dramatic disagreement. It starts with a reasonable-sounding request: "Can we also add this small thing?" For freelancers, consultants and small agencies, the hard part is often not the work itself. The hard part is that nobody has a clean shared record of what changed, what was included originally, what is now extra, and whether the timeline or price should move. The workflow I find most useful is deliberately simple. 1. Write the baseline before work starts Before the project begins, write down: the goal of the work what is included what is excluded assumptions dependencies what the client needs to provide This does not need to be legal language. Plain language is often better because both sides can understand it quickly. 2. Separate included and excluded work Many scope problems happen because "not included" was never written down. For example: Included: one landing page with copy and layout Excluded: email automation, paid ad setup, analytics dashboard, extra page variants That makes later conversations less emotional. You are not saying "no" from nowhere. You are comparing the new request to the original scope. 3. Pause before doing the extra work When a new request appears, write it down before starting. A useful change note can be short: requested change why it is needed impact on price impact on timeline what will be delivered approval status The point is not bureaucracy. The point is to stop invisible work from becoming normal. 4. Make timeline impact explicit Freelancers often talk about price but forget schedule. Even if the client accepts the extra cost, the original delivery date may no longer be realistic. A small change can still interrupt review cycles, dependencies or other client work. 5. Get written approval before starting This can be as simple as: "Confirmed. Please go ahead with this change at the updated price and timeline." The approval does not need to be fancy. It just needs to exist before the addit