今日已更新 247 条资讯 | 累计 31925 条内容
关于我们

标签:#m

找到 9452 篇相关文章

开发者

ACL ARR (May 2026)- Updating Reviewer Score post 17 July AoE Deadline? [D]

Had submitted a paper to ACL ARR May 2026 cycle. Unfortunately, none of the reviewers acknowledged the rebuttal during the author-reviewer discussion I am curious to know from people who had volunteered to review papers this cycle- are you still able to update the ratings, or even your review based on the rebuttal? Also is there any meta-reviewer discussion going on? submitted by /u/Forsaken-Order-7376 [link] [留言]

2026-07-21 原文 →
AI 资讯

Pressure-testing Ota on Open WebUI: proof cleanup ownership, bootstrap truth, and native vs Compose runtime boundaries

Overview Open WebUI exposed a real Ota lifecycle boundary. This was not mainly a parsing or contract-shape repo. The contract was already strong enough to model: source-checkout verification packaged native runtime through uv run open-webui serve frontend development runtime default Docker Compose runtime What the repo exposed was operational truth after proof: a successful native proof still left a host workload alive the first cleanup fix then widened too far and treated a Compose-owned runtime as the same class of host workload That made Open WebUI a valuable pressure repo. It forced Ota to get more precise about cleanup ownership instead of treating all successful runtime proof as one generic teardown problem. The current pressure contract pins released Ota v1.6.24 . Its latest green matrix run proves the release surface at the exact contract and workflow revision linked below. What Open WebUI exposed in Ota This repo exposed four meaningful weaknesses. 1. proof success was weaker than it looked The first issue was not that runtime proof failed. It was that runtime proof succeeded and still left the native workload alive afterward. In this repo, the packaged native workflow launches: serve:native : launch : kind : command exe : uv args : - run - open-webui - serve - --host - 0.0.0.0 - --port - " 8080" Ota proved that workflow, but the launched process tree was still alive after proof completed. In GitHub Actions, that surfaced through setup-uv post-job cleanup, which blocked while the uv cache was still in use. That was an Ota gap. If proof succeeds but leaves behind repo-owned runtime state that later breaks CI cleanup, the proof surface is still incomplete. 2. native service cleanup widened past its real ownership boundary The first core fix made Ota clean selected native service workloads after successful proof. That was directionally correct, but Open WebUI immediately exposed the next boundary. The Docker workflow uses a native task shape to launch Compose:

2026-07-21 原文 →
AI 资讯

The Test That Passes in Staging But Fails When a Customer Runs It

You have been here. The test suite is green. The deployment pipeline reports all checks passed. Then a customer opens a ticket with a screenshot that shows something your test never caught. The test passed in staging. It fails in production. And you cannot reproduce it locally. This is not a flaky test problem. It is a fidelity problem. Your test environment and your production environment are not the same thing. The gap between them is where real bugs live. Let me walk through one concrete example, the fix, and what it teaches about writing tests that survive the handoff to a real user. The Problem: Environment Drift A fintech team I worked with had a checkout flow. The test clicked "Pay Now", waited for a success message, and asserted the text "Payment successful" appeared on screen. It passed every time in staging. Customers reported that after paying, they saw a blank white page for several seconds before the success message appeared. Some of them closed the tab during that blank period, thinking the payment failed. The transaction went through. The customer never saw the confirmation. Support tickets piled up. The test never caught this because the staging environment served the success page in under 200 milliseconds. The blank period did not exist there. Production had a slower downstream service that introduced a three-second delay between the payment confirmation and the page render. The test was correct in what it checked. It was wrong in what it assumed about timing and state. The Fix: Test the Experience, Not Just the Outcome The fix was not to add a longer wait. The fix was to test what the user actually experiences during that gap. Here is a minimal Playwright test in TypeScript that catches this class of problem: import { test , expect } from ' @playwright/test ' ; test ( ' checkout shows loading state before success ' , async ({ page }) => { await page . goto ( ' /checkout ' ); await page . fill ( ' #card-number ' , ' 4111111111111111 ' ); await page

2026-07-21 原文 →
AI 资讯

Is Your BDD Framework Just a Fancy Way to Write Manual Test Cases in Gherkin?

Gherkin is not a test automation tool. It never was. Yet here we are, five years into your SDET career, and you're staring at a feature file that reads like a step-by-step manual for a human tester. Given I log in with username "admin" and password "password123" . When I click the "Submit" button . Then I see the text "Welcome" on the screen . You've written two years of these. Your team calls it BDD. Your manager calls it "living documentation." And somewhere in the back of your mind, a quiet voice whispers: This is just a manual test case with extra steps. That voice is right. Let me say it plainly: if your Gherkin scenarios describe how the system works instead of what it should do, you are not doing BDD. You are writing manual test cases in a structured English format and calling it automation. The only thing you've automated is the illusion of progress. The problem isn't Gherkin. The problem is how we use it. Most teams adopt BDD because someone read a blog post about "collaboration" and "shared understanding." They install Cucumber or SpecFlow. They write feature files. They map steps to Selenium or Playwright code. And they call it a day. But look closely at what happens next. The product owner never reads the feature files. The developer skims them once and goes back to writing code. The QA engineer — that's you — becomes the sole maintainer of a growing pile of Gherkin that nobody else touches. You're not facilitating collaboration. You're translating manual test cases into a format that requires a compiler. Here's the real test. Take any feature file from your project. Hand it to a developer who has never seen it. Ask them to implement the feature using only the Gherkin as a spec. If they can write production code from it, you have real BDD. If they ask you for clarification, you have documentation theater. I've seen teams with hundreds of feature files. Beautifully formatted. Perfect indentation. Tags for every regression cycle. And not a single one of th

2026-07-21 原文 →
AI 资讯

Your First Week of AI-Assisted Automation Will Be a Debugging Nightmare

Most engineers expect AI-assisted automation to be the easy part. You describe a test, the model writes it, you move on. The first week will prove you wrong. Not because the code is bad. Because the code is almost right. And almost-right code is harder to debug than wrong code. Wrong code fails loudly. Almost-right code passes on Monday, fails on Tuesday, passes again on Wednesday, and by Thursday you are questioning whether you understand your own application. I have watched teams adopt AI copilots into their Playwright suites and spend the first five days doing nothing but untangling false passes. If you are about to start this journey, here is what that week actually looks like. The Problem: The Model Does Not Know What "Stable" Means A language model has never waited for a network response. It has never watched a flaky selector survive three CI runs and then collapse on the fourth. It writes tests from a static understanding of your page, not from the dynamic reality of your application. You will ask it to write a test that clicks a button and waits for a confirmation toast. The model will produce something like this: await page . click ( ' button:has-text("Submit") ' ); await page . waitForSelector ( ' .toast-success ' ); Looks fine. Runs fine. Then your team deploys a new build where the toast takes 400ms longer to appear because of an analytics call. The test fails. Not because the feature broke. Because the model assumed a timing that was never guaranteed. This is the core problem. The model writes tests that match the page as it was when the model saw it . It does not write tests that match the page as it will be . The Solution: Treat AI-Generated Tests as Drafts, Not Deliverables The shift is mental before it is technical. You cannot review AI-generated tests the way you review human-written tests. Human tests come with intent. AI tests come with patterns. You need a different review lens. First, look for every hardcoded wait. Replace it with a state-based

2026-07-21 原文 →
AI 资讯

Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40%

Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40% How we moved from "semantic search + hope" to a measured, tunable retrieval pipeline with 95% recall@10 The RAG Reality Check Everyone ships RAG the same way: chunk by 512 tokens, embed with text-embedding-3-small , top-k=5, stuff into context. It works for demos. Then you hit production: Legal contracts: 512 tokens splits clauses mid-sentence API docs: 1000-token chunks drown signal in noise Customer tickets: Conversational context needs overlap, not fixed windows Latency: 500ms embedding + 200ms vector search + 300ms LLM = 1s+ per query We rebuilt our retrieval layer from first principles. Here's what actually moves metrics. Chunking: One Size Fits None # rag/chunking.py from abc import ABC , abstractmethod from dataclasses import dataclass @dataclass class Chunk : text : str metadata : dict token_count : int chunk_id : str class ChunkingStrategy ( ABC ): @abstractmethod def chunk ( self , document : str , metadata : dict ) -> list [ Chunk ]: ... class FixedTokenChunker ( ChunkingStrategy ): """ Baseline. Good for homogeneous content. """ def __init__ ( self , chunk_size = 512 , overlap = 50 ): self . chunk_size = chunk_size self . overlap = overlap class RecursiveChunker ( ChunkingStrategy ): """ Respects structure: markdown headers, code blocks, paragraphs. """ def __init__ ( self , separators = [ " \n ## " , " \n ### " , " \n\n " , " \n " , " " ], chunk_size = 512 ): self . separators = separators self . chunk_size = chunk_size class SemanticChunker ( ChunkingStrategy ): """ Uses embedding similarity to find natural boundaries. """ def __init__ ( self , model = " text-embedding-3-small " , threshold = 0.7 ): self . model = model self . threshold = threshold class AgenticChunker ( ChunkingStrategy ): """ LLM decides boundaries. Expensive but highest quality for complex docs. """ def __init__ ( self , model = " gpt-4o-mini " ): self . model = model Our production config by

2026-07-21 原文 →
AI 资讯

Building Production-Grade LLM Evaluation Pipelines: From Vibes to Metrics

Building Production-Grade LLM Evaluation Pipelines: From Vibes to Metrics How we replaced "looks good to me" with automated evaluation catching 92% of hallucinations before deployment The Problem: Why "Vibe Checks" Fail in Production Three months ago, our team shipped a RAG-based customer support assistant. It worked great in testing — we'd ask it questions, read the answers, and say "yeah, that looks right." Then it hit production. A customer asked about their billing cycle. The assistant confidently cited a policy that didn't exist. Another asked about API rate limits and got numbers from a competitor's documentation. By the time we caught it, 500+ users had seen hallucinated responses. The post-mortem was brutal: we had zero automated evaluation . Our test process was literally "ask 5 questions, read answers, thumbs up." What Production Evaluation Actually Needs Academic benchmarks (MMLU, HellaSwag) don't tell you if your system works for your use case. Production evaluation needs: Domain-specific judges — Your criteria, not generic "helpfulness" Speed — Evaluation must run in CI/CD, not overnight Regression detection — Know immediately when a prompt change breaks things CI/CD integration — Block merges that degrade quality Golden dataset management — Versioned, stratified, growing test cases Architecture: The Evaluation Pipeline ┌─────────────┐ ┌──────────────┐ ┌────────────────────┐ ┌──────────────┐ │ Test Cases │────▶│ LLM Under │────▶│ Judge Ensemble │────▶│ Metrics & │ │ (Golden Set)│ │ Test │ │ - Faithfulness │ │ Regression │ └─────────────┘ └──────────────┘ │ - Instruction F. │ │ Detection │ │ - JSON Schema │ └──────┬───────┘ │ - Custom LLM │ ▼ └────────────────────┘ ┌──────────────┐ │ Dashboard/ │ │ PR Comments │ └──────────────┘ Core Abstractions # eval/base.py @dataclass ( frozen = True ) class TestCase : id : str input : dict [ str , Any ] expected : dict [ str , Any ] | None = None tags : list [ str ] = field ( default_factory = list ) # ["edge-case",

2026-07-21 原文 →
AI 资讯

4 Silent Failures, 2 Undocumented APIs, and a Container That Crashed Because of a Missing User Directive

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry . I spent a week deploying a CrewAI agent to AWS Bedrock AgentCore. The SDK wasn't on PyPI. The error messages were 200 OKs. The container crashed without logs. And the naming regex rejected hyphens without telling me why. This is the full debugging trail. Every failure was silent. Every fix required reading source code nobody documented. Table of Contents The Project Failure 1: The SDK That Doesn't Exist on PyPI Failure 2: The 200 OK That Means Failure Failure 3: The Container That Crashed With No Logs Failure 4: The Naming Regex Nobody Documented The Two-Client Split Nobody Mentions What I Learned The project I built a resume-tailoring AI agent with CrewAI and Amazon Bedrock. It takes a job description, analyzes your resume, identifies gaps, and rewrites bullet points to match what the role actually needs. Locally it worked perfectly. CrewAI orchestrates the agents, Bedrock Nova Pro handles the LLM calls, and the output is solid. Deploying it to production was the problem. AWS launched Bedrock AgentCore in June 2026 as a managed runtime for AI agents. You containerize your agent, push the image, and AgentCore handles scaling, memory, and invocation. Sounds simple. It was not simple. Failure 1: The SDK that doesn't exist on PyPI The docs say to install bedrock-agentcore-client . I ran: pip install bedrock-agentcore-client It installed successfully. No errors. That's because there's a placeholder package on PyPI with that name. It installs, imports fail silently, and your container builds successfully with a broken dependency inside. The real SDK lives in AWS's CodeArtifact registry. You need to configure pip to pull from a private index: aws codeartifact login --tool pip \ --domain amazon-agent-runtimes \ --repository agent-runtimes-pypi \ --domain-owner 600427722194 Then install from there. The PyPI package is a trap. Nobody warns you. Hours lost: 3. The error only appears at runtime

2026-07-21 原文 →