标签：#c

Changing the model for a running AI task is not a settings update. It is a distributed operation: read current task -> prepare credentials/config -> request restart -> receive result -> persist active model If two switches overlap, completion order can differ from request order. The system needs a rule for which intent wins. The concrete case At commit c58bcd4 , MonkeyCode records model-switch attempts with from/to model IDs, request ID, load-session flag, success, message, session ID, and timestamps in TaskModelSwitch . The reviewed task use case creates a switch record, asks taskflow to restart with the target model configuration, and completes the switch record and task model based on the response. The accompanying tests cover success and failure paths. From this source review, I could not establish an explicit compare-and-swap generation or a per-task serialization contract around overlapping requests. That does not prove an exploitable race: serialization may exist elsewhere in the deployment or taskflow boundary. It means concurrency semantics deserve an explicit test and contract. Why last completion is unstable Assume request A selects model A, then request B selects model B: time -> A: request ---- restart ---------------- complete B: request -- restart -- complete If each successful completion writes its model, B applies first and late A overwrites it. Reverse network timing and the result changes. The companion simulator makes that order dependence visible: export function naiveCompletionOrder ( completions ) { let model = " initial " ; for ( const completion of completions ) { if ( completion . success ) model = completion . model ; } return model ; } [A, B] ends on B. [B, A] ends on A. The caller's latest intent is not part of the rule. Add a monotonic generation Assign a generation while accepting each request: A -> generation 41 B -> generation 42 Completion may update active state only when its generation equals the task's current requested generatio

LLM Evaluation System Prompts Scored Rubrics Runtime Guardrails: A Practical Guide for Production Learn how to evaluate LLM outputs in production using system prompts, scored rubrics, and runtime guardrails to prevent hallucinations and ensure quality. TL;DR: To evaluate LLM outputs in production, combine system prompts that define evaluation criteria, scored rubrics using LLM-as-a-judge for dimensions like correctness and relevance, and runtime guardrails that filter or flag unsafe outputs. This approach scales better than human review, adapts via prompt changes, and catches failures that status codes miss, as seen in the Air Canada chatbot case. Why Production LLM Evaluation Demands More Than Status Codes A 200 status code only confirms the server processed the request—it says nothing about whether the generated text is factual, safe, or useful. The Air Canada chatbot that invented a non-existent bereavement discount returned perfectly valid HTTP responses, yet the hallucinated policy led to a tribunal ruling against the airline. Production evaluation must therefore separate operational health (latency, error rates) from output quality (correctness, relevance, harmlessness). Consider a typical API call that succeeds operationally but fails qualitatively: import requests response = requests . post ( " https://api.example.com/v1/chat " , json = { " model " : " gpt-4o " , " messages " : [{ " role " : " user " , " content " : " What is Air Canada ' s bereavement policy? " }]}, headers = { " Authorization " : " Bearer $KEY " } ) print ( response . status_code ) # 200 print ( response . json ()[ " choices " ][ 0 ][ " message " ][ " content " ]) # Output: "Air Canada offers full refunds for bereavement-related cancellations..." A 200 status code and a well-formed JSON body mask a completely fabricated policy. To catch this, you need a separate evaluation layer that scores the output against a rubric. LLM-as-a-judge is a common approach, using a second model to assess the

An activity log tells us what an agent did. A decision log should also tell us what it considered and rejected. Without rejected options, a later reviewer sees a clean path that never existed: model B was selected, the task restarted, the result succeeded. Missing are the reasons model A was unsuitable, why staying put was worse, and what new evidence would change the choice. That information matters for trust and recovery. It lets people challenge a decision without reconstructing the entire session. Execution history is necessary, but different The MonkeyCode model-switch record at commit c58bcd4 stores the task and user, from/to model IDs, request ID, whether to load the session, success, message, session ID, and timestamps. The switch use case creates that switch record, restarts the task with the target configuration, and records the result. That is valuable execution history. It answers “what switch was requested and what happened?” The expanded rejected-options structure below is my design proposal , not a claim about MonkeyCode's current schema or interface. Add the decision before the outcome A reusable record can separate choice from execution: { "decision_id" : "task-42-model-switch-7" , "context" : "The task needs the required tool-call contract." , "chosen" : { "option" : "model-b" , "reason" : "Passed the declared capability contract" , "evidence" : [ "evaluation/capability-model-b.json" ] }, "rejected" : [ { "option" : "model-a" , "reason" : "Required tool-call case failed" , "evidence" : [ "evaluation/capability-model-a.json" ], "revisit_when" : "Adapter version changes" } ], "execution" : { "request_id" : "req-switch-7" , "result" : "success" , "session_id" : "session-9" } } The key field is revisit_when . “Rejected” should not mean universally bad. It should mean unsuitable under a specific context and evidence set. Design the interface for progressive disclosure Do not paste this JSON into the main task timeline. Use three layers: Timeline: Switch

“On-device AI saves battery” and “cloud AI is more efficient” can both sound plausible. Neither is a measurement. The placement decision crosses at least four different budgets: user wait + network transfer + provider spend + device energy Do not collapse them into one vague “cost” number. Measure each with its own unit and evidence boundary. Start by identifying the actual execution path I reviewed MonkeyCode mobile code at commit c58bcd4 . The task stream opens a server-supported WebSocket. The speech-to-text hook also participates in a server-supported streaming path. That reviewed path is not evidence of on-device model inference. So a fair current study would measure a mobile client using remote task and voice services. An on-device alternative would be a separate prototype with its model, runtime, and packaging declared. Record a measurement envelope The included CSV template begins with these fields: sample_id,sample_kind,placement,device,os,framework,model,network,input_tokens,output_tokens,latency_ms,bytes_up,bytes_down,energy_joules,cost_usd Why so many? device , os , and framework make thermal and runtime results interpretable; model and token counts keep workload size visible; network separates offline, Wi-Fi, and cellular behavior; latency is milliseconds, transfer is bytes, energy is joules, and provider spend is currency; sample_kind prevents synthetic examples from masquerading as device measurements. Battery percentage is too coarse for short runs. It is affected by display, radio, background work, battery health, temperature, and OS estimation. If you cannot collect energy with an appropriate platform profiler or external power measurement, leave energy_joules empty. Use matched user flows Compare the same tasks, not unrelated model demos: Flow Cloud case On-device case Short prompt Same input and output cap Same semantic task and cap Voice turn Same audio fixture Same audio fixture Offline Expected failure or queued action Local completion if supp

Downloading an installer and immediately executing it as root collapses three operational decisions into one command: Which artifact? -> Did these bytes arrive intact? -> Should this host execute them? Separate those decisions and the install becomes reviewable, reproducible, and recoverable. A concrete source-review boundary At commit c58bcd4 , the MonkeyCode runner installation template selects x86_64 or aarch64 , checks AVX on x86, requires root, and downloads an architecture-specific installer before executing it. The reviewed template uses curl -4sSLk , so certificate verification is disabled by -k . It also downloads an unversioned path. I could not find a pinned version, digest, or signature check in that template. That is a statement about controls visible in one pinned file—not a claim that the release service is compromised or that no external release control exists. Put a manifest before execution For each release artifact, publish immutable metadata through a separately protected release process: { "version" : "1.2.3" , "architecture" : "x86_64" , "file" : "runner-installer-1.2.3-x86_64" , "sha256" : "<64 lowercase hex characters>" , "size" : 18439210 , "rollback" : { "previous_version" : "1.2.2" , "artifact" : "runner-installer-1.2.2-x86_64" } } SHA-256 detects bytes that differ from the manifest. It does not prove who authored the manifest. Serve the manifest over validated TLS, pin it through deployment configuration, or sign it and verify the signature with a trusted offline public key. Verify as an unprivileged staging step The companion verify-installer.mjs checks filename, exact size, digest, version, architecture, and rollback metadata: node verify-installer.mjs release-manifest.json fixture-installer.sh node test-verifier.mjs Expected output uses the fixture's actual digest: PASS 1.2.3-fixture sha256=<digest> PASS verified fixture; rejected tampered artifact before execution The negative test appends a line to the artifact and requires both size

Two buttons in a confirmation dialog look simple: Cancel and Confirm. Keyboard behavior makes the component a small state machine. A recent MonkeyCode change gives us a concrete example. Issue #862 and PR #863 add these shortcuts to the slash-command confirmation: ArrowLeft -> focus Cancel ArrowRight -> focus Confirm The reviewed implementation at commit c58bcd4 moves focus through button refs. That is a useful extra interaction. It is not a replacement for the dialog's accessibility foundation. Keep the baseline first For an alert-style confirmation, users still need: an accessible name and description; focus moved inside when the dialog opens; Tab and Shift+Tab constrained to dialog controls; Escape to dismiss when cancellation is allowed; visible focus; focus returned to the trigger after close; actual buttons whose labels explain the actions. The WAI-ARIA Authoring Practices Alert Dialog Pattern describes the modal semantics and keyboard foundation. Left/right mapping is a product shortcut, not a required AlertDialog convention. That means we must not steal keys from the established behavior around it. Isolate the extra mapping The companion keyboard.mjs starts with a pure function: export function arrowAction ( key ) { if ( key === " ArrowLeft " ) return " cancel " ; if ( key === " ArrowRight " ) return " confirm " ; return null ; } The event handler ignores unrelated and modified keys: export function handleDialogArrow ( event , controls ) { const action = arrowAction ( event . key ); if ( ! action || event . altKey || event . ctrlKey || event . metaKey ) return false ; event . preventDefault (); controls [ action ]. focus (); return true ; } Notice what is absent: no handler for Tab , Shift+Tab , Escape , or Enter . The native <dialog> and buttons in the minimal demo retain their normal jobs. In a React component, use a well-tested modal/dialog primitive for focus containment and dismissal, then add this narrow handler to its content. A complete minimal dialo

📖 TL;DR GPT-5.6 shipped July 9, 2026 in three tiers Sol (flagship), Terra (balanced), and Luna (cheapest) all tuned for agentic tool calling. All three share a 1M-token context window , 128K max output, and native MCP support in the Responses API. Test any MCP server against Sol, Terra, or Luna in MCP Agent Studio — pick the model, connect a server, and watch each tool call live. OpenAI dropped GPT-5.6 on July 9, 2026 - and this one is aimed squarely at agents. Three models landed at once: Sol , Terra , and Luna . Each is built to call tools, not just chat . That makes testing MCP servers with GPT-5.6 a different exercise than testing a plain chat model. Tool selection is the whole game. I have spent this week pointing all three at MCP servers GitHub, Postgres, Playwright, and multi-server setups. This post is what I learned. You will see which tier to run for which workload , how the new tool-calling features change MCP, and how to test each one free in your browser. Skip it and you will overpay for Sol on jobs Luna handles fine. What Is GPT-5.6? Sol, Terra, and Luna Explained GPT-5.6 is a three-tier model family, not a single model. OpenAI split it by cost and horsepower so you match the model to the job. Here is the lineup, straight from OpenAI's pricing page: Model Built for Input / Output (per 1M) GPT-5.6 Sol Flagship — ambitious agentic work $5.00 / $30.00 GPT-5.6 Terra Balanced — efficient, high-volume work $2.50 / $15.00 GPT-5.6 Luna Fast, affordable — everyday work $1.00 / $6.00 The specs are shared across all three. Every tier gets a 1M-token context window, 128K max output, and a February 16, 2026 knowledge cutoff. So the choice is not about context or capability limits. It is about how much reasoning each task actually needs. New to the protocol these models call? Start with what is Model Context Protocol , then come back. Why GPT-5.6 Changes MCP Tool Calling Here is the part that matters for MCP. GPT-5.6 does not just call tools one at a time it can orc

The live AI employee inside your product, serving every user Discussion | Link

I'll Be Honest: The Internet Already Has Translators I know. Language translation isn't a new idea. There are already huge translation platforms out there. So when I started working on a translator for my tools website, I wasn't thinking: "I'm going to reinvent translation." Not at all. My thought was much simpler: "Can I make quick translation feel less distracting?" My Frustration Was Actually Pretty Simple Sometimes I just need to translate text. That's it. I don't want to: Create an account Open five different menus Break a long text into tiny pieces Jump between multiple tools I want to paste the text... Choose a language... And get the translation. So I Built My Own Version 👉 https://allinonetools.net/language-translator/ The tool currently supports 200+ languages and language variations . You can: Detect the source language Select the target language Translate long text Upload text Use voice input Listen to the result Copy or share the translation And I wanted to keep the text experience simple without forcing users into tiny input limits. Just: Enter → Choose Language → Translate 200+ Languages Sounded Simple Until I Saw the List English. Hindi. Gujarati. Spanish. Arabic. These are the languages most people immediately think about. But then I started going through the full language list. Abkhaz. Acholi. Afar. Alur. Aymara. Baluchi. And many more. Honestly... I hadn't even heard of some of them before building this. That was probably my biggest learning moment. I Realized How Small My Own View of the Internet Was As a developer, it's easy to build around the languages we personally know. For me, seeing English, Hindi, and Gujarati feels normal. But the internet is much bigger than my own experience. Someone somewhere may be trying to understand a sentence in a language I've never even heard spoken. That changed how I looked at this tool. The Hard Part Wasn't Adding a Dropdown A dropdown with 200+ options looks impressive. But that's not the real problem. The

I usually write here about Next.js, caching, and the bugs that steal your sleep. But this post is...

I gotta say, i Spent a Month Testing Chinese AI APIs — Here's What Actually Wins Look, I'm just an indie hacker trying to ship products without going broke. For the past month I've been obsessively running the four biggest Chinese AI model families — DeepSeek, Qwen, Kimi, and GLM — through every test I could think of. And honestly? I wish someone had given me a breakdown like this before I started. So here's my attempt. No corporate fluff, no hand-wavy "it depends" answers. Just real data from someone who actually pays these bills. Why I Even Started Looking at Chinese Models Honestly, I was a GPT-4o loyalist for the longest time. Then I saw my December API bill and nearly choked. $400+ for what amounted to a few chatbot features and some content generation. That's when a friend told me to check out DeepSeek and Qwen. I was skeptical. Like, REALLY skeptical. Chinese models in 2023 were a joke for English tasks. But I kept hearing whispers from other indie hackers about how good things had gotten. So I decided to actually test them properly through Global API's unified endpoint (more on that later). What I found kinda blew my mind. The Quick Cheat Sheet Here's the TL;DR table I wish existed when I started. I'm putting it up top because, lets be real, you probably just want the bottom line: Feature DeepSeek Qwen Kimi GLM Developer DeepSeek (幻方) Alibaba (阿里) Moonshot AI (月之暗面) Zhipu AI (智谱) Price Range $0.25-$2.50/M $0.01-$3.20/M $3.00-$3.50/M $0.01-$1.92/M Best Budget Pick V4 Flash @ $0.25/M Qwen3-8B @ $0.01/M N/A GLM-4-9B @ $0.01/M Best Overall V4 Flash @ $0.25/M Qwen3-32B @ $0.28/M K2.5 @ $3.00/M GLM-5 @ $1.92/M Code Generation ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐ Chinese Language ⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐⭐ English Language ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐⭐ Reasoning ⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐ Speed ⭐⭐⭐⭐⭐ ⭐⭐⭐⭐ ⭐⭐⭐ ⭐⭐⭐⭐ Vision/Multimodal Limited ✅ (VL, Omni) ❌ ✅ (GLM-4.6V) Context Window Up to 128K Up to 128K Up to 128K Up to 128K API Compatibility OpenAI ✅ OpenAI ✅ OpenAI ✅ OpenAI ✅ Alright, now let me act