标签：#m

Part 2 of the Building create-notils series. In my previous article , I explained why I stopped copy-pasting repositories and started building my own project scaffolding tool. However, one major architectural problem remained: I wanted create-notils to support both of these primary project structures. A standalone Next.js application: my-app/ ├── src/ ├── public/ ├── package.json └── components.json And a Turborepo monorepo: my-app/ ├── apps/ │ └── app/ ├── packages/ │ ├── ui/ │ └── config/ ├── turbo.json └── package.json At first glance, the obvious solution is to maintain two separate templates—one for standalone and one for monorepo. Problem solved, right? Except... it isn't. The Hidden Cost of Multiple Templates Every starter template starts out identical. Then one day, you fix a subtle bug in one template and forget to update the other. A week later, you upgrade Next.js in one repository before getting around to the second. A month later, you improve your UI package and find yourself manually copying files back and forth between folders. Eventually, the templates slowly drift apart. The true cost isn't creating templates; the cost is maintaining them forever. What Actually Changes? When I sat down and compared the two project layouts side by side, surprisingly little was different. The actual application code, UI components, theming, and utility functions were 100% identical. The only real differences were the structural project boundaries: Concern Monorepo Standalone UI Package packages/ui src/components/ui Utilities @notils/ui/lib/utils @/lib/utils Configuration Shared workspace package Local configuration Package Manifests Multiple ( package.json files) Single root manifest Workspace Tooling Present ( turbo.json , workspaces) Removed entirely Everything else was effectively the exact same code. That single observation changed the entire architecture of create-notils . A Different Approach: The Canonical Source of Truth Instead of maintaining two templates, I

When we first decided to bring AI into analysts’ work, the task seemed fairly down-to-earth. The company has several development teams and roughly fifteen analysts. They collect requirements, prepare tasks, describe changes in Confluence, think through testing, and help align future implementation with both business and development. A lot of this work is repetitive, so giving analysts a tool that could prepare first drafts felt like an obvious idea. The problem is that a document in this kind of process almost never stands on its own. The same change exists in Jira, in Confluence, in mockups, in test scenarios, and in technical notes. Sometimes there are also separate instructions for making changes in the codebase. Each artifact has its own purpose, but all of them describe the same future system behavior. If the wording drifts apart, that can go unnoticed for several days, until different people begin working from different versions. A typical case looked roughly like this (the details and names are changed, but the mechanism is real). Jira said that a status field could have three values: draft , active , and archived . Confluence still contained an older table with only two values, while the test scenario also checked for disabled , which had been discussed early on and later rejected. The developer implemented what was written in Jira. The tester opened the scenario and filed a defect because disabled was missing. Only after that did the analyst compare the documents and realize that each of them preserved a different version of the decision. Nothing catastrophic happened. We simply had to go through the documentation again, update the tests, clarify the task, explain to the developer that the code did not need to change, and send the package through review one more time. It’s exactly these “nothing serious” moments that add up to delays, when the same task travels two or three times between an analyst, a developer, and a tester. At some point, it becomes diffi

Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40% How we moved from "semantic search + hope" to a measured, tunable retrieval pipeline with 95% recall@10 The RAG Reality Check Everyone ships RAG the same way: chunk by 512 tokens, embed with text-embedding-3-small , top-k=5, stuff into context. It works for demos. Then you hit production: Legal contracts: 512 tokens splits clauses mid-sentence API docs: 1000-token chunks drown signal in noise Customer tickets: Conversational context needs overlap, not fixed windows Latency: 500ms embedding + 200ms vector search + 300ms LLM = 1s+ per query We rebuilt our retrieval layer from first principles. Here's what actually moves metrics. Chunking: One Size Fits None # rag/chunking.py from abc import ABC , abstractmethod from dataclasses import dataclass @dataclass class Chunk : text : str metadata : dict token_count : int chunk_id : str class ChunkingStrategy ( ABC ): @abstractmethod def chunk ( self , document : str , metadata : dict ) -> list [ Chunk ]: ... class FixedTokenChunker ( ChunkingStrategy ): """ Baseline. Good for homogeneous content. """ def __init__ ( self , chunk_size = 512 , overlap = 50 ): self . chunk_size = chunk_size self . overlap = overlap class RecursiveChunker ( ChunkingStrategy ): """ Respects structure: markdown headers, code blocks, paragraphs. """ def __init__ ( self , separators = [ " \n ## " , " \n ### " , " \n\n " , " \n " , " " ], chunk_size = 512 ): self . separators = separators self . chunk_size = chunk_size class SemanticChunker ( ChunkingStrategy ): """ Uses embedding similarity to find natural boundaries. """ def __init__ ( self , model = " text-embedding-3-small " , threshold = 0.7 ): self . model = model self . threshold = threshold class AgenticChunker ( ChunkingStrategy ): """ LLM decides boundaries. Expensive but highest quality for complex docs. """ def __init__ ( self , model = " gpt-4o-mini " ): self . model = model Our production config by

Building Production-Grade LLM Evaluation Pipelines: From Vibes to Metrics How we replaced "looks good to me" with automated evaluation catching 92% of hallucinations before deployment The Problem: Why "Vibe Checks" Fail in Production Three months ago, our team shipped a RAG-based customer support assistant. It worked great in testing — we'd ask it questions, read the answers, and say "yeah, that looks right." Then it hit production. A customer asked about their billing cycle. The assistant confidently cited a policy that didn't exist. Another asked about API rate limits and got numbers from a competitor's documentation. By the time we caught it, 500+ users had seen hallucinated responses. The post-mortem was brutal: we had zero automated evaluation . Our test process was literally "ask 5 questions, read answers, thumbs up." What Production Evaluation Actually Needs Academic benchmarks (MMLU, HellaSwag) don't tell you if your system works for your use case. Production evaluation needs: Domain-specific judges — Your criteria, not generic "helpfulness" Speed — Evaluation must run in CI/CD, not overnight Regression detection — Know immediately when a prompt change breaks things CI/CD integration — Block merges that degrade quality Golden dataset management — Versioned, stratified, growing test cases Architecture: The Evaluation Pipeline ┌─────────────┐ ┌──────────────┐ ┌────────────────────┐ ┌──────────────┐ │ Test Cases │────▶│ LLM Under │────▶│ Judge Ensemble │────▶│ Metrics & │ │ (Golden Set)│ │ Test │ │ - Faithfulness │ │ Regression │ └─────────────┘ └──────────────┘ │ - Instruction F. │ │ Detection │ │ - JSON Schema │ └──────┬───────┘ │ - Custom LLM │ ▼ └────────────────────┘ ┌──────────────┐ │ Dashboard/ │ │ PR Comments │ └──────────────┘ Core Abstractions # eval/base.py @dataclass ( frozen = True ) class TestCase : id : str input : dict [ str , Any ] expected : dict [ str , Any ] | None = None tags : list [ str ] = field ( default_factory = list ) # ["edge-case",

The next time you apply for a job, AI may screen your résumé before any human sees it. But there’s good reason to question whether AI will judge you fairly. Researchers already know that LLMs pick up human biases from their training data. New research suggests that LLMs can also develop their own biases from…

Hey everyone. Just an update on the Salesforce Certification. I am planning on releasing articles...

Liquid syntax error: Variable '{{% raw %}' was not properly terminated with regexp: /\}\}/

Tu prueba pasó el lunes: misma entrada, mismo código y temperature=0 . El martes falló sin cambios en tu aplicación. La aserción esperaba una cadena exacta, pero el modelo devolvió la misma respuesta con una redacción ligeramente distinta. El agente funciona; tu suite de pruebas no. Prueba Apidog hoy Este es el coste de probar sistemas que llaman a modelos de lenguaje. Incluso con temperatura cero, no obtendrás salidas idénticas byte a byte entre ejecuciones. En lugar de probar la redacción exacta, prueba el contrato: estructura, campos, rangos y efectos esperados. Este artículo profundiza en el tercer modo de fallo de nuestra guía sobre por qué los agentes de IA fallan en producción . Por qué temperature=0 no significa determinismo La temperatura controla cómo el modelo selecciona el siguiente token. Con temperature=0 , el modelo elige el token más probable, lo que parece reproducible. Sin embargo, esa configuración no garantiza resultados idénticos. La causa está debajo del modelo: Las operaciones de punto flotante en GPU no son asociativas: el orden de las sumas puede modificar mínimamente el resultado. Esa diferencia puede cambiar cuál token queda en primer lugar. El orden de ejecución puede depender de cómo el proveedor agrupa solicitudes, del hardware, de la región o de la versión del kernel. Los proveedores pueden cambiar GPUs, bibliotecas de inferencia o cuantización de pesos. Una discusión extensa en vLLM explica por qué una semilla fija y temperature=0 no bastan para lograr reproducibilidad bit a bit. La conclusión práctica es simple: el determinismo es una propiedad de toda la pila de servicio, no una opción de tu solicitud. Tu prueba debe aceptar variaciones de redacción cuando el significado y el contrato siguen siendo correctos. Por qué las aserciones de cadena exacta vuelven inestable tu suite Esta prueba es frágil: assert response == " Your order total is $42.00. " Puede pasar hoy y fallar mañana si el modelo responde: Your total comes to $42.00. La

Your test passed on Monday. Same input, same code, temperature=0 . On Tuesday it failed, and you changed nothing. The assertion expected an exact string, but the model returned the same answer with slightly different wording. The agent is fine; the test is not. Try Apidog today This is the cost of testing language-model output. Even at temperature=0 , you cannot rely on byte-identical responses across runs. Instead of testing exact wording, test the API contract: structure, required fields, valid ranges, tool-call payloads, and safety constraints. This is a deeper look at failure mode three in our guide to why AI agents break in production . Why temperature=0 does not mean deterministic Temperature controls token sampling. At zero, the model selects the most probable next token, which sounds reproducible. In practice, the serving stack introduces variation. GPU floating-point math is not associative: adding the same values in a different order can produce small numerical differences. Those differences can change which token ranks highest. Once one token differs, every token after it can diverge. The order of operations can vary based on: Request batching GPU hardware Inference kernels Provider routing and regions Model-serving library versions Weight quantization changes Providers can also update infrastructure without changing your API request. As this vLLM discussion explains, a fixed seed and temperature=0 are not enough for bitwise reproducibility. Determinism is a property of the entire serving stack, not a request parameter. Design tests accordingly. Why exact-string assertions make tests flaky This assertion is fragile: assert . equal ( response , " Your order total is $42.00. " ); It fails if the model returns a correct variation: Your total comes to $42.00. The failure does not indicate a product regression. It indicates that the test is coupled to wording that is expected to vary. Flaky tests create a predictable failure pattern: Developers rerun the suite

Cross-posted from the Loopress docs Loopress is a toolset to make WordPress reproducible and reviewable via Git . Versioned snippets, Composer without SSH, and more coming... Check Loopress WordPress doesn't have a package manager. If you want a PHP library in your project, be it Guzzle for HTTP calls or a PDF generation library in a snippet, you're either vendoring the code by hand or running Composer somewhere the WordPress admin can't see. We built a feature to fix that: a Composer UI inside the WordPress admin. Search Packagist, install a package, audit it for known vulnerabilities, all without SSH access ( full walkthrough here ). Before shipping it, we asked the wordpress.org plugin review team whether it would be acceptable in the official directory. The answer was no. The rule Here's the relevant line from Guideline 8 of the wordpress.org plugin directory: "Plugins may not send executable code via third-party systems." Installing a PHP package from Packagist is, by definition, downloading executable code from a third party and writing it to disk. It doesn't matter that our plugin never calls the installed code's autoloader itself, that the user has to load it deliberately from their own snippet. The review team was clear: the indirection changes nothing. The mere presence of that capability is enough to trigger the rule, whether or not it's ever used. There's no folder you can hide it in that makes it acceptable, they weren't shy about saying that outright. If you want PHP dependencies in a plugin distributed on wordpress.org, the only accepted path is to vendor them at submission time: ship the code, with a compatible license and readable source, not fetch it dynamically. For us, that meant Composer package management could never live in the same plugin that ships on wordpress.org. Full stop, no clever workaround changes that. What we tried first, and undid Our first move was the obvious one: split into two plugins. A "Core" plugin with snippet sync, distri

Running a production database on a minimal two-core PostgreSQL instance presents unique engineering challenges. In resource-constrained environments, default configurations quickly lead to CPU exhaustion, memory thrashing, and high latency. To maintain stable performance, you must aggressively manage resource allocation and optimize query execution plans. The first critical area is connection management. PostgreSQL allocates a separate operating system process for each connection. On a two-core machine, allowing hundreds of concurrent connections will cause severe context switching overhead, degrading performance significantly. You should restrict maximum connections to a low double-digit number and implement a connection pooler like PgBouncer. A pooler ensures that incoming traffic queuing happens outside the database engine, allowing the CPU to focus on executing active queries rather than managing process states. Memory allocation parameters require precise tuning on limited hardware. The shared buffers parameter, which dictates how much memory PostgreSQL uses for caching data, should typically be set to twenty-five percent of total system RAM. However, work memory is where many developers run into trouble. The work memory setting determines the amount of memory used by internal sort operations and hash tables before writing to temporary disk files. Since this memory is allocated per query operation, a complex query with multiple joins can consume many times the configured value. On a two-core instance with limited RAM, setting this value too high can trigger the out-of-memory killer, while setting it too low forces slow disk-based sorting. You must analyze your heaviest queries and find a balanced value that prevents disk thrashing without exhausting system memory. Query optimization becomes a daily necessity when compute resources are scarce. You must regularly inspect query execution plans using the explain analyze command to identify sequential scans on large

We’ve all been there: you are busy coding, and suddenly you get a scary error saying your code conflicts with a teammate's work. Git won't let you pull their changes because it's afraid of overwriting your unsaved files. Instead of panicking, I used git stash. This command acts like a temporary clipboard—it safely hides your current work away so your workspace becomes completely clean. With a clean screen, I was able to safely update the project, fix the conflicting lines of code by hand, and make sure everything compiled perfectly. Once the team's updates were safely pushed, I ran git stash pop to bring my hidden work right back. Git wasn't trying to break my project; it was just protecting my code. Next time you get stuck, remember the golden rule: Stash your work, fix the conflict, and pop it back!

Most people working in offices never think about why the temperature stays comfortable throughout the day. The cooling adjusts automatically. Fresh air increases when occupancy rises. Fans start and stop without anyone touching a switch. Behind all of this is a device that most building occupants have never heard of: the DDC Controller. The Hidden Computer Inside Every Modern Building Walk into a mechanical room and you'll find equipment everywhere: Air Handling Units (AHUs) Chillers Pumps Cooling Towers VAV Boxes All these systems need coordination. If the supply air temperature rises above its target, something has to react. If occupancy increases, fresh air must increase. If a fan trips, alarms must be generated. This is where a DDC controller comes in. Think of it as a small industrial computer dedicated to one job: keeping a building running efficiently. A Typical Day in the Life of a DDC Controller Imagine an AHU supplying air to an office floor. At 9:00 AM employees begin arriving. The return air temperature starts increasing. The DDC controller notices this through a temperature sensor. Within seconds it: Reads the sensor value Compares it against the setpoint Calculates the cooling demand Adjusts the chilled water valve Verifies fan operation Repeats the process No operator is required. No manual intervention is needed. The controller quietly performs these calculations all day. Why Not Just Use a PLC? This is one of the most common questions from engineers entering building automation. PLCs and DDC controllers are both programmable devices. However, they were designed for different worlds. A PLC excels at: Manufacturing lines Packaging machines Process control High-speed sequencing A DDC controller excels at: HVAC control Energy optimization Occupancy schedules Comfort management BACnet communication Both can control equipment. The difference is what they were originally built for. The Four Signals Every BMS Engineer Learns First If you're new to building

Hey! This project was dormant for like 8 years, but now that we have vibe coding, or even better: agentic engineering, I gave it a shot. Of course, I know vibe coding is bad 😉 but I also have a family, so... I'm aware that there are like several dozens of other ways in crypto land to do this somewhat differently, and with different properties. It took two weekends and a few dozen prompts to make it work both in a network simulator and using a command line program and a daemon. The most important parts were written by me back in 2018, I was just having fun. I'm not sure what I want to do with this. It's currently just a toy work in-progress project that I thought may be interesting. I'm actually a software developer working on a communication network simulator in the past 20 years... so this is not entirely new to me. The GitHub link is on the page. Happy hacking! submitted by /u/melevy [link] [留言]

Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40% How we moved from "semantic search + hope" to a measured, tunable retrieval pipeline with 95% recall@10 The RAG Reality Check Everyone ships RAG the same way: chunk by 512 tokens, embed with text-embedding-3-small , top-k=5, stuff into context. It works for demos. Then you hit production: Legal contracts: 512 tokens splits clauses mid-sentence API docs: 1000-token chunks drown signal in noise Customer tickets: Conversational context needs overlap, not fixed windows Latency: 500ms embedding + 200ms vector search + 300ms LLM = 1s+ per query We rebuilt our retrieval layer from first principles. Here's what actually moves metrics. Chunking: One Size Fits None # rag/chunking.py from abc import ABC , abstractmethod from dataclasses import dataclass @dataclass class Chunk : text : str metadata : dict token_count : int chunk_id : str class ChunkingStrategy ( ABC ): @abstractmethod def chunk ( self , document : str , metadata : dict ) -> list [ Chunk ]: ... class FixedTokenChunker ( ChunkingStrategy ): """ Baseline. Good for homogeneous content. """ def __init__ ( self , chunk_size = 512 , overlap = 50 ): self . chunk_size = chunk_size self . overlap = overlap class RecursiveChunker ( ChunkingStrategy ): """ Respects structure: markdown headers, code blocks, paragraphs. """ def __init__ ( self , separators = [ " \n ## " , " \n ### " , " \n\n " , " \n " , " " ], chunk_size = 512 ): self . separators = separators self . chunk_size = chunk_size class SemanticChunker ( ChunkingStrategy ): """ Uses embedding similarity to find natural boundaries. """ def __init__ ( self , model = " text-embedding-3-small " , threshold = 0.7 ): self . model = model self . threshold = threshold class AgenticChunker ( ChunkingStrategy ): """ LLM decides boundaries. Expensive but highest quality for complex docs. """ def __init__ ( self , model = " gpt-4o-mini " ): self . model = model Our production config by

Originally published on the MyTreda engineering blog: read here A support message described what looked like one confusing mobile bug. It turned out to be two independent ones — a cross-site cookie getting blocked by Safari's ITP, and a React Hook Form autofill desync — that just happened to both only show up on mobile. Full breakdown, code, and fixes below.

If you’ve spent any time recently writing detailed product requirement documents or meticulously...

Imagine your AI assistant just produced 200 lines of code. Legally, you may not own a single line of...

Hey any one experience overall score 2.66 and then Meta score 3 in some previous cycle ?? Or meta reviewer just rounded off 2.66 to 2.5?? Any Meta Reviewer here?? because there are some uninterested reviewers doing AI generated reviews and giving noisy scores. For them overall score gets lowered. submitted by /u/Historical_Pause247 [link] [留言]

A helpful comment on my EXIF test suggested using a metadata scrubber. That is useful for ordinary still images, but a mobile upload contract must define every asset it sends. An iOS Live Photo can include both a photo resource and a paired video resource. The failure case is simple: the JPEG derivative has no GPS EXIF, while metadata or an original file associated with the paired video survives in the upload or retry path. Build the resource inventory first: let resources = PHAssetResource . assetResources ( for : asset ) for resource in resources { print ( resource . type , resource . originalFilename ) } Then make privacy assertions per output, not per UI selection: Artifact Required check still derivative no EXIF GPS/location fields paired video derivative no location metadata upload manifest only derived filenames retry queue no path to original resources temporary directory deleted after success or cancellation My lifecycle test would start an upload, force the app into the background after the still image is prepared, kill it while the paired video is processing, and relaunch. Recovery must either regenerate both safe derivatives or delete the incomplete pair. It must never mix a scrubbed still with an original video. Apple’s PHAssetResource API exposes the resources associated with a Photos asset. That enumeration should become evidence in the test: fail when an unexpected resource type is present rather than silently uploading it. Also verify what reaches the server. Device-side inspection alone misses multipart manifests, queued originals, and server-generated previews. Download the stored pair in a test environment and run the same metadata assertions again. A “remove location” button needs a precise scope. For Live Photos, the user reasonably expects it to cover the complete paired asset and every retry copy—not just the JPEG they can see.