标签：#Lua

Sequoia is just one of the top firms that sells same equity at two different prices.

Architectural change cases extend architecture decision record (ADR) thinking by evaluating how decisions may evolve over time. Change cases expose hidden assumptions and help teams estimate the reversibility and cost of change. By Pierre Pureur, Kurt Bittner

What is an LLM evaluation harness? A deep dive into lm-eval-harness You fine-tuned a 7B model. It aced your smoke tests, your colleague ran a few prompts and shrugged approvingly, and the README is now full of cherry-picked outputs that look great in a screenshot. Then someone asks: how good is it, really? — and you realize you have no number to point at. No MMLU score. No HellaSwag. Nothing reproducible, nothing you can defend in a PR review, nothing you can compare to last week's checkpoint. That's the gap an evaluation harness fills. It turns "vibes-based evaluation" into something with a score, a stderr, and a config file you can re-run next Tuesday. Why evaluate LLMs at all? Two reasons that actually matter: Comparability. If you can't put a number on a model, you can't compare it to anything else — not the previous checkpoint, not the open-source baseline, not the commercial API you're trying to replace. Leaderboards are noisy and gaming-prone, but a local leaderboard with the tasks you care about is one of the most useful artifacts a team can build. Regression detection. Most model regressions are silent. A 0.3-point drop on MMLU won't show up in a chat session, but it will show up in CI. People who ship models for a living treat evals the way backend engineers treat unit tests: mandatory, run on every PR, and blocking on regressions. You don't need a hundred benchmarks. You need the three to five tasks that map to your actual use case , plus one or two general capability anchors (MMLU, HellaSwag) so you can sanity-check that you didn't accidentally destroy basic reasoning while you were tuning for your domain. What is an "evaluation harness"? An evaluation harness is the software that sits between a model and a benchmark. It handles the boring-but-critical parts: loading the model weights, tokenizing prompts in the way the benchmark expects, running inference, extracting the answer from a longer generation, scoring it against a ground-truth key, aggregating

Microsoft on Tuesday took the wraps off Adaptive Spec-driven Scoring for Evaluation and Regression Testing, an open source framework for spinning up AI evaluations.