标签：#m

Hey! This project was dormant for like 8 years, but now that we have vibe coding, or even better: agentic engineering, I gave it a shot. Of course, I know vibe coding is bad 😉 but I also have a family, so... I'm aware that there are like several dozens of other ways in crypto land to do this somewhat differently, and with different properties. It took two weekends and a few dozen prompts to make it work both in a network simulator and using a command line program and a daemon. The most important parts were written by me back in 2018, I was just having fun. I'm not sure what I want to do with this. It's currently just a toy work in-progress project that I thought may be interesting. I'm actually a software developer working on a communication network simulator in the past 20 years... so this is not entirely new to me. The GitHub link is on the page. Happy hacking! submitted by /u/melevy [link] [留言]

Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40% How we moved from "semantic search + hope" to a measured, tunable retrieval pipeline with 95% recall@10 The RAG Reality Check Everyone ships RAG the same way: chunk by 512 tokens, embed with text-embedding-3-small , top-k=5, stuff into context. It works for demos. Then you hit production: Legal contracts: 512 tokens splits clauses mid-sentence API docs: 1000-token chunks drown signal in noise Customer tickets: Conversational context needs overlap, not fixed windows Latency: 500ms embedding + 200ms vector search + 300ms LLM = 1s+ per query We rebuilt our retrieval layer from first principles. Here's what actually moves metrics. Chunking: One Size Fits None # rag/chunking.py from abc import ABC , abstractmethod from dataclasses import dataclass @dataclass class Chunk : text : str metadata : dict token_count : int chunk_id : str class ChunkingStrategy ( ABC ): @abstractmethod def chunk ( self , document : str , metadata : dict ) -> list [ Chunk ]: ... class FixedTokenChunker ( ChunkingStrategy ): """ Baseline. Good for homogeneous content. """ def __init__ ( self , chunk_size = 512 , overlap = 50 ): self . chunk_size = chunk_size self . overlap = overlap class RecursiveChunker ( ChunkingStrategy ): """ Respects structure: markdown headers, code blocks, paragraphs. """ def __init__ ( self , separators = [ " \n ## " , " \n ### " , " \n\n " , " \n " , " " ], chunk_size = 512 ): self . separators = separators self . chunk_size = chunk_size class SemanticChunker ( ChunkingStrategy ): """ Uses embedding similarity to find natural boundaries. """ def __init__ ( self , model = " text-embedding-3-small " , threshold = 0.7 ): self . model = model self . threshold = threshold class AgenticChunker ( ChunkingStrategy ): """ LLM decides boundaries. Expensive but highest quality for complex docs. """ def __init__ ( self , model = " gpt-4o-mini " ): self . model = model Our production config by

Originally published on the MyTreda engineering blog: read here A support message described what looked like one confusing mobile bug. It turned out to be two independent ones — a cross-site cookie getting blocked by Safari's ITP, and a React Hook Form autofill desync — that just happened to both only show up on mobile. Full breakdown, code, and fixes below.

If you’ve spent any time recently writing detailed product requirement documents or meticulously...

Imagine your AI assistant just produced 200 lines of code. Legally, you may not own a single line of...

Hey any one experience overall score 2.66 and then Meta score 3 in some previous cycle ?? Or meta reviewer just rounded off 2.66 to 2.5?? Any Meta Reviewer here?? because there are some uninterested reviewers doing AI generated reviews and giving noisy scores. For them overall score gets lowered. submitted by /u/Historical_Pause247 [link] [留言]

A helpful comment on my EXIF test suggested using a metadata scrubber. That is useful for ordinary still images, but a mobile upload contract must define every asset it sends. An iOS Live Photo can include both a photo resource and a paired video resource. The failure case is simple: the JPEG derivative has no GPS EXIF, while metadata or an original file associated with the paired video survives in the upload or retry path. Build the resource inventory first: let resources = PHAssetResource . assetResources ( for : asset ) for resource in resources { print ( resource . type , resource . originalFilename ) } Then make privacy assertions per output, not per UI selection: Artifact Required check still derivative no EXIF GPS/location fields paired video derivative no location metadata upload manifest only derived filenames retry queue no path to original resources temporary directory deleted after success or cancellation My lifecycle test would start an upload, force the app into the background after the still image is prepared, kill it while the paired video is processing, and relaunch. Recovery must either regenerate both safe derivatives or delete the incomplete pair. It must never mix a scrubbed still with an original video. Apple’s PHAssetResource API exposes the resources associated with a Photos asset. That enumeration should become evidence in the test: fail when an unexpected resource type is present rather than silently uploading it. Also verify what reaches the server. Device-side inspection alone misses multipart manifests, queued originals, and server-generated previews. Download the stored pair in a test environment and run the same metadata assertions again. A “remove location” button needs a precise scope. For Live Photos, the user reasonably expects it to cover the complete paired asset and every retry copy—not just the JPEG they can see.

A SQLite service can keep answering health checks while its WAL grows without a successful checkpoint. One forgotten read transaction is enough to reproduce the risk. Run a controlled drill: Enable WAL mode and create a small table. Open connection A, begin a read transaction, and keep it open. From connection B, commit batches of writes for 60 seconds. Sample WAL bytes and checkpoint results every second. Release A and observe recovery. PRAGMA journal_mode = WAL ; PRAGMA wal_checkpoint ( PASSIVE ); Record the three checkpoint counters plus duration, WAL file size, oldest transaction age, write latency, and free disk bytes. A green SELECT 1 says nothing about checkpoint progress. My fault-injection gate looks like this: Given: one reader holds its transaction When: 10,000 writes commit Then: writes remain bounded And: WAL growth triggers an alert before the disk budget When: the reader closes Then: checkpoint progress resumes and WAL size converges Do not start with an automatic TRUNCATE loop. A busy result is evidence of contention; aggressive checkpoints can add latency without fixing the reader lifecycle. First identify long transactions, ensure result iterators close, and define a maximum transaction age. SQLite’s WAL documentation explains that a checkpoint must stop when it reaches pages beyond an active reader’s end mark. This is why the drill needs concurrency rather than a synthetic file-growth assertion. Operational thresholds should be tied to a disk budget: alert when projected WAL growth reaches the remaining safe window, not at an arbitrary file size. The runbook should name how to find the oldest reader, how to shed writes, and when a restart is safer than waiting. A health check is useful only when it measures the subsystem that is failing.

Suppose leadership rewards teams for increasing the percentage of “AI-assisted pull requests.” The dashboard rises. Did productivity improve, or did people learn which box to tick? Before launching that metric, I would run a consequence-mapping session: Intended behavior Plausible adaptation Counter-metric try useful assistance label trivial PRs as assisted retained task outcome ship faster split work into tiny PRs lead time per task share adoption avoid difficult non-AI work task-mix distribution accept suggestions reduce review scrutiny rollback and defect rate The metric card should make disagreement possible: name : ai_assisted_pr_share purpose : detect workflow adoption, not productivity owner : developer-experience known_game : self-label inflation counter_metrics : [ task_mix , review_minutes , rollback_rate ] review_date : 2026-08-19 retire_when : classification cannot be audited Then interview both high and low scorers without treating the score as performance. Ask what work disappeared, what new verification appeared, and what behavior the dashboard encouraged. Include an anonymous channel: a metric cannot reveal pressure if challenging it carries career risk. The SPACE framework argues that developer productivity cannot be captured by one dimension. That is especially relevant when AI telemetry is easy to count but verification and rework are harder to observe. My launch gate is not “the metric is accurate.” It is: teams can inspect its definition, challenge its interpretation, and show where it changes behavior. If the counter-metrics diverge, pause incentives before refining the chart. What behavior would your current AI dashboard accidentally reward?

An accepted AI suggestion is an event, not a durable outcome. If the code is rewritten tomorrow, acceptance rate still calls it a success. For an adoption review, I would connect three timestamps: suggestion : accepted_at : 2026-07-19T09:00:00Z repository : api task_type : test change : retained_lines_24h : 31 rewritten_lines_24h : 9 reverted_at : null review : human_minutes : 12 incident_link : null Then report a funnel rather than one flattering percentage: shown → accepted → merged → retained_24h → retained_14d A useful unit is cost per retained task : (tool cost + review labor + rework labor) / retained tasks “Retained” needs a written contract. For example: the change remains merged after 14 days, passes required checks, and has not caused a linked rollback. Line survival alone is weak because formatting and refactoring can change lines without rejecting the solution. Segment the result by task type and repository. Boilerplate tests and unfamiliar security changes should not be blended into one portfolio average. Also publish counter-metrics: review time, escaped defects, rollback rate, and developer-reported interruption. GitHub documents available fields and limitations in its Copilot metrics API . Those product metrics can be inputs, but the retained-task join belongs to the adopting organization and should be versioned like any other analytics contract. My pilot gate would be simple: expand only if retained-task cost beats the existing workflow without worsening rollback rate. Otherwise, change the workflow before buying more seats. Record the baseline before enabling the tool, and keep one comparable task cohort outside the rollout; without that counterfactual, a rising retention rate may only reflect easier work entering the sample. What retention window would make an accepted change meaningful for your team?

24 hours. That's how long a contract lasted before someone pulled it because I was running Windows,...

A consumer health app isn't one product. It's eleven products stitched together into a single...

I recently spent some time profiling a Python code analysis pipeline that processes large repositories into a structured knowledge bundle. Rather than trying random optimizations, I profiled each stage, fixed the biggest bottleneck, and then profiled again. What surprised me was that every optimization exposed a completely different bottleneck. The benchmark was run on a real workspace containing: 23 GB source tree ~18,000 source files ~41,000 extracted concepts Timeline Stage Before After ---------------------------------------- Filesystem Walk 58.0s 0.65s Parsing 16.7s 3.23s Cross-reference Link 9.5s 0.61s Bundle Write 98.0s 4.10s ---------------------------------------- Total 157.0s 12.0s The biggest improvements came from removing unnecessary work rather than simply adding more parallelism: Replaced Path.rglob() with os.walk() and directory pruning (avoiding hundreds of thousands of unnecessary filesystem visits). Parallelized parsing with ProcessPoolExecutor because parsing was genuinely CPU-bound. Replaced repeated list scans with precomputed indexes, caching, and set-based lookups in the linker. Profiled inside the write stage and discovered that yaml.safe_dump() accounted for nearly 95% of render time for a fixed front matter schema. Replacing it with a schema-specific serializer became one of the biggest remaining wins. One takeaway that surprised me is how misleading optimization can be without profiling. I initially assumed parsing would dominate, but after each fix, a completely different stage became the bottleneck. I documented the entire optimization journey—including benchmark methodology, implementation details, trade-offs, and the optimizations that didn't matter—in this write-up: Performance write-up: performance The implementation is part of an open-source project called OKF Generator , but I tried to make the article useful even if you're not interested in the project itself: okf-generator I'd love feedback from anyone who has worked on compiler

As someone who studied stats undergrad and industrial engineering operations research grad, and who thinks about the practical business of ML components in software.... I get lost and a bit hopeless when I think about how to make useful software out of ML models in a reasonable amount of time, and in the current business environment. And when I look at the businesses where I have worked that have mountains of middle management running tiny bits of the ML model lifecycle (think feature extraction, data ingestion and integration, training infra, hosting infra, more hosting infra, applied science)... that only makes my head hurt even more. How do you go about making practical software out of ML components? Edit: I should mention that I mean from scratch ML components, not just a call to a third party hosted tool. submitted by /u/ConstructionBoth6461 [link] [留言]

https://mdooai.com AI 모델을 다루다 보면 코드는 돌아가는데 왜 성능이 안 나오는지 이해하지 못하는 순간이 옵니다. 그 답은 대부분 수학에 있습니다. 실무 AI 개발자로 성장하기 위해 반드시 잡아야 할 최소한의 수학 개념과, 그것을 어떻게 공부해야 하는지를 구체적으로 살펴보겠습니다. 코딩만으로는 해결할 수 없는 AI 모델 성능의 열쇠 많은 주니어 개발자들이 비슷한 경험을 합니다. PyTorch나 TensorFlow로 튜토리얼 코드를 복사하고, 학습도 돌리고, 결과도 나옵니다. 그런데 모델이 왜 이렇게 예측하는지, 왜 학습이 멈추는지, 어떻게 하면 성능이 올라가는지는 설명할 수가 없습니다. 이 간극의 정체가 바로 수학입니다. 가령 모델이 과적합(overfitting)될 때 L2 정규화를 쓰라는 조언을 듣습니다. 코드 한 줄로 해결되지만, 그게 왜 작동하는지는 가중치 벡터의 크기를 제한한다는 선형대수와 확률론적 추론이 얽혀 있습니다. 이 원리를 이해하는 개발자와 모르는 개발자가 새로운 문제에서 만들어내는 솔루션의 질은 다를 수밖에 없습니다. 다행인 건, AI 개발에서 필요한 수학은 대학원 수준의 수리해석학이 아닙니다. 핵심 개념 몇 가지를 실용적 도구로 이해하는 것으로 충분합니다. 선형대수와 확률통계, 어떤 개념부터 공부해야 할까? 선형대수와 확률통계는 AI 수학의 두 축입니다. 그런데 이 두 분야 전체를 공부하려 들면 끝이 없습니다. 개발자 관점에서 실제로 반복해서 등장하는 개념만 추려내면 다음과 같습니다. 선형대수에서 우선해야 할 개념 벡터와 행렬의 연산 : 데이터는 사실상 전부 벡터와 행렬로 표현됩니다. 이미지는 픽셀값 행렬이고, 텍스트는 임베딩 벡터입니다. 행렬 곱(matrix multiplication)이 신경망의 순전파(forward pass) 그 자체입니다. 내적(dot product)과 유사도 : 추천 시스템, 어텐션 메커니즘(Transformer의 핵심)이 내적 연산 위에 서 있습니다. 고유값(eigenvalue)과 고유벡터(eigenvector) : PCA(주성분 분석)처럼 차원을 줄이는 기법을 이해하려면 반드시 필요합니다. 행렬 분해(SVD 등) : 추천 시스템과 자연어 처리의 기반 기술에 등장합니다. 확률통계에서 우선해야 할 개념 확률분포와 기댓값 : 모델의 출력이 확률인 이유, 소프트맥스(softmax)가 하는 일을 이해하는 기반입니다. 베이즈 정리 : 사전 지식을 데이터로 업데이트하는 논리 구조로, 생성 모델과 불확실성 추정에 직접 연결됩니다. 최대 우도 추정(MLE, Maximum Likelihood Estimation) : 모델 학습이 왜 손실 함수를 최소화하는 방향인지를 설명하는 원리입니다. 크로스 엔트로피 손실 : 분류 모델에서 가장 자주 쓰이는 손실 함수로, 확률론과 정보이론이 만나는 지점입니다. 분야 핵심 개념 AI 실무 연결 지점 선형대수 행렬 곱, 내적 신경망 순전파, 어텐션 선형대수 고유값·고유벡터 PCA, 차원 축소 선형대수 행렬 분해(SVD) 추천 시스템, NLP 확률통계 확률분포, 기댓값 소프트맥스, 분류 출력 확률통계 베이즈 정리 생성 모델, 불확실성 추정 확률통계 최대 우도 추정 손실 함수의 논리적 근거 확률통계 크로스 엔트로피 분류 모델 학습 활성화 함수와 역전파에서 미적분은 어떻게 작동하는가? 미적분은 신경망 학습의 핵심 메커니즘인 역전파(backpropagation)를 이해하는 데 필요합니다. 겁먹을 필요는 없습니다. AI 개발에서 실제로 필요한 미적분 개념은 크게 두 가지입니다. 1. 편미분과 기울기(gradient) 손실 함수는 모델의 가중치를 변수로 가지는 다변수 함수입니다. 학습이란 이 함수의 값을 줄이는 방향으로 가중치를 조금씩 조정하는 과정입니다. 이 '방향'을 계산하는 도구가 편미분이고, 모든 가중치에 대한 편미분을 모아놓은 것이 기울기(gradient)입니다. 경사하강법(gradient descent)은 이 기울기의 반대 방향으로 가중치를 업데이트합니다. 수식으로는 다음과 같습니다. 새 가중치 = 기존 가중치 − 학습률(lr) × 기

I run several AI coding agents at once and never really knew what they were doing — what it cost, which one was stuck, what they just changed. So I built agentglass : real-time mission control and a workspace for AI coding agents, across every provider and every project on your machine. ▶ Live demo (no install) · ⭐ GitHub What it does Point any agent at it — Claude Code hooks or any OpenTelemetry exporter (Codex, Gemini, Bedrock…) — and watch everything move live: 💰 Cost per event / session / model ⏱️ Tool latency (real p50 / p95) 🔴 Error timelines + a "what needs you" alert center 💾 Persists (SQLite) — history's there the moment you open it More than a viewer — a workspace All one keystroke away, same cockpit: 🔬 Diff & review — every Edit/Write, syntax-highlighted 🌿 Source control — lazygit in the browser 🐳 Docker — lazydocker in the browser ▶ Real terminal — an actual PTY shell, not an emulation 💬 Chat — drive local Claude sessions Open source (MIT), local-first Built solo. Small stack: Bun + SQLite, React/Vite, Tauri desktop, a stdlib-only Python hook forwarder. If you run more than one agent at a time, I'd love your feedback 👇 ⭐ https://github.com/SirAllap/agentglass

Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40% How we moved from "semantic search + hope" to a measured, tunable retrieval pipeline with 95% recall@10 The RAG Reality Check Everyone ships RAG the same way: chunk by 512 tokens, embed with text-embedding-3-small , top-k=5, stuff into context. It works for demos. Then you hit production: Legal contracts: 512 tokens splits clauses mid-sentence API docs: 1000-token chunks drown signal in noise Customer tickets: Conversational context needs overlap, not fixed windows Latency: 500ms embedding + 200ms vector search + 300ms LLM = 1s+ per query We rebuilt our retrieval layer from first principles. Here's what actually moves metrics. Chunking: One Size Fits None # rag/chunking.py from abc import ABC , abstractmethod from dataclasses import dataclass @dataclass class Chunk : text : str metadata : dict token_count : int chunk_id : str class ChunkingStrategy ( ABC ): @abstractmethod def chunk ( self , document : str , metadata : dict ) -> list [ Chunk ]: ... class FixedTokenChunker ( ChunkingStrategy ): """ Baseline. Good for homogeneous content. """ def __init__ ( self , chunk_size = 512 , overlap = 50 ): self . chunk_size = chunk_size self . overlap = overlap class RecursiveChunker ( ChunkingStrategy ): """ Respects structure: markdown headers, code blocks, paragraphs. """ def __init__ ( self , separators = [ " \n ## " , " \n ### " , " \n\n " , " \n " , " " ], chunk_size = 512 ): self . separators = separators self . chunk_size = chunk_size class SemanticChunker ( ChunkingStrategy ): """ Uses embedding similarity to find natural boundaries. """ def __init__ ( self , model = " text-embedding-3-small " , threshold = 0.7 ): self . model = model self . threshold = threshold class AgenticChunker ( ChunkingStrategy ): """ LLM decides boundaries. Expensive but highest quality for complex docs. """ def __init__ ( self , model = " gpt-4o-mini " ): self . model = model Our production config by

Building Production-Grade LLM Evaluation Pipelines: From Vibes to Metrics How we replaced "looks good to me" with automated evaluation catching 92% of hallucinations before deployment The Problem: Why "Vibe Checks" Fail in Production Three months ago, our team shipped a RAG-based customer support assistant. It worked great in testing — we'd ask it questions, read the answers, and say "yeah, that looks right." Then it hit production. A customer asked about their billing cycle. The assistant confidently cited a policy that didn't exist. Another asked about API rate limits and got numbers from a competitor's documentation. By the time we caught it, 500+ users had seen hallucinated responses. The post-mortem was brutal: we had zero automated evaluation . Our test process was literally "ask 5 questions, read answers, thumbs up." What Production Evaluation Actually Needs Academic benchmarks (MMLU, HellaSwag) don't tell you if your system works for your use case. Production evaluation needs: Domain-specific judges — Your criteria, not generic "helpfulness" Speed — Evaluation must run in CI/CD, not overnight Regression detection — Know immediately when a prompt change breaks things CI/CD integration — Block merges that degrade quality Golden dataset management — Versioned, stratified, growing test cases Architecture: The Evaluation Pipeline ┌─────────────┐ ┌──────────────┐ ┌────────────────────┐ ┌──────────────┐ │ Test Cases │────▶│ LLM Under │────▶│ Judge Ensemble │────▶│ Metrics & │ │ (Golden Set)│ │ Test │ │ - Faithfulness │ │ Regression │ └─────────────┘ └──────────────┘ │ - Instruction F. │ │ Detection │ │ - JSON Schema │ └──────┬───────┘ │ - Custom LLM │ ▼ └────────────────────┘ ┌──────────────┐ │ Dashboard/ │ │ PR Comments │ └──────────────┘ Core Abstractions # eval/base.py @dataclass ( frozen = True ) class TestCase : id : str input : dict [ str , Any ] expected : dict [ str , Any ] | None = None tags : list [ str ] = field ( default_factory = list ) # ["edge-case",

Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40% How we moved from "semantic search + hope" to a measured, tunable retrieval pipeline with 95% recall@10 The RAG Reality Check Everyone ships RAG the same way: chunk by 512 tokens, embed with text-embedding-3-small , top-k=5, stuff into context. It works for demos. Then you hit production: Legal contracts: 512 tokens splits clauses mid-sentence API docs: 1000-token chunks drown signal in noise Customer tickets: Conversational context needs overlap, not fixed windows Latency: 500ms embedding + 200ms vector search + 300ms LLM = 1s+ per query We rebuilt our retrieval layer from first principles. Here's what actually moves metrics. Chunking: One Size Fits None # rag/chunking.py from abc import ABC , abstractmethod from dataclasses import dataclass @dataclass class Chunk : text : str metadata : dict token_count : int chunk_id : str class ChunkingStrategy ( ABC ): @abstractmethod def chunk ( self , document : str , metadata : dict ) -> list [ Chunk ]: ... class FixedTokenChunker ( ChunkingStrategy ): """ Baseline. Good for homogeneous content. """ def __init__ ( self , chunk_size = 512 , overlap = 50 ): self . chunk_size = chunk_size self . overlap = overlap class RecursiveChunker ( ChunkingStrategy ): """ Respects structure: markdown headers, code blocks, paragraphs. """ def __init__ ( self , separators = [ " \n ## " , " \n ### " , " \n\n " , " \n " , " " ], chunk_size = 512 ): self . separators = separators self . chunk_size = chunk_size class SemanticChunker ( ChunkingStrategy ): """ Uses embedding similarity to find natural boundaries. """ def __init__ ( self , model = " text-embedding-3-small " , threshold = 0.7 ): self . model = model self . threshold = threshold class AgenticChunker ( ChunkingStrategy ): """ LLM decides boundaries. Expensive but highest quality for complex docs. """ def __init__ ( self , model = " gpt-4o-mini " ): self . model = model Our production config by

Building Production-Grade LLM Evaluation Pipelines: From Vibes to Metrics How we replaced "looks good to me" with automated evaluation catching 92% of hallucinations before deployment The Problem: Why "Vibe Checks" Fail in Production Three months ago, our team shipped a RAG-based customer support assistant. It worked great in testing — we'd ask it questions, read the answers, and say "yeah, that looks right." Then it hit production. A customer asked about their billing cycle. The assistant confidently cited a policy that didn't exist. Another asked about API rate limits and got numbers from a competitor's documentation. By the time we caught it, 500+ users had seen hallucinated responses. The post-mortem was brutal: we had zero automated evaluation . Our test process was literally "ask 5 questions, read answers, thumbs up." What Production Evaluation Actually Needs Academic benchmarks (MMLU, HellaSwag) don't tell you if your system works for your use case. Production evaluation needs: Domain-specific judges — Your criteria, not generic "helpfulness" Speed — Evaluation must run in CI/CD, not overnight Regression detection — Know immediately when a prompt change breaks things CI/CD integration — Block merges that degrade quality Golden dataset management — Versioned, stratified, growing test cases Architecture: The Evaluation Pipeline ┌─────────────┐ ┌──────────────┐ ┌────────────────────┐ ┌──────────────┐ │ Test Cases │────▶│ LLM Under │────▶│ Judge Ensemble │────▶│ Metrics & │ │ (Golden Set)│ │ Test │ │ - Faithfulness │ │ Regression │ └─────────────┘ └──────────────┘ │ - Instruction F. │ │ Detection │ │ - JSON Schema │ └──────┬───────┘ │ - Custom LLM │ ▼ └────────────────────┘ ┌──────────────┐ │ Dashboard/ │ │ PR Comments │ └──────────────┘ Core Abstractions # eval/base.py @dataclass ( frozen = True ) class TestCase : id : str input : dict [ str , Any ] expected : dict [ str , Any ] | None = None tags : list [ str ] = field ( default_factory = list ) # ["edge-case",