标签：#provenance

A generated image looks finished until review starts. Someone approves the first version. Someone else crops it. A branded copy goes out. Another edit changes the prompt. A week later, the useful question is simple: which prompt, model, seed, size, parent image, and publishing settings produced the version on screen? In a content studio, I put those answers in the PostgreSQL row that stores the image. Logs explain what happened during a run, then rotate away. Object storage keeps the bytes and forgets why they exist. The row is the only one of the three that survives edits, review, and publishing. 1. The row is the receipt The table in apps/api/src/database/init-ai-images-table.js treats generated and edited images as one record type. An original image gets its own row. An edit gets another row, with original_image_id pointing back to the parent. CREATE TABLE IF NOT EXISTS ai_generated_images ( id SERIAL PRIMARY KEY , image_url TEXT NOT NULL , -- what produced it prompt TEXT NOT NULL , model VARCHAR ( 100 ) DEFAULT 'fal-ai/imagen4' , model_version VARCHAR ( 100 ), seed BIGINT , width INTEGER , height INTEGER , -- how it derives from another row is_edited BOOLEAN DEFAULT FALSE , original_image_id INTEGER REFERENCES ai_generated_images ( id ), edit_prompt TEXT , edit_strength DECIMAL ( 3 , 2 ), -- what actually shipped branded_url TEXT , branding_options JSONB , metadata JSONB , tags TEXT [], created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); That self-reference is the design choice. It makes the image table append-only-ish: new variants are inserted as new rows instead of overwriting the earlier state. The cost is more rows and more discipline at write time. The benefit is editable history that product screens and debugging queries can follow. flowchart TD original["Original row: prompt, model, seed, dimensions"] editA["Edited child: edit_prompt, edit_strength, edit_steps"] editB["Edited child: edit_prompt, edit_guidance_scale"] brandedA["Branded output: branded_url,

Most "data lineage" you've seen answers a schema question: table B comes from table A , or column B.total comes from columns A.price and A.qty . That's genuinely useful, and tools like OpenLineage do it well. But notice what it doesn't tell you: it says which columns can influence an output. It never says which values actually did . That gap is the whole subject of this post. I built a small, self-contained reference pipeline that captures provenance at the row and field level — "the value in this destination row, this field, was computed from these specific source (row, field) pairs" — and I want to walk through two things: why you'd ever want provenance at that granularity, and why it's genuinely hard once you commit to it. Repo (dbt-core + DuckDB, no server, no cloud, runs on a clean checkout): https://github.com/stevenblough/row-level-prov The one distinction everything follows from Here's the sentence the entire project turns on: Column-level lineage is a schema-sized, static fact you can derive from code. Value-level provenance is a data-sized, dynamic fact you must capture at execution. Put it in complexity terms and the consequences become obvious: Column lineage is O(schema) . It scales with how many columns you have. You can compute it by parsing SQL, offline, without ever looking at a single row. Value provenance is O(rows × fan-in) . It scales with your data volume times how many source values feed each output value. It does not exist anywhere until the query runs, and it can only be captured there , piggybacked on the query that actually produced the values. You cannot "reconstruct" value provenance later by re-querying the sources — the moment the source changes, you'd reconstruct a different answer than what really happened. That single exponent change ( schema → rows × fan-in ) is why value-level provenance has an entire class of problems that column lineage never faces. Why bother? The reasons for this level of granularity Granularity is expensive,