标签：#Data

IBM and Red Hat have announced an expansion of Lightwell, introducing new commercial offerings designed to help organizations establish trusted, verifiable software supply chains for the age of AI-assisted software development. By Craig Risi

Imagine working as a Data Analyst in a healthcare Non-Governmental Organization (NGO) implementing HIV and AIDS programmes across several communities. The organization has limited resources. There may not be enough funding, healthcare workers, testing kits, transport, outreach teams, or community programmes to serve every community at the same intensity. This creates an important question: How can we use data and machine learning to direct limited programme resources to communities with the greatest need? This is where Machine Learning (ML) can become valuable. Rather than distributing resources equally across all communities, an NGO can use historical programme data to identify communities experiencing greater HIV-related service gaps or higher levels of need. Resources can then be prioritized based on evidence. What Is Machine Learning? Machine Learning is a branch of Artificial Intelligence that enables computers to learn patterns from data and use those patterns to make predictions or support decisions. Instead of manually creating rules for every situation, you provide the algorithm with historical data and allow it to identify relationships within that data. For example, the NGO could have this information about different communities: Community HIV Testing Coverage ART Coverage Missed Appointments Outreach Activities Community A 85% 90% 5% High Community B 52% 61% 25% Low Community C 70% 75% 15% Medium Community D 40% 55% 32% Low Looking at this data, Community D appears to have greater programme gaps than Community A. However, in a real programme, the decision should not be based on one indicator alone. Machine learning can analyse many variables simultaneously to identify communities that may require greater attention. Why Resource Allocation Matters in HIV Programmes HIV programmes operate in environments where resources are often limited. An NGO may have: A limited number of community health workers A fixed outreach budget Limited HIV testing supplies Limi

The Quest Begins (The "Why") I still remember the first time I saw LeetCode 323 “Number of Connected Components in an Undirected Graph”. I stared at the adjacency list, thought “I’ll just run a DFS from every node”, and coded it up in ten minutes. The solution passed the easy tests, but when the hidden test cases hit a graph with 10⁵ nodes and 10⁵ edges, my DFS started to choke—stack overflows, repeated visits, and a sinking feeling that I was brute‑forcing a problem that deserved a smarter tool. That night, after a few too many coffees, I stumbled upon a tiny comment in a discussion thread: “Union‑Find can do this in almost O(1) per operation”. My curiosity sparked like a power‑up in a retro arcade game. I had to know why this seemingly simple data structure could turn a nightmare into a breeze. The Revelation (The Insight) At its heart, Union‑Find (aka Disjoint Set Union, DSU) maintains a collection of elements partitioned into disjoint subsets. It supports two operations: Find(x) – returns the representative (root) of the set containing x . Union(x, y) – merges the sets containing x and y . The magic lies in two simple heuristics: Path Compression – when we walk up the tree to find a root, we make every node on that path point directly to the root. Future finds become flat, almost constant‑time. Union by Rank/Size – we always attach the smaller tree under the root of the larger one, keeping the overall tree shallow. Why does this give us near‑O(1) amortized time? Think of each Find as paying a small “tax” to flatten the path. The tax is paid only a few times per node before it becomes a direct child of the root. Over a sequence of m operations, the total work is bounded by O(m α(n)) , where α is the inverse Ackermann function—so slow‑growing it’s practically a constant for any realistic n . In plain English: every time we climb up, we leave a shortcut behind. The next climber benefits from that shortcut, and the structure keeps getting better. It’s like building

I was watching The Mandalorian the other day when it struck me that I don't know Pedro Pascal, which...

You can load-test search latency with a script and a graph. Relevance has no such gauge by default, so most teams ship a new ranking rule, eyeball a handful of queries, and hope nothing important regressed. The fix is a small, boring relevance test suite: a fixed set of queries, human-judged expected results, and a metric you compute the same way every time — so "did this ranking change help?" becomes a number you can diff, not an argument you have in Slack. This post is a build guide. By the end you'll have a judgments file, a scorer that outputs precision@k, MRR, and nDCG, and a before/after comparison you can wire into CI. The examples use Postgres full-text search, but the harness is engine-agnostic — Elasticsearch, Meilisearch, or a vector store all slot into the same shape. Why can't I just load-test relevance the way I load-test latency? Latency is a property of the system. Relevance is a property of the match between a query and what a human expected to see — and that judgment lives outside the database. A commenter on an earlier post about running Postgres search in production put it well: latency can be load-tested, but quality needs query sets, expected result buckets, bad-query examples, and a way to compare changes before shipping a new ranking rule. That's the whole job, and none of it comes for free with your index. The trap is thinking a passing query proves relevance. SELECT ... WHERE tsv @@ query returning rows tells you the index matched. It says nothing about whether the right rows landed in the top 5, which is all a user ever sees. The takeaway: relevance is measured against human judgments, not row counts — so the first artifact you build is the judgments, not the query. Building the golden query set Start with 20–50 real queries. Pull them from your search logs if you have them (the head terms plus a long tail of specific ones), or write them from real user intents if you don't. For each query, mark which documents should come back and how rel

Context: I'm a data scientist/analyst (in Python and R) learning development from scratch. Inevitably, I am learning these through the lens of what I already know. If you have a similar background and are a beginner developer, I hope these analogies help! Any comments, especially if you spot any misunderstanding, are appreciated. Commenting is caring <3 Motivation: I was building a mock data layer for a fitness social app — simulating what happens when users fetch new posts from a feed. The function needs to return mock posts after a delay, simulating a real network request. Working Code: `function fakeFetchPosts() { return new Promise((resolve) => { setTimeout(() => { resolve(posts); }, 2000); }); } async function main() { console.log("Fetching..."); const fetchedPosts = await fakeFetchPosts(); console.log("Fetched posts:", fetchedPosts); } main(); console.log("Sync code ran");` What do you expect to see as an output? I first confused the logic with blocking. For example, in webscraping, something like time.sleep() or Selenium's WebDriverWait(driver, 10).until(EC.presence_of_element_located(...)) . In this case, output will be Fetching..., Fetched posts: ..., then Sync code ran. However, the output gives Fetching..., Sync code ran, and then Fetched posts. In the former, the whole script (single thread) pauses and does nothing else until the wait ends or the condition is met. The latter is different in that the rest of your program keeps running during the wait, and thus the output where Sync code ran is printed first before the fetchedPosts. By the way, posts are arrays. const posts = [{ author: "j1wonkim", text: "Testing Physical", likes: 100, }, {author: "onewc0218", text: "Love love", likes: 55, }, {author: "gakbca", text: "You are good", likes: 10, } ];

The Quest Begins (The "Why") I still remember the first time I got hit with a sorting question in an interview. The interviewer slid a whiteboard marker across the table and said, “Sort this array of a million integers – and tell me why you chose your method.” My brain went straight to the trusty old bubble sort I’d learned in CS101. I started writing nested loops, feeling like Neo dodging bullets in slow motion, only to realize the runtime was creeping toward O(n²). After a few painful minutes, I could see the interviewer’s eyes glaze over – not because I was wrong, but because I was using a sledgehammer to crack a nut. That moment sparked a quest: What makes a sorting algorithm truly efficient, and how do I know when to reach for it? I dove into textbooks, blog posts, and late‑night YouTube deep dives. The answer kept pointing back to one algorithm that felt like discovering a hidden cheat code: Merge Sort . The Revelation (The Insight) So why does Merge Sort work so well? It’s not just about splitting and merging; it’s about guaranteeing that each level of recursion does a linear amount of work, no matter how the input is arranged. Think of an unsorted array as a messy pile of LEGO bricks. Merge Sort first divides the pile into two halves, then halves again, until each sub‑pile contains a single brick – which is, by definition, sorted. The magic happens in the merge step: we take two already‑sorted sub‑arrays and walk through them with two pointers, always picking the smaller front element and appending it to the result. Because each sub‑array is sorted, we never need to look back; we simply advance one pointer at a time. That walk is O(n) for the merge: each element is examined exactly once as it gets placed into the output array. Since we split the array log₂ n times (each level halves the size), we perform an O(n) merge at each of those log₂ n levels. Multiply them together and you get O(n log n) worst‑case time, with O(n) extra space for the temporary buffer

I was building a runtime N+1 query detector for Node. The detection part worked on the first afternoon. Getting it to tell you which line of your code caused the problem took considerably longer, and taught me something about how ORMs execute queries that I had not thought about before. This is that story, and the fix. The symptom The detector instruments your database driver. When the same query shape runs many times inside one request, it reports it — along with the file and line that issued it, which is the part that actually saves you time: nplusone 1 finding in GET /orders — 51 queries, 840ms N+1 query 50× SELECT * FROM items WHERE order_id = ? at src/routes/orders.ts:47:38 (loadOrdersPage) 612ms spent here That worked. Then I pointed it at an app using Drizzle and got this instead: N + 1 query 12 × select "id" , "order_id" from "items" where "items" . "order_id" = $ 1 < unknown call site > Detected, counted, and attributed to nothing. Do not theorise. Dump the stack My first instinct was that my frame filter was too aggressive — it skips node_modules , node:internal , and the library's own frames, so maybe it was eating something it should not have. Rather than guess, I printed the whole stack at the exact moment the driver was called: const originalQuery = pg . Client . prototype . query ; pg . Client . prototype . query = function (... args ) { const previous = Error . stackTraceLimit ; Error . stackTraceLimit = 100 ; const stack = new Error (). stack . split ( " \n " ). slice ( 1 ); Error . stackTraceLimit = previous ; console . log ( " FRAMES: " , stack . length ); stack . forEach (( line , i ) => { const mine = ! /node_modules|node:internal/ . test ( line ); console . log ( ` ${ String ( i ). padStart ( 3 )} ${ mine ? " >>> " : " " } ${ line . trim ()} ` ); }); return originalQuery . apply ( this , args ); }; Here is what came back for a single await db.select().from(items).where(...) : FRAMES: 12 0 at Proxy.<anonymous> (.../nplusone/dist/adapters/postgre

I remember the old days of manual scaling. You'd jump into a CLI, check your metrics, realize you needed another node or a capacity adjustment, log into a web console, navigate three layers deep into some proprietary dashboard, and hope you didn't click the wrong thing while trying to find a specific cluster ID. Now we have AI agents. But most people are using them wrong. They treat Claude or Cursor as just better search engines for code, rather than giving them hands. If you're running high-availability workloads on something like TiDB Cloud, the friction isn't in writing the SQL—you already know how to do that. The friction is in the operational visibility: knowing exactly what’s happening across your serverless instances versus your dedicated clusters without leaving your IDE. The Gap Between Code and Infrastructure The reason I spend so much time building things like MCPFusion is precisely because of this disconnect. An LLM might help you write a complex join perfectly, but if it doesn't know whether the target TiDB X instance is actually healthy or which project ID handles your staging environment, it's basically flying blind. You end up copy-pasting JSON blobs from your terminal into the chat window just to give the model context. That's slow, prone to error, and frankly, beneath what modern tooling should look like. This is why we released the TiDB Cloud (Serverless Distributed SQL) MCP server on Vinkius. It closes that loop. What This Actually Does (And Doesn't) Let's be very clear about what this tool allows you to do through an agent like Claude or Cursor. We aren't looking for "magic" here; we want predictable utility. The current implementation focuses on discovery and inspection. In DevOps terms, it provides a controlled read-only view of your topology. Here is what's available: Organization Discovery: You can call list_projects to see everything sitting under your umbrella and pull metadata via get_project . This solves the "what was that project ID ag

Three semantic layers, three architectures, three very different bills. All three will define what a metric means. None of them proves an AI agent is allowed to run it. Quick orientation dbt Semantic Layer Cube AtScale Core idea Metrics as version-controlled code Headless API in front of metrics OLAP-style aggregate acceleration Strongest when You want engineering discipline Many apps consume the same numbers Heavy, stable aggregate workloads Modelling Hand-authored YAML Hand-authored data model Hand-authored cubes Cost driver Plan tier + query volume Pre-aggregation builds + compute Quote-based licence + compute Governance Upstream, in the warehouse In front of the API On the cube Each is competent at what it was built for. If your consumers are dashboards and analysts, any of the three will serve you. The question none of them answers An agent doesn't arrive with a metric name. It arrives with an intent in English and has to work out which entities, which grain, which joins, and whether it's entitled to any of it. That exposes two gaps every one of these shares: Undefined intent has no answer. Coverage is whatever someone remembered to model. Business questions don't respect that boundary. Authorisation is checked around the query, not inside it. A filter applied after execution means the data already moved. What to actually evaluate on Ignore feature matrices and score these five: Answer a question nobody modelled, on your schema Show why one join path was chosen over two others Same question, two users with different entitlements — show both SQL statements Ask something ambiguous. Refusal or guess? Reproduce a number from six months ago with the definitions then in force Most evaluations stop at 1. Numbers 3 and 5 are the ones that decide whether the thing ships in a regulated business. The full breakdown — architecture-by-architecture comparison, cost profiles, and the migration implications of each — is here: 👉 dbt Semantic Layer vs Cube vs AtScale: Choosing a

I wanted to see earthquakes the way they actually happen: as points lighting up on a spinning planet, in near real time. That became Earthquake: Live Seismic Monitor , an iOS app that renders a 3D globe of recent quakes straight from the USGS feed. No backend of my own - just the public data and the device. Here's how it comes together. The data source The USGS publishes earthquake data as GeoJSON feeds, updated continuously, at several time/magnitude cutoffs (past hour, past day, 2.5+, 4.5+, etc). Each feature has coordinates, magnitude, depth and time. That's everything you need to place a quake on a globe - no custom API required. The app polls the appropriate feed, diffs against what it already has, and updates the scene. Because USGS does the heavy lifting, the whole thing is effectively serverless from my side. Putting quakes on a globe The core mapping problem is turning (latitude, longitude) into a point on a sphere. Once you have that, each earthquake becomes a marker whose size and color encode magnitude and depth, so a glance tells you "big and shallow" vs "small and deep". Design decisions that mattered: Encode magnitude visually. Radius and color do more than any label. A magnitude 6 should look like a magnitude 6. Cluster sensibly. Active regions produce swarms; markers need to stay readable when dozens land in one area. Keep the globe interactive. Rotate, zoom, tap a quake for details. It should feel like an object, not a chart. Real-time without a server Every network-dependent app has to answer: what happens offline, and how fresh is "live"? My rules: Cache the last good feed so the globe still renders with no connection. Refresh on foreground and on an interval, and show the data's own timestamp so "live" is honest. Never block the UI on the network - render what you have, then update. Why no backend It's tempting to proxy the feed through your own server "for control". But USGS is reliable, public, and built for exactly this. Skipping a backend me

Most teams don't consciously choose their messaging infrastructure. They inherit it. Someone used Service Bus on the last project, it worked fine, and now it's the default answer for every async communication problem that comes up. Two years later, you're bending it into shapes it was never designed for, and the operational pain gets blamed on "distributed systems being hard" rather than on the actual culprit: a tool being asked to do a job it doesn't fit. The problem isn't that Service Bus, Event Grid, or Kafka are bad. It's that they solve genuinely different problems, and conflating them doesn't just create technical debt — it creates architectural liability that compounds over time. The Real Difference Is the Communication Contract, Not the Feature List When you put these three tools side by side in a comparison table, you'll find overlapping columns. All three move messages between systems. All three have some delivery guarantee story. That's where the surface-level comparison breaks down and people make bad decisions. The more useful question is: what contract does your system need to uphold with the data it moves? Service Bus is fundamentally about reliable, ordered processing with strong delivery guarantees. It's designed for the case where every message matters individually, where you need competing consumers pulling from a queue, where poison message handling and dead-lettering are first-class concerns. If you're coordinating business process steps or handling financial transactions where exactly-once semantics matter, this is the right shape of tool. Event Grid is about reactive routing. Something happened in your infrastructure or your application, and you want other things to respond to it. It's push-based, fan-out-friendly, and optimized for low-latency notification rather than high-volume throughput. It's not trying to be a buffer. If you're triggering downstream workflows in response to blob uploads, resource state changes, or custom application even

Companies that rely on Ceva Logistics for shipping their physical goods to customers say their personal data was taken during a recent cyberattack.

By the end of this guide you will have a published Tableau dashboard and a three-point story, built on a real dataset. It lives on a public URL you can put in an application. You build four small sheets. Each one makes exactly one point. You arrange them on a single screen, then walk a reader through them in three steps that end with a recommendation. Every step says what to click and what you should see afterwards. Four small sheets, rather than a wall of charts, because a dashboard has to argue for something. A screen holding everything you could build leaves the reader to work out what matters. Most readers will not do that work. Dashboard vs Story, in one line. A dashboard puts several charts on one screen so someone can explore. A story is a sequence of views with captions, clicked through in order, so someone is walked to a conclusion. Build both: the dashboard is what a hiring manager glances at, the story is what proves you can think. The original carries a diagram here. In words: Four separate worksheets stack on the left: a big single number, a set of vertical bars, a set of horizontal bars, and a scatter of circles. An arrow points right to one dashboard panel that holds all four of them arranged on a single screen: the number across the top, the two bar charts side by side in the middle, the scatter along the bottom. A second arrow points right to three story cards numbered one, two and three, each showing one of those views with a caption line above it. The worked example. Every instruction below is written against a real, free dataset: the Telco Customer Churn file on Kaggle, 7,043 customers, one row each. A finished analysis of it, including the Python script that shapes the data, is public at telco-churn-analysis . Swap in your own dataset and the steps do not change, only the field names do. Step 1: Shape the data before you open Tableau Tableau is a display layer. Deriving something inside it takes longer than deriving it upstream in SQL, Python or

This guide gives you a repeatable loop for the days when nothing works, and four true stories showing it used on real problems. Here is what a working day actually contains. A website's firewall blocks you for no reason. A table that visibly exists tells your script it does not. A query runs for thirty minutes with no end in sight. A fix you know is correct changes nothing at all. None of that means you are doing it wrong. That is the job. What separates people who ship analyses from people who stop is technical tenacity : staying methodical when the tools fight back. It is not a personality trait you either have or lack. It is a small procedure, and you can learn it in the next ten minutes. The diagnosis loop (tenacity is a method, not a mood) Think back to the last time a tool beat you for an hour. What was the first thing you did when it failed, and what did you do second? Most people can name the first move and not the second, and the second is where the method lives. Gritting your teeth and re-running the same thing harder is not tenacity; it's frustration with extra steps. What experienced people actually run is a loop: Step Move 1. Read the actual message Not "it's broken" — the words. Error messages name the symptom precisely, even when the cause is elsewhere. 2. Form ONE hypothesis "The table isn't in the file the script reads." Specific enough to be wrong. 3. Run the cheapest test of it Prefer checks that take seconds — list the tables, count the rows, print one value. 4. Verify from a second vantage point Don't ask the tool that's confusing you whether it's confused. Check the file from outside, the data from a different program, the value with a different query. 5. Change ONE thing, re-run Change three things and you'll never know which one mattered — or which one broke something new. 6. Timebox, then change strategy If the current approach has eaten 30 minutes with no progress, stopping is a decision, not a defeat. There's usually a second road. Four tr

COUNT looks like the simplest function in SQL, and it is the one that quietly trips up the most people in interviews and on the job. The confusion is almost always the same: COUNT(*) , COUNT(column) , and COUNT(DISTINCT column) look nearly identical but count three different things. Once you can say out loud what each one counts, a lot opens up. You can verify a data migration, find duplicates, and measure how complete a column is, all with the same little function. This guide is that explanation, with lots of small examples you can copy. The one-sentence version. COUNT(*) counts rows . COUNT(column) counts rows where that column is not NULL . COUNT(DISTINCT column) counts how many different non-NULL values that column has. Everything below is just that sentence, slowed down. The three forms of COUNT and what each one counts Picture one small table, customers , with a region column where two rows were never filled in: id name region 1 Maya North 2 Jordan South 3 Alex North 4 Sam NULL 5 Taylor NULL Now run the three forms on it: SELECT COUNT(*) AS all_rows, COUNT(region) AS rows_with_region, COUNT(DISTINCT region) AS different_regions FROM customers; all_rows rows_with_region different_regions 5 3 2 COUNT(*) = 5. Every row, no exceptions. The * means "the row itself," so NULLs never matter. COUNT(region) = 3. Only the rows where region has a value. Sam and Taylor are skipped because their region is NULL. COUNT(DISTINCT region) = 2. The different values are just North and South . The two Norths collapse to one, and NULL is not counted. The NULL rule that makes them disagree Predict it first. A table has 100 rows. Twenty of them have no email address. What does counting the email column give you? Say the number before you read on. Here is the whole trick in one line: COUNT(*) counts rows. COUNT(something) counts non-NULL values of that something. So the moment a column has any NULLs, COUNT(column) comes back smaller than COUNT(*) . That gap is not a bug, it is informat

This guide walks through five steps for working out which records are the same real thing, and merging them without wrecking your data. It runs on real chart data, and it includes the two times the rules came out wrong. Here is the problem in one example. Count the distinct artists in Billboard's public chart history and the number is wrong. "Elvis Presley" and "Elvis Presley With The Jordanaires" are the same man, and so are five other credit strings. One real-world entity , seven database strings . Every dataset with human-entered names has this. Customers who signed up twice. "IBM" against "I.B.M." against "International Business Machines". The same supplier in two systems, spelled two ways. The work of fixing it is called entity resolution . Matching across two datasets is record linkage . Removing duplicates inside one is deduplication . They are the same skill pointed at different situations, and it is one of the most common tasks an analyst actually gets handed. The vocabulary map Term Meaning Entity The real-world thing: one artist, one customer, one company Entity resolution Figuring out which records refer to the same entity Record linkage The same problem across two datasets. "Is row 5 in file A the same person as row 90 in file B?" Formalized by Fellegi & Sunter (1969) Deduplication The same problem inside one dataset Normalization / standardization Transforming values toward a canonical form (lowercasing, trimming, cutting suffixes) so equal things become equal strings Match key The cleaned column(s) you actually join on Match rate The share of records that found their counterpart. This is the number that keeps the whole exercise honest Clerical review Human eyes on the records the rules could not decide. This is a formal stage of the classic framework, not an admission of failure Step 1: measure the fragmentation before fixing anything The worked example is Billboard Hot 100 history, 1958 to present. The goal is one clean row per artist. Before writing

I have been building a thing that lets a language model propose an UPDATE , then executes it for real inside a transaction, measures the actual before and after values, and always rolls back. A human reads the measurement and decides. Only then does anything commit. The pitch is one sentence: what you approve is not the model's description of its SQL, it is what the database did when the SQL ran. Last week I found that the thing showing you that measurement was showing you a subset of it, and had been since the first release. The failure Real output, from @hyuga/llm-safe-sql@0.4.0 installed from npm. One row: name = 'Tanaka' , postcode = '00100' . UPDATE customers SET name='Sato', postcode='00100' WHERE id=1 What this touches customers — Customer records. The postcode is used for billing address and delivery. 1 row would change, across 1 column: name Measured by running the statement and rolling it back id = 1 name: 'Tanaka' -> 'Sato' One row, one column. postcode is not mentioned, and that is correct — it is being assigned the value it already holds, so nothing about it changes. The card is describing the diff accurately. Approve it. Then, before it is applied, somebody else notices the postcode is wrong and fixes it: UPDATE customers SET postcode = '90210' WHERE id = 1 ; Now apply the approved plan: Applied: UPDATE on customers, 1 row(s), at 2026-08-10T09:49:12.049Z. DB now: [{"name":"Sato","postcode":"00100"}] The fix is gone. Zero warnings. The word postcode never appeared on the approval card, never appeared in the audit record, and never appeared in the comparison the tool makes before it commits. One variable doing two jobs The diff was built like this: const changed : string [] = []; for ( const c of Object . keys ( before )) { if ( same ( before [ c ], after [ c ])) continue ; // drop what did not move if ( auto . has ( lower ( c ))) continue ; // drop what the DB maintains itself changed . push ( c ); } That is a correct answer to "what should the card sho

Kennedy Torkura discusses practical GenAI red teaming techniques to safeguard LLMs and knowledge bases against security threats like data poisoning and LLMjacking on AWS. He explains how engineering leaders and architects can bridge traditional cloud security with MITRE ATLAS frameworks to proactively identify vulnerabilities, implement guardrails, and secure production AI applications. By Kennedy Torkura

In Blog 5, we built a SQL parser. It can take this: INSERT INTO payments VALUES ( 500 , payment_1 , pending , 1 ) and turn it into a struct: InsertIntoTable { TableName : "payments" , ColumnValues : [] string { "500" , "payment_1" , "pending" , "1" }, } But this is still not enough for the storage engine. Our storage engine only knows how to store key-value pairs. It does not know what a table is. It does not know what a column is. It does not know that 500 is an integer, pending is a string, and 1 is a boolean. So, in this post we solve the missing bridge of persisting these in our key-value store. CREATE and INSERT are PUT operations This is the first major realisation. A key-value store is extensible to store literally anything. This is what we have been saying from the first post itself. But now we will be taking actual examples to prove that. CREATE TABLE Example Let's start with the create table example and see what should be the key and the value. Serialisation The key should be something that uniquely identifies the table, which is straightforward enough in this case as the table name . The value becomes everything else except the key, which is the schema of the table. So, in order to store the table name, we can append a reserved keyword as prefix like schema as a unique identifier. The structure of the key becomes _schema:<table_name> . The next question to answer is: How do we store a struct like below into our key value store where the value is always string? CreateTable { TableName : "payments" , ColumnDetails : [] Column { { ColumnName : "amount" , DataType : Int }, { ColumnName : "id" , DataType : String }, { ColumnName : "status" , DataType : String }, { ColumnName : "captured" , DataType : Bool }, }, PrimaryKeyColumnPosition : 1 , } One way is to serialise the entire struct into a string and store that directly. But in that case, deserialisation is a complex logic. JSON or struct serialisation and deserialisation is both space-heavy and compute inte