标签：#base

I run a small self-hosted observability tool on the cheapest VPS I could find on purpose: 2 cores, 2 GB RAM, 20 GB SATA SSD . It ingests errors, traces and metrics from two low-traffic sites of mine. The stack is three containers — a Go app, PostgreSQL, and ClickHouse. One evening docker stats showed ClickHouse sitting on 880 MB of its 1 GB limit and the box swapping, with basically zero events coming in. So I went looking for where the memory and disk had gone. The answer turned out to be a good lesson in how a database can spend almost all of its I/O talking to itself. 543 KB of my data, 579 MB of ClickHouse talking about ClickHouse First thing I checked: how much data had my app actually stored versus how much ClickHouse had stored about itself . My application database: 543 KB, 16k rows The system database: 579 MB, 46.3M rows Roughly a thousand to one. Disk was 12 GB used out of 20 — on a tool that had recorded half a megabyte of real telemetry. The culprit was ClickHouse's own system logs, several of which have no TTL by default and therefore grow forever: trace_log — 404 MB, 26M rows (the query profiler writes here; it's on by default, sampling once per second) asynchronous_metric_log — 16.6M rows text_log — 132 MB plus query_log , latency_log Only metric_log , processors_profile_log and part_log ship with a TTL. Everything else just accumulates. Then I looked at the insert rate over 30 seconds: trace_log — 227 rows/s asynchronous_metric_log — 157 rows/s text_log — 44 rows/s my application — about 5 rows/s 98.8% of all inserts were ClickHouse narrating its own internals. The part that's expensive beyond disk Here's the number that made me stop. Over the same 30 seconds: rows inserted : 16,222 rows merged : 11,007,643 That's a 1 : 678 ratio. For every row written, the engine rewrote 678 already-sitting rows. The mechanics: MergeTree drops every insert into its own data part, then merges parts into bigger ones so reads stay fast. When the table is small this is

A normal secondary index can answer: status = "active" It cannot efficiently answer: documents containing "distributed database" LioranDB therefore has a dedicated text-segment architecture. Tokenization Text is split on non-alphanumeric characters. Depending on index options, tokens can be normalized to lowercase and filtered through stopwords. "Building Distributed Databases" becomes ["building", "distributed", "databases"] Segment contents A LioranDB text segment can contain several files and structures: Term dictionary Posting lists Document map Document-length norms Optional term positions Bloom filter Segment metadata A posting connects a term to the local documents containing it. "database" → [doc 2, doc 8, doc 19] Positions can record where the term appears inside each document. That enables more advanced query behaviour and phrase-aware features. Global and local document IDs Each segment assigns compact local IDs to its documents. A separate document map translates them back to global document IDs. This keeps postings smaller while preserving the external identity of the record. Bloom filters Each segment also maintains a Bloom filter for terms. Before reading a segment's postings, the query path can test whether the term might exist there. A negative answer is definitive. A positive answer means the segment may contain the term and should be checked. Query modes The text query layer supports modes such as: AND OR It also emits scored documents and metrics including: Query time Postings read Candidate documents Segments searched Full-text search is essentially a specialized database living beside the document database. Its data structures, compaction behaviour, scoring, and caching needs are different enough that treating it as a plain secondary index would be a mistake. Built by Swaraj Puppalwar under Lioran Group . Learn more: LioranDB Lioran Developer Solutions Lioran Group

Disk structures are durable, but updating them for every write is expensive. LioranDB uses memtables to absorb writes before flushing them to the on-disk B+ tree. What is a memtable? A memtable is an ordered in-memory map. In LioranDB, each entry contains either: Value ( bytes ) or: Tombstone A tombstone represents a deletion. The memtable also tracks: Approximate memory usage Minimum LSN Maximum LSN Entry count Put count Delete count The write path A simplified write path looks like this: Application write ↓ WAL durability ↓ Mutable memtable ↓ Immutable memtable queue ↓ Background flush ↓ Disk B+ tree The active mutable memtable accepts new writes. When it crosses a size limit, the engine rotates it into an immutable memtable. That immutable table is no longer modified and can safely be flushed in the background. Why ordered maps? LioranDB uses an ordered map for memtable entries. This helps because the flush process can emit keys in sorted order, which is friendly to the B+ tree and bulk-write paths. It also simplifies range merging between: Mutable data Immutable data On-disk pages Backpressure Background flushing cannot be allowed to fall behind forever. LioranDB therefore tracks limits such as: Maximum immutable memtables Maximum immutable bytes Partition-wide queue limits Maximum writer stall duration If the disk cannot drain the backlog quickly enough, the foreground write path slows down. That may sound undesirable, but controlled backpressure is much safer than consuming memory until the process dies. A memtable is not merely a cache. It is a pressure valve between CPU-speed writes and disk-speed persistence. Without that valve, the engine would either become slow on every commit or dangerously accumulate unbounded work. Built by Swaraj Puppalwar under Lioran Group . Links: LioranDB Lioran Developer Solutions Lioran Group

Most developers think of LioranDB as a document database. Internally, however, its storage engine does not understand documents, objects, fields, or JSON. It understands only: table + key bytes + value bytes That separation is intentional. The architecture LioranDB is split into two major layers: Application ↓ Document DBMS ↓ Transactional key-value engine ↓ WAL, memtables, B+ tree, pager and disk The engine exposes operations such as: get ( table , key ) put ( table , key , value ) delete ( table , key ) scan ( table , range ) The DBMS layer then adds document-oriented features: Collections JSON encoding Queries Updates Secondary indexes Text indexes Transactions For example, a secondary index can be represented as: idx:status:active → document_id A text index can be represented as: inv:database → posting_list The storage engine does not need to know what status , active , or database means. It only stores ordered bytes. Why this matters This architecture keeps the core engine small and reusable. The engine focuses on difficult low-level concerns: Durability Page management Transactions Recovery Ordering Concurrency Range scans The DBMS focuses on application-level semantics. This also makes it possible to build different data models over the same engine in the future. A document database is therefore not one giant component. It is a collection of carefully separated layers. That separation is one of the most important architectural decisions inside LioranDB. LioranDB is being developed by Swaraj Puppalwar under Lioran Group . Learn more: LioranDB Lioran Developer Solutions Lioran Group

The headline going around is that Spark 4.2 can retire your vector database. That's half true, which is the most dangerous kind of true. Spark did add real vector search, and for some workloads it genuinely removes a whole system from your stack. For others, you'd regret dropping your vector DB. Here's the honest version. The short answer If your vectors already live in your data platform and your searches are batch or analytical, Spark 4.2 can absolutely replace a separate vector database. If you're serving live, low-latency retrieval for a chatbot or search box, you probably still want a dedicated one. It's a "depends on the workload" answer, and the details matter. What Spark 4.2 actually added Spark 4.2 brought vector search into plain SQL. No bolt-on library, no separate engine. The new primitives include vector distance and similarity functions, vector normalization, vector aggregation like sum and average, and the headline one, NEAREST BY, a top-K ranking join that finds the closest matches by distance. In practice that means you can store embeddings in a Spark table and run a similarity search with SQL you already know. Those operations cover the real use cases: retrieval, recommendations, entity resolution, and candidate generation. Databricks is openly framing this as Spark becoming an AI serving layer, not just a batch engine. Why this is a big deal The value isn't that Spark invented vector search. Plenty of tools do it. The value is that you can keep your retrieval pipeline on one platform. Think about the normal setup today. Your data sits in a lakehouse or warehouse. To do vector search, you spin up a separate vector database, then build a pipeline to copy and sync embeddings into it, and keep the two in step forever. That's a second system to run, secure, pay for, and debug at 2 a.m. Spark 4.2 lets you skip that for a lot of cases. The embeddings stay where your data already is, and the search runs right there. Fewer moving parts is a real win, and i

How I Built an SSR Valorant Tracker with React, Supabase and Live Esports Data I recently built VALTRAIN , a player-focused Valorant platform that combines a Valorant Tracker, VCT match database and weapon skins explorer. The project started as a simple match history lookup page. It eventually became a much larger SSR application with player statistics, esports schedules, match detail pages, replay discovery, multilingual routes and searchable cosmetic data. The Main Product Areas VALTRAIN is divided into three primary areas. 1. Valorant Tracker The Valorant Tracker accepts a Riot ID and region. It can display: Current rank and RR Recent competitive and unrated matches KDA and combat score Headshot, body shot and leg shot data Competitive RR movement Lifetime performance statistics Individual match details and team compositions One challenge was handling incomplete or delayed data from an external player API. The interface needed useful loading, empty and error states instead of leaving users with an endless spinner. 2. VCT Match Database The VCT esports database stores upcoming and completed matches. Each public match can have: Tournament and stage information Team names and series scores Map-level results Player statistics Recent team form Official replay availability Related matches and internal links The project also includes an original VCT performance report generated from completed match records. 3. Valorant Weapon Skins The weapon skins database allows players to browse skins by collection, weapon type, rarity, price and chroma. The main performance challenge was preventing high-resolution media from slowing down the initial page load. Why I Moved the Site to SSR The original version relied heavily on client-side rendering. That worked for user interaction, but it created several problems: Public pages had limited initial HTML Search engines had to execute JavaScript Metadata was harder to control Dynamic routes occasionally returned weak fallback pages Firs

Redis gets reached for reflexively, "just add Redis," as if it were a single fix for slowness. It's genuinely one of the most useful tools in a backend engineer's kit, but using it well starts with understanding what it actually is: an in-memory data structure store, not just a cache. Once you see it as a fast, versatile store of real data structures, the range of problems it solves cleanly (caching, rate limiting, queues, sessions, leaderboards, locks) stops looking like a grab bag and starts looking like one idea applied many ways. This is the opening article of the Redis Masterclass, and it builds on the PostgreSQL series : Redis usually sits alongside a primary database like Postgres, not instead of it. In-memory is the whole point Redis keeps its data in RAM. That single fact explains most of its character. Reading from memory is orders of magnitude faster than reading from disk, so Redis operations typically complete in well under a millisecond, and a single instance handles a very high request rate. That speed is why it's the default choice for anything on the hot path, where a database round trip would be too slow. The tradeoff is that RAM is smaller and more expensive than disk, and volatile. Redis addresses durability with persistence options we'll cover later, but the mental model to start with is: Redis is fast because it's in memory, and you use it for data that benefits from being fast to access, not as the permanent home for everything. It's a data structure store, not a key-value blob The common misconception is that Redis is a simple key-value store, strings in and strings out. It's much more. Redis stores real data structures as values, each with its own commands: Strings for simple values, counters, and cached blobs. Hashes for objects with fields, like a user record. Lists for ordered sequences and simple queues. Sets for unique collections and membership checks. Sorted sets for ranked data like leaderboards and priority queues. Plus streams, bit

Many databases use B+tree indexes, but they all differ. It's a sorted structure. The leaf pages are logically sorted so that a specific key value belongs to one page. A lookup by value reaches a single leaf page and either directly finds an entry for that value or immediately knows there's no entry with that key. When a page becomes full, it is split into two pages, each covering its own dedicated range. To find the right page, an internal page holds the range of values for the pages below. This internal page can become full, and a new level is added above it. Finally, at the highest level, there's a single internal page that is the root. A lookup always starts at the root and goes down to the leaves, following the branches of internal pages. In a traditional B+tree lookup, the cost is proportional to the height of the tree because the search starts at the root and descends to a leaf: 1 page to read when all fits in one leaf that is also the root (0 levels of internal pages, total height is 1). With small keys, this level can typically index hundreds of rows. 2 pages to read when there's one root that can list all leaf pages (1 level of internal page, total height is 2). With small keys, this level can typically index tens or hundreds of thousands of rows. 3 pages to read when there's one level of branches under the root (so 2 levels of internal pages, total height is 3). With small keys, this level can typically index millions of rows. This means that finding one key within ten million rows may require traversing 3 index pages, where most of them are probably in cache given the small number of branches compared to the leaves. For a given index size, whatever the value you are looking for, it's always the same number of pages to read because the index is balanced (the commonly accepted meaning of the B in B+tree). This property is maintained because any page can split, but only splitting the root adds another level. I've described how the height of an index can incr

You have a row in Postgres. You want that same row in MongoDB — not tonight in a batch job, but the instant it changes. And next month you'll want it in Redis too, and maybe POSTed to some webhook. Today that's a Kafka cluster, a Debezium connector, a sink connector, a schema registry, and a weekend. For a job that is, at its heart, one sentence: a source → one or more destinations → kept in sync. Syncle is an open-source tool that does exactly that sentence, and nothing you didn't ask for. Connect your databases, draw a bridge from a source to one or more destinations, and the moment a row changes in the source it's written to every destination you linked. Any engine to any engine — PostgreSQL · MySQL/MariaDB · SQLite · MongoDB · Redis — plus HTTP endpoints when you need them. This post is a tour of what it does and, for the curious, how it's built. The core idea: a bridge A bridge reads rows from a source and writes each one to its destinations . A destination is either: another database — the headline feature. Postgres → MongoDB, MySQL → SQLite, MongoDB → Redis. One bridge can fan out to several databases at once, and bridges can chain (A → B → C). an HTTP endpoint — POST/PUT/PATCH each row to a URL with a payload you design, for feeding a service instead of a database. The interesting part isn't that it copies data — plenty of things copy data. It's the guarantees around how . No duplicates, ever Every database write is an idempotent upsert , keyed by columns you choose. So replays, retries, and at-least-once redeliveries never double-write. Under the hood each engine does it with its own native atomic operation: Engine Upsert PostgreSQL / SQLite INSERT ... ON CONFLICT MySQL INSERT ... ON DUPLICATE KEY UPDATE MongoDB updateOne(filter, ..., { upsert: true }) Inserts, updates, and deletes all propagate — a delete routes to a keyed delete on each target. Missing table? It builds it If the destination table or collection doesn't exist, Syncle creates it from the sou

Jeremy Edberg & Qian Li discuss why external orchestrators decrease reliability and how to use your existing database for durable execution. They share how DBOS Transact uses standard tables, SKIP LOCKED queues, and unique primary keys to manage complex, fault-tolerant AI workflows with minimal latency, all without the operational overhead of separate distributed systems. By Jeremy Edberg, Qian Li

Every morning, an online store receives the previous day’s orders from a marketplace partner. The file comes in JSON format. The company needs to add those orders to its main MongoDB orders collection. The sales manager also needs a CSV report that can be opened in Excel. That sounds like a small task. Import the file, copy the documents, export the report. But in practice, a few things can break the process. A date can be imported as a string. A field can have the wrong name. One batch may use total , while the main collection uses totalAmount . A temporary collection can keep old records and trigger duplicate key errors. A CSV export can create null values because the mapping points to fields that do not exist. And then there is customer data. The manager may need the sales numbers, but they probably do not need real customer names or internal customer IDs. This article walks through a real daily workflow: Import marketplace JSON ↓ Store the batch in a temporary MongoDB collection ↓ Copy the orders into the main orders collection ↓ Mask customer fields during export ↓ Create a CSV report The goal is not just to move data from JSON to CSV. The goal is to make the process repeatable, easier to check, and safer to share. The workflow The workflow has three jobs: Import Yesterday Orders ↓ Add Orders to Main ↓ Export Daily Sales Report The important part is the parent relationship between the jobs. Add Orders to Main depends on Import Yesterday Orders , so it only runs after the JSON file is imported successfully. Export Daily Sales Report depends on Add Orders to Main , so the CSV is created only after the main orders collection has been updated. This prevents the report from being generated when data is missing or incomplete. The incoming JSON file The partner sends a file with yesterday’s completed orders. A single order looks like this: { "orderId" : "ORD-2026-07-201" , "customerId" : "CUST-1003" , "customerName" : "Sofia Rossi" , "orderDate" : "2026-07-21T08:20:00

There's a class of bug reports that keeps coming back in the NocoBase community, especially in the Chinese-language forum: "all my times are off by 8 hours" or "dates show up as the day before." China is UTC+8, so the shift is 8 hours there. I run my instances at UTC+9, and sure enough — my shift is 9 hours. Whatever your offset is, that's the size of your shift. That pattern is a strong hint that this isn't random corruption. It's a mechanism. I set up NocoBase 2.x against both PostgreSQL and MySQL and measured what actually gets stored and how it gets reinterpreted, until the mystery had a concrete answer. Test setup: NocoBase 2.0.51 and 2.1.23 (official Docker images) × PostgreSQL 16 and MySQL 8.4. All data written and read through the REST API, with the server timezone controlled via the container's TZ environment variable. I'm deliberately ignoring the browser-side rendering here — this is about what the server stores and how it interprets it. Background: 2.x has four datetime field types NocoBase 2.x collections offer four datetime-ish field types ( official list — though several of the per-type detail pages still say "To be added", which is exactly why I measured instead): Type What it's for Datetime (with time zone) Absolute instants — event start times, logs Datetime (without time zone) Wall-clock times you want preserved as-is Date only Birthdays, due dates, anniversaries Unix timestamp System integration Measurement 1: what each type actually stores I imported "2026-07-12 09:00" via xlsx and looked at the raw values in each database (identical on 2.0.51 and 2.1.23): Field type PostgreSQL MySQL Datetime (with TZ) timestamptz → 2026-07-12 09:00:00+09 ( an absolute instant, offset included ) DATETIME → 2026-07-12 09:00:00 ( wall clock only — no offset information ) Datetime (without TZ) timestamp → 09:00:00 DATETIME → 09:00:00 Date only date → 2026-07-12 date → 2026-07-12 The first row is the whole story. The same field type — "Datetime (with time zone)" — i

Using Firestore as "the app's primary database" is easy at first. Writes and reads complete in one place, and onSnapshot gives you realtime push for free. But as a service grows, keeping Firestore as the SoT (source of truth) exposes some fatal constraints. Where SoT-Firestore hurts Weak transaction boundaries, missing complex queries, the cost structure, the difficulty of migrating away. The harshest one: you cannot narrow related updates with a WHERE . For the use case "update a set of documents matching a condition, consistently, in one go," Firestore is structurally weak. Redefining it as a public cache In one project, I made Cloud SQL the SoT and treated Firestore as a "read-optimized projection." Writes go through the backend (Next.js / Cloud Run), and the SoT (Cloud SQL via Data Connect) and Firestore are both updated within the same write path . Separately, a Cloud Run reconciliation job runs and checks and repairs consistency against the SoT as authoritative. This reconciliation job is enqueued at the start of the request, before the DB updates. Because it's queued first, even if the write dies halfway, the consistency check always runs afterward and repairs the state. From the client's perspective, Firestore is a rebuildable cache — in the worst case it can be rebuilt from the SoT. // Writes go through the backend. The backend updates both the SoT // (Data Connect → Cloud SQL) and Firestore. Clients never write Firestore directly. await backend . updateEntity ({ id , title }); // Changes are pushed back in realtime via Firestore's onSnapshot unsubscribe = onSnapshot ( entityRef , ( snap ) => render ( snap . data ())); Benefits and costs The benefits are clear. The SoT side (SQL) brings transactional consistency and complex queries; the read side (Firestore) brings freedom in data modeling and realtime push; and consistent syncing with external systems (OpenSearch / BigQuery, etc.) coexists naturally. Since the backend updates Firestore at write time, the f

-- looks routine in review, locks your table in production: CREATE INDEX idx_orders_customer ON orders ( customer_id ); ALTER TABLE users ALTER COLUMN email SET NOT NULL ; Both of those statements hold a lock that blocks traffic while they scan or build against the table. On a small table you never notice. On a busy production table it is downtime, and a diff review does not catch it because the SQL looks fine. pg-migration-guard is a free tool that reads the migration and flags these before they reach production: $ pg-migration-guard V42__orders.sql WARN IDX01 CREATE INDEX without CONCURRENTLY [becomes a blocker on a large or busy table] Why: Plain CREATE INDEX takes a lock that blocks all writes until the build finishes. Safe: CREATE INDEX CONCURRENTLY idx ON tbl (col); -- outside a transaction WARN NN01 SET NOT NULL scans the whole table under ACCESS EXCLUSIVE Safe: ADD CONSTRAINT c CHECK (col IS NOT NULL) NOT VALID; VALIDATE CONSTRAINT c; then SET NOT NULL; Summary: 0 blocker, 2 warn, 0 advisory Why migrations bite Whether a Postgres migration is safe has almost nothing to do with intent and everything to do with locks. A migration is dangerous when it holds a strong lock while it scans or rewrites a large table. ACCESS EXCLUSIVE blocks reads and writes; SHARE blocks writes. The change that reads fine in review is the one that quietly takes that lock. Put it in CI The most useful place for this is the pull request, so a risky migration fails the build before anyone merges it. There is a GitHub Action: name : migration-guard on : pull_request : paths : [ " db/migration/**.sql" ] jobs : guard : runs-on : ubuntu-latest steps : - uses : actions/checkout@v4 - uses : actions/setup-python@v5 with : python-version : " 3.x" - uses : technical-turtle/pg-migration-guard@v1 with : path : db/migration fail-on : blocker Each finding shows up as an inline annotation on the diff, so the author sees exactly which line is the problem and what the safe rewrite is. Or run it locall

When a gbase database cluster experiences an unclean shutdown — a sudden power loss or kernel panic — some metadata that was still in memory may never make it to disk. This can leave the critical gbase.table_distribution table with missing rows, causing queries against certain tables to fail with the error: ERROR 1149 (42000): (GBA-02SC-1001) Can't get distribution attribute of table `testdb`.`t2_test`, please check your gbase.table_distribution. The Symptom The affected table enters a contradictory state: Dropping it returns Unknown table . Re‑creating it returns Table already exists . The data files exist on disk, but the metadata record that describes how the table is distributed is gone. Root Cause GBase 8a stores critical table metadata — distribution type (hash or random), replication status, hash column — in the system table gbase.table_distribution . After an abrupt shutdown, this table may have missing entries, causing the cluster to lose track of the table's distribution properties. Recovery Options Option 1: Re‑insert the Metadata Row (Keeps Data Intact) This is the recommended approach when the data must be preserved. Insert the missing row directly into gbase.table_distribution . -- Insert the lost metadata record INSERT INTO gbase . table_distribution VALUES ( 'testdb.t2_test' , 'testdb' , 't2_test' , 'NO' , NULL , NULL , NULL , 'NO' , 2 ); -- Verify the row is now present SELECT * FROM gbase . table_distribution WHERE index_name LIKE '%t2_test%' ; Key fields: isReplicate : YES if the table is replicated, otherwise NO . hash_column : The distribution key column name (if hash‑distributed); NULL for random distribution. data_distribution_id : A numeric ID. Copy the value from another healthy table in the same database to keep it consistent. After inserting, restart the gcluster service so the metadata takes effect (the table_distribution table is managed by gcluster, not gnodes): gcluster_services gcluster restart The table should now be queryable again.

AI coding assistants have transformed the way we build software. They can understand your code, generate features, and help with debugging but they usually have no idea what's inside your database. That's where Neo4j Laravel Boost comes in. It integrates the official Neo4j MCP server directly into Laravel Boost , giving any MCP-compatible AI client access to your live Neo4j database and graph tooling. Instead of managing multiple MCP servers, everything is exposed through your existing Laravel Boost server. Why Use Neo4j Laravel Boost? Once configured, your AI assistant can: 🔍 Inspect your live Neo4j schema 📝 Execute read and write Cypher queries 🧠 Query your Laravel container dependency graph 📊 Access Graph Data Science (GDS) procedures 🤖 Work through a single Laravel Boost MCP server Instead of relying solely on static code analysis, your AI assistant gains access to your application's graph data and architecture, making it much more capable. Installation Install the package using Composer. composer require --dev neo4j/laravel-boost Configure your Neo4j connection. NEO4J_URI=bolt://localhost:7687 NEO4J_USERNAME=neo4j NEO4J_PASSWORD=your-password Run the interactive setup. php artisan neo4j-boost:setup The setup wizard validates your Neo4j connection, configures the package, optionally installs the official Neo4j MCP binary, and can even spin up a local Neo4j Docker instance for you. Connect Your MCP Client Your AI client only needs a single MCP server configuration. { "mcpServers" : { "laravel-boost" : { "command" : "php" , "args" : [ "artisan" , "boost:mcp" ], "env" : { "APP_ENV" : "local" } } } } Whether you're using Cursor , Claude Code , or another MCP-compatible client, Neo4j tools become available alongside your existing Laravel Boost tools. Explore Your Laravel Dependency Graph One of the most interesting features of Neo4j Laravel Boost is the ability to export your Laravel service container directly into Neo4j. php artisan container:graph Once exported, yo

A SQLite service can keep answering health checks while its WAL grows without a successful checkpoint. One forgotten read transaction is enough to reproduce the risk. Run a controlled drill: Enable WAL mode and create a small table. Open connection A, begin a read transaction, and keep it open. From connection B, commit batches of writes for 60 seconds. Sample WAL bytes and checkpoint results every second. Release A and observe recovery. PRAGMA journal_mode = WAL ; PRAGMA wal_checkpoint ( PASSIVE ); Record the three checkpoint counters plus duration, WAL file size, oldest transaction age, write latency, and free disk bytes. A green SELECT 1 says nothing about checkpoint progress. My fault-injection gate looks like this: Given: one reader holds its transaction When: 10,000 writes commit Then: writes remain bounded And: WAL growth triggers an alert before the disk budget When: the reader closes Then: checkpoint progress resumes and WAL size converges Do not start with an automatic TRUNCATE loop. A busy result is evidence of contention; aggressive checkpoints can add latency without fixing the reader lifecycle. First identify long transactions, ensure result iterators close, and define a maximum transaction age. SQLite’s WAL documentation explains that a checkpoint must stop when it reaches pages beyond an active reader’s end mark. This is why the drill needs concurrency rather than a synthetic file-growth assertion. Operational thresholds should be tied to a disk budget: alert when projected WAL growth reaches the remaining safe window, not at an arbitrary file size. The runbook should name how to find the oldest reader, how to shed writes, and when a restart is safer than waiting. A health check is useful only when it measures the subsystem that is failing.

Prisma and Drizzle connect as the postgres role and bypass Supabase RLS entirely, so your policies never protect ORM queries. Here's the fix. TL;DR: No. Prisma and Drizzle open their own direct Postgres connection and log in as the postgres role, which owns your tables and carries BYPASSRLS — so Row Level Security is skipped on every ORM query. Point your app's connection at a dedicated, non-owner NOBYPASSRLS role (and keep the auth check in your code), not at postgres . If you built your Supabase project assuming RLS is a safety net on the data itself, adding an ORM quietly punches a hole straight through it. Your policies are still there. They just never run for the ORM's connection. Here's the mechanism, the myth to unlearn, and three fixes in order of how much you should reach for them. Does Prisma respect Supabase RLS? No. RLS is not a global property of the database — Postgres enforces it per role, per statement . A policy only bites for a role that is (a) not the table's owner, (b) has no BYPASSRLS attribute, and (c) is not a superuser. The Supabase JS client satisfies all three because it reaches Postgres through PostgREST, which runs your query as the unprivileged anon or authenticated role. Prisma and Drizzle satisfy none of them: they read DATABASE_URL and open a raw SQL connection as postgres , which owns virtually every table you migrated and holds BYPASSRLS . Either fact alone is enough for Postgres to skip your policies. So the same query that returns one tenant's rows through supabase-js returns every tenant's rows through Prisma. That is not a bug in your policy — it's the connection role. Two doors into the same database There are two completely different paths to your data, and they authenticate as different roles. The supabase-js path (RLS enforced). supabase-js talks HTTP to PostgREST, not to Postgres directly. PostgREST connects as authenticator , validates the request JWT, and does a SET ROLE into anon or authenticated for the statement. Those

Rds challenge lab devto post 🗄️🐬 aws #rds #database #tutorial Build Your DB Server and Interact With Your DB INTRO Did a hands-on AWS challenge lab on Amazon RDS. Task: spin up a managed database, connect from a Linux server, and run real SQL — create tables, insert data, join across tables. No hand-holding here, just requirements to figure out myself. Here's the walkthrough. SCENARIO Service: Amazon RDS Role: Cloud/DB Admin Goal: Launch RDS under set constraints, connect via EC2, run SQL (create, insert, select, join) ARCHITECTURE LinuxServer (EC2) sits in the Lab VPC — this is the client RDS instance (Aurora or MySQL) in the same VPC Security group lets LinuxServer talk to RDS Flow: LinuxServer -> MySQL client (port 3306) -> RDS -> tables STEP 1: LAUNCH THE RDS INSTANCE Constraints for this lab: Engine: Aurora (Provisioned) or MySQL — no serverless Template: Dev/Test or Free tier No standby instance (single-AZ only) Instance size: db.t3.micro to db.t3.medium Storage: gp2, up to 100 GB — no Provisioned IOPS Network: Lab VPC Security group must allow LinuxServer access MySQL only: turn off Enhanced Monitoring On-Demand only These limits keep costs in check — Provisioned IOPS and Multi-AZ are the fastest ways to blow up an RDS bill. Noted the master username, password, and endpoint — needed next. STEP 2: CONNECT TO THE LINUX SERVER Downloaded the PEM key, grabbed the LinuxServer address, connected over SSH: chmod 400 labsuser.pem ssh -i labsuser.pem ec2-user@<LinuxServer-address> This box is just the SQL client — it needs network access to RDS, nothing more. STEP 3: INSTALL MYSQL CLIENT AND CONNECT On the LinuxServer: sudo yum install mysql -y Connect using the master credentials from Step 1: mysql -h <rds-endpoint> -u <master-username> -p If it hangs, it's almost always the security group — check port 3306 inbound. STEP 4: CREATE THE RESTART TABLE CREATE DATABASE lab_db ; USE lab_db ; CREATE TABLE RESTART ( StudentID INT , StudentName VARCHAR ( 100 ), RestartCity VA

Python quickstart: nutrition data in 10 lines You can search Dietly's public nutrition catalog with one GET request to https://api.getdietly.com/search . The response is a JSON array of foods, not an object with a results property. Install requests , run the example below, and you have a working nutrition lookup. The rest of this page turns that single call into something dependable: a client class, correct field handling, and retries that respect the rate limit. Make your first request The only required parameter is q (minimum two characters). Use limit to cap results between 1 and 50. Ten is plenty while you explore. import requests r = requests . get ( " https://api.getdietly.com/search " , params = { " q " : " greek yogurt " , " limit " : 5 }, timeout = 10 ) r . raise_for_status () foods = r . json () for food in foods : print ( food [ " name " ], food . get ( " protein_g " )) Understand the fields you get back Each result is a flat object. Nutrient values are per 100 g of the product unless noted, and any nutrient can be null when the source label did not list it. Treat null as unknown, never as zero. Field Meaning id Stable Dietly food ID, used for direct lookups name , brand Product name and brand when known calories_kcal Energy per 100 g protein_g , fat_g , carbs_g Macronutrients per 100 g fiber_g , sugar_g , saturated_fat_g , sodium_mg Detail nutrients per 100 g, often null serving_size_g , serving_desc One serving in grams and its label wording, when provided source , confidence Where the record came from and how sure Dietly is static_url Path to the food's page on getdietly.com, when one exists To convert a per-100 g value to a portion, multiply by grams and divide by 100. For a 150 g serving of a food with 10 g protein per 100 g, that is 10 * 150 / 100 = 15 g. Read the result defensively An empty search is [] , which is a normal outcome rather than an error. Guard for it, and keep null nutrients as null in your own model. food = foods [ 0 ] if foods else