标签：#base

In the rush to build AI agents, we defaulted to complex vector databases. But high-traffic platforms are converging on a simpler, more robust foundation: plain files. Most long-term agent memory setups are massively over-engineered. When developers start building LLM applications, the default prescription is almost always: "Spin up a managed vector database and build a RAG pipeline." But if you look at the highest-traffic production agent platforms (like Claude Code, Manus, and OpenClaw), a quieter trend has emerged. They are bypassing the enterprise embeddings store and using plain markdown files as their primary memory substrate. This is not a regression to simplicity. Done well, it is a stronger engineering foundation because files are inspectable, diffable, portable, and git-native. But a folder of plain text notes with no structure is just a slow, poorly indexing database. To make a file-first architecture work at scale, you must follow a fundamental system design principle: separate storage from search . The Core Invariant: Storage vs. Search The single highest-leverage decision you can make in agent memory design is treating your storage layer and search indexes as completely separate systems. Storage (Canonical Source of Truth): Versioned, human-readable files (Markdown + YAML frontmatter). Search (Derived Index): Derived search structures (vector databases, full-text BM25 indexes, entity graphs, keyword indexes). In this architecture, every search index is treated as a disposable artifact. You can delete your vector embeddings database or rebuild your entity graph at any time, with zero loss of underlying memory. This buys you three advantages: Auditability for free: By storing memories in text files, you can version-control them using Git. Every memory update, supersession, or correction is diffable, attributable, and reversible without any custom database versioning logic. Algorithmic freedom: Swap your embedding models, adjust your chunking strategies, o

AWS released Managed Knowledge Bases for Amazon Bedrock on 17 June 2026. The feature significantly reduces the operational complexity of building Retrieval-Augmented Generation (RAG) solutions by allowing Bedrock to manage the vector storage, indexing, embeddings, and retrieval infrastructure on your behalf. For teams looking to deliver an Agent Core proof of concept or their first production RAG workload quickly, this can be a compelling option. However, there are some important trade-offs to understand before committing to the managed approach. Traditionally, a Bedrock Knowledge Base required a customer-managed vector store such as: OpenSearch Serverless OpenSearch Managed Clusters Aurora PostgreSQL with pgvector Pinecone DocumentDB Other supported vector databases With a Managed Knowledge Base, Bedrock handles the underlying vector infrastructure and embedding model selection for you. Creating one from the AWS CLI is straightforward: aws bedrock-agent create-knowledge-base \ --name "my-managed-kb" \ --role-arn "arn:aws:iam:: ${ AWS_ACCOUNT_ID } :role/service-role/AmazonBedrockExecutionRoleForKnowledgeBase_ihv1p" \ --knowledge-base-configuration '{ "type": "MANAGED", "managedKnowledgeBaseConfiguration": { "embeddingModelType": "MANAGED" } }' Advantages Lower operational overhead There is no need to provision, secure, monitor, patch, or scale a separate vector database. Lower costs S3 storage is cheaper than database storage. Pay only for each ingestion and retrieval operation. Indexing and searching compute is free. No 24/7 server costs. Faster time-to-value Managed Knowledge Bases make it possible to stand up a RAG solution in minutes rather than days. Automatic embedding management Bedrock manages embedding selection and indexing, reducing the number of architectural decisions required from development teams. Cost-effective for smaller workloads The managed model can be attractive for: Proofs of Concept Departmental knowledge bases Agent Core pilots Workloads wi

Sign-in worked flawlessly in dev. Then a real user pasted a real code and got "invalid format" — before the code ever reached Supabase. The credential was fine. My regex was wrong. Here's the one-line assumption that broke auth for every human who wasn't me. I run a Discord-native Company Brain. Teams /save docs and /ask grounded answers; access is gated by a magic-link claim that emails a one-time code. Standard GoTrue OTP flow. The client shows a box, you paste the code, the server verifies it. Boring — which is exactly what auth should be. The bug: a six-digit assumption in a validation guard The claim handler did a cheap client-side sanity check before calling verifyOtp : // The bug. Looks reasonable. Rejects every real code. const OTP = /^ \d{6} $/ ; function normalize ( input : string ): string { const code = input . trim (); if ( ! OTP . test ( code )) throw new Error ( " Enter the 6-digit code from your email. " ); return code ; } Every OTP tutorial uses \d{6} . Every code demo shows six digits. So I typed six digits into the test and it passed. In dev I was generating my own codes and never actually reading the email. Supabase's GoTrue emits an eight-digit code on this project. ^\d{6}$ rejects eight digits outright. The user's perfectly valid credential got thrown out by my own front door with a lie for an error message — "enter the 6-digit code" when the email plainly showed eight. Why it happens: OTP length is a setting, not a constant The length of a GoTrue email OTP is configurable — GOTRUE_MAILER_OTP_LENGTH (Dashboard → Authentication → Email). It defaults to six in many setups and to eight in others depending on when and how the project was provisioned. The number in the tutorial is that author's project setting , not a property of OTPs. Hardcoding 6 couples your client to a server config you don't control and might change. Bump the length for security later and every client silently starts rejecting valid codes. No error in your logs — the rejection

Quick Answer (TL;DR) Modifying an RDS instance class in place causes 5 to 15 minutes of downtime while AWS reboots the database. To right-size without downtime, use RDS Blue/Green Deployments (fastest, cleanest), a read-replica promotion (works on older engines), or a Multi-AZ failover to a resized standby. Blue/Green is the 2026 default for most workloads on MySQL, MariaDB, Postgres, and now SQL Server. Why this happens RDS instances are Managed EC2 hosts running the DB engine, and a class change (say db.m6i.large to db.m6i.xlarge ) requires stopping the process, migrating the EBS volumes to a new host, and restarting. AWS's default "modify" workflow does this in place and warns you about downtime. The workarounds exist because that reboot is unacceptable for user-facing services, so you build the new instance alongside the old one and cut over. Fix #1: Use RDS Blue/Green Deployments The 2026 default. Available for RDS MySQL, MariaDB, PostgreSQL, and SQL Server (added mid-2025). Steps: In the RDS console, select the instance and choose Actions → Create Blue/Green Deployment . Set the Green instance to your target instance class. AWS creates a full standby using logical replication, keeps it in sync, and validates health. When ready, click Switch over . Cutover typically takes under 60 seconds. Applications reconnect using the same endpoint. Command-line equivalent: aws rds create-blue-green-deployment --blue-green-deployment-name resize-prod --source arn:aws:rds:... --target-db-instance-class db.m6i.xlarge Best when: your engine supports it and you can tolerate the extra cost of running two instances for the sync window. Fix #2: Read-replica promotion For engines or versions that do not yet support Blue/Green, or for cross-region resizing. Steps: Create a read replica with the desired new instance class. Wait for the replica to catch up (near-zero lag). Point application writes to the read replica endpoint (requires connection-string change or DNS switch). Promote

Data Modeling, Relationships, and Schemas in Data Analytics In the fields of data analytics, data warehousing, and database management, modeling and schema design are the fundamental pillars used to organize and query information efficiently. This article provides a comprehensive guide to these core concepts. 1. Data Modeling Data modeling is the architectural process of designing how data is stored, interconnected, and accessed within a system. Core Questions Addressed: Storage: What specific data points need to be captured? Structure: How should individual tables be organized? Connectivity: How do these tables interact with one another? Levels of Data Models: Conceptual Model: A high-level business perspective focusing on entities and their relationships, devoid of technical specifications. Logical Model: Defines specific attributes, keys, and relationships. It is independent of the Database Management System (DBMS). Physical Model: The actual implementation within a database, including technical details like indexes, partitions, and storage requirements. 2. Relationships Relationships define the logic of how data in one table corresponds to data in another. One-to-One (1:1): A single record in Table A relates to exactly one record in Table B. One-to-Many (1:M): The most common relationship; for example, one Customer can place many Orders . Many-to-Many (M:M): Multiple records in one table relate to multiple records in another. This requires a Junction Table (Bridge Table) to function. Example: One Student can enroll in many Courses, and one Course contains many Students. 3. SQL Joins Joins are used to combine rows from two or more tables based on a related column. Join Type Description Inner Join Returns only the records that have matching values in both tables. Left Join Returns all records from the left table and the matched records from the right. Right Join Returns all records from the right table and the matched records from the left. Full Outer Join Returns

I've been building on DynamoDB since around 2015, and these days I build tools for it: dynoxide , a DynamoDB engine, and Nubo , a native client. So I'm not neutral about it. It's the first database I reach for, and with reason - the operational overhead is close to nil, no connections to pool or instances to size, and it holds the same single-digit-millisecond reads whether the table has a thousand items or a billion. The data modelling is a craft, and a satisfying one. It's also one of the harder databases to learn, and that's the part I keep coming back to. DynamoDB punishes the instincts you bring from SQL. You don't normalise and join at read time; you work out the questions your app will ask first, and shape the data around those access patterns, until one table answers all of them. It's a real shift in how you think, and it's where a lot of people bounce off - it feels backwards right up until it clicks. The people who teach it best all teach it the same way. Alex DeBrie's The DynamoDB Book , the arc.codes team's examples, Rick Houlihan's re:Invent talks - the legendary ones, where he models half a dozen access patterns onto a single table at a hundred miles an hour - none of them hand you rules to memorise. They show you patterns and make you run them. I learned a lot of my DynamoDB from all three, and it stuck because I was building as I went. That last part is the bit that's hard to come by on your own. Reading about an access pattern and having it in your fingers are different things, and to run one - build the table, write the items, fire the query and see what comes back - you need an AWS account, or a local emulator installed and seeded. Enough friction that plenty of people read about single-table design without ever building one. There was a second thing pulling the same way. dynoxide had learned to run in the browser only last month, compiled to WebAssembly with no server behind it, and it was a preview I didn't fully trust. What it needed was a real

"That would be an awesome capability."

My dashboard had a useMemo doing arithmetic it had no business doing. It was a Pokémon TCG Pocket collection tracker, but that part doesn't matter. What matters is that the home page needed to show three things: overall completion, completion per set, and which set you were closest to finishing. The way I'd built it, the browser was fetching every card and every owned record, then grinding through the math on each render to figure all of that out. It worked. It also got slower and harder to read every time the data grew or I added a metric. So I moved the aggregation out of React and into Postgres, and the surprising result was that my frontend got boring . Fewer hooks, less state, almost nothing left to break. That's the whole argument of this post: aggregation belongs in the database, and when you put it there, the React code that's left over is the kind of boring you actually want in the layer your users touch. What "fetching everything into React" actually looks like Here's the shape of the original dashboard. Load all the cards, load the user's owned rows, then derive everything on the client. const [ cards , setCards ] = useState < Card [] > ([]); const [ owned , setOwned ] = useState < OwnedCard [] > ([]); useEffect (() => { ( async () => { const { data : allCards } = await supabase . from ( " cards " ). select ( " * " ); const { data : ownedRows } = await supabase . from ( " user_cards " ) . select ( " card_id " ); setCards ( allCards ?? []); setOwned ( ownedRows ?? []); })(); }, []); const ownedIds = useMemo (() => new Set ( owned . map (( o ) => o . card_id )), [ owned ]); const perSet = useMemo (() => { const groups : Record < string , { total : number ; have : number } > = {}; for ( const card of cards ) { const g = ( groups [ card . set_id ] ??= { total : 0 , have : 0 }); g . total += 1 ; if ( ownedIds . has ( card . id )) g . have += 1 ; } return groups ; }, [ cards , ownedIds ]); const overall = useMemo (() => { const total = cards . length ; const ha

If you've ever worked with e-commerce data, you know that "a pair of jeans" is never just one product. A single style might come in 5 washes, 8 sizes, and 3 inseam lengths. That's 120 potential SKUs. Handling this correctly in an API can be tricky, so let me share a pattern I've used for structuring product variants. The core problem is balancing flexibility with performance. You want customers to filter by size, color, and fit without making dozens of API calls. Here's a simple but effective approach using a normalized database schema with a flat query layer: -- Products table (the &quot;parent&quot;) CREATE TABLE products ( id UUID PRIMARY KEY , name TEXT NOT NULL , description TEXT , base_price DECIMAL ( 10 , 2 ), category TEXT ); -- Variants table (the actual sellable items) CREATE TABLE variants ( id UUID PRIMARY KEY , product_id UUID REFERENCES products ( id ), sku TEXT UNIQUE NOT NULL , size TEXT , color TEXT , wash TEXT , inseam TEXT , price DECIMAL ( 10 , 2 ), -- can override base price stock_quantity INT , image_url TEXT ); The key insight? Keep the product metadata (description, care instructions, brand story) in the products table, but put all the sellable attributes in variants. This lets you run queries like: -- Find all size 28 jeans in &quot;mid wash&quot; under $80 SELECT p . name , v . color , v . wash , v . price , v . stock_quantity FROM products p JOIN variants v ON p . id = v . product_id WHERE p . category = &# 039 ; women - jeans &# 039 ; AND v . size = &# 039 ; 28 &# 039 ; AND v . wash LIKE &# 039 ; % mid %&# 039 ; AND v . price & lt ; 80 AND v . stock_quantity & gt ; 0 ORDER BY v . price ; For the frontend, I usually return a flattened structure: { &quot;product&quot;: { &quot;id&quot;: &quot;abc -123 &quot; , &quot;name&quot;: &quot;Classic Straight Leg Jean&quot; , &quot;description&quot;: &quot;High-rise fit in stretch denim...&quot; , &quot;availableSizes&quot;: [ &quot; 24 &quot; , &quot; 25 &quot; , &quot; 26 &quot; , &quot; 27 &quot;

Introduction As applications grow, traditional relational databases such as MySQL may struggle with analytical workloads involving millions of records and complex aggregations. While MySQL excels at Online Transaction Processing (OLTP), ClickHouse® is purpose-built for Online Analytical Processing (OLAP), enabling lightning-fast analytical queries on massive datasets. Migrating data from MySQL to ClickHouse® allows organizations to build high-performance reporting systems, dashboards, and real-time analytics without impacting transactional workloads. In this guide, you'll learn several approaches to migrate data from MySQL to ClickHouse®, along with their advantages, limitations, and ideal use cases. Why Migrate from MySQL to ClickHouse®? MySQL and ClickHouse® are designed for different workloads. Feature MySQL ClickHouse® Storage Model Row-based Columnar Best For Transactions (OLTP) Analytics (OLAP) Query Speed Fast for row lookups Extremely fast for large scans Aggregation Performance Moderate Extremely fast Scalability Primarily Vertical Optimized for analytical scaling Typical Use Cases Applications and transactional systems Reporting, dashboards, and analytics Migrating from MySQL to ClickHouse® makes sense when: Analytical queries are becoming slow in MySQL. You need real-time dashboards over large datasets. Reporting queries are impacting your production database. You regularly process millions or billions of rows. Migration Architecture MySQL │ ▼ Export / Synchronization │ ▼ Data Transformation │ ▼ ClickHouse® │ ▼ Dashboards / Analytics Migration Methods There are multiple ways to migrate data depending on your requirements. Method 1: CSV Export and Import (Recommended for Beginners) This is the simplest approach for performing a one-time migration of historical data. Step 1: Export Data from MySQL Run the following command inside MySQL: SELECT * INTO OUTFILE '/tmp/employees.csv' FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY ' \n ' FROM employ

As the AI Engineer World’s Fair kicked off officially on Monday, the halls were filled for the...

Modern web application architectures typically run each layer in its own process, like a NodeJS server and database both running in their own processes. They communicate via a network connection or localhost socket. This separation introduces protocol overheads, TCP stack latency, and data serialization/deserialization costs on every single query. Planck is designed around the concept of Zero-Distance Architecture that co-locates data and application code. It combines the database engine and a WebAssembly application runtime into a single, unified process. By running your application code directly inside the database process, database calls become direct in-memory function calls rather than network round-trips. This article provides a practical guide to getting started with Planck. We will look at the core toolchain, walk through setting up a self-contained local benchmark, compare its performance against a NodeJS, ExpressJS, MongoDB stack, and look at how to build more complex features. The Toolchain: Planck, planctl, and Workbench Running and managing a zero-distance app requires three main components. Planck itself is the core binary. It functions as both the storage engine (a WiscKey-style, LSM-tree-based engine) and the WebAssembly host. Instead of running a database in one process and your application server in another, you run a single Planck process. It loads your compiled WebAssembly application directly into its memory, running it in the same process space as the database. To manage this runtime, you use planctl. This is the command-line tool for developers. It handles the compilation of your code, packages it, and deploys it to the Planck host. It also allows you to perform database operations, like creating stores and defining indexes, export/import, backup/restore directly from your terminal. Finally, there is the Workbench. This is a web console that comes built into the platform. It provides a visual dashboard to monitor your applications, view databa

Wix-owned vibe coding platform Base44 has started rolling out its own AI model — with hopes that it will eventually outperform frontier models.

The Database resource in Oracle Cloud Infrastructure Database service provides an optional auto_backup_window option in its API during creation ( Terraform resource: oci_database_database ). The database resource can be used in an OCI Base DB system or Exadata Cloud VM Cluster pluggable database (PDB) for example. The time window enum value selected for initiating automatic backup for the database system is available in twelve two-hour UTC time windows as the following: Slot Description SLOT_ONE 12:00AM - 2:00AM UTC SLOT_TWO 2:00AM - 4:00AM UTC SLOT_THREE 4:00AM - 6:00AM UTC SLOT_FOUR 6:00AM - 8:00AM UTC SLOT_FIVE 8:00AM - 10:00AM UTC SLOT_SIX 10:00AM - 12:00PM UTC SLOT_SEVEN 12:00PM - 2:00PM UTC SLOT_EIGHT 2:00PM - 4:00PM UTC SLOT_NINE 4:00PM - 6:00PM UTC SLOT_TEN 6:00PM - 8:00PM UTC SLOT_ELEVEN 8:00PM - 10:00PM UTC SLOT_TWELVE 10:00PM - 12:00AM UTC Timezone used for the slots is always UTC regardless of the timezone used in the database. For example, if the user selects SLOT_TWO from the enum list, the automatic backup job will start in between 2:00 AM (inclusive) to 4:00 AM (exclusive) If no option is selected, a start time between 12:00 AM to 7:00 AM in the region of the database is automatically chosen. Reference Terraform resource: oci_database_database OCI API Reference: Database DbBackupConfig Safe harbor statement The information provided on this channel/article/story is solely intended for informational purposes and cannot be used as a part of any contractual agreement. The content does not guarantee the delivery of any material, code, or functionality, and should not be the sole basis for making purchasing decisions. The postings on this site are my own and do not necessarily reflect the views or work of Oracle or Mythics, LLC. This work is licensed under a Creative Commons Attribution 4.0 International License (CC-BY 4.0) .

The GitHub Advisory Database is processing more vulnerability reports than ever before. Here's what's driving the surge, how we're responding, and how the community can help. The post Inside the Advisory Database and what happens when vulnerability volume breaks records appeared first on The GitHub Blog .

Target built a generative AI system to improve marketing campaign forecasting by retrieving and ranking similar historical campaigns. Using embeddings, vector search, and LLM ranking, it replaces rule-based workflows. Evaluation shows 75% top-1 and 100% top-3 coverage. The system reduces manual effort, improves consistency, and uses feedback loops to refine retrieval using campaign outcomes. By Leela Kumili

Denormalization has been the standard approach to analytical data modeling for good reason. Moving joins, lookups, and business rules out of query time and into ingestion gives you the fastest possible reads for a known access pattern. For most of the past decade, it was often the practical default for latency-sensitive analytics. Earlier columnar engines and distributed query processors could execute joins, but many workloads paid for them through higher latency, higher compute cost, spill-to-disk, or distributed coordination overhead. That constraint has loosened. Modern columnar databases with advanced join algorithms have reduced the cost of runtime joins enough that normalization is now a genuinely viable option for many analytical workloads. Denormalization still delivers faster reads, but normalization can bring operational benefits: simpler pipelines, flexible schemas, and cleaner governance. Engineers can now make the decision based on their actual workload characteristics, rather than being forced into one approach by engine limitations. This guide is a decision framework for making that choice in ClickHouse. It starts with why denormalization became the default, explains what has changed in join performance, then compares the tradeoffs on both sides so you can decide where to denormalize, where to join, and where to use ClickHouse primitives that bridge the gap. For a broader evaluation framework covering latency, concurrency, ingest throughput, SQL flexibility, and cost across real-time OLAP options, see our guide to choosing a database for real-time analytics in 2026 . For a deeper comparison of how ClickHouse executes star schema joins against Druid, Pinot, and cloud DWHs, see our star schema and fast joins guide . TL;DR Denormalization and normalization are both valid modeling strategies. The right choice depends on your workload. Denormalization's tradeoffs are primarily operational : pipeline complexity, write-path overhead, data freshness lag, back

Have you ever wondered how apps like university portals know which courses a student is enrolled in, or how they pull up an instructor's full schedule in seconds? The answer lies in database relationships - one of the most important concepts in backend development. In this article, we'll explore: What database relationships are and why they matter The three types of relationships: One-to-One, One-to-Many, and Many-to-Many How relationship schemas work (primary keys, foreign keys) How SQL Joins let you pull connected data from multiple tables To keep things grounded, we'll use one running example throughout: a University Management System . By the end, you won't just understand the theory, you'll see exactly how these concepts connect in a real-world scenario. What Are Database Relationships? A database relationship defines how data in one table connects to data in another. Instead of storing the same information repeatedly, relational databases organize data into separate tables and link them using keys . Think about our university system. We have a table for students and another for courses . A student can enroll in multiple courses, and each course can have many students. Rather than storing a student's full details on every course record, we store the student's info once and create a relationship between the two tables. This keeps data clean, reduces duplication, and makes updates easy. If a student's email changes? Update it in one place - done. Here's a simple visual of what that looks like: +------------------+ +------------------+ | Students | | Courses | +------------------+ +------------------+ | student_id (PK) | | course_id (PK) | | name | | title | | email | | credits | +------------------+ +------------------+ \ / \ / \ / Enrollments (links students ↔ courses) Now let's look at the three types of relationships you'll encounter. Types of Database Relationships 1. One-to-One (1:1) Each record in Table A matches exactly one record in Table B and vice versa

This article covers debugging and deploying a Rust backed WASM module with a Firebase hosted web app...

The decision to migrate an on-premises database to the cloud is rarely a simple one. It touches infrastructure, budget, team workflows, and application uptime all at once. Yet for most organizations, moving your on-premises database to the cloud is no longer a question of if but when — and more importantly, how to do it without bringing production systems to their knees. Whether you're running PostgreSQL on bare metal or SQL Server in a company-owned data center, the path to cloud-hosted data has predictable pitfalls and, when navigated carefully, significant payoffs. This guide walks through the full migration process: from pre-migration assessment to post-migration validation, with real configuration examples and the kind of hard-won detail that vendor documentation tends to skip. Why Organizations Make the Move On-premises databases carry hidden costs that compound over time. Hardware refresh cycles, licensing renewals, physical security, power, and cooling — these expenses are easy to underestimate when they're already baked into the operating budget. Cloud databases shift that burden to a consumption model, where you pay for what you use and scale without a procurement cycle. Beyond cost, there's the question of availability. Cloud providers like AWS, Google Cloud, and Azure run managed database services with built-in replication, automated failover, and point-in-time recovery that would take a dedicated DBA weeks to configure manually. A startup can get enterprise-grade durability out of the box. A mid-sized company can stop worrying about what happens when a hard drive fails at 2 a.m. Performance is another draw — though it comes with caveats. Managed cloud databases excel at read-heavy workloads and benefit from proximity to cloud-native application services. Latency-sensitive operations or highly custom storage configurations sometimes perform better on dedicated hardware. Knowing which category your workload falls into is the first real question to answer.