标签：#data

Originally published at nlocoding.com 41% of enterprise database engineers already use AI tools daily to generate, optimize, or review SQL—up from just 14% in 2023 (Gartner, 2026). The new database arms race is invisible. Enterprises process 7.4x more data per person than they did five years ago. That’s not a typo. AI-assisted database development isn’t just about speed; it’s about not drowning in schema drift and query chaos. If you’re not automating, you’re lagging by $8,200 per developer per year (Forrester, 2026). 73%of data teams say AI reduced query errors (Redgate, 2026) AI-assisted database development is rewriting the rules in 2026 AI-assisted database development is now the backbone for 52% of Fortune 500 engineering departments, slashing schema build time by 48% on average (Stack Overflow Developer Survey, 2026). Developers no longer waste days hand-writing migration scripts or debugging malformed indexes. Instead, GPT-5-powered copilots like Tabnine and DataPilot draft DDL, suggest denormalization strategies, and catch performance anti-patterns before they hit production. The result: projects ship 23% faster, according to Fivetran’s 2026 benchmark. If you’re still relying on manual SQL, you’re not just slower—you’re more expensive. Find one workflow, automate it, and measure the delta. That’s how the best teams start. ⚠️ Common Mistake: Treating AI-generated schema suggestions as gospel. Blind trust leads to silent data loss or bloated tables. Always review before merging. Schema design is now a conversation, not a bottleneck Most people get this wrong: schema design is not just a technical hurdle—it’s a communication bottleneck. In 2026, 64% of product teams report that AI-driven schema prototyping (using tools like dbdiagram.io+AI Assist, $7/month) reduced handoff time between engineering and product by 58% (LinearB, 2026). Instead of four revision meetings, you get a Slack thread with three alternative schemas, clear tradeoffs, and a side-by-side diff

By Michael Nocito , data analyst · Published August 9, 2026 By the end of this page you can add an extension to a dashboard, tell the two hosting kinds apart, and read the permission box well enough to know what you're agreeing to. You'll also know the one behavior that surprises people after publishing, which is what an extension looks like in a PDF. It's about twelve minutes. Here's what to do before you add your first one. Find out where it runs. An extension you drop onto a dashboard is a web application, and some of them are hosted on Tableau-managed servers while others are hosted by whoever built them. That single fact decides how much thought the rest of the decision needs. The short version: an extension is a third-party web application running inside a dashboard object, and one of the two permission levels gives it your full underlying data along with table and field names. Where the code actually runs is the thing the panel doesn't show you, so it gets the picture. The original carries a diagram here. In words: A large rectangle labeled your dashboard contains four panels that all look alike. Three of them are shaded the same and marked as ordinary views. The fourth, in the lower right and outlined in a warning color, is labeled extension. A line runs from that fourth panel, crosses the boundary of the dashboard rectangle, and continues out to a separate box drawn outside and to the right labeled third-party host. The three ordinary views have no lines leaving the rectangle. The drawing shows that the extension panel sits inside the dashboard visually while its code and its data traffic reach outside it, which the other three panels never do. 1. What an extension actually is Before the explanation: you drop an extension onto a dashboard and it draws a chart type Tableau doesn't have. Where did that chart come from? From a web application, written by somebody else, running inside a panel on your dashboard. Tableau's own description is that extensions "let

A database client supporting 90+ engines sounds like a dependency-management problem disguised as a UI. My late-night question was simpler: how much of that complexity does t8y2/dbx carry before the first connection? The interesting claim is its small footprint—around 20 MB—combined with desktop, CLI, Docker, AI, and MCP Server modes. That is a much different architecture from shipping one heavy client per database vendor. The real test is not today’s +420 stars; it is startup latency, resident memory, and whether an unused adapter stays out of the hot path. Under the Hood The likely execution model is a shared core with database-specific drivers around it. The desktop interface, CLI, Docker image, and MCP endpoint become different front doors to the same connection and query layers. That design has two useful consequences: Connection handling and query behavior can stay consistent across interfaces. New database support does not require duplicating authentication, result formatting, or export logic. The edge case is driver loading. If all 90+ integrations initialize eagerly, startup and memory usage will grow quickly. Lazy loading is therefore more important than the headline database count. A Minimal Measurement Pass After downloading a release binary, I used this deliberately boring check: chmod +x ./dbx /usr/bin/time -v ./dbx --help 2>&1 \ | grep -E 'Elapsed|Maximum resident' For a source checkout, the first useful inspection is: git clone https://github.com/t8y2/dbx.git cd dbx find . -maxdepth 2 \( -name 'go.mod' -o -name 'Cargo.toml' -o -name 'Dockerfile' \) -print This avoids guessing the build system and immediately exposes whether the advertised modes are separate binaries, containers, or wrappers. Trade-offs I Would Watch A compact binary does not guarantee a compact running process. TLS libraries, database drivers, schema introspection, query history, and result grids can dominate memory after startup. MongoDB and Redis also do not fit neatly into a relat

JupyterGIS is a GIS-focused extension for Jupyter notebooks. The recent 0.16 release enhances collaborative features, real-time editing, and support for large-scale data processing, including remote sensing. It introduces better visualisation tools and extends compatibility to R users. Community feedback highlights practical concerns and a desire for improved portability. By Olimpiu Pop

Background : In migration projects involving different databases, incompatibility of SQL syntax is often encountered. Question : If there is a large amount of code that needs to be rewritten, manual processing would be time-consuming and prone to errors. Is it possible to achieve automatic conversion of code syntax in large quantities through tools? Solution : The open-source tool ZGLanguage can be utilized to perform automated conversion of SQL code in large batches. For example： Suppose 'ORACLE START WITH CONNECT' syntax code（ start_with_connect.sql ）： SELECT * FROM tree START WITH id = 1 CONNECT BY NOCYCLE PRIOR id = parentid ; By configuring the conversion rules, the above code can be directly converted into the following code（convert to "with recursive" syntax）： with recursive wr_tree as ( SELECT id , parentid , 1 as level from tree where id = 1 union SELECT tree . id , tree . parentid , level + 1 from tree , wr_tree where tree . parentid = wr_tree . id ) SELECT * from wr_tree order by id ; Conversion rule (STATR_WITH_CONNECT_SQL_REPLACE.syn) is as follows： __DEF_FUZZY__ Y __DEF_DEBUG__ N __DEF_CASE_SENSITIVE__ N __DEF_LINE_COMMENT__ -- __DEF_LINES_COMMENT__ /* */ __DEF_STR__ __IF_KW__ <1,100> [1,1]ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz [0,100]ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_ __DEF_PATH__ __START_WITH_CONNECT__ 1 : sel @ %__IF_KW__ | select : cc @ | * : frm @ | from : srctab @ | __NAME__ : sta @ %__IF_KW__ | start : wth @ %__IF_KW__ | with : swp @ | __NAME__ : dy1 @ | = : int @ | __INT__ : str @ + __STRING__ : cnn @ %__IF_KW__ | connect : by @ %__IF_KW__ | by : ncy @ %__IF_KW__ CAN_SKIP | nocycle : prr1 @ %__IF_KW__ CAN_SKIP | prior : col1 @ | __NAME__ : dy @ | = : col2 @ | __NAME__ : end @ | ; ----------------------------------------------------------------------- 1 : sel @ | with : sel @ | recursive : sel @ | wr_ : srctab @ \ __NAME__ : sel @ STRING | as : sel @ | __\n__ : sel @ | ( : sel @ | __\n__ : sel @ | selec

A senator's letter confirms the U.S. military moved to prevent the tracking after foreign adversaries used location data to target troops.

Rank tracking tools love to notify you the instant a number changes. We deliberately don't — our drop alert only fires once an app falls 10 spots or more between two measurements. The tempting, wrong version A 1-spot threshold sounds like the more attentive product. In practice it turns every notification channel into noise: App Store search rank has real day-to-day jitter that has nothing to do with anything you did — a competitor's own rank shifting, a re-index, sampling timing. Alert on every 1-spot move and within a week the alert is something people mute, which defeats the entire point of having one. Why 10, specifically 10 spots is large enough to almost never be pure noise and small enough to still catch a real problem while it's still cheap to fix — a keyword field edit, a screenshot swap, a review-response push. Wait for a 30-spot collapse before alerting and you've waited past the point where the fix is simple. The threshold is symmetric: the same 10-spot rule fires on a jump upward, so a keyword field change you made on purpose gets confirmed by the same mechanism that would have warned you if it went the other way. The trade-off we're making explicit This means small real movements — 3 spots, 5 spots — genuinely don't page anyone. That's intentional, not a limitation we're hiding: an alert system tuned to catch everything catches nothing anyone still trusts by week three. A threshold set high enough that every alert is worth opening is worth more than a lower one that trains you to ignore your own notifications. If you're building anything similar — uptime, price, rank, any noisy time series — the question worth asking isn't "how sensitive can I make this," it's "what's the smallest move that's still cheaper to catch early than to catch late." That number is rarely 1. We build Storelift , where this threshold governs both the in-app alert and the rank-drop email.

By Michael Nocito , data analyst · Published August 9, 2026 By the end of this page you can turn a chart that says E, W, N and S into one that says East, West, North and South, in about thirty seconds, without editing the data or writing a calculation. You'll also know exactly why the Aliases option is missing on some fields, which is the part that sends people looking for a workaround they don't need. It's about ten minutes. Here's the move. Right-click a dimension in the Data pane, choose Aliases, and type the name you want beside each value. The chart updates, the stored data doesn't change, and every view built on that field picks up the new labels. The short version: an alias renames the members of a discrete dimension. Only discrete dimensions have members, which is why measures, dates and continuous dimensions can't have one. An alias sits in a specific place, between what's stored and what's shown, and that placement explains everything else here. So it gets the picture. The original carries a diagram here. In words: Three stacked panels connected left to right. The left panel is labeled stored and holds four small cells reading E, W, N and S. The middle panel is a narrow vertical band labeled alias, holding four arrows. The right panel is labeled shown and holds four cells reading East, West, North and South. A solid arrow runs from the stored panel through the alias band to the shown panel, indicating the direction labels travel. A second arrow attempting to run backwards from the shown panel to the stored panel is crossed through with a heavy X, showing that renaming the label never changes the stored value. The stored cells still read E, W, N and S after the change. This is on the certification. Aliases sit in Section 2, Exploring and Analyzing Data, which is 37% of the Tableau Desktop Foundations exam and the largest section on it. The questions people get wrong are almost always about which field types accept an alias, which is section 2 below. 1. What

Onur Satici explains how Vortex, an open-source columnar file format under the Linux Foundation, revolutionizes high-throughput data loading. He details how cascading lightweight encodings, layout-based segment pruning, and zero-copy memory pipelines eliminate CPU/NVMe bottlenecks to stream S3 data straight to GPUs at speeds up to 60 Gbps without requiring upfront data reprocessing. By Onur Satici

This eMag examines how architects can lead with clarity in a rapidly evolving engineering world, distilling industry insights into field-tested practices for teams. Together, these stories reveal a core theme: the technology leader’s role is expanding from building systems to guiding how tech behaves and learns, while enabling engineers and organizations to bring out their best. By InfoQ

Turning Data Into Decisions Bar charts, histograms, scatter plots, subplots, and plotting straight from pandas Previously learned to draw a line — literally. we now know how to create a figure, style it, and save it. But real analyst work rarely stops at trends over time. You'll need to compare categories , understand distributions , spot relationships between variables , and show several views of the data at once . That's exactly what today covers. Grab a coffee — let's turn raw numbers into charts that actually tell a story. 1. Bar Charts: Comparing Categories When to use one Bar charts are your go-to whenever you're comparing discrete categories against each other — regions, products, departments, months. If someone asks "which one is bigger?", a bar chart answers it instantly. The code import matplotlib.pyplot as plt regions = [ " North " , " South " , " East " , " West " ] revenue = [ 420 , 380 , 510 , 290 ] fig , ax = plt . subplots ( figsize = ( 7 , 5 )) ax . bar ( regions , revenue , color = " teal " ) ax . set_title ( " Revenue by Region " ) ax . set_xlabel ( " Region " ) ax . set_ylabel ( " Revenue ($K) " ) plt . show () A useful variant: horizontal bars When category names are long, flip the chart with barh() — it's far easier to read than squeezing labels sideways: fig , ax = plt . subplots ( figsize = ( 7 , 5 )) ax . barh ( regions , revenue , color = " darkorange " ) ax . set_title ( " Revenue by Region " ) ax . set_xlabel ( " Revenue ($K) " ) plt . show () Rule of thumb: categories on the x-axis → bar() . Long labels or many categories → barh() . 2. Histograms: Understanding Distributions Bar chart vs. histogram — don't mix them up This trips up almost every beginner: a bar chart compares separate categories. A histogram shows how continuous numeric data is distributed by grouping values into ranges called bins . There are no gaps between histogram bars by convention, because the x-axis is continuous, not categorical. The code import matplotlib.pyplot

Every charting tool eventually gets the same feature request: "show me other times this stock looked like this." It sounds like a lookup. It is not. The retrieval is the easy half. The hard half is that a correct implementation can still produce results that are quietly meaningless, and nothing in the code will tell you. Here is the method, and the failure modes worth knowing before you ship it. No claims about predictive power anywhere in this piece — the last section explains why that is a deliberate choice, not a hedge. The naive version, and why it fails immediately The obvious first attempt: take the last 30 days of closing prices as a query vector, slide it across history, compute Euclidean distance, return the closest matches. import numpy as np def naive_search ( history , query , k = 5 ): m = len ( query ) windows = np . lib . stride_tricks . sliding_window_view ( history , m ) dists = np . linalg . norm ( windows - query , axis = 1 ) idx = np . argsort ( dists )[: k ] return idx , dists [ idx ] Run this and you get garbage — but instructively specific garbage. Every match comes from whatever period had a similar price level . Query a stock trading at $180 and you get back the other times it traded near $180. The shape is irrelevant to the metric; the offset dominates it. Scale is the same problem in a different coat. A stock that moved 2% over the window and one that moved 40% can trace an identical shape, and raw distance calls them unrelated. Normalize per window, not globally The fix is to z-normalize each window independently: def znorm ( x , axis =- 1 , eps = 1e-8 ): mu = x . mean ( axis = axis , keepdims = True ) sd = x . std ( axis = axis , keepdims = True ) return ( x - mu ) / ( sd + eps ) Per-window is the load-bearing part. Normalizing the whole series once preserves the relative offsets you were trying to remove. Each candidate window has to be centered and scaled on its own terms before it's compared. There's a satisfying identity waiting here.

Our dataset said the average loan was 2.3 million kroner. The number that actually mattered was 255,000. Both were correct. Only one of them was true. This is a writeup of three ways a dashboard can be arithmetically perfect and still lie, using real figures from an analysis of 1,000 Norwegian debt consolidation applications. If you build reporting for anyone, you have probably shipped at least one of these. 1. Summing a field that contains two different things A debt consolidation loan pays off your expensive credit card debt. It also, if you own property, rolls your existing mortgage into the same new loan. Same column in the database. Same loan_amount . Utterly different meaning. SELECT AVG ( loan_amount ) FROM applications ; -- 2,300,000 That query is right and the answer is useless. Of that 2.3 million, roughly 1.9 million is an existing mortgage being moved from one lender to another. The expensive debt, the part the customer actually has a problem with, averages 255,000 . So the headline figure overstates the thing you care about by a factor of nine. Nothing in the schema warns you. loan_amount is a number, AVG is a function, the result renders fine. The bug is that one column is holding two concepts and only a human who understands the domain will notice. -- what you actually wanted SELECT AVG ( unsecured_debt ) FROM applications ; -- 255,000 If a column can mean two things depending on another column, split it. Every time. 2. Reporting the mean when the distribution has a tail Income in this dataset runs from ordinary salaries up to about five million kroner. A handful of very high earners drag the mean upward: Mean income: ~635,000 Median income: 647,000 for homeowners, 550,000 for renters Look at what happens there. The mean sits between the two medians and describes neither group. Someone reading only the mean concludes the typical applicant earns 635,000. Nobody earns 635,000. It is an artefact. df . groupby ( ' housing ' )[ ' income ' ]. agg ([ ' mean

Have you ever wondered how your smartwatch actually knows you're stressed? Most of us treat the "Stress Score" on our wrists as a source of truth, but the logic remains hidden behind proprietary algorithms. Today, we are pulling back the curtain. We are going beyond basic heart rate tracking to perform PPG signal processing and HRV frequency domain analysis using Python. By the end of this guide, you’ll know how to ingest raw data via Bluetooth Low Energy (BLE) , apply digital filters with SciPy , and calculate the elusive LF/HF ratio to determine autonomic nervous system balance. If you are interested in advanced biometric algorithms or Python signal analysis , you’re in the right place. 🚀 The Architecture: From Photons to Stress Metrics Unlike standard heart rate (BPM), which just counts peaks, Stress Scores rely on Heart Rate Variability (HRV) —the millisecond-level variations between heartbeats. We'll be moving from raw light intensity data to a frequency-based stress index. graph TD A[Wearable Sensor / PPG] -->|Raw BLE Stream| B[Data Acquisition - Bleak] B --> C[Preprocessing - Bandpass Filter] C --> D[Peak Detection - Find R-R Intervals] D --> E[Cubic Spline Interpolation] E --> F[Fast Fourier Transform - FFT] F --> G[LF/HF Ratio Calculation] G --> H[Final Stress Score] 🛠 Prerequisites To follow this advanced tutorial, you’ll need: Hardware : A pulse oximeter or wearable that exposes raw PPG via BLE (e.g., Polar OH1, MAX30102 with an ESP32). Stack : NumPy & SciPy : For heavy-duty math and signal processing. Bleak : For cross-platform Bluetooth Low Energy communication. Matplotlib : To visualize the pulse waves. Step 1: Capturing the Raw PPG Stream (BLE) Photoplethysmography (PPG) works by shining green or red light into the skin and measuring the light absorption. First, let's grab that raw stream. import asyncio from bleak import BleakClient # UUID for the Raw PPG Characteristic (Device specific) PPG_CHAR_UUID = " 00002a37-0000-1000-8000-00805f9b34fb " def no

Hi Folks , In this article, I'm going to share my learnings in SQL . Install MySQL Workbench editor from **MySQL Downloads **found in internet. After Installation, Open MySQL Workbench Click on '+' icon to create new sql editor Click on import icon present on top in sql editor workbench After importing, See the data, schema tables present in left side panel We see the sql editor workspace, result Grid,output view. How to display data present in result table ? Use SELECT '*' as Keyword FROM table_name Ex: SELECT * FROM moviesdb.movies; Instead of moviesdb.movies we can use movies as name. Follow the steps below: Go to the left panel where movie data present, select movie data table -> Right click on default schema it highlights in bold -> Then, Write query as, SELECT * FROM movies Method - 2 USE moviesdb SELECT * FROM movies; Here we're using ' USE ' Keyword How can we check how many rows are present after writing query ? Just go to output view present in MySQL Workbench editor (present below to the result Grid) you can see there (or) Check with Excel sheet using filter option to check how many rows are present. Suppose we're writing query but we don't know recieved output is right or wrong. We can check with our excel sheet data using filter option if it matches with our output rows then our query is correct this is how we can debug and check ✅ output. There are some inbuilt clauses present in SQL 1.WHERE 2.COUNT 3.DISTINCT (for unique values) 4.LIKE 5.% - Wildcard Search Ex: %movies% Gives the movies data present anywhere in the sentence %movies - Returns movies data present starting at the sentence %movies - Returns of movies present at last in the sentence Ctrl + Scroll UP - Zoom in SQL Editor Database - Collection of rows and columns in tabular format with schema it has duplicate ids, unique ids in different formats like excel, .sql etc. How to save all written queries ? Write all queries at one place and highlight each query, do the next steps mentioned below,

Every backtest has to answer a boring question: when the strategy says "buy," what price does it actually get? Most backtesting frameworks answer this question badly by default, and the badness is almost always in the strategy's favor. Here are the four assumptions that do the most damage, roughly in order of how often they show up. Mid-price fills If your backtest fills orders at the midpoint of the bid-ask spread, you are assuming you trade for free. You don't. A market order pays at least half the spread to cross it; a marketable limit order pays something close to that too, once you're honest about how often it actually gets hit versus sitting unfilled while the market moves away. Mid-price fills are the single most common way a backtest manufactures edge that doesn't exist, because the effect compounds with trade frequency — a strategy that trades often looks great on mid-price fills and mediocre-to-negative once it pays the spread on every round trip. Zero slippage Slippage is the gap between the price your signal fired at and the price your order actually executed at, and it's not just a queuing artifact — it's partly information. If your strategy is buying because something changed, other participants are reacting to the same thing, and the price you wanted is often gone by the time your order reaches the book. A backtest with zero slippage is quietly assuming the market waits for you. Unlimited size at the touch Backtests routinely assume you can execute your full position size at the best bid or ask, no matter how large the order is relative to the visible size there. In practice, a large order walks the book, and the average fill price is worse than the touch price by an amount that depends on how thin the book is. This one is invisible until you try to size up, which is exactly when a strategy that looked fine in testing starts bleeding. Commissions omitted or averaged Commissions and fees are usually small per trade and therefore easy to skip or fold in

Cohere has launched Parse 5, a multimodal foundation model designed to extract structured data from complex enterprise documents. The 2.3-billion-parameter system converts visually rich PDFs into Markdown while providing bounding box coordinates for visual grounding. It has been evaluated against over 2,000 enterprise pages, achieving an average score of 79.2 in key performance areas. By Olimpiu Pop

Is a blank cell signal, or just missing? Sometimes an empty cell is the most informative thing in the row. The trouble is that you usually only know which case you're in by reading the data dictionary — and that doesn't scale to 800 columns named f_0347 . So we measure it instead, then check the answer against the literature. Ames housing · 1,460 sales · 79 columns · 19 of them contain blanks "Drop any column that's more than 70% missing." I've written that line into more pipelines than I can count. On Ames it deletes four columns — and three of them have real price signal sitting in the gap. The blanks in this dataset are structural . A blank GarageQual doesn't mean the value was lost; it means the house has no garage. A blank Alley means no alley access. The emptiness is the measurement. That's easy to see here because the columns have English names and a published data dictionary. It is not easy to see on a vendor feed of anonymised features, which is what most real projects look like. So the question worth answering isn't "does missingness carry signal" — it's can you tell, without knowing what the column means? 0.41 R² from the blank/not-blank pattern alone — every value discarded 1.00 AUC recovering the garage blanks from other columns' values ±0.9% Total spread across five strategies — inside a ±1.5% CV noise band 1 · A blank cell has a price tag Start with the crude check: does sale price differ between rows where a column is blank and rows where it isn't? Columns that go blank on the same rows describe one fact, so the five garage columns collapse into one. Fig 1. Median sale price, blank rows vs. valued rows. No garage is a $68k median discount on a $163k median house. Note the sign flip: houses that have an alley or fence are the cheaper ones — those features mark older, denser blocks. "Blank = worse" is not a rule you can assume. Then the harder test. Throw away every value in the table and keep only a 19-column matrix of True / False — was this cell emp

An identity theft search site claimed to have more than 150 million driver's license photos stolen from an ID verification service. The crime site has now shut down.

By Michael Nocito , data analyst · Published August 9, 2026 By the end of this page you can compute quartiles by hand, build the standard outlier fence from them, and run that fence over any column to get back a short list of rows worth looking at. On the sixteen orders below, one mistyped quantity gets flagged automatically while every honest large order stays inside the fence. Here is what to actually do today. On the column you care about most, get four numbers: the 25th percentile, the 75th, their difference, and 1.5 times that difference added to the 75th. Anything above that last number is a row to open and read. It is one query, and it turns "is this data clean" into a list of specific rows. The short version: a percentile is a value with a known share of the data below it. The interquartile range is the width of the middle half. Values more than one and a half of those widths beyond the middle half get flagged. The fence is easier to see than to read, so it gets the picture. The original carries a diagram here. In words: A horizontal line with a row of small filled dots along it, spaced unevenly and thinning out towards the right. A tall rectangle is drawn around the dots in the middle of the row, covering the central half of them, with a thick vertical bar inside it. The rectangle's left edge is labelled Q1, its right edge Q3, and the bar inside it median. From each edge of the rectangle a horizontal whisker line runs outward to a short vertical cap, reaching the furthest dot on that side that still lies within range. To the right of the right-hand cap stands a tall dashed vertical line labelled fence, drawn one and a half rectangle-widths beyond the rectangle's right edge, with a small double-headed measuring arrow underneath showing that distance against the rectangle's own width. One lone dot sits well to the right of that dashed line, drawn as a hollow ring instead of a filled dot, so it reads as picked out rather than belonging with the rest. Every oth