标签：#data

By Michael Nocito , data analyst · Published August 9, 2026 By the end of this page you can say what a p-value measures in one sentence, compute one by hand with no distribution theory at all, and name the four things people routinely claim a p-value says that it does not. The worked example is nine real orders where two regions differ by 168.50, and the answer comes out of counting rather than out of a table. Here is what to actually do today. Any time you are about to report that two groups differ, write down the two group sizes first. If either is under about twenty, a p-value will almost certainly come back large no matter how real the difference is, and the honest report is the difference, the sizes, and an interval, not a verdict. The short version: a p-value is the share of results at least as extreme as yours that you would get if the thing you are testing had no effect at all. Small means your result would be unusual under nothing-happening. It does not mean the effect is large, and it does not mean the effect is real. That definition is doing a lot of work in one sentence, so it gets the picture. The original carries a diagram here. In words: A histogram built from small dots, one dot per outcome, arranged in ten vertical columns of different heights standing on a horizontal baseline. The columns rise from one dot at the far left to a peak of twenty-seven dots just right of centre, then fall away to a single dot at the far right, giving the whole shape a rounded hump centred slightly left of the middle of the picture. Two vertical dashed lines cut down through the shape, one on the left of the hump and one on the right, placed symmetrically about the hump's centre. The dots lying in the two tails beyond those lines are drawn in a darker, warmer shade, and the dots in the bulk between them are drawn in a lighter blue, so the tails stand out from the middle. In the two columns the dashed lines pass through, the darker dots are stacked at the bottom of the co

By Michael Nocito , data analyst · Published August 9, 2026 By the end of this page you can build a moving average by hand, choose a window length on purpose rather than by habit, explain the lag it introduces, and spot the one data problem that silently corrupts every moving average you will ever compute. On the sixteen weeks below, the raw series wobbles with a standard deviation of 237. A three-week average brings that to 101, and a five-week to 62. Here is what to actually do today. Add a moving average to your busiest weekly or daily chart, using a window that matches your cycle: 7 for daily data with a weekday pattern, 4 or 13 for weekly, 12 for monthly. Then plot it on top of the raw series rather than instead of it, so the reader can see both the noise and the trend. The short version: a moving average replaces each point with the average of it and the points around it. Wobble cancels out, trend survives, and the price is that the smoothed line reacts late. The effect is easier to see than to describe, so it gets the picture. The original carries a diagram here. In words: A line chart with two lines drawn on the same axes over sixteen evenly spaced points. The first line is thin and violently jagged, swinging from near the top of the chart down to near the bottom and back again several times, with one especially deep trough about a quarter of the way along and one especially tall spike just past the middle. The second line is thicker and starts two points later than the first. It stays in a narrow band through the middle of the chart, rising and falling only gently, and never comes close to either the peaks or the troughs of the jagged line. Where the jagged line spikes to its highest point, the thick line rises only slightly and does so one point later than the spike. The word actual labels the jagged line near its left end, and the phrase three-week average labels the thick line. Every number on this page is real. The same sixteen orders used across these

By Michael Nocito , data analyst · Published August 9, 2026 By the end of this page you can compute both averages by hand, say in one sentence which one your question needs, and show what each does when one number in your data is wrong. On the sixteen orders below, a single mistyped quantity moves the mean by 337.50 and the median by 30. That gap is the whole reason both words exist. Here is what to actually do today. Next to every average you report, put the median beside it and look at the two numbers together. If they are close, report the mean and move on. If they are far apart, you have found something worth saying out loud, and this page is about what. The short version: the mean shares the total out equally, so every value pulls on it. The median is the middle value once you sort, so only the position of a value matters, not its size. That difference is easiest to see rather than read, so it gets the picture. The original carries a diagram here. In words: Two horizontal number lines, one above the other, drawn on the same scale. On the top line, sixteen small filled dots sit in a loose cluster across the left and middle of the line. Two markers sit almost on top of each other just past the centre of that cluster: a solid triangle pointing up, labelled mean, and a short vertical bar, labelled median. On the bottom line the same sixteen dots appear, except one dot from the middle of the cluster has moved far to the right and off the end of the line, shown by an arrow leaving the line through a small break mark at the right edge. The median bar on the bottom line has barely shifted from where it was on the top line, moving only a hair to the right. The mean triangle on the bottom line has slid a long way to the right, roughly ten times further than the median moved, and a horizontal dotted guide connects its old position on the top line to its new position on the bottom line to show the size of the slide. Every number on this page is real. One sixteen-row order

Why “Behind the Scenes” Isn’t Optional in Machine Learning There’s a version of Machine Learning that treats models as interchangeable black boxes: Import the library. Call .fit() . Tune a few hyperparameters. Ship it. It works — until it doesn’t. When a model fails, the real question isn’t just “How do I fix it?” It’s “Why is it failing in the first place?” The engineers who can answer that question are the ones who understand what’s happening beneath the API. Not because knowing the internals is intellectually satisfying — although it is — but because that understanding is what separates applying a model from engineering one . Knowing that Gradient Descent exists is not the same as understanding why it becomes preferable to closed-form solutions as problems scale. Knowing that Softmax outputs sum to one is not the same as understanding why that alone doesn’t make its outputs true probabilities. Knowing what a technique does is useful. Understanding why it works, when it works, and what assumptions it depends on is what allows you to adapt it when the problem doesn’t look like the textbook example. And you can’t meaningfully customize something you don’t understand. That’s the idea behind #MLUnboxed — a series where I’ll break down Machine Learning and Statistics concepts, including many that are already well documented, but without stripping away the mechanics that make them work. The goal isn’t to reinvent established ideas. It’s to understand them deeply enough to use them intelligently. This series is for people who already know the “what” and are ready to spend more time with the “why.” If that sounds like you, let’s get into it.

Preventing Cache Penetration in Spring Boot Using Redis and Bloom Filters Cache penetration occurs when high-frequency requests query non-existent keys, bypassing the Redis cache completely and hitting the relational database directly. Here is how we set up a Bloom Filter guard layer in front of Redis and PostgreSQL. 1. The Bloom Filter Guard Concept A Bloom Filter is a space-efficient probabilistic data structure that tests whether an element is definitely NOT in a set or MIGHT be in a set. @Component public class CachePenetrationGuard { private final BloomFilter < String > accountFilter ; public CachePenetrationGuard () { // Expected insertions: 500,000, False positive probability: 0.01 (1%) this . accountFilter = BloomFilter . create ( Funnels . stringFunnel ( StandardCharsets . UTF_8 ), 500000 , 0.01 ); } public void registerKey ( String accountId ) { accountFilter . put ( accountId ); } public boolean mightContain ( String accountId ) { return accountFilter . mightContain ( accountId ); } } 2. Service Layer Verification Before querying Redis or PostgreSQL, verify with the Bloom Filter: @Service public class AccountService { private final CachePenetrationGuard guard ; private final RedisTemplate < String , AccountDto > redisTemplate ; private final AccountRepository repository ; public AccountDto getAccount ( String accountId ) { // Step 1: Bloom filter pre-check if (! guard . mightContain ( accountId )) { return null ; // Instant rejection, saves DB from unnecessary lookups } // Step 2: Redis lookup AccountDto cached = redisTemplate . opsForValue (). get ( "acc:" + accountId ); if ( cached != null ) return cached ; // Step 3: DB fetch and cache populate AccountDto dbResult = repository . findByAccountId ( accountId ); if ( dbResult != null ) { redisTemplate . opsForValue (). set ( "acc:" + accountId , dbResult , Duration . ofMinutes ( 30 )); } return dbResult ; } } 3. Summary Combining Bloom Filters with TTL jitter in Redis shields backend databases from cache

Most data engineers write pipelines the way most people write shell scripts: run it, eyeball the output, ship it. That works right up until a schema changes upstream, a null slips through a join, or someone "fixes" a transformation and silently breaks three downstream tables. By then the bug isn't your problem anymore — it's a bad number in someone's dashboard. Software engineers solved this problem decades ago with automated testing. Data engineering has been slower to adopt the habit, partly because our code touches messy external reality (files, databases, clusters) in a way a typical web app doesn't. But that's exactly why testing matters more here, not less. This article is a practical, DE-flavored crash course in pytest — the dominant Python testing framework — plus the patterns you actually need for pandas, Polars, and PySpark pipelines. Why bother testing a data pipeline? A few concrete failure modes that tests catch before production does: A column gets renamed upstream and your join silently produces all-null matches instead of erroring. A "cleaning" function that's supposed to drop duplicates accidentally drops valid rows too. A date-parsing function works on your local machine's locale and breaks in the CI environment. A refactor changes an aggregation from sum to mean and nobody notices until finance asks why revenue looks 90% smaller. None of these require exotic testing techniques. They require the habit of writing small, deterministic checks against small, deterministic inputs — which is exactly what pytest is built for. Where pytest fits — and where it doesn't Before diving in, it's worth being precise about scope, because "testing a data pipeline" actually covers two different questions, and conflating them is a common source of confusion: Is my code correct? Given a known input, does the transformation logic produce the right output? This is a property of your code , and it doesn't change based on what day it is or what a source system decided to

A new identity theft service launched on the dark web this week is selling digital scans of more than 153 million drivers licenses from people in the United States and Canada. Based on interviews with individuals whose licenses are available for purchase on this service, it appears to be siphoning images collected by a widely-used identity verification company based in Louisiana. KrebsOnSecurity also has learned that the New Orleans field office of the Federal Bureau of Investigation (FBI) today launched an official inquiry into the source of the images.

Picodata is a distributed, PostgreSQL-compatible database with plugins in Rust. Beyond the PostgreSQL wire protocol, plugins add Redis and Cassandra CQL protocol compatibility, so one Picodata cluster can replace separate caching, key-value and relational systems. It is open source and self-hosted. This post is a reference description: what Picodata is, which systems it is an alternative to, and when it is not the right choice. Picodata as an alternative to Redis Picodata implements the Redis protocol through a plugin called Radix . Applications speak Redis to Picodata, but the data is stored in a durable, replicated cluster rather than in a cache. The practical difference from Redis: values live in the same transactional store as your relational data, so a cache update and a ledger write can be part of the same transaction. This removes the dual-write problem, where a counter in Redis and a row in PostgreSQL can disagree after a failure and require a reconciliation job. Durability is WAL-based rather than best-effort. Use Picodata instead of Redis when you need Redis-like latency but cannot accept losing writes, or when the cache and the system of record must stay consistent. Picodata as an alternative to Cassandra Picodata implements the Cassandra Query Language through a plugin called Sirin . Applications issue CQL against Picodata. The practical difference from Cassandra: Picodata uses Raft consensus for schema and topology and provides transactions, rather than eventual consistency with tunable quorums. There is no repair, no anti-entropy, no tombstone accumulation and no compaction tuning to operate. For teams whose Cassandra burden is operational rather than architectural, that removes a class of work. Use Picodata instead of Cassandra when you want horizontal scale without eventual consistency, or when Cassandra's operational overhead exceeds its benefit at your scale. Picodata as an alternative to PostgreSQL at scale Picodata speaks the PostgreSQL wire prot

Disclaimer: This is a side project, not a production story. The slow-query problem is real, but the database is synthetic data I generated to make it show up on demand. I have no connection to PgCache. Everything here is in a repo you can clone and run. I tested version 0.6.2. A handful of dashboard queries on one of my projects were fine for a year and then weren't: count users by tier, revenue grouped by country, best-selling products per category. Nothing exotic, just aggregates and joins over tables that had gotten big. The usual fixes didn't sit right with me. A materialized view means picking a refresh interval and serving slightly stale numbers in between. Redis in front of Postgres means writing and maintaining code that knows which cache entries to throw away on every write. A read replica just runs the same slow query on another machine. PgCache offers a different trade. It's a proxy that talks the Postgres wire protocol, so your app connects to it as if it were the database. It caches reads. And instead of expiring entries on a timer, it follows Postgres's replication stream and refreshes a cached result when the rows behind it change. That stream is the same feed Postgres uses to copy data to a standby server , a running log of every insert, update, and delete. The "no timers, no manual invalidation" part is the interesting claim. Here's how it held up. A database big enough to be slow First I needed a database where "slow" was real and not a rounding error. I wrote a seed script for a small e-commerce schema and filled it to about 16 million rows: Table Rows Notes users 1,000,000 10 countries; tiers 50% free / 33% pro / 17% enterprise products 2,000 10 categories orders 5,000,000 four statuses, random totals, spread over two years order_items 10,000,000 about two per order I added indexes on every foreign key and on every column the test queries filter or group by. That was on purpose. I wanted to compare PgCache against a Postgres that had been tuned p

What Rebuilding Kafka From Scratch Actually Teaches You There's a gap between using a system and understanding it. Most engineers never close that gap, and honestly, most of the time that's fine. Kafka works. Topics, producers, consumers, pull the levers, ship the data. Done. But then you hit a weird latency spike, or a consumer group stalls in a way that doesn't match the docs, or replication starts behaving like it has feelings. And suddenly "I know the terminology" doesn't cut it anymore. That's exactly why this rebuild post is worth your time. The Abstraction Tax Every framework you use charges you an abstraction tax. The tax isn't the dependency. It's the mental model debt you carry when something goes wrong and you don't know what layer to blame. Kafka's tax is particularly sneaky because its concepts sound simple: topics are channels, partitions are buckets, offsets are counters. You can get productive fast. And then that simplicity starts lying to you. Why does lag spike when throughput looks fine? Why does adding consumers past the partition count do nothing? Why does a rebalance tank your throughput for 30 seconds? These aren't Kafka quirks. They're direct consequences of how the log is actually structured, consequences that become obvious the second you implement it yourself. What the Rebuild Exposes When you write the log append yourself, the offset model stops being abstract. An offset isn't just a cursor, it's a byte position in a segment file. Consumers aren't "reading from a partition," they're replaying a structured log from a known position. Replication isn't a background checkbox, it's a follower explicitly fetching and acknowledging write positions. A few things that tend to click when you go through this kind of exercise: Segment files and retention , Kafka doesn't delete old messages by scanning. It deletes whole segment files once they're past the retention boundary. If you've ever been surprised by how Kafka handles disk, this is why. Why par

What building a warehouse management system taught me about the data operational software leaves behind — and the engineering it takes to make that data trustworthy. The second that outlives itself A picker holds a handheld scanner, points it at a carton, and pulls the trigger. There's a beep. They type 10, confirm, and move to the next location. The whole thing takes about a second. For a long time I thought of my job as making that second work. I built the screen, the endpoint behind it, the repository behind that. My definition of done was that the user completed the workflow, the API returned success, and the right rows landed in the database. What changed my thinking was noticing what was still there afterwards. The screen closes, the session ends, the app ships a new version, the picker changes jobs, the device is replaced. The row stays — and the row isn't a record of a UI interaction. It's a durable claim about the physical world: at this time, this person, on this device, ten units of this product moved. The application is the instrument. The data is the measurement. A measurement is only ever worth what the instrument's precision allows. This article is about the gap between those two definitions of done, and the specific decisions — retry semantics, timestamps, identity, status codes, conflict resolution — that determine which side of it you land on. Almost all of them get made by application developers, inside feature work, long before anyone tries to analyze anything. What warehouse owners actually do with this data now Worth being concrete about the stakes first, because "data quality matters" is the kind of statement everyone agrees with and nobody acts on. What's changed isn't that owners suddenly became analytical. It's that operational systems started producing enough granular, attributed, time-stamped movement data that previously unanswerable questions became answerable. Inventory accuracy is a working-capital decision. Stock you can't trust is s

A raw sold-listings search is not automatically a usable comp set. Search for a phone and you may also get cases, chargers, broken screens, empty boxes, and nearby models. Search for a camera lens and you may get caps, adapters, or a different focal length. If those rows go directly into a median, the result can describe the search noise instead of the product. I wanted a larger measurement than a single convenient example, so I ran a fixed 100-product study through CompSniper, the sold-price API I own. The goal was not to prove that an automated classifier is always correct. The goal was narrower: Measure what the production relevance cleaner removed and how the product-level median changed on one predeclared sample. The protocol I selected the products before making the first request: 20 smartphones and tablets 20 gaming and computing products 20 cameras and lenses 20 audio and music products 20 collectibles and luxury products Every search used the same settings: Marketplace: ebay.com Sold window: 2026-06-02 through 2026-08-31 Page: 1 Requested rows: 240 Sort: ended recently Condition: any Relevance cleaning: enabled Each relevance-enabled response contained the raw sample count and raw median captured before classification, followed by the cleaned rows and deterministic price summary from the same fetched page. That meant one production request per product, not separate raw and cleaned fetches. All 100 requests succeeded with unique request IDs. The headline results Across the study: 19,220 priced raw rows were parsed 11,942 priced rows remained after cleaning 7,278 rows were classified out The weighted removal rate was 37.87% 34 of 100 product medians changed by at least 10% 15 of 100 changed by at least 25% 11 of 100 changed by at least 50% The direction was not always upward: 73 medians increased 21 medians decreased 6 medians stayed unchanged That is important. The cleaner is not instructed to raise prices. It tries to retain listings for the requested produ

The company, which distributes medicines and medical devices to hospitals and healthcare practices across the U.S., said it was hacked and expects intermittent service degradation.

Adding an index stops working eventually. Here's what we did when a nationwide logistics platform's core table crossed a billion rows — and the parts nobody warns you about. There's a specific moment in a backend engineer's life when the usual advice stops working. A query gets slow. You check the execution plan, you add an index, it gets fast again. This works for years. It works so reliably that it starts to feel like a law of nature. Then one day you add the index and nothing happens. Or worse — the index takes six hours to build, locks the table while it does, and the query is still slow at the end of it. That's roughly where we were on a nationwide logistics platform processing tens of thousands of orders a day. The tracking events table — one row per scan, per parcel, per status change — had crossed a billion rows. Every parcel generated a dozen or more events on its journey. The table only ever grew. This is what we did about it, and more usefully, what nobody told us beforehand. First: are you sure you need this? Partitioning is not a performance trick you reach for when a query feels sluggish. It carries real operational cost, and most tables that people want to partition should just be indexed properly. Some honest signals that you're actually at the boundary: Your indexes no longer fit comfortably in memory, so index reads hit disk Index maintenance — REINDEX, VACUUM, ANALYZE — takes so long you can't schedule it Deleting old data is impossible in practice, because a DELETE of a hundred million rows will destroy your write throughput for hours Your queries almost always filter on a single obvious dimension, usually time That last one matters more than the rest. Partitioning only helps if your access pattern lines up with how you split the data. If your queries hit every partition anyway, you have added complexity and gained nothing. For us the alignment was clean: nearly every query on the events table was scoped to a date range. Operations dashboards loo

Nvidia invests $3.5 billion into Taiwanese chipmaker MediaTek. The deal shows how Nvidia plans to stay essential to AI infrastructure as Big Tech begins to build its own AI chips.

Dealing with multiple Excel or CSV files is a common task in business. Whether it is sales reports from different regions, customer data across various campaigns, or financial records by month, the need to combine these files into a single, cohesive dataset is constant. Manually copying and pasting or even using complex formulas can quickly become a time sink, prone to errors, and a source of frustration. What if you could automate this repetitive process? Imagine setting up a workflow that automatically merges your Excel files for you. That is where Microsoft Power Automate comes in. And when your data is messy, inconsistent, or riddled with duplicates, AI tools can take your automation to the next level. This guide will walk you through building robust workflows in Power Automate to combine your Excel workbooks. We will also explore how AI can address the often overlooked challenge of data cleaning and standardization, turning disparate data into a clean, unified source. Why Automate Excel Merges? The benefits of automating Excel data consolidation extend beyond simply saving time. Consider these advantages: Time Savings: Free up hours spent on manual data handling, allowing you to focus on analysis and strategic tasks. Reduced Errors: Eliminate human error from copy-pasting, formula mistakes, or missing data. Consistency: Ensure data is merged and formatted uniformly every time, regardless of who runs the process. Scalability: Easily handle increasing volumes of files without proportional increases in manual effort. Timeliness: Get up-to-date consolidated reports faster, enabling quicker decision-making. The Old Way: Manual Merges and VBA Limitations For years, consolidating data meant either painstaking manual copy-pasting, using VLOOKUP or INDEX/MATCH across sheets, or resorting to VBA (Visual Basic for Applications) scripts. Manual methods are slow and error-prone, especially with large datasets or many files. VBA offered a significant improvement, providing c

I had a PySpark job joining a large transactions table with a customer dimension table. Nothing exotic — a standard join, then an aggregation. On paper, it looked like it should scale fine across the cluster. In practice, the job would race through most of its tasks and then stall. The Spark UI told the real story: almost every task finished in a few minutes, but one or two tasks ran for over 40 minutes while their executors sat at 80–90% CPU, and the rest of the cluster sat mostly idle waiting for them to finish. This post walks through what data skew actually is, how I confirmed it was the cause, and the fix that brought the job back under control. What data skew actually is Data skew happens when one or a few keys hold a disproportionate share of the data. When Spark distributes work across partitions — usually via hash partitioning on a join or group-by key — all the rows for a given key land in the same partition. If one key has millions of rows and most others have a few thousand, that one partition (and the single task processing it) ends up doing far more work than every other partition combined. The result is a job where 95% of tasks look completely healthy, and the remaining 5% become the actual bottleneck. Total job time is dictated by the slowest task, not the average one — so a single skewed key can dominate your entire runtime even if it represents a tiny fraction of your total row count. How I confirmed it The Spark UI's stage view was the first clue: a chart of task durations with almost every bar clustered together, and one or two bars stretching far beyond the rest. That pattern — uniform short tasks plus one long outlier — is close to a signature for skew. To confirm which key was responsible, I ran a simple aggregation on the join key before doing anything else: from pyspark.sql import functions as F df . groupBy ( " customer_id " ) \ . count () \ . orderBy ( F . desc ( " count " )) \ . show ( 20 ) The output made it obvious: a small number of cu

If you've done LeetCode's Design HashMap , you've implemented put , get , and remove . What that exercise usually skips is the part that actually breaks in production: what happens when someone mutates the map while another piece of code is iterating over it. I ran into this directly while building MyHashMap , a from-scratch single-threaded HashMap (separate chaining, resize on load factor). Getting put / get / remove right was the easy 80%. Getting entrySet().iterator() to correctly detect concurrent mutation — including the case where a second, completely separate iterator is the one that should notice — took three wrong turns before landing on the pattern the JDK actually uses. The problem, concretely Iterator < Entry < K , V >> it = map . entrySet (). iterator (); it . next (); map . put ( someNewKey , someValue ); // structural change, mid-iteration it . next (); // ??? — undefined behavior if we don't guard against this Without a guard, next() might return a stale entry, skip entries entirely, or throw an unrelated exception depending on internal bucket-array state. Java's real collections handle this with ConcurrentModificationException (CME) — but the interesting part isn't the exception, it's the mechanism that detects when to throw it. First idea: a boolean "dirty" flag Obvious first attempt: a boolean modified field on the map, flipped to true on any put / remove , checked by the iterator. This works for exactly one iterator. It falls apart the moment two iterators are alive at once: Iterator A calls next() , sees modified == false , proceeds. Something else mutates the map. modified flips to true . Iterator B — created after that mutation — checks the same shared modified flag, sees true , and incorrectly throws, even though nothing has changed since B was created. A single shared boolean can't represent "changed since this specific iterator was created" for more than one iterator at a time. Resetting it on read doesn't help either — now the other iterat

Every lottery player knows the saying: "One line is no line, you have to play a few more to boost your chances!" That's why the average player often fills out a complete ticket with 8 lines. Sounds like a solid strategy, right? Wrong. It’s actually the fastest way to systematically burn through your cash. To prove it, we wrote a Python simulation that uses historical payout data and cold, hard combinatorics to see who actually has any money left in their account at the end. What exactly is a Monte Carlo simulation? Named after the famous casino in Monaco, the Monte Carlo simulation is basically the brute-force approach to probability theory. Usually, mathematicians use a single, elegant formula to calculate the expected value. That formula will dryly inform you: "You lose an average of 1 Euro per Eurojackpot line." That might be mathematically correct, but emotionally, it's a bit of a snooze. It doesn't capture the true pain of slowly bleeding out financially. The Monte Carlo simulation throws that elegant formula right out the window. Its core concept is pure, raw computing power. Instead of just calculating the theoretical outcome, we let the computer simply play through reality thousands of times. It’s not an equation; it’s a simulation and iteration of real events. The computer spawns 1,000 fictional players. For every player and every draw over the last 10 years, it generates a random number based on the actual Eurojackpot probabilities. It simulates the real-world winning and (mostly) losing, step by step. At the end, we aren't looking at some abstract theoretical number, but at the very real, blood-red bank accounts of 1,000 ruined clones. The Setup We're using Polars for lightning-fast data processing, NumPy to simulate millions of random draws, and Matplotlib to visualize our financial doom. First, grab our historical Eurojackpot database and drop it into the same folder. The Script Here is the complete Python code. It calculates the exact mathematical odds

As I have started my Data analysis and science curriculum, Microsoft excel is the first tool we have learnt. In my life before getting into this course, I used to underestimate the power of Microsoft excel. Definition of Microsoft Excel This is a spreadsheet software developed by Microsoft that allows you to collect, organize, analyze, calculate and visualize data efficiently. The Excel Interface When open Microsoft excel, you first an interface made of various tools We start with the ribbon. The ribbon contains a toolbar that contains all the commands organized into tabs such, Home, Insert, Page Layout , Formulas, Charts and Data There is a formula Bar that content and formulas appear. In addition, we have Quick access toolbar that contains icons for Save, Undo and Redo. A row is a horizontal line of cells that runs from left to right across the grid while a column is a vertical line of cells that runs from top to bottom down the grid. A cell is where row and column intersect and form a single box. Cleaning A Dirty Dataset As we learnt about Data cleaning with Excel on dataset that was a bit very disorganized. We first started with sorting that data which basically means arranging the data in specific order. The sorting can be Text sorting which is A-Z or Z-A, Number sorting which is from largest to small and vice versa, Date sorting which is Oldest to Newest and vice vera. Next was filtering which allows you display data that meet certain criteria and hide the rest temporarily. Next, Number formatting changes how numbers display without altering their actual value. Conditional Formatting highlights cells automatically based on rules or criteria, helping spot trends or outliers. The dataset we were cleaning had a lot of duplicates, so it was important for us to remove the duplicates. Text functions for Cleaning Data Text functions are used to remove extra spaces , standardize text ,extract parts of strings or find and replace sub strings. text functions include UPP