TRUE Coverage: How We Achieved 90% Faster CI by Measuring What Tests Actually Do

TL;DR: Use per-test coverage data to build a reverse map (file → tests that touch it). Git diff + map lookup = run only relevant tests. 43min → 4min CI time. The Problem Everyone Has Your test suite runs for 45 minutes. You change one file. Should you: Run all 2,000 tests? (Safe but slow) Run tests with matching filenames? (Fast but broken) Manually tag tests? (Gets stale immediately) We tried all three. They all failed. The Insight: Stop Guessing, Start Measuring What if we just measured what each test actually executes? Coverage tools have done this for years: { "src/MyComponent.js" : { "statements" : { "0" : 5 , "1" : 12 , "2" : 0 } } } Key insight: Coverage tools report per test run, not per test file. All we needed: save coverage after each individual test. The Architecture (5 Steps) Step 1: Instrument the Code // Build config (webpack/vite/rollup/etc) export default { plugins : [ process . env . COLLECT_COVERAGE && coveragePlugin ({ include : ' src/**/* ' , exclude : [ ' **/*.test.* ' ] }) ] } Works with: Istanbul (JS), coverage.py (Python), SimpleCov (Ruby), JaCoCo (Java) Step 2: Collect Coverage Per Test // Test framework afterEach hook afterEach ( async () => { const coverage = getCoverageData (); saveCoverage ( `coverage- ${ testName } .json` , coverage ); }); Step 3: Build the Reverse Map const map = {} ; for (const coverageFile of allCoverageFiles()) { const testName = extractTestName(coverageFile); const coverage = JSON.parse(read(coverageFile)); map [ testName ] = [] ; for (const [ sourceFile , data ] of Object.entries(coverage)) { if (wasExecuted(data)) { map [ testName ] .push(sourceFile); } } } Example output: { "tests/user-profile.test.js" : [ "src/components/Profile.jsx" , "src/components/Card.jsx" , "src/utils/formatters.js" ] } The magic: The test told us what it executed. No parsing, no guessing. Step 4: Detect Tests CHANGED = $( git diff origin/main...HEAD --name-only ) TESTS = $( lookupTestsInMap " $CHANGED " ) npm test $TESTS Step 5: Auto-Up