Building an Enterprise Football Data Pipeline: Decoding Flashscore's Protocol for xG & Referee Analytics

Most football data scrapers on the market only extract high-level final scores (e.g. 2-1 ). But quantitative sports analysts, data scientists, and predictive betting modelers need granular data: Expected Goals (xG) , Official Referee Assignments , Goal Scorers paired with Assist Providers , and Half-Time vs Full-Time (1H/2H) statistical breakdowns . When I set out to build a professional-grade Flashscore scraper on Apify, I ran into two major engineering challenges: The Memory Problem : Keeping Puppeteer running to scrape hundreds of historical matches consumes over 1.5GB of RAM per run. The Protocol Problem : Flashscore serves its deep statistical feeds using a proprietary pipe-delimited data format ( ~ , ¬ , ÷ ) over CDN endpoints, rather than standard REST APIs. In this tutorial, I'll explain how I engineered the Flashscore Elite Statistics Extractor , how the hybrid Browser + HTTP/2 streaming pipeline drops RAM footprint from 1.5GB to 70MB , how to parse Flashscore's custom feed protocol, and how to pipe the resulting datasets directly into Python and Pandas. 🏛️ The Hybrid Pipeline Architecture To achieve zero proxy reliance for standard runs and ultra-low compute costs, the Actor splits execution into a 2-Phase Hybrid Pipeline : [ League & Season Selection ] │ ▼ ┌───────────────────────────────────────────┐ │ Phase 1: Browser Handshake (Puppeteer) │ │ - Captures x-fsign security tokens │ │ - Extracts countryId & tourId │ └─────────────────────┬─────────────────────┘ │ [ Immediate Browser Shutdown ] (RAM drops from 1.2GB -> 70MB) │ ▼ ┌───────────────────────────────────────────┐ │ Phase 2: Parallel HTTP/2 Feed Workers │ │ - got-scraping with JA3 TLS matching │ │ - Decodes df_st_1_ (Stats) & df_sui_1_ │ └─────────────────────┬─────────────────────┘ │ ▼ ┌───────────────────────────────────────────┐ │ Self-Healing Recovery Pass │ │ - Auto-retries skipped/failed matches │ └─────────────────────┬─────────────────────┘ │ ▼ ┌───────────────────────────────────────────┐