今日已更新 240 条资讯 | 累计 23823 条内容
关于我们

My idle ClickHouse was merging 11 million rows every 30 seconds

OtezVikentiy 2026年07月25日 05:12 0 次阅读 来源:Dev.to

I run a small self-hosted observability tool on the cheapest VPS I could find on purpose: 2 cores, 2 GB RAM, 20 GB SATA SSD . It ingests errors, traces and metrics from two low-traffic sites of mine. The stack is three containers — a Go app, PostgreSQL, and ClickHouse. One evening docker stats showed ClickHouse sitting on 880 MB of its 1 GB limit and the box swapping, with basically zero events coming in. So I went looking for where the memory and disk had gone. The answer turned out to be a good lesson in how a database can spend almost all of its I/O talking to itself. 543 KB of my data, 579 MB of ClickHouse talking about ClickHouse First thing I checked: how much data had my app actually stored versus how much ClickHouse had stored about itself . My application database: 543 KB, 16k rows The system database: 579 MB, 46.3M rows Roughly a thousand to one. Disk was 12 GB used out of 20 — on a tool that had recorded half a megabyte of real telemetry. The culprit was ClickHouse's own system logs, several of which have no TTL by default and therefore grow forever: trace_log — 404 MB, 26M rows (the query profiler writes here; it's on by default, sampling once per second) asynchronous_metric_log — 16.6M rows text_log — 132 MB plus query_log , latency_log Only metric_log , processors_profile_log and part_log ship with a TTL. Everything else just accumulates. Then I looked at the insert rate over 30 seconds: trace_log — 227 rows/s asynchronous_metric_log — 157 rows/s text_log — 44 rows/s my application — about 5 rows/s 98.8% of all inserts were ClickHouse narrating its own internals. The part that's expensive beyond disk Here's the number that made me stop. Over the same 30 seconds: rows inserted : 16,222 rows merged : 11,007,643 That's a 1 : 678 ratio. For every row written, the engine rewrote 678 already-sitting rows. The mechanics: MergeTree drops every insert into its own data part, then merges parts into bigger ones so reads stay fast. When the table is small this is

本文内容来源于互联网,版权归原作者所有
查看原文