ClickHouse Merge Queue: Why It Grows & How to Fix It

A growing merge queue means ClickHouse is creating parts faster than it can merge them. Left unchecked, this leads to the dreaded "too many parts" error that blocks inserts. Monitor system.merges and alert when queue depth exceeds 50 per table.

Monitoring the Merge Queue

-- Current active merges
SELECT database, table, elapsed, progress,
    result_part_name, source_part_names
FROM system.merges
ORDER BY elapsed DESC;
 
-- Part count per table (leading indicator)
SELECT database, table, count() AS part_count
FROM system.parts
WHERE active = 1
GROUP BY database, table
HAVING count() > 50
ORDER BY part_count DESC;

What Causes Merge Queue Growth?

High insert rate — Inserts create new parts. If your insert rate exceeds ClickHouse's merge throughput, the queue grows.

Large parts — Very large merges take a long time, blocking the queue.

Limited background pool — The background_pool_size setting controls concurrent merge threads. Default is 16, but high-throughput clusters may need more.

Disk I/O saturation — Merges are I/O heavy. A saturated disk slows merge throughput.

Fixing a Deep Merge Queue

-- Force an immediate manual merge
OPTIMIZE TABLE database.table_name FINAL;
 
-- Check background_pool_size
SELECT name, value FROM system.settings
WHERE name = 'background_pool_size';
 
-- See merge progress
SELECT database, table, round(progress * 100, 1) AS pct,
    formatReadableSize(total_size_bytes_compressed) AS size
FROM system.merges
ORDER BY progress;

Prevention

Use batch inserts instead of many small inserts. Each INSERT statement creates at least one part — batching 1000+ rows per insert reduces part creation rate dramatically.

Clustersight monitors merge queue depth and part count per table with automatic alerts before "too many parts" errors occur.

Read more: How to Monitor ClickHouse in Production