Cassandra Compaction Strategies: STCS vs LCS vs TWCS vs UCS

Why architecture matters here

Compaction strategy choice matters because it's one of the highest-impact per-table tuning decisions in Cassandra -- the difference between a workload performing well and badly often comes down to matching the compaction strategy to the workload. Cassandra's LSM structure (accumulating SSTables) requires compaction (merging them), and the strategy determines how -- occupying very different points in the read/write/space tradeoff. A well-matched strategy (STCS for write-heavy, LCS for read-heavy, TWCS for time-series) makes the workload perform well (the right amplification tradeoff for the workload); a mismatched strategy (e.g., STCS for read-heavy -- high read amplification; STCS for time-series TTL -- tombstone accumulation) makes it perform badly (the wrong tradeoff -- slow reads, or tombstone problems). And because it's a per-table choice, each table can be tuned for its workload. So understanding the strategies (their tradeoffs, their fit) and choosing correctly is one of the most impactful Cassandra tuning decisions, and getting it wrong is a common, significant performance problem. For operating Cassandra, understanding compaction strategies is essential to good performance.

The read/write/space tradeoff is the analytical framework, and each strategy is a different point in it. The fundamental LSM tradeoff (read vs write vs space amplification) is navigated differently by each strategy. STCS optimizes write amplification (merging similar-sized SSTables -- each piece of data rewritten relatively few times -- low write amplification) at the cost of read amplification (a row's data can be in many size-tiers -- a read checking many SSTables -- high read amplification) and space (large SSTables of partially-obsolete data linger until they have similar-sized peers to compact with -- space amplification). So STCS is write-optimized (low write amplification -- good for write-heavy) but read- and space-costly. LCS optimizes read amplification (organizing SSTables into levels of non-overlapping SSTables -- so a row is in at most one SSTable per level, and there are few levels -- a read checking few SSTables -- low read amplification) and space (the leveled structure reclaiming obsolete data efficiently -- low space amplification) at the cost of write amplification (maintaining the leveled, non-overlapping structure requires more rewriting -- high write amplification). So LCS is read-optimized (low read amplification -- good for read-heavy) but write-costly. TWCS optimizes for time-series with TTL (grouping SSTables into time windows and compacting within a window -- so each window's data, sharing a time range, expires together and the whole window's SSTable can be dropped when expired -- efficient TTL expiry) -- avoiding the tombstone accumulation that STCS/LCS hit with time-series TTL. So each strategy is a different point in the read/write/space tradeoff (STCS write-optimized, LCS read-optimized, TWCS time-series-optimized), and understanding this framework (each strategy's tradeoff) is understanding how to choose.

And the tombstone-and-time-series reality is a crucial, often-painful consideration, especially motivating TWCS. In Cassandra, deletes and TTL expiry create tombstones (markers -- the data in immutable SSTables not removed in place, reclaimed by compaction). For time-series data with TTL (a very common Cassandra pattern -- e.g., metrics, events, expiring after a retention period), STCS and LCS have a serious problem: the expiring data (and its tombstones) is spread across SSTables that also contain non-expiring data (mixed by the size-tiered or leveled compaction -- not grouped by time), so the expired data can't be efficiently dropped (it's mixed with live data in the SSTables -- the tombstones accumulating, and the expired data lingering -- causing tombstone problems: read performance degradation from scanning tombstones, and space from un-reclaimed expired data). TWCS solves this by grouping SSTables into time windows (so each SSTable contains data from one time range -- and when that time range's data all expires -- TTL -- the whole SSTable can simply be dropped -- efficient, no tombstone scanning or mixed-SSTable problem). So for time-series with TTL, TWCS is the right strategy (efficient time-based expiry -- the whole time-window SSTable dropped when expired), and using STCS/LCS instead causes tombstone problems (accumulation, read degradation, space). Understanding the tombstone/time-series issue (and TWCS as its solution) is understanding a crucial, common Cassandra consideration -- the time-series TTL workload needing TWCS.

The architecture: every piece explained

Top row: the problem and strategies. The LSM problem: writes accumulate into immutable SSTables, requiring compaction (merging them to bound read amplification and reclaim space). STCS (Size-Tiered): merges SSTables of similar size into larger ones -- write-optimized (low write amplification) but higher read amplification (data in many tiers) and space (large obsolete SSTables lingering). LCS (Leveled): organizes SSTables into levels of non-overlapping SSTables -- read-optimized (few SSTables per read) and space-efficient, but higher write amplification (maintaining the levels). TWCS (Time-Window): groups SSTables into time windows, compacting within a window -- ideal for time-series with TTL (each window's data expires together, the window's SSTable dropped efficiently).

Middle row: the amplifications and tombstones. Read amplification: the SSTables checked per read (STCS high -- data across tiers; LCS low -- few SSTables per read) -- affecting read performance. Write amplification: the rewrite cost (STCS low -- few rewrites; LCS high -- maintaining the levels) -- affecting write cost and compaction I/O. Space amplification: obsolete data lingering (STCS higher -- large obsolete SSTables; LCS lower -- efficient reclamation) -- affecting disk usage. Tombstone handling: how expiring/deleted data (tombstones) is reclaimed -- STCS/LCS mixing expiring data across SSTables (tombstone accumulation for time-series TTL), TWCS grouping by time (efficient window-based expiry) -- crucial for time-series/TTL workloads.

Bottom rows: matching and unification. Workload matching: choosing the strategy for the workload -- STCS for write-heavy (low write amplification), LCS for read-heavy (low read amplification, worth the write cost), TWCS for time-series/TTL (efficient time-based expiry) -- the per-table strategy matched to the table's workload. UCS (Unified): a newer, tunable compaction strategy (behaving like STCS or LCS via configuration -- unifying the choice into a tunable spectrum -- so the tradeoff can be tuned rather than choosing a fixed strategy) -- the modern unified approach. The ops strip: strategy choice (choosing the compaction strategy per table -- matched to the workload -- the high-impact decision), monitoring (monitoring the compaction -- SSTable counts per read (read amplification), compaction I/O (write amplification), space, tombstones -- to verify the strategy is working and catch problems), and tuning (tuning the strategy's parameters -- e.g., the size tiers for STCS, the level sizes for LCS, the time window for TWCS -- and switching strategies if the workload changes).

End-to-end flow

Trace matching strategies to workloads. A write-heavy table (high write throughput, relatively few reads -- e.g., an ingestion/logging table): STCS is chosen (write-optimized -- low write amplification, minimizing the rewrite cost for the heavy writes) -- the higher read amplification acceptable (few reads). A read-heavy table (frequent reads, needing low read latency -- e.g., a serving table): LCS is chosen (read-optimized -- low read amplification, a read touching few SSTables -- fast reads) -- the higher write amplification (maintaining the levels) accepted for the read performance. A time-series table with TTL (metrics/events expiring after a retention period -- e.g., 30-day retention): TWCS is chosen (grouping SSTables into time windows -- e.g., daily windows -- so each day's data is in its own SSTables, and when the 30-day TTL expires a day's data, the whole day's SSTable is dropped -- efficient expiry, no tombstone accumulation). Each table's strategy is matched to its workload (STCS write-heavy, LCS read-heavy, TWCS time-series TTL) -- the per-table matching optimizing each.

The mismatch and tombstone vignettes show the failure modes. A mismatch case: a read-heavy table was left on STCS (the default) -- suffering high read amplification (a read checking many SSTables across the size tiers -- slow reads). Switching it to LCS (read-optimized -- few SSTables per read) dramatically improved the read latency (the read amplification reduced) -- the strategy match fixing the read performance. A tombstone case: a time-series table with TTL was on STCS -- hitting tombstone problems (the expiring data spread across SSTables mixed with non-expiring data -- so the expired data couldn't be efficiently dropped, the tombstones accumulating -- degrading reads (scanning tombstones) and wasting space (un-reclaimed expired data)). Switching to TWCS (grouping by time window -- so each window's data expires together and the window's SSTable is dropped) solved the tombstone problem (efficient time-based expiry -- no tombstone accumulation) -- the TWCS match fixing the time-series TTL problem.

The UCS and tuning vignettes complete it. A UCS case: on newer Cassandra, the team uses UCS (the unified strategy -- tunable to behave like STCS or LCS) -- so instead of choosing a fixed strategy, they tune UCS's parameters to the desired tradeoff (more write-optimized or more read-optimized) -- the unified, tunable approach (a spectrum rather than fixed strategies). A tuning case: the team monitors the compaction (SSTable counts per read -- read amplification; compaction I/O -- write amplification; tombstones; space) -- verifying each table's strategy is working (the expected tradeoff) and catching problems (e.g., a table whose workload shifted, now mismatched -- prompting a strategy switch) -- the monitoring and tuning keeping the strategies matched. The consolidated discipline the team documents: choose the compaction strategy per table matched to the workload (STCS for write-heavy -- low write amplification; LCS for read-heavy -- low read amplification; TWCS for time-series/TTL -- efficient time-based expiry), understand the read/write/space tradeoff (each strategy a different point), use TWCS for time-series with TTL (avoiding the tombstone accumulation STCS/LCS hit), consider UCS on newer Cassandra (the tunable unified strategy), monitor the compaction (read/write/space amplification, tombstones -- verifying the strategy and catching mismatches), and tune/switch strategies as workloads change -- because the compaction strategy is a high-impact per-table decision (each strategy a different point in the read/write/space tradeoff), and matching it to the workload (STCS/LCS/TWCS for write/read/time-series) is one of the most impactful Cassandra tuning decisions.

How often does Cassandra compaction run?

Cassandra compaction has no timer and no fixed frequency. A background check runs after an event: a memtable flush adds an SSTable, streaming or an import adds SSTables, or another compaction finishes. There is also a backstop that checks every table once a minute, starting five minutes after the node starts, and the comment in CassandraDaemon says it exists only for missed scheduling. Each check asks the table's strategy whether anything qualifies, so how often compaction actually does work depends on the write rate and on the rule below.

STCS has one more trigger. When no bucket qualifies, it considers a single SSTable whose droppable tombstone ratio is above tombstone_threshold (0.2) and that is older than tombstone_compaction_interval (86,400 seconds), provided it passes the overlap check.

The frequency follows from arithmetic. This script uses the bucket rule from SizeTieredCompactionStrategy on a steady insert load for Belgavi's Book Store, where the book_orders table receives 800 new rows a second at 1 KiB each and flushes 100 MiB at a time. Every number is arithmetic from those inputs, not a benchmark, and real flush sizes vary.

"""How often does Size-Tiered compaction run? Cassandra's bucket rule, on a simple load.

Sizes are in MiB. Every row is new: no overwrites, deletes or TTL.
Book Store example: the book_orders table flushes 100 MiB at a time.
"""
MIN_THRESHOLD, MAX_THRESHOLD = 4, 32
BUCKET_LOW, BUCKET_HIGH, MIN_SSTABLE_SIZE = 0.5, 1.5, 50
FLUSH_MIB = 100
ROWS_PER_SEC, ROW_KIB = 800, 1


def buckets(sizes):
    groups = []  # each group is [average size, [member sizes]]
    for size in sorted(sizes):
        for group in groups:
            avg, members = group
            similar = BUCKET_LOW * avg < size < BUCKET_HIGH * avg
            both_small = size < MIN_SSTABLE_SIZE and avg < MIN_SSTABLE_SIZE
            if similar or both_small:
                group[0] = (avg * len(members) + size) // (len(members) + 1)
                members.append(size)
                break
        else:
            groups.append([size, [size]])
    return groups


def compact_once(sstables):
    """Merge one qualifying bucket. Cassandra picks the hottest; this picks the smallest."""
    for avg, members in sorted(buckets(sstables), key=lambda g: g[0]):
        if len(members) >= MIN_THRESHOLD:
            chosen = members[:MAX_THRESHOLD]
            for size in chosen:
                sstables.remove(size)
            sstables.append(sum(chosen))
            return len(chosen), sum(chosen)
    return None


def run(flushes):
    sstables, rewritten, compactions = [], 0, []
    for n in range(1, flushes + 1):
        sstables.append(FLUSH_MIB)
        while (done := compact_once(sstables)) is not None:
            rewritten += done[1]
            compactions.append((n, done[0], done[1]))
    return sstables, rewritten, compactions


seconds_per_flush = FLUSH_MIB * 1024 / (ROWS_PER_SEC * ROW_KIB)
sstables, rewritten, compactions = run(64)
print(f"one flush every {seconds_per_flush:.0f} s at {ROWS_PER_SEC} rows/s of {ROW_KIB} KiB")
for n, count, size in compactions[:6]:
    print(f"flush {n:2d} (t={n * seconds_per_flush:5.0f} s): merge {count} SSTables into {size} MiB")
flushed = 64 * FLUSH_MIB
print(f"after 64 flushes: {len(compactions)} compactions, SSTables on disk {sorted(sstables)}")
print(f"data flushed {flushed} MiB, rewritten by compaction {rewritten} MiB")
print(f"write amplification {(flushed + rewritten) / flushed:.1f}x")

one flush every 128 s at 800 rows/s of 1 KiB
flush  4 (t=  512 s): merge 4 SSTables into 400 MiB
flush  8 (t= 1024 s): merge 4 SSTables into 400 MiB
flush 12 (t= 1536 s): merge 4 SSTables into 400 MiB
flush 16 (t= 2048 s): merge 4 SSTables into 400 MiB
flush 16 (t= 2048 s): merge 4 SSTables into 1600 MiB
flush 20 (t= 2560 s): merge 4 SSTables into 400 MiB
after 64 flushes: 21 compactions, SSTables on disk [6400]
data flushed 6400 MiB, rewritten by compaction 19200 MiB
write amplification 4.0x

So this table compacts about every fourth flush, and every sixteenth flush does a second, larger merge. The 64 flushes pass through three tiers, so each byte is rewritten three times on top of its flush, which gives 4.0x. Doubling the write rate halves the time between compactions. It does not change the count per flush. Compaction work therefore follows write volume, and the same table on a quiet day may not compact for hours, which is normal.

Best practices that follow from this

• Do not schedule compaction. Leave it automatic. A nightly nodetool compact on an STCS table builds one very large SSTable that has no similar-sized peers, so it can sit unmerged for a long time and hold obsolete data with it.
• Check the throttle before adding threads. The setting is compaction_throughput from 4.1 (default 64 MiB/s), compaction_throughput_mb_per_sec in 4.0 (64) and 3.11 (16). The comment in cassandra.yaml suggests 16 to 32 times your insert rate. Here, 800 KiB/s on one node gives about 12.5 to 25 MiB/s, so the default is ample. Validation for repair shares the same limit. 0 disables throttling.
• Change concurrent_compactors second. By default it is the smaller of disks and cores, kept between 2 and 8. The yaml comment recommends the number of cores on SSD.
• Watch the backlog, not the clock. Rising pending tasks in nodetool compactionstats and rising SSTables per read in nodetool tablehistograms mean compaction is behind.
• Leave free disk. A merge writes its output before the input files are removed, so the largest possible merge needs room for its output.
• Keep TWCS for append-only TTL data. Updates or deletes that reach old windows overlap across SSTables and block whole-file expiry.

ALTER TABLE bookstore.book_orders WITH compaction = {
  'class': 'SizeTieredCompactionStrategy',
  'min_threshold': 4,
  'max_threshold': 32
};