Why architecture matters here
Immutability is the decision everything else follows from. Once an SSTable's components are written and finalised, not one byte of them changes for the rest of the file's life. That single constraint buys an enormous amount: no locking and no page latches, because nothing can be concurrently modified; free memory-mapping and sharing of the same file across every reader thread; checksums computed once at write time that stay valid forever; a snapshot that is nothing more than a directory of hard links; and node-to-node streaming that is a byte copy rather than a logical replay. None of that is available to an engine that mutates pages in place.
The bill arrives as amplification. Because nothing is edited, an update to one column is a new cell in a new file, and a delete is a tombstone — a marker that is strictly additive. Deleting a terabyte makes the on-disk footprint grow before it shrinks. Every version of every cell you have ever written is still on disk until some compaction rewrites the file holding it into a new file that omits it. There is no vacuum process, no in-place free list, no truncation of a live file. Compaction is the only reclamation mechanism in the entire storage engine, which is why a stalled compaction backlog is a disk-space incident and not merely a latency one.
So the component files exist to make "check N files" cheap. Read the file set as a ladder of progressively more expensive filters: SSTable-level metadata dismisses a file for free, the bloom filter dismisses it without I/O, the summary narrows the on-disk search, the partition index resolves one offset, and only then does anything touch the data file. A read that has to descend the full ladder on twenty files behaves nothing like the same read against three, which is why SSTables consulted per read is the number that predicts p99 latency far better than row count, node count, or heap size.
Two tables can share a schema, a hardware profile and a latency SLA and diverge completely on this one number. Diagnosing the divergence means reading the SSTable metadata — not the application code, and not the query.
The architecture: every file explained
Walk the diagram top to bottom. A flush or a compaction produces the whole set below in a single pass, and the set is atomic: it is complete or it is discarded.
MemTable flush. When a memtable crosses its threshold it is serialised to disk as a new SSTable. All of the files below are written together, from data already in sorted order.
SSTable Writer. Iterates the sorted memtable once, streaming the data file while simultaneously accumulating the index, the bloom filter, the compression chunk offsets and the statistics.
Immutable Files. Once written, an SSTable never changes. Deletes are tombstones in newer SSTables; updates are newer cells in newer SSTables.
Data.db. The partition-ordered, clustering-ordered rows. Compressed in fixed-size chunks. Every other component is, in the end, a way of not reading this one.
Index.db. Every partition key in the file, in token order, each mapped to its byte offset in Data.db — plus, for large partitions, an embedded clustering-column index. Binary-searchable, but on disk.
Summary.db. A sparse in-memory sample of Index.db (every Nth entry). Binary-search the small resident structure to get a narrow byte range, then search only that range of Index.db.
Filter.db. The bloom filter over this file's partition keys. Answers "definitely not here" or "possibly here" with no disk I/O at all. The sizing and false-positive tradeoff is covered in Cassandra bloom filters.
Statistics.db. Per-SSTable metadata: min and max write timestamps, min and max clustering values, estimated droppable-tombstone ratio, partition-size and cell-count histograms, compaction level, repair state. The cheapest filter of the lot.
CompressionInfo.db. The compressed offset of every chunk in Data.db, plus the compressor and chunk length. Without it a compressed data file cannot be seeked into at all.
TOC.txt / Digest.crc32. The manifest of components belonging to this SSTable, and a digest over the data file for integrity verification.
File names, and which components are disposable
Components share a filename prefix and differ only in the suffix, so a single SSTable looks like nb-1-big-Data.db, nb-1-big-Index.db, nb-1-big-Filter.db and so on, all sitting in <data_dir>/<keyspace>/<table>-<table_uuid>/. The leading two-letter code is the format version, bumped whenever the on-disk layout changes, which is precisely why parsing these names in a script is a trap across upgrades. The middle field is the generation; Cassandra 5.0 can instead issue globally unique identifiers there, which removes the generation collisions that make streaming files between nodes awkward.
The components are not equally precious. Summary.db, Filter.db, Digest.crc32 and the compression metadata are all derived — they can be regenerated by re-reading the index and data files, which is why a node with a missing or stale summary starts up slower but starts up. Data.db and Index.db are not derivable from anything else on that node. Never hand-edit or partially copy a component set; if you must move SSTables, move every file sharing the prefix, and consult TOC.txt for the list.
Partition-level indexing versus row-level indexing
For a small partition the index entry is trivial: the partition key and one byte offset into Data.db. That is the whole of partition-level indexing, and it is all most tables ever need — find the partition, read it, done.
Row-level indexing appears only when a partition grows past column_index_size_in_kb (64 KB by default, renamed to column_index_size with explicit units in more recent releases). Beyond that threshold the writer additionally emits a list of index-info entries for that partition: one per block of roughly that size, each recording the first and last clustering key in the block and the block's offset. A query with a clustering restriction binary-searches this list and seeks straight to the block that can contain the requested range, instead of scanning the partition from its start.
That is the entirety of row-level indexing in the base format: within one partition, in clustering order, at block granularity. Non-key columns get nothing here at all — querying by a non-key column is what secondary indexes and storage-attached indexing exist for.
What a wide partition does to all of this
The clustering index lives inside the partition's index entry, so its size grows linearly with the partition. A 2 GB partition at 64 KB granularity carries on the order of thirty thousand index-info entries, and naively deserialising all of them to answer a single-row lookup is both slow and a heavy allocation on the read path. That mechanism — not some vague notion of "big is bad" — is the concrete reason wide partitions hurt reads.
Cassandra mitigates it with column_index_cache_size_in_kb (2 KB by default): index entries whose serialised form exceeds that are not held in the key cache and are instead binary-searched on disk in place, trading a couple of extra seeks for bounded memory. The bound is real but the read is still doing work proportional to how badly the partition was modelled. Watch compacted partition maximum bytes in nodetool tablestats; it is the earliest honest warning you get.
Compression happens at the chunk level - and that forces an offset table
Cassandra does not compress a row, and it does not compress the data file as one stream. It cuts Data.db into fixed-size uncompressed chunks of chunk_length_in_kb and compresses each chunk independently. The reason is seekability: a single compression stream over a whole file would have to be decompressed from the beginning to reach byte 900,000,000, which makes random access impossible. Independent chunks restore random access at the cost of a slightly worse ratio, because each chunk's dictionary starts cold.
That choice is exactly why CompressionInfo.db must exist. A logical offset in the uncompressed file no longer tells you where the bytes are, so the file stores the compressed start offset of every chunk. Reading logical byte X means integer-dividing by the chunk length to get a chunk number, looking that chunk's physical offset up in the offset array, reading and decompressing that chunk, and then indexing into the result. Each chunk also carries a checksum verified according to crc_check_chance; uncompressed tables get a CRC.db with the same per-chunk checksums, because integrity checking needs chunking even when compression does not.
Two consequences matter operationally. First, the chunk is the minimum unit of read: fetching a 200-byte row from a table with 64 KB chunks decompresses 64 KB. Smaller chunks waste far less on point reads and compress slightly worse; larger chunks favour scans. The default was lowered from 64 KB to 16 KB in Cassandra 4.0 for exactly this reason, and raising it back is defensible for scan-heavy tables only.
Second, the offset array is resident memory proportional to file size divided by chunk length. At eight bytes per chunk, a terabyte of data on one node at 16 KB chunks is roughly half a gigabyte of chunk offsets held off-heap, before you count bloom filters and summaries. Halving the chunk length doubles that. The compression ratio reported by nodetool tablestats is stored-over-original, so 0.30 means you are getting a little over 3x — the storage saving is real, but the memory it costs is a line item you have to budget.
Statistics.db - the metadata that skips whole files
Statistics.db is the cheapest filter in the ladder because consulting it involves no I/O at all: it is loaded when the SSTable is opened and kept with the reader. What it holds is a compact summary of everything in the file — minimum and maximum write timestamps, minimum and maximum local deletion times and TTLs, minimum and maximum clustering values, estimated partition-size and cell-count histograms, an estimated droppable-tombstone ratio, the compaction level for leveled strategies, the compression ratio, and the repair state of the data.
Four short-circuits fall out of it directly:
Timestamp ordering. Candidate SSTables are considered newest-first by maximum timestamp. Once the merge has a complete answer that no older file could possibly improve on, the remaining files are never opened. This is why a table whose partitions are written once and never updated reads so much faster than its SSTable count suggests.
Clustering bounds. A slice such as WHERE event_time > ? can be compared against the file's maximum clustering value and dismissed outright — before the bloom filter, before any I/O. Time-ordered clustering keys make this filter extremely effective, and it is a large part of why time-series schemas behave well on Cassandra.
Deletion times and TTLs. Recording the maximum local deletion time lets compaction identify a fully-expired SSTable and drop the entire file without merging it, which is the mechanism behind time-window compaction's efficiency on TTL'd data.
Droppable tombstone ratio. When the estimate crosses tombstone_threshold (0.2 by default) a single-SSTable compaction can be triggered purely to purge tombstones from that one file.
The repair state is worth a separate note: incremental repair marks SSTables as repaired, and repaired data is never compacted together with unrepaired data. That partition of the file set is invisible in the schema but plainly visible in the SSTable count, and it surprises people who change repair strategy and then wonder why their file counts moved. The tool that prints all of this for one file is sstablemetadata. Read it before theorising.
End-to-end read flow inside one SSTable
Pick up the story at the point where a replica has already decided this file is a candidate for a single-partition read. Everything upstream of that — coordinator routing, replica selection, cross-replica reconciliation — belongs to the read path article.
1. SSTable metadata. If the query's clustering restriction cannot intersect the file's min/max clustering bounds, or the merge already holds an answer newer than this file's maximum timestamp, the file is dropped here. Cost: zero I/O.
2. Bloom filter. Filter.db is resident, so this is a handful of hash computations. "Definitely not present" ends the file's involvement with no disk access whatsoever; "possibly present" continues, occasionally wastefully, at the configured false-positive rate.
3. Key cache. If this partition key was read recently, the cached index entry supplies the data-file offset directly and steps 4 and 5 are skipped entirely. A healthy key-cache hit rate is worth two structures' worth of lookup on every read.
4. Partition summary. Summary.db samples Index.db at min_index_interval (128 by default, allowed to relax toward max_index_interval at 2048 for very large tables so that summary memory stays bounded). Binary-searching the sample yields a narrow byte range of Index.db to scan.
5. Partition index. Scan that range of Index.db on disk to find the exact key and read its index entry: the offset into Data.db, and for a large partition the clustering index described above.
6. Clustering seek. Binary-search the clustering index for the block whose first/last clustering keys bracket the requested range, giving a byte offset inside the partition rather than at its start.
7. Data file. Translate that logical offset through CompressionInfo.db into a chunk number and a physical offset, read the chunk from the page cache or the disk, verify its checksum according to crc_check_chance, decompress it, and deserialise the rows.
What comes back is a fragment, not an answer. It is merged with the memtable and with every other SSTable that survived the ladder, newest timestamp winning per cell and tombstones shadowing whatever they cover. A read that touches three SSTables does the above three times; a read that touches thirty does it thirty times, and that is the whole of the SSTable-count story.