Weaviate Math + Architecture

What a collection actually stores per object

A Weaviate collection is not one index, it is four things sharing a shard. There is an object store (an LSM-tree keyed by UUID holding the JSON properties), an inverted index for filters and BM25, the vector index itself, and the id maps that tie an external UUID to the internal integer the graph actually uses.

The consequence people miss: the vector is persisted twice. It is written alongside the object so a shard can be rebuilt from scratch, and held again in the index that serves queries. Disk carries roughly N · (4d + |props|); RAM carries the index copy plus graph links. And named vectors multiply this cleanly — giving one object a title and a body embedding means two vector spaces, two graphs, two sets of links. Cost scales as k · (4d + links · id_width) per object, not as one index that happens to be wider.

The HNSW knobs, and the ef Weaviate computes for you

Weaviate exposes the usual trio under its own names: maxConnections (edges per node above layer 0; layer 0 gets twice that), efConstruction (build-time beam width, paid once and stored as graph quality), and ef (search-time beam width, paid every query). The Qdrant article derives why the total link count per node is close to 2M — that arithmetic carries over unchanged.

What is Weaviate’s own is that ef defaults to -1, meaning dynamic: it is derived from the query, not fixed on the collection.

ef = min( max(dynamicEfMin, dynamicEfFactor · limit), dynamicEfMax )
with factor 8, min 100, max 500:
  limit =  10  →  max(100, 80)  = 100
  limit =  50  →  max(100, 400) = 400
  limit = 100  →  min(800, 500) = 500

The trap is that recall now depends on limit. Asking for 10 results and asking for 100 run different searches, so the object ranked 12th in one need not be ranked 12th in the other — and deep pagination silently saturates at the ceiling.

Thresholds counted in objects, not bytes

Two Weaviate limits are denominated in object counts, and both are therefore blind to dimensionality. flatSearchCutoff (commonly 40,000) says: if a filter’s allow-list is smaller than this, abandon the graph and brute-force the matching vectors instead. vectorCacheMaxObjects caps how many vectors the index keeps resident before it starts reading them from disk during a walk.

Convert the first one and the blind spot appears. At d = 384 float32, 40,000 objects is 61 MB — a trivial scan. At d = 1536 the same count is 40000 × 6144 B = 246 MB, four times the memory traffic for an identical setting. Qdrant makes the opposite choice and denominates its threshold in kilobytes, precisely because a linear scan is bandwidth-bound. The rule: a count-based default is calibrated for some assumed dimension, so lower it on wide vectors, and treat the cache limit the same way.

The dynamic index: flat until it is worth a graph

Weaviate can start a collection as a flat index — no graph, a disk-backed brute-force scan, optionally over binary-compressed vectors — and convert it to HNSW once the object count crosses a threshold (on the order of 10,000). The crossover arithmetic explains why the switch point sits so low.

N = 10,000   d = 1536   float32
flat scan   : 10^4 × 6144 B = 61 MB touched, 1.5e7 mul-adds
HNSW @ ef=100: a few thousand distance evals ≈ 5e6 mul-adds
               + 10^4 × 65 links × 8 B ≈ 5 MB graph
               + O(N · efConstruction · log N) build
→ graph wins by ~3× here, but by ~100× at N = 10^6

So the crossover is a wide band, not a point: below it the graph buys under an order of magnitude while costing build CPU, memory and index lag; above it the scan is hopeless. Defaulting to the low end is the right asymmetry — over-building a small index is cheap, and a 100 ms scan is not.

Multi-tenancy: a shard each, and the cost of many small tenants

Weaviate’s multi-tenancy is physical. Each tenant gets its own shard — its own LSM store, inverted index and HNSW graph. A tenant-scoped query never touches another tenant’s vectors, which means the filtered-search problem that dominates single-index designs simply does not arise: there is no induced subgraph to fragment, because the graph was never shared.

You pay for that in fixed per-shard overhead — memtables, segment metadata, bloom filters, file descriptors, index structs. Call it C bytes per active shard:

50,000 tenants × 2,000 objects, d = 768
vectors if all hot : 10^8 × 3072 B = 307 GB
fixed overhead     : 50,000 × C;  C = 2 MB → 100 GB
                     — before a single vector is stored

Which is why tenants have activity states: hot in memory, cold on local disk, or offloaded to object storage and lazily reloaded. The number that sizes your cluster is the count of simultaneously active tenants, never the count of tenants.

Hybrid search: alpha, and two fusions that disagree

Weaviate’s hybrid runs a BM25 query and a vector query and blends them with alpha: alpha = 1 is pure vector, alpha = 0 is pure keyword. What matters more than alpha is which fusion consumes it. rankedFusion throws the scores away and combines 1/(60 + rank) per list. relativeScoreFusion (the modern default) min-max normalizes each list, then blends the normalized scores. Same alpha, different answers:

vector sim : X .95  Y .94  Z .60   → norm  X 1.00  Y .971  Z 0
bm25       : Y 20    Z 3     X 2     → norm  Y 1.00  Z .056  X 0

relativeScore, α=0.5 :  Y .986   X .500   Z .028
rankedFusion,  α=0.5 :  Y .01626 X .01613 Z .01600

Both rank Y first, but rank fusion compresses the field into a 1.6% spread while relative-score preserves the margins — X’s near-tie on vector and Y’s BM25 blowout both survive. Rank fusion is robust to a miscalibrated retriever; relative-score is faithful to a well-calibrated one, and lets one runaway keyword hit dominate even at α = 0.5.

Product quantization and the lookup table that pays for it

Weaviate’s PQ splits each vector into s segments, clusters each segment’s subspace into 256 centroids with k-means over a training sample, and stores one byte per segment. The compression is 4d / s. But the reason PQ is fast is asymmetric distance computation: the query is never compressed. Instead it is turned into a table.

d = 1536, s = 256 (6 dims/segment), 256 centroids
code       : 256 B/vector   (24× vs 6144 B)
codebook   : 256 × 256 × 6 × 4 B = 1.57 MB
query prep : 256 × d = 393k mul-adds → 256×256 table = 256 KB
per cand.  : s = 256 lookups+adds  vs  d = 1536 mul-adds
break-even : 393k / d ≈ 256 full-precision comparisons

An ef = 100 walk evaluates thousands of candidates, so the table amortizes many times over: 6× fewer operations and 24× less memory traffic per comparison. The honest cost is not arithmetic but the random reads into that 256 KB table, and the recall loss — repaired by rescoring the top candidates against full vectors, which now live on disk. Binary and scalar quantization are also offered; the Qdrant article derives both.

Replication: every replica builds its own graph

Replication in Weaviate is not a file copy. Each replica of a shard ingests the objects itself and constructs its own HNSW graph. Because insertion order differs and level assignment is randomized, no two replicas hold byte-identical graphs even when they hold identical data.

Two consequences follow, and only the first is obvious. Memory multiplies: factor r costs r × (4d + links) per object, and adding a replica to a 100M-object collection is O(N · efConstruction · log N) of index-building CPU — hours, not a transfer. The subtler one is that at read consistency ONE, two identical queries routed to different replicas can return slightly different neighbours, because approximate search over two different graphs is genuinely two different searches. That is expected, not corruption; raise consistency if reproducibility matters. Weaviate offers tunable ONE/QUORUM/ALL per operation, and the w + c > r overlap condition is derived in the Qdrant piece.

Putting it together: a sizing example

Take 20 million objects at d = 1536, maxConnections = 32, one named vector, no replication. Link count per node is 2M plus a thin upper-layer tail, ≈ 65; the byte width of an internal id is an implementation detail, so carry it as a parameter and assume 8 bytes.

vectors : 2e7 × 1536 × 4 B          = 122.9 GB
graph   : 2e7 × 65 × 8 B            =  10.4 GB
                                       ---------
                                        133.3 GB  (+ objects on disk)

with PQ, s = 256:
codes   : 2e7 × 256 B                =   5.1 GB
graph   : unchanged                    =  10.4 GB
                                       ---------
                                         15.5 GB

The codes compress 24× but the total only improves 8.6×, because the graph does not compress at all — and it is now the larger half of your RAM. That flips which knob matters: uncompressed, maxConnections is noise against 123 GB of floats; compressed, halving it to 16 saves about 5 GB of 15.5. Multiply the whole table by r for replicas and by k for named vectors, and 133 GB versus 15.5 GB is the difference between a sharded cluster and one commodity box.