A search engine with vectors, not a vector store with filters
Most vector databases start from one primitive — approximate nearest neighbour over a dense index — and add filtering, text matching, and reranking around the edges. Vespa starts from a general matching engine: a query is a tree of operands, and a nearest-neighbour operand sits in that tree next to a term match, a range predicate, and a boolean filter, all evaluated by the same matcher over the same document set.
That single design decision is why the math in this article looks different from the math in the rest of this cluster. There is no ‘vector search, then rerank in a separate service’ pipeline to price, because the ranking expression, the machine-learned model, and the raw tensors all live in the same process as the postings. The costs that dominate are therefore not network hops but per-document CPU multiplied by candidate count — and Vespa gives you explicit knobs to control both factors independently, which is the whole point of what follows.
The tensor data model: indexed, mapped, and mixed
A Vespa field can be a tensor with a declared type, and the type tells you the memory bill exactly. An indexed dimension is dense with a fixed size: tensor<float>(x[384]) is a plain 384-vector costing 384 × 4 = 1536 bytes. A mapped dimension is sparse and keyed by label: tensor<float>(token{}) stores only the entries you populate, which is how a learned sparse representation fits the same type system.
The interesting case is mixed: tensor<float>(chunk{}, x[384]) holds one embedding per chunk of a document in a single field. Twenty chunks costs 20 × 384 × 4 ≈ 30.7 kB per document, and declaring the cell type bfloat16 halves that to 15.4 kB with a precision loss far below the noise floor of the embedding itself. The point is that a document is not forced to be one vector, so you do not have to explode it into twenty rows and deduplicate afterwards.
Ranking expressions: the math runs where the data is
Tensors would be storage trivia if you could not compute on them in place. Vespa’s ranking expressions are a small tensor language built on two operations: join, which multiplies tensors cell-wise over shared dimensions, and reduce, which collapses a dimension by sum, max, or average. A dot product is a join followed by a sum-reduce; a max-over-chunks score is a nested reduce.
first-phase: bm25(text) + 8 * closeness(field, emb)
second-phase: reduce(reduce(query(q) * attribute(chunks),
sum, x), max, chunk)The inner reduce scores every chunk against the query vector; the outer one keeps the best chunk. Doing this in the engine is not a convenience. Shipping the raw tensors to an external reranker would move 1000 × 30.7 kB ≈ 31 MB per query across the network, which at any realistic query rate is the whole cost of the system. Computing on resident memory makes it a few milliseconds.
Multi-phase ranking: the cost equation
Every ranking design obeys the same equation. Let N be the candidates a query matches on one content node, R the rerank-count handed to the second phase, and G the count handed to the global phase after merging. With per-document costs c1 < c2 < c3:
T ≈ N·c1 + R·c2 (per content node)
+ G·c3 (once, in the container)Take 20M documents over 8 content nodes, a query matching N = 100,000 per node, a cheap first phase at c1 = 0.4 µs and a model at c2 = 20 µs. Single-phase scoring costs 100,000 × 20 µs = 2.0 s per node. Two-phase with R = 200 costs 40 ms + 4 ms = 44 ms — roughly 45× cheaper for a ranking that differs only in the tail. The asymmetry to hold onto: rerank-count is per node, so 8 nodes rerank 1600 documents cluster-wide while each pays for only 200.
Sizing the rerank count
Cost in R is linear. Benefit is not. The gain from a larger R is the probability that the document the second phase would rank first actually survived the first phase, and that probability saturates: doubling from 50 to 100 usually moves NDCG@10 visibly, doubling from 400 to 800 usually does not move it at all. So the sizing method is empirical and cheap — measure NDCG@10 at R = 50, 100, 200, 400, 800 and stop at the knee.
Two corrections to the naive reading. First, because R is per node, adding nodes widens the global candidate pool for free: R = 100 on 8 nodes already reranks 800 documents. Second, a useful floor is 10–20× the number of results you actually display, because the second phase can only reorder what it is given — too small an R caps quality silently, with no error and no log line, while too large an R is a pure latency tax on a flat curve.
Where a model can afford to run
Vespa will evaluate a gradient-boosted tree ensemble or an ONNX model inside the ranking expression, so the real question is which tier can pay for it. A GBDT with 500 trees of depth 6 is about 3000 branch evaluations per document, memory-latency bound at roughly 5 µs. At R = 200 that is 1 ms per node — comfortably a second-phase model.
A cross-encoder is a different universe. Take 6 layers, d = 384, T = 256 tokens. Non-embedding parameters are 12d^2 per layer = 1.77M, so 10.6M total; the token embedding table is a lookup, not a matmul, and does not count. Forward cost is 2 × 10.6e6 × 256 ≈ 5.4 GFLOP, plus attention at 4T^2 d per layer ≈ 0.6 GFLOP — call it 6 GFLOP per document. At 50 GFLOP/s per core that is 120 ms for one document. This is why it belongs in the global phase, over tens of documents, in the stateless tier you can scale separately.