Choosing Vector Dimensionality for Spatial Retrieval

Dimensionality is chosen by picking a model, and it is the single number that determines the memory footprint of every index and every replica for the life of the system. Choosing it deliberately means measuring recall at several sizes on your own corpus and taking the smallest that holds — which is usually far below the largest available. This guide does that, as the sizing decision within spatial embedding models.

When to Use This Approach

Measure before committing, and re-measure whenever the corpus grows by an order of magnitude or the model changes. Between those events the number is settled and should be left alone.

The second row is where most spatial corpora sit, and it is also where the largest available model is most tempting and least justified: the recall difference between 768 and 1536 dimensions on technical prose is typically under a point, and the memory difference is a factor of two.

Implementation

The measurement sweeps dimensionality, records recall and memory together, and returns the cheapest configuration that clears the target.

import logging
from dataclasses import dataclass
from typing import Callable, Sequence

log = logging.getLogger("dimensionality")


@dataclass(frozen=True)
class Point:
    dim: int
    recall: float
    index_gib: float
    note: str = ""


def index_gib(n_chunks: int, dim: int, m: int = 16, bytes_per_link: int = 4) -> float:
    """Vectors plus graph links. Metadata and identifiers are on top of this."""
    vectors = n_chunks * dim * 4
    links = n_chunks * m * 2 * bytes_per_link
    return round((vectors + links) / 1024 ** 3, 2)


def sweep(dims: Sequence[int], n_chunks: int,
          measure_recall: Callable[[int], float]) -> list[Point]:
    """Measure filtered recall at each dimensionality on the real corpus."""
    out: list[Point] = []
    for dim in sorted(dims):
        try:
            recall = measure_recall(dim)
        except Exception as exc:                    # a failed measurement is not a good one
            log.warning("recall measurement failed at dim=%d: %s", dim, exc)
            out.append(Point(dim, 0.0, index_gib(n_chunks, dim), f"measurement failed: {exc}"))
            continue
        out.append(Point(dim, round(recall, 4), index_gib(n_chunks, dim)))
    return out


def cheapest_meeting(points: Sequence[Point], target_recall: float,
                     memory_budget_gib: float | None = None) -> Point | None:
    """Smallest index that clears the recall target and fits the budget."""
    if not 0.0 < target_recall < 1.0:
        raise ValueError("target recall must be a fraction between 0 and 1")
    eligible = [p for p in points if p.recall >= target_recall
                and (memory_budget_gib is None or p.index_gib <= memory_budget_gib)]
    if not eligible:
        log.info("no dimensionality met recall %.3f within the memory budget", target_recall)
        return None
    return min(eligible, key=lambda p: (p.index_gib, p.dim))

Measuring recall under the real filter is what makes this exercise meaningful. Unfiltered recall is nearly flat across dimensionality on most corpora, so an unfiltered sweep concludes that the choice does not matter — which is true of the measurement and not of the workload.

Returning None rather than the best available when nothing meets the target is deliberate. That condition is a real finding: it means the corpus needs either a better model, a different chunking strategy, or a larger machine, and returning the highest-recall option quietly papers over the choice.

Validation & Testing

def test_cheapest_meeting_prefers_smaller_over_better():
    points = [Point(384, 0.91, 6.5), Point(1536, 0.97, 24.1)]
    assert cheapest_meeting(points, 0.90).dim == 384


def test_memory_budget_excludes_a_configuration_that_meets_recall():
    points = [Point(384, 0.88, 6.5), Point(1536, 0.97, 24.1)]
    assert cheapest_meeting(points, 0.90, memory_budget_gib=10.0) is None


def test_failed_measurement_scores_zero_and_is_noted():
    def raising(_dim):
        raise RuntimeError("encoder unavailable")
    points = sweep([384], 1_000_000, raising)
    assert points[0].recall == 0.0 and "failed" in points[0].note


def test_target_must_be_a_fraction():
    try:
        cheapest_meeting([Point(384, 0.9, 1.0)], 90)
    except ValueError:
        return
    raise AssertionError("a target of 90 must be rejected, not read as 90%")


def test_memory_model_scales_with_both_inputs():
    assert index_gib(2_000_000, 768) > index_gib(1_000_000, 768)
    assert index_gib(1_000_000, 1536) > index_gib(1_000_000, 768)

The second test is the one that produces a useful conversation rather than a silent compromise. When nothing fits, somebody has to decide between a bigger machine, a smaller corpus and a lower target, and a function that quietly returns the best available option removes that decision from view.

Run the sweep against the same evaluation set used to choose the model, so the two decisions are measured on comparable ground. A dimensionality chosen against a different query set than the model was chosen against produces a configuration that is optimal for neither, and the discrepancy is invisible because both numbers look reasonable in isolation.

Gotchas & Edge Cases

Memory estimated from vectors alone. Graph links, identifiers, deletion markers and per-record metadata are all on top, and on short vectors the overhead can rival the data. Measure the built index and treat the model as planning arithmetic.

Recall measured on a uniform sample. Real embeddings are clustered, and clustered data is where approximate indexes lose recall. Sample by region — everything from a few areas — rather than scattering across the corpus.

Dimensionality chosen with the model and never revisited. The two are separable in more cases than people expect, and a model that supports shortened outputs makes the choice a slice rather than a migration. Ask about that capability before committing.

Quantization applied before the baseline exists. Compressing first makes a later recall problem impossible to attribute. Measure uncompressed, then compress and measure again.

Replicas forgotten in the budget. Three replicas of a 30 GiB index is 90 GiB of memory, and the sizing conversation usually happens about one copy. Multiply before comparing against a host.

Quantized and unquantized configurations compared on one axis. They are different points on a different curve, and plotting them together invites a comparison of memory figures that were produced under different assumptions about recall. Sweep uncompressed first, choose, then evaluate compression against that choice.

A target set higher than the workload needs. Chasing ninety-nine per cent recall costs disproportionately and rarely changes answers, because the reranking stage reorders the top candidates anyway. Set the target from measured answer quality, not from the metric.

Frequently Asked Questions

Related

• Up to the parent topic: Spatial Embedding Models
• Benchmarking Spatial Embedding Models for Vector GIS
• Related technique: Sizing HNSW Parameters for Spatial Recall
• Related topic: Spatial Vector Store Selection