Caching Spatial Tool Results Across Agent Turns

Within one conversation the same place resolves, the same region filters and the same statistic is computed repeatedly, and none of it changes between turns. Caching that is the largest cost saving available in a spatial agent, and it delivers something more valuable as a side effect: the same question asked twice gets the same answer. This guide covers the keys, which is where caches succeed or fail — the practical half of cost and latency budgets for spatial agents.

When to Use This Approach

Cache anything deterministic that costs more than the cache lookup. In a spatial agent that is nearly every tool result, with one important exception.

The last row is absolute. A cached result for a step that notified something, wrote something back or triggered a downstream process turns a repeated call into a silently skipped effect, which is a much worse failure than the cost it saved.

Implementation

The key builder is the whole design. It rounds, sorts, and includes every version that could change the answer.

import hashlib
import json
import logging
from typing import Any, Callable, Mapping, Optional

log = logging.getLogger("spatial_cache")

EXTENT_PLACES = 3          # ~100 m: far finer than the difference between two answers


def cache_key(kind: str, versions: Mapping[str, str], **parts: Any) -> str:
    """Deterministic across processes: rounded extents, sorted keys, versions included."""
    normalised: dict[str, Any] = {}
    for name, value in parts.items():
        if name.endswith("_bbox") and value is not None:
            normalised[name] = [round(float(v), EXTENT_PLACES) for v in value]
        elif isinstance(value, (set, frozenset)):
            normalised[name] = sorted(value)
        elif isinstance(value, str):
            normalised[name] = value.strip().lower()
        else:
            normalised[name] = value
    payload = json.dumps({"kind": kind, "versions": dict(sorted(versions.items())),
                          "parts": normalised},
                         sort_keys=True, separators=(",", ":"), default=str)
    return f"{kind}:{hashlib.sha256(payload.encode()).hexdigest()[:20]}"

Rounding the extent is what turns a per-request miss into a per-neighbourhood hit. Three decimal places is roughly a hundred metres, which is far finer than the difference between two answers about the same area and coarse enough that two requests about the same place share a key.

Including versions in the key rather than relying on expiry is what makes a data correction propagate immediately. When the index is rebuilt, every key changes, and the stale entries expire on their own schedule without ever being served.

def cached(kind: str, cache, compute: Callable[[], Any], ttl_s: int,
           versions: Mapping[str, str], **parts: Any) -> tuple[Any, bool]:
    """Read through the cache. A cache outage degrades performance, never correctness."""
    key = cache_key(kind, versions, **parts)
    try:
        hit = cache.get(key)
        if hit is not None:
            return hit, True
    except Exception as exc:
        log.warning("cache read failed for %s: %s", key, exc)

    value = compute()
    try:
        if _is_cacheable(value):
            cache.set(key, value, ttl=ttl_s)
    except Exception as exc:
        log.warning("cache write failed for %s: %s", key, exc)
    return value, False


def _is_cacheable(value: Any) -> bool:
    """Never cache a failure, an empty result, or a flagged low-confidence answer."""
    if value is None:
        return False
    if isinstance(value, dict):
        if value.get("error") or value.get("low_confidence"):
            return False
        if value.get("truncated"):
            return False              # a truncated result depends on a limit that may change
    return True

Refusing to cache failures and low-confidence results is the rule that prevents a transient outage from freezing a bad answer in place for the whole time-to-live. It is also what stops a flagged fallback from being served as though it were a determination on every subsequent turn.

Validation & Testing

def test_rounded_extents_share_a_key():
    a = cache_key("region", {"index": "v7"}, region_bbox=[-3.19999, 55.94001, -3.18, 55.96])
    b = cache_key("region", {"index": "v7"}, region_bbox=[-3.20001, 55.93999, -3.18, 55.96])
    assert a == b


def test_version_change_invalidates():
    a = cache_key("region", {"index": "v7"}, region_bbox=[-3.2, 55.94, -3.18, 55.96])
    b = cache_key("region", {"index": "v8"}, region_bbox=[-3.2, 55.94, -3.18, 55.96])
    assert a != b


def test_keys_are_stable_across_processes():
    key = cache_key("place", {"gazetteer": "2026-06"}, name="Kirkby Lonsdale")
    assert key == EXPECTED_KEY_FROM_A_PREVIOUS_RUN


def test_failures_are_not_cached(fake_cache):
    value, hit = cached("place", fake_cache, lambda: {"error": "gazetteer down"},
                        ttl_s=60, versions={"gazetteer": "2026-06"}, name="x")
    assert not hit and fake_cache.writes == 0


def test_cache_outage_does_not_fail_the_call(broken_cache):
    value, hit = cached("place", broken_cache, lambda: {"place_id": "p1"},
                        ttl_s=60, versions={}, name="x")
    assert value["place_id"] == "p1" and not hit

The third test is the one that catches the subtlest bug. A key built from a Python dictionary’s iteration order, or from a hash that varies between processes, is stable within one worker and different in another — so the cache appears to work in development and hits almost never in production.

Write the key builder once and use it everywhere. Two call sites that construct keys slightly differently produce two caches for the same data, each with half the hit rate, and the symptom is a hit rate that looks merely disappointing rather than obviously broken.

Gotchas & Edge Cases

Rounding that is too coarse. Two decimal places is roughly a kilometre, which merges genuinely different regions into one key and serves an answer about the wrong area. Three places is a reasonable default; derive it from how precisely regions actually differ in your workload.

A version omitted from the key. A cache keyed on the region and the filters but not the index version serves pre-rebuild answers after a rebuild. Every input that can change the answer belongs in the key, including ones that feel like infrastructure.

Caching a truncated result. A result limited to fifty rows is a function of the limit as well as the query, and serving it for a request that asked for two hundred is silently wrong. Either include the limit in the key or refuse to cache truncated results.

Per-conversation and shared caches conflated. A place resolution is shared safely across conversations; a session extent is not. Namespace them, or one user’s context leaks into another’s resolution.

Time-to-live chosen by habit. An hour is a number, not a policy. Derive it from how quickly the underlying data changes: a gazetteer entry is stable for months, a coverage statistic for days, and a live sensor reading for seconds.

A key that includes the user. Namespacing everything per user feels safe and destroys the shared benefit: a place resolution is the same for everyone, and keying it per user turns a high-hit-rate global cache into thousands of cold ones.

Cache hit rate measured in aggregate. One cache collapsing to zero disappears into the average of the others. Track the rate per kind, and alert on a step change in any of them.

Frequently Asked Questions

Related

• Up to the parent topic: Cost and Latency Budgets for Spatial Agents
• Estimating Token Cost of a Geoprocessing Plan
• Related topic: Geocoding and Place-Name Resolution
• Related topic: Fallback Routing for Geospatial Queries