Retrieval at Scale | Drop for 2025-12-18

TL;DR

Quiet but meaningful couple of weeks (Dec 8–18, 2025). Four updates: (1) OpenSearch 3.4 shipped with a redesigned agentic‑search UX plus performance and API gains (notably Lucene bulk‑collection for faster aggregations and richer gRPC query types); (2) Weaviate delivered a 1.34.5 stability patch and opened 1.35.0‑rc.0 with new operational controls (useful for tail‑latency and ops at scale); (3) a new theory paper analyzes when high‑dim retrieval avoids the “curse of dimensionality,” with concrete implications for multi‑vector, filtered, and sparse search; (4) Exqutor proposes a cardinality‑aware optimizer for vector‑augmented analytical queries, showing large speedups in pgvector/DuckDB/VBASE integrations.

OpenSearch 3.4: new agentic‑search UX, aggregation speedups, and gRPC query expansion

  • Key facts and current state of the topic
    • OpenSearch 3.4 was released Dec 16–17, bringing a no‑code agentic‑search experience, new search‑relevance workbench tools, and performance improvements. (opensearch.org)
  • Important context and background information
    • While 3.3 delivered notable vector‑search enhancements, 3.4 focuses on usability and infra: Lucene’s bulk‑collection API integration yields 5–40% faster aggregations; percentiles and matrix‑stats also see sizable gains. The Search gRPC API now supports additional query types (e.g., MatchBoolPrefix, Prefix, Fuzzy), simplifying service integration. These changes matter for hybrid stacks where aggregations and filters sit alongside vector/lexical stages. (opensearch.org)
  • Recent developments or changes
    • Available for download now; check the blog and release notes for details before planning upgrades or A/Bs in managed environments. (opensearch.org)

Weaviate December updates: 1.34.5 stability; 1.35.0‑rc.0 previews operational modes and TTL

  • Key facts and current state of the topic
    • Weaviate 1.34 made ACORN the default filtered‑ANN strategy and added Flat‑index Rotational Quantization and server‑side dynamic batching (preview). On Dec 11, 1.34.5 shipped a stability patch; on Dec 12, 1.35.0‑rc.0 introduced “Operational Modes” and preview “Object TTL.” (weaviate.io)
  • Important context and background information
    • For multi‑vector/late‑interaction or heavy‑filter workloads, ACORN‑by‑default plus quantization controls reduce memory and stabilize latency; the 1.34.5 fixes address lock contention and tombstone cleanup—relevant to p95/p99. Operational Modes in 1.35 RC can simplify environment‑specific tuning; TTL can bound storage/refresh costs in feed‑heavy corpora. (github.com)
  • Recent developments or changes
    • Apply 1.34.5 in production clusters; pilot 1.35.0‑rc.0 in non‑prod to validate mode/TTL impacts alongside batching and SPFresh indexing from 1.34. (github.com)

Theory: when high‑dim retrieval is stable (and when it isn’t)

  • Key facts and current state of the topic
    • A Dec 13 arXiv paper frames practical vector retrieval through “stability” (small query perturbations → similar neighbors), extending results to three settings: multi‑vector, filtered vector search, and sparse search. (arxiv.org)
  • Important context and background information
    • Results suggest Chamfer‑style late‑interaction (ColBERT‑like MaxSim) preserves single‑vector stability, while average‑pooling can break it; sufficiently strong penalties for filter violations can induce stability in filtered search; and the authors derive sufficient conditions for stability in sparse retrieval. This provides actionable guidance for model/scoring choices and filter coupling. (arxiv.org)
  • Recent developments or changes
    • Consider stability‑oriented scoring (e.g., MaxSim/Chamfer) for multi‑vector stages and explicit penalty design in filtered‑ANN to reduce recall/latency variance under distribution shift. (arxiv.org)

Exqutor: cardinality‑aware optimization for vector‑augmented analytical queries

  • Key facts and current state of the topic
    • New (Dec 10) optimizer proposes accurate cardinality estimation for queries that blend vector search with relational analytics; integrated into pgvector, VBASE, and DuckDB prototypes. (arxiv.org)
  • Important context and background information
    • Hybrid pipelines often mis‑estimate vector operator selectivity, yielding suboptimal join orders or excessive rescans; Exqutor uses index‑aware estimation (HNSW/IVF) and adaptive sampling when indexes are absent. (arxiv.org)
  • Recent developments or changes
    • Reported speedups up to four orders of magnitude on vector‑augmented analytical queries. If you run hybrid (vector + SQL) ranking/analytics, evaluate Exqutor‑style estimation to improve plan stability and cost. (arxiv.org)

If you were expecting more: beyond these items, we did not find additional retrieval‑relevant releases or papers between December 8–18, 2025 that meet the bar for inclusion.