TL;DR
Five notable retrieval updates since your baseline: (1) WARP dramatically speeds up multi‑vector (late‑interaction/XTR) retrieval; (2) ColBERT‑serve slashes RAM via memory‑mapped indexes and adds multi‑stage serving; (3) principled, near‑lossless token pruning cuts late‑interaction index size to ~30% of tokens with minimal quality loss; (4) Lucene 10.x lands sizable HNSW and classic‑sparse speedups (including filtered ANN via ACORN‑1) and hardware‑efficiency gains; (5) learned‑sparse retrieval advances with inference‑free Li‑LSR and LLM‑based CSPLADE, targeting query‑time cost and scale.
WARP: a faster engine for multi‑vector/late‑interaction retrieval
- Key facts and current state of the topic
- Multi‑vector (e.g., ColBERT/XTR) keeps token‑level interactions but historically paid high latency due to decompression and heavy scoring. (arxiv.org, deepmind.google)
- Important context and background information
- PLAID accelerated ColBERTv2 with centroid interaction/pruning; XTR improved token selection to avoid full document vector access, but reference stacks still had overhead. (arxiv.org)
- Recent developments or changes
- WARP introduces dynamic similarity imputation, implicit decompression, and a two‑stage reduction with optimized C++ kernels—reporting 41× lower latency than the XTR reference and ~3× over the official PLAID engine, while preserving quality. If you’re evaluating late‑interaction at ad‑scale, WARP is a prime candidate for head‑to‑head latency/throughput tests. (arxiv.org)
ColBERT‑serve: memory‑mapped, multi‑stage serving for late‑interaction
- Key facts and current state of the topic
- Serving late‑interaction to many concurrent users is memory‑intensive; indexes often don’t fit in RAM on cost‑efficient servers. (arxiv.org)
- Important context and background information
- Prior work (PLAID) optimized scoring; productionization still needed better memory behavior and concurrency. (arxiv.org)
- Recent developments or changes
- ColBERT‑serve applies memory‑mapping to late‑interaction indexes (reporting ~90% RAM reduction) and adds a multi‑stage, hybrid‑scoring architecture to cut latency under high concurrency—making it more deployable on commodity instances. (arxiv.org)
Toward lossless token pruning in late‑interaction
- Key facts and current state of the topic
- Index footprint is a key blocker for wide deployment of token‑level (multi‑vector) retrieval. Heuristic/statistical token pruning existed but could hurt scoring. (arxiv.org)
- Important context and background information
- ColBERT/PLAID/SPLATE families trade memory for accuracy; principled pruning that preserves scores has been missing. (arxiv.org)
- Recent developments or changes
- New work formalizes conditions for pruning without changing query–document scores and introduces regularization losses plus pruning strategies that keep effectiveness while using only ~30% of tokens—promising major index‑size cuts for late‑interaction stacks. (arxiv.org)
Lucene 10.x: big efficiency wins for both vector and classic sparse search
- Key facts and current state of the topic
- Lucene underpins Elasticsearch/OpenSearch; its HNSW kNN and classic postings performance directly impact large‑scale retrieval stacks. (elastic.co)
- Important context and background information
- Lucene 10 raised the Java baseline (JDK 21) and moved closer to hardware with foreign memory/FFI and better vectorization. (elastic.co)
- Recent developments or changes
- ACORN‑1–style “filtered HNSW fast mode” yields up to ~5× speedups for kNN with metadata filters; early‑termination and HNSW‑build memory reductions landed in 10.2.x; classic sparse got faster via bit‑set encoding of dense postings blocks—useful for hybrid pipelines and high‑QPS ad retrieval requiring filters. (elastic.co, lucene.apache.org)
Learned‑sparse retrieval: inference‑free queries and LLM backbones
- Key facts and current state of the topic
- LSR (e.g., SPLADE) keeps inverted‑index efficiency but query encoding can dominate latency; recent works explore LLM backbones and making query time “lookup‑only.” (arxiv.org)
- Important context and background information
- Inference‑free variants aim to pre‑compute token scores to avoid online transformer passes; others scale LSR with decoder‑only/encoder‑decoder LMs. (web3.arxiv.org)
- Recent developments or changes
- Li‑LSR relaxes regularization and learns per‑token scores so query encoding becomes a table lookup, reporting +1 mRR@10 on MS MARCO and +1.8 nDCG@10 on BEIR over strong baselines (e.g., SPLADE‑v3‑Doc), directly attacking query‑time cost. Complementary 2025 works (CSPLADE; decoder/encoder‑decoder analyses) probe large‑LM backbones for LSR, offering new quality/efficiency trade‑offs. Consider Li‑LSR‑style indexing when latency budgets are tight and traffic is spiky. (arxiv.org, arxiv.org)