intermediate9 min readUpdated 2026-06-08

Vector Databases

Vector databases store and query high-dimensional vector embeddings using approximate nearest neighbor (ANN) search, enabling semantic similarity search.

Quick answer: Vector Databases

Vector Databases is covered here as a practical system design concept. Vector databases store and query high-dimensional vector embeddings using approximate nearest neighbor (ANN) search, enabling semantic similarity search. Use it to understand the Databases idea, where it appears in production systems, and how to discuss it clearly in interviews.

Focus on the problem being solved, the main design choice, the failure mode, and the tradeoff you would defend in front of a senior engineer. This page is part of the DosiBridge system design knowledge graph, so it links related concepts, interview problems, tradeoffs, and references instead of leaving the topic isolated.

Aspect	Details
What it is	A database that stores high-dimensional vectors (embeddings) and provides fast approximate nearest neighbor search for semantic similarity queries
When to use	RAG for LLM applications, semantic search, recommendation systems, image similarity, anomaly detection, and any use case requiring meaning-based retrieval
When NOT to use	Exact-match lookups, transactional data, structured filtering without a vector component, or small datasets where brute-force distance computation is fast enough
Real-world example	Spotify uses vector similarity search to find songs that sound similar based on audio embeddings, powering 'Songs Like This' and Discover Weekly recommendations
Interview tip	Explain the full pipeline: data → embedding model → vectors → indexed in vector DB → query embedding → ANN search → top-K results. Interviewers want to see you understand the end-to-end flow
Common mistake	Assuming vector search always returns relevant results — the quality depends entirely on the embedding model; a bad model produces bad vectors, and no index can fix that
Key tradeoff	ANN algorithms trade perfect accuracy (finding the true nearest neighbors) for speed — typical recall is 95-99%, meaning 1-5% of true nearest neighbors may be missed

Why This Matters

Vector databases are critical because modern AI applications represent data as embeddings — dense numerical vectors where semantic similarity maps to geometric proximity. An embedding model converts text, images, or audio into vectors, and the fundamental operation is finding the K most similar vectors to a query. Brute-force search is O(N*D) per query (N vectors, D dimensions), which is unusable at scale. Vector databases use approximate nearest neighbor (ANN) indexes to make this sub-linear, enabling semantic search across billions of vectors in milliseconds. With the explosion of LLM-powered applications, vector databases have become essential infrastructure for RAG, where they retrieve relevant context to ground LLM responses in factual data.

System architecture diagram for Vector Databases showing how services, databases, and caches connect — System architecture for Vector Databases

The Building Blocks

Embeddings: Numerical vector representations (typically 384-1536 dimensions) generated by models like OpenAI's text-embedding-ada-002 or sentence-transformers, where similar concepts have nearby vectors.
ANN Indexes: Approximate nearest neighbor index structures like HNSW (hierarchical navigable small world graphs) and IVF (inverted file indexes) enable sub-linear similarity search at the cost of slight recall loss.
Distance Metrics: Similarity is measured by cosine similarity (angle between vectors), Euclidean distance (L2 norm), or dot product. The choice depends on how embeddings were trained.
Metadata Filtering: Vector databases support hybrid queries — combining vector similarity with metadata filters (e.g., 'find similar documents where category = finance and date > 2024') for precise retrieval.
HNSW Graphs: The most popular ANN structure: a multi-layer graph where each layer is a navigable small-world network. Search starts at the top (sparse) layer and greedily descends to find nearest neighbors.

Under the Hood

When a vector is inserted, it's added to an ANN index that enables fast similarity search. HNSW (Hierarchical Navigable Small World) is the most widely used index. It builds a multi-layer graph where each vector is a node with edges to its nearest neighbors. The top layer is sparse (few nodes, long-range connections for coarse navigation), and each successive layer is denser. Inserting a vector involves finding its nearest neighbors in each layer via greedy traversal and adding bidirectional edges.

Step-by-step diagram showing how Vector Databases processes a request from start to finish — How Vector Databases works step by step

At query time, search begins at the top layer's entry point and greedily moves to the node nearest to the query vector. It drops to the next layer using that node as the entry point and repeats, refining the search. At the bottom layer (containing all vectors), it explores a neighborhood of ef_search candidates and returns the top-K. This achieves O(log N) search complexity with high recall (typically 95-99%), compared to brute-force O(N).

IVF (Inverted File) indexes take a different approach: they partition the vector space into clusters using k-means, then at query time only search the nearest nprobe clusters. Product quantization (PQ) compresses vectors by splitting each into subvectors and quantizing them to codebook entries, reducing memory by 4-64x at the cost of some accuracy. In practice, many vector databases combine IVF for coarse search with PQ for compression and HNSW for fine-grained search within partitions, balancing speed, memory, and recall.

How Companies Actually Do This

Spotify Encodes songs as audio embedding vectors and uses approximate nearest neighbor search to power similarity-based recommendations, finding songs that 'sound like' a user's favorites based on audio features, not just metadata.

Comparison table for Vector Databases contrasting approaches, tradeoffs, and when to use each — Comparing key aspects of Vector Databases

Pinterest Uses vector embeddings for visual search — when a user clicks 'More like this' on a pin, Pinterest encodes the image as a vector and searches billions of pin embeddings to find visually similar content in real-time.

Notion Uses vector search for their AI Q&A feature, embedding workspace documents into vectors and using RAG to retrieve relevant context when users ask questions, enabling answers grounded in their actual documents.

Common Pitfalls

Embedding model mismatch — using an embedding model trained on general English text to embed code or medical documents produces poor vectors; choose a model trained on data similar to your domain
Ignoring recall vs latency tradeoff when tuning ANN parameters — setting HNSW's ef_search too low makes queries fast but misses relevant results; too high gives perfect recall but defeats the purpose of ANN
Not updating embeddings when the model changes — if you retrain or switch embedding models, all existing vectors must be re-embedded; mixing vectors from different models in the same index produces garbage results

Data flow diagram for Vector Databases showing how requests and responses move through the system — Data flow through Vector Databases

Interview Questions Worth Practicing

How would you design a RAG system that retrieves relevant context for an LLM from 10 million company documents?
Explain how HNSW works and why it achieves sub-linear query time for nearest neighbor search.
How would you handle a scenario where vector search returns semantically similar but factually irrelevant results?

The Tradeoffs

Recall vs Speed: ANN indexes sacrifice perfect recall (finding the absolute nearest neighbors) for dramatically faster queries — typical configurations achieve 95-99% recall at 100x the speed of brute-force
Memory vs Accuracy: Quantization compresses vectors 4-64x to fit more in RAM but introduces approximation error that reduces search quality
Generality vs Relevance: Generic embedding models work across domains but domain-specific models produce much better search results for specialized content like medical, legal, or code search

Component diagram for Vector Databases showing each building block and its responsibility — Key components of Vector Databases

How to Explain This in an Interview

Here is how I would explain Vector Databases in a system design interview:

Start with the core idea: modern AI represents data as high-dimensional vectors (embeddings) where semantic similarity corresponds to geometric proximity. The fundamental operation is finding the K vectors most similar to a query vector. Brute-force comparison against every vector is O(N), which is impractical at scale. Vector databases solve this with ANN indexes — HNSW builds a multi-layer navigable graph achieving O(log N) search, while IVF partitions the space into clusters and only searches nearby ones. Walk through the RAG pipeline: documents are chunked, each chunk is embedded into a vector, vectors are indexed. At query time, the user's question is embedded, the ANN index finds the most similar document chunks, and those chunks are passed as context to the LLM. Mention the key tuning decisions: embedding model choice (determines vector quality), distance metric (cosine for normalized embeddings), and HNSW parameters (ef_construction for index quality, ef_search for query recall vs speed). The biggest pitfall is a bad embedding model — no index can compensate for poor vectors.

Interview preparation checklist for Vector Databases with key points to mention and mistakes to avoid — Interview tips for Vector Databases

Original DosiBridge notes for Vector Databases

This section is written specifically for this page, not copied from a general system-design checklist. The useful way to study Vector Databases is to connect the concept to a data path where correctness, access pattern, and future migrations matter more than benchmark numbers. That framing changes the design discussion from definitions to operating choices.

What makes this topic distinct

Query Shape: For Vector Databases, decide the query shape before choosing tools; otherwise the design looks clean but fails when vector changes.
Write Amplification: For Vector Databases, decide the write amplification before choosing tools; otherwise the design looks clean but fails when databases changes.
Replication Lag: For Vector Databases, decide the replication lag before choosing tools; otherwise the design looks clean but fails when vector changes.
Migration Safety: For Vector Databases, decide the migration safety before choosing tools; otherwise the design looks clean but fails when databases changes.
Data Ownership: For Vector Databases, decide the data ownership before choosing tools; otherwise the design looks clean but fails when vector changes.

Production review question

If a team adopts Vector Databases, what observable signal proves it helped instead of only making the architecture diagram more fashionable?

Practical metric to track

Track vector-databases health as a named dashboard or review item: combine one user-facing metric, one saturation metric, and one correctness metric. For Vector Databases, this keeps the discussion grounded in behavior that can be measured after deployment instead of generic advice that sounds good in an interview but never gets verified.