A vector database is a store for embeddings that returns the most similar stored entries for a query. It solves a task classic database indexes are not designed for: searching for proximity in a space with hundreds or thousands of dimensions.
Why a normal index is not enough
A classic index speeds up searches for exact values or ranges. Both presuppose an ordering that does not exist in a high-dimensional vector space: there is no meaningful sort by similarity, because proximity only arises relative to a specific query. Without a specialised index, every query would have to be compared with every stored vector — unusably slow on large corpora.
Approximate search as the norm
Vector databases solve this with approximate methods. They do not guarantee the exactly most similar hits but return very similar ones with high probability — and orders of magnitude faster. That trade-off is controlled by parameters balancing speed against hit quality. For most applications it is unproblematic, because a narrowly missed third-best hit does not matter. Where completeness counts, the trade-off must be set deliberately differently.
Metadata is the real lever
Pure similarity search rarely delivers what is needed. In practice a restriction is almost always added: only this tenant’s documents, only the current version, only content from a certain date. That metadata filtering matters more in many applications than the similarity calculation itself — and it is where systems differ in practice. In multi-tenant applications in particular, filtering by tenant is not a convenience feature but a security requirement.
Standalone database or extension
There are two routes. Specialised vector databases are optimised for this one task and scale correspondingly far. Extensions to existing relational databases bring vector search into a system that is already running. For many applications the second route is the more pragmatic: the vectors then sit next to the business data, metadata filtering is an ordinary condition in the same query, and no second system arrives with its own operations, backups and consistency question.
The consistency question
Where vectors live in a separate system, a synchronisation problem arises: when a document is deleted or changed, the associated vector must be deleted or recomputed. If that does not happen reliably, search returns hits on content that no longer exists — with personal data, a deletion problem rather than merely a quality problem.
Practical consequence
Choosing the vector database is rarely the most important decision in a RAG system. Chunking, the metadata model, and whether vectors and business data stay consistent matter more. Anyone starting small is usually better served by extending the existing database than by adding a second system.