A year ago, “vector database” was niche infrastructure most enterprise data architects had only abstractly heard of. In early 2024, it has become a category most serious enterprise data strategies have a position on. The reason is straightforward: every interesting application of generative AI on internal data — search, retrieval-augmented generation, recommendation, similarity, classification — leans on the same basic operation, fast similarity search over high-dimensional embeddings, and someone has to provide that capability at the scale and reliability the rest of the stack expects.

The interesting question is no longer whether vector databases matter. It’s where they fit in the larger data architecture, and which of several approaches enterprises should actually adopt.

What Vector Databases Are For

Stripped of the marketing, a vector database does one thing well: given a high-dimensional vector and a corpus of pre-indexed vectors, return the most similar ones quickly. Around that core, the category has accumulated features — metadata filtering, hybrid keyword-plus-vector search, namespacing, replication, point-in-time recovery, role-based access — that make it usable as production infrastructure.

The workloads that drive demand are familiar by now: semantic search over documents, RAG for AI assistants, recommendation systems, anomaly detection, image and audio similarity, content moderation. None of these are new problems; what changed is the availability of high-quality embeddings to represent them as vector-similarity problems, and the volume of data being processed in this way.

The Three Architectural Choices

Enterprises in 2024 face a real choice among three credible approaches:

Dedicated vector databases. Pinecone, Weaviate, Qdrant, Milvus, Chroma. Purpose-built systems optimized for vector search, often with the most advanced index types and tuning options. Best fit when vector search is the primary workload, scale is high, or the additional features (hybrid search, multi-tenant isolation, real-time updates at scale) actually matter.

Vector capabilities in existing databases. PostgreSQL with pgvector, Elasticsearch, Redis, MongoDB Atlas Vector Search, Cassandra, and most major cloud-native stores have added vector indexing. The pitch is operational simplicity: keep your data and its embeddings in one place, with one set of operational tooling, one access control story, and one backup strategy. For a wide range of enterprise workloads, this is the right answer.

Vector capabilities in data warehouses. Snowflake, BigQuery, Databricks, and others have all added vector indexing or are in the process. The pitch here is analytical: vectors live alongside the structured data they augment, queries can join across them, and the warehouse’s governance and lineage applies. This is becoming the default for analytics-heavy use cases.

The choice between these isn’t religious. It’s a function of where the data already lives, what the operational team can support, and what the workload actually requires.

Sizing the Decision

A few practical heuristics from teams that have shipped real systems:

For small-to-medium corpora (single-digit millions of vectors) with modest update rates, almost any option works. Choose for operational fit.

For large corpora (tens of millions plus) with frequent updates, the dedicated vector databases tend to win on tuning surface. The performance and cost differences become real at scale.

When the workload is “vector search alongside relational data,” a vector-capable relational database (especially Postgres with pgvector) is often the simplest answer. The integration story beats the small performance gap on most workloads.

When the workload is multi-tenant SaaS with strong tenant isolation requirements, the dedicated vector databases generally have more mature primitives (namespaces, per-tenant filtering, isolation guarantees).

When the workload is “embed and search alongside warehouse data,” the warehouse-native options remove a substantial integration burden.

What Often Gets Missed

A few factors that tend not to surface in early evaluations and become decisive later:

Update semantics. How quickly does a new or updated vector become searchable? Some systems are minutes-fresh; others are seconds. For RAG over rapidly changing content, the freshness gap is the user experience.

Hybrid search quality. Most real workloads need both vector and keyword retrieval. Systems vary substantially in how well they combine them. The reranking approach (retrieve from each, then merge) is often more effective than dense alone, but the implementation quality differs.

Filter performance under cardinality. Filtering vector results by metadata is fast on most systems for simple filters and falls off cliff-style for complex ones. Workloads that combine vector similarity with sophisticated filtering (multi-tenant, time-bounded, permission-aware) need to test this carefully on real data.

Backup, replication, and disaster recovery. Several of the dedicated vector databases have made significant progress on operational maturity, but it’s still uneven. For critical workloads, the operational story matters as much as raw performance.

Cost at scale. The pricing models differ enough that the right answer at 100K vectors and the right answer at 100M vectors can be different products. Re-evaluation is often worthwhile when scale changes by an order of magnitude.

Where the Category Is Going

A few clear directions through 2024:

The “vector database vs. database with vector support” debate is largely resolved by having both win in different segments. Most enterprises will end up with multiple — pgvector for application-tier workloads, a dedicated vector database for high-scale RAG, warehouse vectors for analytics.

The performance gap between dedicated systems and added-vector systems is narrowing. Postgres’ pgvector with HNSW (added in 2023) is a credible option for far more workloads than it was a year before.

Hybrid search is becoming a first-class feature rather than a bolt-on. The systems that don’t ship strong hybrid will lose ground.

Governance and observability are emerging as differentiators. As vector workloads carry sensitive content, lineage, access control, and audit get harder to retrofit and easier to design in.

Conclusion

Vector databases in 2024 are settling into a recognizable layer in the enterprise data architecture rather than a contested frontier. Most enterprises will run multiple vector stores, chosen by workload rather than by central edict, and integrated with their broader data tooling. The strategic question for data leaders is less “which vector database” and more “what’s our pattern for adding vector capabilities where they belong.” Teams that answer this clearly are shipping AI features faster; teams treating it as a new platform decision are still in evaluation.

References

Johnson, J. et al. (2017). Billion-Scale Similarity Search with GPUs. (FAISS).

Malkov, Y. & Yashunin, D. (2018). Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs (HNSW). IEEE TPAMI.

Pinecone (2024). Vector Database Architecture Whitepaper.

PostgreSQL (2024). pgvector Extension Documentation.

Snowflake (2024). Cortex Vector Search: Technical Overview.