Adding Semantic Search to Drupal 11 with Solr

When we search for something online, we usually type in keywords and hope the system finds an exact or close match. But modern search is evolving beyond keyword matching, as it looks deeper into semantic meaning and context. That’s where text embeddings, or vectors, come in. A vector is a numerical representation of a piece of text, capturing not just the words themselves but their relationships and semantic meaning. Using models like Bidirectional Encoder Representations from Transformers (BERT) or OpenAI’s embedding APIs, you can turn a sentence into a dense vector: a list of numbers that encode its meaning in a multi-dimensional space.

Many services specialize in vector storage. Pinecone, Milvus, and Qdrant are some leaders in the space, while Postgres, MySQL, and SQLite databases can also store vectors (albeit through plugins or extensions currently). Search engine technologies like Solr, TypeSense, Meilisearch, and Algolia can now do it too. Since Solr is extremely well supported in Drupal and we use it on most of our projects, I set out to add support for one of Solr’s new vector features.

Solr 9 introduced support for dense vector fields, a major development in its search capabilities. With these, you can store and query vector representations directly in your Solr index. This opens the door to semantic search and enables use cases like Retrieval-Augmented Generation (RAG), where relevant information is pulled based on meaning rather than strict keyword overlap. For Drupal 11 sites, this means that instead of just returning content that contains the words “climate change,” Solr can find content that’s about climate change even if the content uses different language to describe it. 

It’s in this area that vectors really shine. Where older search implementations relied on “exact matches” or “close to exact matches” for relevancy scoring, vectors allow users to enter more natural language searches and receive similar or more semantic results based on what they may be trying to search for. If you think about the first case, you must have a good idea of what you want, whereas with the latter, you can get as good or better results. This should help users find what they’re looking for, or perhaps something they didn’t know they were looking for but has now surfaced in this type of search.

Solr Dense Vector Module for Drupal 11

To make this next-generation search capability accessible within Drupal CMS, I created the Search API Solr Dense Vector module. It extends the powerful Search API and Solr ecosystem by allowing Drupal sites to index and query dense vector fields directly. With it, you can generate vector embeddings from your content using any embedding provider you like and send them to Solr 9+ for storage and similarity search. The result is a seamless integration that brings vector-based, semantically aware search to Drupal 11. It enables smarter content discovery and even RAG-style question answering when paired with a large language model (LLM). 

The backbone of the AI support is provided by the Drupal AI module framework for generating the embedding values. If you install an AI provider (like OpenAI) that supports generating text embeddings, you’ll be able to use it on the search index of your choosing. The size of the vector will be inferred from the provider, as most services will return vectors in various sizes depending on the model used. The UI allows you to select the provider and model.

At this point, you can add a field to your search index to store the vector. Any text-based field will be sufficient. For my example, I chose an aggregated field of title and body, concatenated into a single value for evaluation. When indexing, the module generates embeddings and stores them in the Solr documents. When a user initiates a query, their input is converted into an embedding vector and then compared in Solr, returning a list of items that best meet their search.

Scaling Dense Vector Search with Solr in Drupal

This is a starting point, but it’s exciting to bring into Drupal as either a low-cost alternative to commercial enterprise search platforms or as a means of adding on to existing Solr implementations for sites to enhance searches and search-based workflows in a modern way. Additional features can be added on top, such as summarizing result sets with AI (like Google) or connecting with chatbots for support, enhancing admin management of content, or other possible ideas. We’re looking forward to deploying this implementation on projects this year and seeing how the search improvements impact users. 

Ready to bring smarter, AI-powered search to your Drupal site? Velir can help you implement dense vector search with Solr in Drupal 11 to deliver more meaningful, intuitive search experiences for your users. Contact us today to explore how we can modernize your site’s search capabilities.