Embeddings for product search, without the hype

The first time I replaced keyword search with embeddings, relevance went down immediately. Users searching for an exact product SKU or model number like A2337 got poetic, semantically adjacent nonsense like sleek silver laptops instead of the exact replacement battery they needed. Vector search alone lacks precision. Lexical search alone lacks semantic understanding. Hybrid search is the pragmatic middle ground.

Why hybrid beats pure vectors

Lexical search (BM25 or Postgres Full-Text) is precise and literal. Vector search (using cosine distance or inner product) is fuzzy, conceptual, and forgiving of typos or synonyms. Real e-commerce and product search queries are almost always a mix of both intent types.

To get the best of both worlds, score both lexical relevance and embedding similarity, then blend them using reciprocal rank fusion (RRF) or weighted linear combinations directly in Postgres via pgvector:

select
  id,
  title,
  (0.6 * (1 - (embedding <=> $1))) +
  (0.4 * ts_rank(search_vector, plainto_tsquery($2))) as score
from documents
where search_vector @@ plainto_tsquery($2)
   or (embedding <=> $1) < 0.35
order by score desc
limit 20;

By filtering out extreme outliers early and weighting vector closeness slightly higher than lexical matches, you ensure that exact SKU queries still hit while conceptual queries like "quiet mechanical keyboard for coding" return accurate products.

Chunking matters more than the model

Developers often spend weeks benchmarking OpenAI, Cohere, or local HuggingFace embedding models, hoping for a magic relevance jump. In practice, how you split your document chunks impacts accuracy far more than model choice.

• Chunk on semantic boundaries — Split text at section headings or logical component boundaries rather than raw character/token counts.
• Maintain overlap windows — Keep a 10–15% overlap between adjacent chunks so critical context isn't sliced right down the middle.
• Inject metadata into chunks — Store the parent title, category path, and heading hierarchy alongside the chunk vector. This doubles as an inline citation and enriches vector density.

Swapping the embedding model gives you a few accuracy points. Fixing your chunking strategy gives you tens.

Evaluate with 40 real queries

Stop relying on generic synthetic benchmarks like MTEB. Your domain has specific vocabulary, acronyms, and user query quirks that general benchmarks will never capture.

You don't need a complex evaluation suite. Pull 40 real queries from your production analytics logs, hand-label the ideal top 3 product results, and compute Recall@5 and MRR (Mean Reciprocal Rank) after every indexing change. It takes single afternoon to set up and provides the only ground-truth metric that actually impacts user conversion.