Add Semantic Search to Any App Using Embeddings

Keyword search has a fundamental problem: it matches words, not meaning. A user searching "how to cancel my subscription" won't find your article titled "ending your plan" unless those exact words overlap. Semantic search understands that those two phrases mean the same thing.

The underlying technology is embeddings — numerical representations of text that capture meaning. Similar text produces similar vectors. Searching becomes a geometry problem: find the vectors closest to the query vector.

This post walks through a complete implementation you can drop into any existing app.

The architecture

User query → Embed query → Vector similarity search → Ranked results
                                    ↑
                          Pre-embedded documents
                          stored in your DB

You pre-compute embeddings for all your content once (and re-run whenever content changes). At search time, you embed the query and find the closest matches. That's the whole thing.

Step 1: Choose your embedding model

For most applications, OpenAI's text-embedding-3-small is the right call:

• 1536 dimensions (good accuracy)
• $0.02 per million tokens (cheap)
• Fast API, no infrastructure to manage

For production at scale or if you need to keep data on-prem, look at sentence-transformers/all-MiniLM-L6-v2 — runs locally, still very good.

Step 2: Set up pgvector

If you're already using PostgreSQL (you probably should be), pgvector adds vector storage and similarity search natively. No separate vector database needed.

-- Enable the extension
CREATE EXTENSION IF NOT EXISTS vector;

-- Add a vector column to your existing table
ALTER TABLE articles ADD COLUMN embedding vector(1536);

-- Create an index for fast similarity search
CREATE INDEX articles_embedding_idx 
ON articles USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 100);

The ivfflat index makes similarity search fast at scale. For under ~100k rows you can skip it and use exact search instead.

Step 3: Embed your content

import OpenAI from 'openai'
import { Pool } from 'pg'

const openai = new OpenAI()
const db = new Pool({ connectionString: process.env.DATABASE_URL })

async function embedText(text: string): Promise<number[]> {
  const response = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: text.slice(0, 8000), // token limit safety
  })
  return response.data[0].embedding
}

async function embedAllArticles() {
  const { rows } = await db.query(
    'SELECT id, title, content FROM articles WHERE embedding IS NULL'
  )

  console.log(`Embedding ${rows.length} articles...`)

  for (const article of rows) {
    // Combine title + content for richer embedding
    const text = `${article.title}\n\n${article.content}`
    const embedding = await embedText(text)

    await db.query(
      'UPDATE articles SET embedding = $1 WHERE id = $2',
      [`[${embedding.join(',')}]`, article.id]
    )

    // Rate limit: OpenAI allows ~3000 RPM on embeddings
    await new Promise(r => setTimeout(r, 20))
  }

  console.log('Done.')
}

Run this once to backfill, then call embedText and update the row whenever content is created or updated.

Step 4: The search function

async function semanticSearch(query: string, limit = 10) {
  // Embed the query using the same model
  const queryEmbedding = await embedText(query)

  // Find the closest articles by cosine similarity
  const { rows } = await db.query(`
    SELECT 
      id,
      title,
      excerpt,
      1 - (embedding <=> $1::vector) AS similarity
    FROM articles
    WHERE embedding IS NOT NULL
    ORDER BY embedding <=> $1::vector
    LIMIT $2
  `, [`[${queryEmbedding.join(',')}]`, limit])

  return rows
}

// Usage
const results = await semanticSearch("how do I cancel my account")
// Returns articles about cancellation, ending subscriptions, etc.
// even if they don't contain those exact words

The <=> operator is cosine distance in pgvector. Lower distance = more similar. We subtract from 1 to get a similarity score (1.0 = identical, 0.0 = unrelated).

Step 5: Add a search API endpoint

import express from 'express'
const app = express()

app.get('/api/search', async (req, res) => {
  const query = req.query.q as string
  
  if (!query || query.length < 2) {
    return res.json({ results: [] })
  }

  try {
    const results = await semanticSearch(query, 8)
    res.json({ results })
  } catch (err) {
    console.error('Search error:', err)
    res.status(500).json({ error: 'Search failed' })
  }
})

Step 6: Hybrid search (the production upgrade)

Pure semantic search can miss exact matches. A user searching for a product SKU like "INV-2024-003" wants exact match, not semantic similarity. The solution is hybrid search: run both and merge the results.

async function hybridSearch(query: string, limit = 10) {
  const queryEmbedding = await embedText(query)

  const { rows } = await db.query(`
    WITH semantic AS (
      SELECT id, 1 - (embedding <=> $1::vector) AS score
      FROM articles
      WHERE embedding IS NOT NULL
      ORDER BY embedding <=> $1::vector
      LIMIT 20
    ),
    keyword AS (
      SELECT id, ts_rank(search_vector, plainto_tsquery('english', $2)) AS score
      FROM articles
      WHERE search_vector @@ plainto_tsquery('english', $2)
      LIMIT 20
    )
    SELECT 
      a.id, a.title, a.excerpt,
      COALESCE(s.score * 0.7, 0) + COALESCE(k.score * 0.3, 0) AS combined_score
    FROM articles a
    LEFT JOIN semantic s ON a.id = s.id
    LEFT JOIN keyword k ON a.id = k.id
    WHERE s.id IS NOT NULL OR k.id IS NOT NULL
    ORDER BY combined_score DESC
    LIMIT $3
  `, [`[${queryEmbedding.join(',')}]`, query, limit])

  return rows
}

This weights semantic results at 70% and keyword results at 30%. Tune the weights for your use case.

Performance notes

• Embedding API calls take ~100–300ms. Cache results aggressively (query → embedding pairs don't change).
• The ivfflat index makes similarity search sub-10ms on millions of rows.
• For real-time search (as-you-type), debounce the query by 300ms and show results when the user pauses.

When to use it

Semantic search is worth adding when:

• Users search in natural language ("best way to export data")
• Your content uses different terminology than users do
• You have more than ~500 documents
• Keyword search produces too many zero-result searches

It's probably overkill when:

• Users are searching for exact IDs, codes, or names
• Your search corpus is tiny
• Search isn't a core part of the product experience

---

Need semantic search in your product? It's usually a 1–2 day addition. Let's talk.