What Are Embeddings? How Web Teams Use Them for Search, RAG and Cleanup

Embeddings are lists of numbers that stand for the meaning of a piece of text, so a computer can tell which pages, questions or records are about the same thing. Two passages with similar meaning get similar numbers, even when they share no words. That one trick sits under semantic site search, "related posts" widgets, RAG chatbots and a lot of quiet data cleanup.

Our guide to what a large language model is mentions embeddings only in passing, as the search step behind RAG. This piece goes deeper on that one question. It covers what embeddings are, how similarity is measured without heavy math, five jobs web teams use them for, the main models and prices in one dated table, where the vectors live and what it all costs.

The short version: an embedding model turns text into a vector, a fixed-length list of numbers. You embed your content once, store the vectors, then embed each new query and look for the nearest ones. Embedding is cheap. A whole site often costs pennies. The real decisions are which model to pick, where to store the vectors and how to keep them current.

What embeddings are, in plain terms

OpenAI’s embeddings guide puts it simply: "An embedding is a vector (list) of floating point numbers." Anthropic’s embeddings page adds the purpose. Text embeddings "enable measuring semantic similarity."

Think of each number as a position on one hidden scale that the model learned in training. A real model uses hundreds or thousands of these scales at once.

Every embedding from a given model has the same length. OpenAI’s text-embedding-3-small returns 1,536 numbers. Google’s Gemini embedding models return 3,072 by default. A tweet and a 10-page policy both come back as a list of that fixed length.

That fixed length lets you compare any two pieces of text by comparing their lists. A member’s question about "renewing my dues" lands near your page titled "Membership renewal," because the model learned that those phrases mean nearly the same thing.

An embedding model does not write text. It reads text and returns numbers, and it is much cheaper to run than a chat model.

Where embeddings came from: word2vec

The idea took off with a 2013 paper by Tomas Mikolov, Kai Chen, Greg Corrado and Jeffrey Dean, Efficient Estimation of Word Representations in Vector Space. It introduced the method later known as word2vec. The authors proposed two model designs, CBOW and Skip-gram, for learning word vectors from very large text collections at low cost.

The paper’s best-known example is word math. The authors report that vector("King") minus vector("Man") plus vector("Woman") "results in a vector that is closest to the vector representation of the word Queen." Relations such as a country and its capital showed up as consistent directions in the space.

Word2vec gave one vector per word. Modern models read whole passages and handle context, so "bank" in a finance FAQ and "bank" in a river guide end up in different places. The core idea is the same: meaning becomes position, and close positions mean similar meaning.

How similarity works: cosine, without the math

Once text is a list of numbers, you need a way to say how close two lists are. The usual answer is cosine similarity.

Picture each embedding as an arrow pointing from the center of the space. Cosine similarity asks one question: do two arrows point the same way? Arrows pointing the same way score near 1. Unrelated arrows score lower. The length of the arrow does not matter, only its direction.

OpenAI’s guide uses cosine similarity in its search example. It also states the rule of thumb: "Small distances suggest high relatedness and large distances suggest low relatedness."

Two practical notes:

• Normalized vectors make the math cheaper. Anthropic’s page notes that Voyage AI embeddings "are normalized to length 1, so dot-product and cosine similarity are the same." The pgvector README says the same of OpenAI embeddings and suggests inner product for speed.
• Scores are relative. A similarity of 0.8 means nothing on its own. What matters is ranking, and any duplicate threshold has to be tuned on your own data.

What web teams use embeddings for

Most uses follow one pattern: embed your content, embed the thing you are comparing, find the nearest matches.

Site search. Keyword search misses a visitor who types "how do I get CEUs" when your page says "continuing education credits." Semantic search with embeddings finds it. Hosted platforms cover this too; our What Is Algolia? post describes one.

Related content. Embed every post once and list each post’s five nearest neighbors. That gives a "related reading" block based on meaning instead of shared tags, and it updates when you add posts.

RAG. Retrieval-augmented generation uses embeddings to find the passages a chat model should read before it answers. The quality of the answer depends heavily on this search step. Bad retrieval means answers from the wrong source.

Deduplicating records. Association databases collect near-duplicates: "Natl. Council of Teachers," "National Council of Teachers," a typo in a school name. Embedding names, titles or descriptions and flagging very close pairs gives staff a short review list. A person still decides what to merge.

Clustering member feedback. Embed 3,000 open-ended survey answers, then group the vectors that sit close together. You get piles such as "event pricing," "app login trouble" and "more webinars." OpenAI’s guide lists clustering, classification and anomaly detection among common uses.

Embedding models and prices, as of October 2026

This is the one section that will date. Figures come from vendor pages checked on October 21, 2026. Prices are per million input tokens. Embedding models charge only for input.

Sources: OpenAI’s model pages for text-embedding-3-small and text-embedding-3-large; Google’s Gemini embeddings docs and Vertex AI pricing; and MongoDB’s Voyage AI billing page.

Anthropic. Anthropic’s docs say plainly: "Anthropic does not offer its own embedding model." The page names Voyage AI by MongoDB as one provider and walks through Voyage’s models. It also tells readers to assess a variety of embeddings vendors. Claude projects pair Claude with another vendor’s embeddings.

Google. The Gemini docs call gemini-embedding-2 "the first multimodal embedding model in the Gemini API," taking images, audio, video and PDFs as well as text. The page says the Batch API runs at 50% of the default embedding price.

Where the vectors live: pgvector or a vector database

Once you have vectors, you need somewhere to store and search them. There are two common routes.

pgvector in Postgres. pgvector describes itself as "Open-source vector similarity search for Postgres." It adds a vector column type and distance operators, including <=> for cosine distance. As of October 2026 its README shows version 0.8.7. If a site already runs on Postgres or Supabase, this keeps vectors next to the content they describe. Our pgvector on Supabase walkthrough builds a full RAG pipeline this way.

A dedicated vector database. Products built only for vector search add managed scaling, filtering and hybrid search. Our What Are Vector Databases? explainer covers the category and when it earns its cost.

One pgvector detail trips up many builds. The README says vectors can have up to 16,000 dimensions, but HNSW and IVFFlat indexes on the standard type stop at 2,000. A 3,072-dimension embedding from text-embedding-3-large or Gemini will store but will not index as-is. The fixes are to shorten the embedding, which OpenAI and Google both support, or to use half-precision vectors, which index up to 4,000 dimensions.

For most nonprofit and association sites, pgvector is enough.

What embeddings cost in practice

The embedding bill is usually the smallest line. A 2,000-page site at about 1,000 tokens a page is 2 million tokens. At $0.02 per million, the whole site costs about four cents to embed. At $0.13, about 26 cents. Queries are short, so embedding each search adds almost nothing.

The other costs are larger:

• Storage. pgvector stores each standard vector in 4 bytes per dimension plus 8. A 1,536-dimension vector is about 6 KB. 100,000 chunks come to roughly 600 MB before indexes. Half-precision cuts that in half.
• Re-embedding. Vectors from different models cannot be compared. Switching models means embedding everything again. Plan for it when a vendor retires a model.
• The chat model in RAG. Each answer sends retrieved passages to a chat model, billed per token. That call costs far more than the embedding step. Our pillar on what AI actually costs covers that side.

Limits to plan for

Embeddings capture meaning, not exact facts. They can rank a page about "2025 dues" close to a query about "2026 dues." Pair semantic search with filters for dates, content types and member-only status.

They also miss exact strings. Part numbers, order IDs and acronyms often match better with keyword search. That is why hybrid search, keyword plus vector, is common.

Long documents need splitting. One vector for a 40-page PDF blurs its topics together, so most RAG builds embed passages of a few hundred tokens.

Some models expect hints. Voyage asks you to mark text as a document or a query, and Gemini’s older model takes a task type.

Finally, embedding text sends it to the vendor, under the same data terms as any API call.

Frequently Asked Questions