# Semantic Vector Search

Use `lakebase_vector` for approximate nearest-neighbor retrieval over embeddings. It retains pgvector's vector types, distance operators, and query syntax; the index access method is `lakebase_ann`.

## Contents

- [Create the extension](#create-the-extension) — enable `lakebase_vector` and its `pgvector` dependency
- [Prepare embeddings](#prepare-embeddings) — generate vectors with the AI Gateway and match the column dimension
- [Build the index](#build-the-index) — match the distance metric, operator class, and query operator
- [Tune the index](#tune-the-index) — configure index-build options and concurrent index management
- [Query](#query) — rank by vector distance or filter by a similarity radius
- [Tune search](#tune-search) — inspect the index and tune recall against latency
  - [Use prefilter selectively](#use-prefilter-selectively) — apply selective filters before ANN scoring

## Create the Extension

Lakebase Search requires Postgres 16 or later. Enable the extension before creating vector columns or indexes:

```sql
CREATE EXTENSION IF NOT EXISTS lakebase_vector CASCADE;
```

`lakebase_vector` installs `pgvector` through `CASCADE`. It relies on a preloaded library that Neon enables by default; if the project customized its preloaded-library list, confirm the library remains enabled.

## Prepare Embeddings

Neon's [AI Gateway embeddings endpoint](https://neon.com/docs/ai-gateway/embeddings) is an OpenAI-compatible way to generate vectors for Lakebase Search. This table uses 1024 dimensions as an example; set the column dimension to the output of the model and any `dimensions` parameter you choose:

```sql
CREATE TABLE documents (
  id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
  title text NOT NULL,
  body text NOT NULL,
  embedding vector(1024)
);
```

Enable the AI Gateway on the branch and get `NEON_AI_GATEWAY_TOKEN` and the bare-host `NEON_AI_GATEWAY_BASE_URL`:

- With `neon.ts`, declare `aiGateway: true`, run `neon deploy`, then run `neon env pull --file .env`.
- Without `neon.ts`, run `neon env pull --service ai-gateway --file .env` from a directory linked to the intended project and branch. This requests the gateway variables explicitly and skips other services.
- Alternatively, get both values from the Neon Console. Neon Functions receive them automatically.

### AI SDK (Recommended)

For TypeScript apps, use Neon's `@neon/ai-sdk-provider` 1.0.0+ with the [Vercel AI SDK](https://ai-sdk.dev) (`ai`, 6+) (`npm install @neon/ai-sdk-provider ai dotenv`). The provider reads `NEON_AI_GATEWAY_TOKEN` and `NEON_AI_GATEWAY_BASE_URL` directly and sends embedding requests to `/v1/embeddings`. Load `.env` in a standalone app; Neon Functions inject the variables automatically.

```typescript
import "dotenv/config";
import { neon } from "@neon/ai-sdk-provider";
import { embed, embedMany } from "ai";

const embeddingModel = neon.embeddingModel("qwen3-embedding-0-6b");
const providerOptions = { neon: { dimensions: 1024 } };
// Qwen3 query prompt adapted for Neon documentation search; documents stay unprefixed.
const queryPrompt =
  "Instruct: Given a question about Neon, retrieve documentation passages that answer the question\nQuery: ";

const { embeddings: documentEmbeddings } = await embedMany({
  model: embeddingModel,
  values: documentBodies, // non-empty document batch; store vectors by input index
  providerOptions,
  maxParallelCalls: 1,
});
const { embedding: queryEmbedding } = await embed({
  model: embeddingModel,
  value: `${queryPrompt}${queryText}`,
  providerOptions,
}); // bind to $1 below
```

Skip empty imports before calling `embedMany`. It returns vectors in input order and splits requests according to the provider's item limit. For large imports, also split document text within the endpoint's token limits and keep database writes bounded.

For Neon documentation search, the example adapts the task instruction in Qwen3's stored `query` prompt from [config_sentence_transformers.json](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B/blob/main/config_sentence_transformers.json). Its `document` prompt is empty, so document text stays unprefixed. Neither SDK applies SentenceTransformers prompts automatically. Keep the original `queryText` for BM25, and use the selected model's own prompt conventions if you switch models.

### OpenAI SDK Alternative

For an existing OpenAI SDK integration, install `openai` and `dotenv` (`npm install openai dotenv`) and configure the client explicitly:

```typescript
import "dotenv/config";
import OpenAI from "openai";

const apiKey = process.env.NEON_AI_GATEWAY_TOKEN;
const gatewayBaseUrl = process.env.NEON_AI_GATEWAY_BASE_URL;
if (!apiKey || !gatewayBaseUrl) {
  throw new Error("Missing Neon AI Gateway credentials");
}

const client = new OpenAI({
  apiKey,
  baseURL: `${gatewayBaseUrl}/v1`,
});

async function embedBatch(inputs: string[]): Promise<number[][]> {
  if (inputs.length === 0) return [];
  const response = await client.embeddings.create({
    model: "qwen3-embedding-0-6b",
    input: inputs,
    dimensions: 1024,
    encoding_format: "float",
  });
  const vectors: number[][] = [];
  for (const item of response.data) vectors[item.index] = item.embedding;
  return inputs.map((_, index) => {
    const embedding = vectors[index];
    if (!embedding) throw new Error(`Missing embedding for input ${index}`);
    return embedding;
  });
}

const documentEmbeddings = await embedBatch(documentBodies);
const [queryEmbedding] = await embedBatch([`${queryPrompt}${queryText}`]);
if (!queryEmbedding) throw new Error("Missing query embedding");
```

Use the same `queryPrompt` defined above, only for query inputs. Keep each batch within the endpoint's input and rate limits, and pair each vector with its source row using the response `index`. The [OpenAI JavaScript SDK v6](https://github.com/openai/openai-node/blob/v6.0.0/src/resources/embeddings.ts) defaults to requesting base64 and decodes the response as base64. If a compatible endpoint returns float arrays instead, that decoder can silently turn them into zeros. Pass `encoding_format: "float"` explicitly for the AI Gateway; the SDK then returns the float arrays directly. Base64 can reduce response size when an endpoint actually returns base64, but do not assume that format for this gateway.

Use the same model and output dimensions for stored documents and queries. Follow the model's input conventions: some models recommend a query instruction while document text remains unprefixed. If the model supports shorter vectors, compare retrieval quality, storage, and latency before changing dimensions; set the column and both embedding requests to the chosen size. Switching models requires re-embedding the corpus; changing dimensions also requires migrating the vector column and index. Generate embeddings in the application, outside SQL, unless the architecture already provides an in-database embedding function. Other embedding providers are also valid when their output matches the schema and index.

## Build the Index

Choose the operator class and query operator as a matched pair:

| Metric         | Common use                                                   | Operator class      | Distance operator |
| -------------- | ------------------------------------------------------------ | ------------------- | ----------------- |
| Cosine         | Most text embeddings                                         | `vector_cosine_ops` | `<=>`             |
| L2 / Euclidean | Absolute distance matters; vectors do not need normalization | `vector_l2_ops`     | `<->`             |
| Inner product  | Unit-normalized vectors; matches cosine for unit vectors     | `vector_ip_ops`     | `<#>`             |

Choose the metric recommended for the selected model, and keep its index operator class and query operator aligned. The [Qwen3 Embedding model card](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B) demonstrates cosine similarity and normalized dot product for retrieval. For unit-normalized vectors, cosine, inner product, and L2 have the same exact ranking; approximate indexes can still differ in recall and latency, so benchmark on your corpus.

```sql
CREATE INDEX documents_embedding_ann ON documents
  USING lakebase_ann (embedding vector_cosine_ops);
```

## Tune the Index

The default index options suit most workloads:

- `build_mode = 'standard'` balances recall and index build time. Use `quality` for better recall when a longer build is acceptable.
- `lists = 'auto'` chooses the IVF partition layout from the number of indexed vectors. Choose between `auto` and a manual value case by case: test both on the target dataset and use the value that better meets recall and performance targets.

To prioritize recall over index build time:

```sql
CREATE INDEX documents_embedding_ann_quality ON documents
  USING lakebase_ann (embedding vector_cosine_ops)
  WITH (build_mode = 'quality');
```

To override the automatic partition layout instead:

```sql
CREATE INDEX documents_embedding_ann_lists ON documents
  USING lakebase_ann (embedding vector_cosine_ops)
  WITH (lists = '1024');
```

For a large table, use `CREATE INDEX CONCURRENTLY` to avoid locking out writes while creating the index. For a frequently changing table, periodically use `REINDEX INDEX CONCURRENTLY` to rebuild the index with minimal write locking.

## Query

Generate the query embedding with the same model and dimensions used for stored documents, following the model's query-input conventions (using the Qwen3 query prompt shown above), then bind it as a parameter:

```sql
SELECT id, title, embedding <=> $1::vector AS distance
FROM documents
ORDER BY distance
LIMIT $2;
```

Distance sorts ascending: a smaller value is a closer match. Keep the query operator consistent with the index operator class.

To filter by a similarity radius, use the matching boolean range operator in `WHERE` and the distance operator in `ORDER BY`:

```sql
SELECT id, title
FROM documents
WHERE embedding <<=>> sphere($1::vector, 0.5)
ORDER BY embedding <=> $1::vector
LIMIT $2;
```

The cosine range operator `<<=>>` returns a boolean; do not use it as the ranking expression.

## Tune Search

Inspect the index before overriding defaults:

```sql
SELECT lakebase_ann_index_info('documents_embedding_ann');
```

This reports `lists`, `default_probes`, and `default_epsilon`. Small datasets use exact flat search before IVF lists are built. In that state, `lists` and `default_probes` are empty. Leave `lakebase_ann.probes` set to its default of `'auto'`; `lakebase_ann.epsilon` still controls full-precision reranking during flat search.

For an IVF index, `lakebase_ann.probes` controls how many partitions are searched at each level. Higher values generally improve recall at the cost of speed. Its default is `'auto'`. When `lists` is not empty, the shape of `probes` must match the shape of `lists`: use one value for a one-level index or two comma-separated values for a two-level index. At each level, the `probes` value must be no larger than the corresponding `lists` value. A mismatched or out-of-range value causes an error.

`lakebase_ann.epsilon` controls how many candidates are reranked using full-precision distances. Higher values rerank more candidates and take longer. Its default is `'auto'`, which works well for most workloads.

### Use Prefilter Selectively

By default, PostgreSQL applies non-vector filters after the ANN index returns candidate rows. Enable prefilter when a filter is cheap to evaluate and removes most rows. Leave it off for loose or expensive filters.

```sql
BEGIN;
SET LOCAL lakebase_ann.prefilter = on;

SELECT id, title
FROM documents
WHERE id % 100 = 0
ORDER BY embedding <=> $1::vector
LIMIT $2;
COMMIT;
```

Start with `probes` and `epsilon` set to `'auto'`. Benchmark manual probe values against representative query embeddings and choose the smallest values that satisfy recall and tail-latency targets. Keep session settings and the query in the same transaction when using a connection pool or stateless driver.

Source: [`lakebase_vector` documentation](https://neon.com/docs/extensions/lakebase-vector).