Embeddings

Embeddings turn text into vectors that capture meaning, so you can match text by what it means rather than by the exact words. They power semantic search, recommendations, clustering, and retrieval-augmented generation (RAG).

The Neon AI Gateway embeddings endpoint generates those vectors so you can store and search them in Postgres. It's compatible with the OpenAI Embeddings API: the same OpenAI SDK you use elsewhere works against the gateway, using the same Neon credential as the other AI Gateway endpoints.

Base URL: https://<branch-host>/v1

This endpoint is also reachable at the longer /ai-gateway/mlflow/v1/embeddings path. Both behave identically and neither is deprecated. See Shorter paths.

Models

Two embedding models are available today. Both return 1024-dimensional vectors and are billed on input tokens only.

Use the short model IDs shown above. The databricks- prefixed forms (databricks-qwen3-embedding-0-6b, databricks-gte-large-en) are also accepted.

Setup

The examples use two environment variables:

NEON_AI_GATEWAY_TOKEN=nt_live_...
NEON_AI_GATEWAY_BASE_URL=https://br-...

Run neon env pull --file .env to write both to your .env for the current branch (neon deploy and neon config apply also pull them). You can also copy them from the Neon Console's Connect panel, on the AI Gateway tab. Inside Neon Functions they're injected automatically. See Authentication for details.

The examples below use the openai SDK. Install it with npm install openai or pip install openai.

Basic request

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.NEON_AI_GATEWAY_TOKEN,
  baseURL: `${process.env.NEON_AI_GATEWAY_BASE_URL}/v1`,
});

const response = await client.embeddings.create({
  model: 'qwen3-embedding-0-6b',
  input: 'The quick brown fox jumps over the lazy dog.',
  encoding_format: 'float', // required on the openai SDK v6; harmless on v7+
});

console.log(response.data[0].embedding.length); // 1024

Response shape

The response is OpenAI-compatible: a list of embedding objects, one per input, each with the vector and its position.

{
  "id": "...",
  "object": "list",
  "model": "qwen3-embedding-0-6b-112025",
  "data": [
    {
      "index": 0,
      "object": "embedding",
      "embedding": [-0.0149, 0.0172, -0.0119, "..."]
    }
  ],
  "usage": { "prompt_tokens": 11, "total_tokens": 11 }
}

The model field reports the resolved, versioned name that served the request (qwen3-embedding-0-6b-112025 here), which is handy for logging. Keep sending the short model ID (qwen3-embedding-0-6b or gte-large-en) in your requests.

Use with the Vercel AI SDK

For TypeScript apps and agents, @neon/ai-sdk-provider exposes the gateway's embedding models to the Vercel AI SDK. It reads NEON_AI_GATEWAY_BASE_URL and NEON_AI_GATEWAY_TOKEN (the same variables from Setup), so no extra configuration is needed. Install it with npm install @neon/ai-sdk-provider ai.

import { neon } from '@neon/ai-sdk-provider';
import { embed } from 'ai';

const { embedding } = await embed({
  model: neon.embeddingModel('qwen3-embedding-0-6b'),
  value: 'The quick brown fox jumps over the lazy dog.',
});

console.log(embedding.length); // 1024

Use embedMany() to embed a batch in one call. The gateway accepts up to 150 inputs per request (see Batch input).

Batch input

Pass an array of strings to embed many inputs in one request. The response returns one embedding per input, in the same order, with a matching index.

const response = await client.embeddings.create({
  model: 'qwen3-embedding-0-6b',
  input: ['First document.', 'Second document.', 'Third document.'],
  encoding_format: 'float',
});

for (const item of response.data) {
  console.log(item.index, item.embedding.length);
}

Shorter vectors withdimensions

qwen3-embedding-0-6b supports the dimensions parameter to return a shorter, Matryoshka-style vector (for example dimensions: 256), which trades some accuracy for smaller storage and faster search. gte-large-en ignores this parameter and always returns 1024 dimensions.

const response = await client.embeddings.create({
  model: 'qwen3-embedding-0-6b',
  input: 'A shorter embedding.',
  dimensions: 256,
  encoding_format: 'float',
});

console.log(response.data[0].embedding.length); // 256

Choosing a distance operator

The two models differ in how their vectors are scaled, which affects the pgvector operator class you index and query with:

• qwen3-embedding-0-6b is unit-normalized (every vector has L2 norm 1), so cosine distance and inner product rank results identically. Cosine (vector_cosine_ops, <=>) is a safe default.
• gte-large-en is not normalized (L2 norm ≈ 24). Use cosine distance (vector_cosine_ops, <=>), which is scale-invariant, or normalize the vectors before storing them.

For an end-to-end example that stores AI Gateway embeddings in Postgres and searches them, see Get started with Lakebase Search.

Not supported

• Streaming. Embeddings don't stream. A request with stream: true returns 400.
• Wrong endpoint. An embedding model called on a chat endpoint (or a chat model on this endpoint) is rejected. Send embedding requests to /v1/embeddings.

Error handling

Next steps

• Get started with Lakebase Search: store and search AI Gateway embeddings in Postgres
• Models: full model catalog
• Chat completions: the OpenAI-compatible chat endpoint