Chat completions

The chat completions endpoint is the recommended way to use Neon AI Gateway. It's fully compatible with the OpenAI Chat Completions API and works with every model in the AI Gateway catalog. Switch models by changing a single field.

Base URL: https://<branch-host>/v1

This endpoint is also reachable at the longer /ai-gateway/mlflow/v1/chat/completions path. Both behave identically and neither is deprecated. See Shorter /v1 paths for the full list of aliases.

If you're using an OpenRouter-compatible client that asks for a base URL, set it to https://<branch-host>/v1 and call /chat/completions.

Setup

Set these environment variables. See Get started for how to obtain them.

NEON_AI_GATEWAY_TOKEN=nt_live_...
NEON_AI_GATEWAY_BASE_URL=https://br-winter-pond-aptw82ef-api.ai.c-2.us-east-2.aws.neon.tech

Basic request

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.NEON_AI_GATEWAY_TOKEN,
  baseURL: `${process.env.NEON_AI_GATEWAY_BASE_URL}/v1`,
});

const response = await client.chat.completions.create({
  model: 'gpt-5-mini',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'What is Neon?' },
  ],
  max_tokens: 256,
});

console.log(response.choices[0].message.content);

Streaming

Add stream: true to receive a server-sent events response.

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.NEON_AI_GATEWAY_TOKEN,
  baseURL: `${process.env.NEON_AI_GATEWAY_BASE_URL}/v1`,
});

const stream = await client.chat.completions.create({
  model: 'gpt-5-mini',
  messages: [{ role: 'user', content: 'Explain branching in Postgres.' }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '');
}

Switching models

Change the model field to use a different provider. Everything else stays the same.

// OpenAI
model: 'gpt-5-4'

// Google
model: 'gemini-3-flash'

// Alibaba
model: 'qwen3-next-80b-a3b-instruct'

For a few models, message.content comes back as an array of content blocks instead of a plain string. See Content shape varies by model before swapping in a model you haven't used yet.

See Models for the full list.

Rate limiting

There are two separate rate limit tiers:

• Neon account quota: enforced by Neon. Returns 429 with error code REQUEST_LIMIT_EXCEEDED. See Rate limits for current limits.
• Upstream provider limit: enforced by the Databricks workspace serving the model. Returns 429 with forwarded rate limit headers.

When the upstream provider rate-limits a request, AI Gateway forwards the relevant headers so your client can back off correctly:

Error handling

Error responses are a JSON object with an error.message field:

{
  "error": {
    "message": "unknown model \"<model-id>\""
  }
}

Next steps

• Models: full model catalog
• OpenAI Responses API: Responses API endpoint
• Authentication: credential scopes and branch binding