Overview

Bifrost provides complete OpenAI API compatibility through protocol adaptation. The integration handles request transformation, response normalization, and error mapping between OpenAI’s API specification and Bifrost’s internal processing pipeline. This integration enables you to utilize Bifrost’s features like governance, load balancing, semantic caching, multi-provider support, and more, all while preserving your existing OpenAI SDK-based architecture. Endpoint: /openai

Setup

Python
JavaScript

import openai

# Configure client to use Bifrost
client = openai.OpenAI(
    base_url="http://localhost:8080/openai",
    api_key="<YOUR-BIFROST-VIRTUAL-KEY>"  # Replace with your actual Bifrost virtual key.
)

# Make requests as usual
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai";

// Configure client to use Bifrost
const openai = new OpenAI({
  baseURL: "http://localhost:8080/openai",
  apiKey: "<YOUR-BIFROST-VIRTUAL-KEY>", // Replace with your actual Bifrost virtual key.
});

// Make requests as usual
const response = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello!" }],
});

console.log(response.choices[0].message.content);

Provider/Model Usage Examples

Use multiple providers through the same OpenAI SDK format by prefixing model names with the provider:

Python
JavaScript

import openai

client = openai.OpenAI(
    base_url="http://localhost:8080/openai",
    api_key="<YOUR-BIFROST-VIRTUAL-KEY>"  # Replace with your actual Bifrost virtual key.
)

# OpenAI models (default)
openai_response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello from OpenAI!"}]
)

# Anthropic models via OpenAI SDK format
anthropic_response = client.chat.completions.create(
    model="anthropic/claude-3-sonnet-20240229",
    messages=[{"role": "user", "content": "Hello from Claude!"}]
)

# Google Vertex models via OpenAI SDK format
vertex_response = client.chat.completions.create(
    model="vertex/gemini-pro",
    messages=[{"role": "user", "content": "Hello from Gemini!"}]
)

# Azure models
azure_response = client.chat.completions.create(
    model="azure/gpt-4o",
    messages=[{"role": "user", "content": "Hello from Azure!"}]
)

# Local Ollama models
ollama_response = client.chat.completions.create(
    model="ollama/llama3.1:8b",
    messages=[{"role": "user", "content": "Hello from Ollama!"}]
)

import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "http://localhost:8080/openai",
  apiKey: "<YOUR-BIFROST-VIRTUAL-KEY>", // Replace with your actual Bifrost virtual key.
});

// OpenAI models (default)
const openaiResponse = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello from OpenAI!" }],
});

// Anthropic models via OpenAI SDK format
const anthropicResponse = await openai.chat.completions.create({
  model: "anthropic/claude-3-sonnet-20240229",
  messages: [{ role: "user", content: "Hello from Claude!" }],
});

// Google Vertex models via OpenAI SDK format
const vertexResponse = await openai.chat.completions.create({
  model: "vertex/gemini-pro",
  messages: [{ role: "user", content: "Hello from Gemini!" }],
});

// Azure models
const azureResponse = await openai.chat.completions.create({
  model: "azure/gpt-4o",
  messages: [{ role: "user", content: "Hello from Azure!" }],
});

// Local Ollama models
const ollamaResponse = await openai.chat.completions.create({
  model: "ollama/llama3.1:8b",
  messages: [{ role: "user", content: "Hello from Ollama!" }],
});

Adding Custom Headers

Pass custom headers required by Bifrost plugins (like governance, telemetry, etc.):

Python
JavaScript

import openai

client = openai.OpenAI(
    base_url="http://localhost:8080/openai",
    api_key="<YOUR-BIFROST-VIRTUAL-KEY>",  # Replace with your actual Bifrost virtual key.
    default_headers={
        "x-bf-vk": "<YOUR-BIFROST-VIRTUAL-KEY>",  # Replace with your actual Bifrost virtual key.
    }
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello with custom headers!"}]
)

import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "http://localhost:8080/openai",
  apiKey: "<YOUR-BIFROST-VIRTUAL-KEY>", // Replace with your actual Bifrost virtual key.
  defaultHeaders: {
    "x-bf-vk": "<YOUR-BIFROST-VIRTUAL-KEY>", // Replace with your actual Bifrost virtual key.
  },
});

const response = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{ role: "user", content: "Hello with custom headers!" }],
});

Async Inference

Submit inference requests asynchronously and poll for results later using the x-bf-async header. This is useful for long-running requests where you don’t want to hold a connection open. See Async Inference for full details.

Async inference requires a Logs Store to be configured and is not compatible with streaming.

Chat Completions

Python
JavaScript

import openai
import time

client = openai.OpenAI(
    base_url="http://localhost:8080/openai",
    api_key="<YOUR-BIFROST-VIRTUAL-KEY>"  # Replace with your actual Bifrost virtual key.
)

# Submit async request
initial = client.chat.completions.create(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Tell me a short story."}],
    extra_headers={"x-bf-async": "true"}
)

# If choices are present, the request completed synchronously
if initial.choices:
    print(initial.choices[0].message.content)
else:
    # Poll until completed
    while True:
        time.sleep(2)
        poll = client.chat.completions.create(
            model="openai/gpt-4o-mini",
            messages=[{"role": "user", "content": "Tell me a short story."}],
            extra_headers={"x-bf-async-id": initial.id}
        )
        if poll.choices:
            print(poll.choices[0].message.content)
            break

import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "http://localhost:8080/openai",
  apiKey: "<YOUR-BIFROST-VIRTUAL-KEY>", // Replace with your actual Bifrost virtual key.
});

// Submit async request
const initial = await openai.chat.completions.create(
  {
    model: "openai/gpt-4o-mini",
    messages: [{ role: "user", content: "Tell me a short story." }],
  },
  { headers: { "x-bf-async": "true" } }
);

// If choices are present, the request completed synchronously
if (initial.choices?.length > 0) {
  console.log(initial.choices[0].message.content);
} else {
  // Poll until completed
  while (true) {
    await new Promise((r) => setTimeout(r, 2000));
    const poll = await openai.chat.completions.create(
      {
        model: "openai/gpt-4o-mini",
        messages: [{ role: "user", content: "Tell me a short story." }],
      },
      { headers: { "x-bf-async-id": initial.id } }
    );
    if (poll.choices?.length > 0) {
      console.log(poll.choices[0].message.content);
      break;
    }
  }
}

Responses API

Python
JavaScript

import openai
import time

client = openai.OpenAI(
    base_url="http://localhost:8080/openai",
    api_key="<YOUR-BIFROST-VIRTUAL-KEY>"  # Replace with your actual Bifrost virtual key.
)

# Submit async request
initial = client.responses.create(
    model="openai/gpt-4o-mini",
    input="Tell me a short story.",
    extra_headers={"x-bf-async": "true"}
)

# If status is "completed", the request completed synchronously
if initial.status == "completed":
    print(initial.output_text)
else:
    # Poll until completed
    while True:
        time.sleep(2)
        poll = client.responses.create(
            model="openai/gpt-4o-mini",
            input="Tell me a short story.",
            extra_headers={"x-bf-async-id": initial.id}
        )
        if poll.status == "completed":
            print(poll.output_text)
            break

import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "http://localhost:8080/openai",
  apiKey: "<YOUR-BIFROST-VIRTUAL-KEY>", // Replace with your actual Bifrost virtual key.
});

// Submit async request
const initial = await openai.responses.create(
  { model: "openai/gpt-4o-mini", input: "Tell me a short story." },
  { headers: { "x-bf-async": "true" } }
);

// If status is "completed", the request completed synchronously
if (initial.status === "completed") {
  console.log(initial.output_text);
} else {
  // Poll until completed
  while (true) {
    await new Promise((r) => setTimeout(r, 2000));
    const poll = await openai.responses.create(
      { model: "openai/gpt-4o-mini", input: "Tell me a short story." },
      { headers: { "x-bf-async-id": initial.id } }
    );
    if (poll.status === "completed") {
      console.log(poll.output_text);
      break;
    }
  }
}

Async Headers

Header	Description
`x-bf-async: true`	Submit the request as an async job. Returns immediately with a job ID.
`x-bf-async-id: <job-id>`	Poll for results of a previously submitted async job.
`x-bf-async-job-result-ttl: <seconds>`	Override the default result TTL (default: 3600s).

Decisions API

OpenAI’s Decisions API works through Bifrost with the same base_url, at /openai/v1/decisions (or /openai/decisions). Any decision model can answer, on any provider that serves one: OpenAI’s gpt-6-luna, TypeSafe’s jev models, Clef on Cloudflare, Perplexity’s deciders, and the others listed under Decisions. A chat model returns unsupported_operation unless convert_decision_to_responses is on, which emulates the answer through its Responses API (details). Answers come back in OpenAI’s response shape, with name: null for a question sent without a name, plus Bifrost’s extra_fields as on the other OpenAI routes; when raw responses are enabled and OpenAI served the request, OpenAI’s own response body is returned instead. The request is OpenAI’s: a top-level state and a field a question, choice, or level does not define, such as TypeSafe’s criteria, are rejected with a 400 rather than translated, while other top-level fields a newer SDK sends are forwarded unchanged. For TypeSafe-only features, use /v1/decisions. client.decisions.create requires an SDK with the Decisions resource: openai>=3.26.0 in Python or openai>=7.30.0 in Node.

response = client.decisions.create(
    model="gpt-6-luna",  # or "typesafe/jev-1.13.0"
    input="I was charged twice for my order and nobody has replied.",
    questions=[
        {"type": "predicate", "name": "is_frustrated", "instructions": "Is the customer frustrated?"},
        {
            "type": "choice",
            "name": "category",
            "instructions": "Pick the ticket category",
            "choices": [{"value": "billing"}, {"value": "bug"}, {"value": "other"}],
        },
    ],
)
for answer in response.answers:
    print(answer.type, answer.name)

An SDK version without the decisions resource can send the same body with its generic request method, for example client.post("/decisions", body={...}, cast_to=httpx.Response) in Python or client.post("/decisions", { body: {...} }) in Node. See Decisions for the request fields.

Supported Features

The OpenAI integration supports all features that are available in both the OpenAI SDK and Bifrost core functionality. If the OpenAI SDK supports a feature and Bifrost supports it, the integration will work seamlessly.

Next Steps

Files and Batch API - File uploads and batch processing
Anthropic SDK - Claude integration patterns
Google GenAI SDK - Gemini integration patterns
Configuration - Bifrost setup and configuration
Core Features - Advanced Bifrost capabilities

Runbooks

CLI Agents & Editors

SDKs & Frameworks

Identity Providers (SSO)

Content Safety (Guardrails)

Observability

Vector Databases

Overview

Overview

Setup

Provider/Model Usage Examples

Adding Custom Headers

Async Inference

Chat Completions

Responses API

Async Headers

Decisions API

Supported Features

Next Steps

​Overview

​Setup

​Provider/Model Usage Examples

​Adding Custom Headers

​Async Inference

​Chat Completions

​Responses API

​Async Headers

​Decisions API

​Supported Features

​Next Steps

Overview

Setup

Provider/Model Usage Examples

Adding Custom Headers

Async Inference

Chat Completions

Responses API

Async Headers

Decisions API

Supported Features

Next Steps