Built-in Observability

Overview

Bifrost includes built-in observability, a powerful feature that automatically captures and stores detailed information about every AI request and response that flows through your system. This provides structured, searchable data with real-time monitoring capabilities, making it easy to debug issues, analyze performance patterns, and understand your AI application’s behavior at scale. All LLM interactions are captured with comprehensive metadata including inputs, outputs, tokens, costs, and latency. The logging plugin operates asynchronously with zero impact on request latency.

What’s Captured

Bifrost traces comprehensive information for every request, without any changes to your application code.

Request Data

Input Messages: Complete conversation history and user prompts
Model Parameters: Temperature, max tokens, tools, and all other parameters
Provider Context: Which provider and model handled the request
Prompt Tracking: When the Prompts plugin is active, the log captures the selected prompt name, version number, and ID for full traceability

Response Data

Output Messages: AI responses, tool calls, and function results
Performance Metrics: Latency and token usage
Status Information: Success or error details

Guardrail Redaction

When Enterprise guardrail redaction is enabled, Bifrost logs store the redacted form of any content that the guardrail provider detected:

runtime mode stores the same redacted content sent at runtime.
logs_only mode leaves runtime content raw but stores reversible placeholders in logs.
runtime_reversible mode stores the same reversible placeholders used at runtime.

Log detail responses include redaction_mapping only for users with Logs:Reveal. If disable_content_logging is enabled, Bifrost skips request/response content and does not persist reveal data for that log. For the full mode matrix, see Guardrail Redaction.

Retry & Key Selection ^{v1.5.0-prerelease4+}

When Bifrost retries a request (per-key failure or transient network/5xx error) the following fields are recorded:

Field	Meaning
`selected_key_id` / `selected_key_name`	The API key that successfully served the request. `null` when all attempts failed - use `attempt_trail` to see which keys were tried.
`number_of_retries`	Total number of attempts minus one. Does not indicate which key was used on each attempt.
`attempt_trail`	Ordered array of every attempt. Each entry contains `attempt`, `key_id`, `key_name`, `fail_reason` (set on every failed attempt, including the terminal one, except one the caller cancelled; omitted from the JSON on a successful or cancelled attempt), `failure_class` and `status_code` (set on every failed attempt, including transport failures where the provider never answered: those are classed `transient` and carry Bifrost’s synthetic `502`; an attempt that ran out of time is classed `timeout` with `504`, and one the caller cancelled is classed `cancelled` with `499` and no `fail_reason`; see how failures are classified), `retry_after_ms` (the provider’s own wait for that attempt, omitted when it gave none; see provider retry hints), and `triggered_rotation` (always present; `true` only when this attempt’s per-key failure (a rate limit, a rejected credential, an exhausted quota, a model this key cannot reach, a retired model, or a region block on this key’s project or account) caused the next retry to switch to a different key, `false` otherwise).

fail_reason is the class’s label for the classes that name what the provider refused: rate_limit_error, authentication_error, billing_error, model_access_error, model_retired_error, region_blocked_error. A timeout is request_timed_out, or ttft_timeout when a stream’s first-token limit cut it. A cancelled attempt has none. For other failures it is the provider’s own error type, falling back to unknown. Example attempt_trail — two rate-limit rotations then success on a third key:

"attempt_trail": [
  { "attempt": 0, "key_id": "key-a", "key_name": "Key A", "fail_reason": "rate_limit_error", "failure_class": "rate_limit", "status_code": 429, "triggered_rotation": true },
  { "attempt": 1, "key_id": "key-b", "key_name": "Key B", "fail_reason": "rate_limit_error", "failure_class": "rate_limit", "status_code": 429, "triggered_rotation": true },
  { "attempt": 2, "key_id": "key-c", "key_name": "Key C", "triggered_rotation": false }
]

Permanent per-key failures also rotate to a different key, since the failure is bound to the key, and they do so even at max_retries: 0. A rejected credential:

"attempt_trail": [
  { "attempt": 0, "key_id": "key-a", "key_name": "Key A", "fail_reason": "authentication_error", "failure_class": "credential", "status_code": 401, "triggered_rotation": true },
  { "attempt": 1, "key_id": "key-b", "key_name": "Key B", "triggered_rotation": false }
]

A model the first key cannot reach (an Azure deployment that exists only on the second key’s resource):

"attempt_trail": [
  { "attempt": 0, "key_id": "key-a", "key_name": "Key A", "fail_reason": "model_access_error", "failure_class": "model_access", "status_code": 404, "triggered_rotation": true },
  { "attempt": 1, "key_id": "key-b", "key_name": "Key B", "triggered_rotation": false }
]

Network/5xx retries reuse the same key: they are transient server issues, not per-key problems. triggered_rotation is therefore false on network-error attempts even when a retry follows (a network error carries Bifrost’s own provider_connection_failed type and a synthetic 502):

"attempt_trail": [
  { "attempt": 0, "key_id": "key-a", "key_name": "Key A", "fail_reason": "provider_connection_failed", "failure_class": "transient", "status_code": 502, "triggered_rotation": false },
  { "attempt": 1, "key_id": "key-a", "key_name": "Key A", "fail_reason": "rate_limit_error", "failure_class": "rate_limit", "status_code": 429, "triggered_rotation": true },
  { "attempt": 2, "key_id": "key-b", "key_name": "Key B", "triggered_rotation": false }
]

For terminal failures (no retry happens, including non-retryable errors), the trail has a single entry with fail_reason set and triggered_rotation false:

"attempt_trail": [
  { "attempt": 0, "key_id": "key-a", "key_name": "Key A", "fail_reason": "invalid_request_error", "failure_class": "caller_fault", "status_code": 400, "triggered_rotation": false }
]

attempt_trail is null / absent when the request succeeded on the first try without retries.

Custom Metadata

Logging Headers: Capture configured request headers (e.g., X-Tenant-ID) into log metadata
Ad-hoc Headers: Any x-bf-lh-* prefixed header is automatically captured into metadata
See Logging Headers below for full details

Multimodal & Tool Support

Audio Processing: Speech synthesis and transcription inputs/outputs
Vision Analysis: Image URLs and vision model responses
Tool Execution: Function calling arguments and results

How It Works

The logging plugin intercepts all requests flowing through Bifrost using the plugin architecture, ensuring your LLM requests maintain optimal performance:

PreLLMHook: Captures request metadata (provider, model, input messages, parameters).
Async Processing: Logs are written in background goroutines with sync.Pool optimization.
PostLLMHook: Updates log entry with response data (output, tokens, cost, latency, errors).
Real-time Updates: WebSocket broadcasts keep the UI synchronized.

All logging operations are non-blocking, ensuring your LLM requests maintain optimal performance.

Configuration

Configure request tracing to control what gets logged and where it’s stored.

Using Web UI
Using API
Using config.json
Using Go SDK

Navigate to http://localhost:8080
Go to “Settings”
Toggle “Enable Logs”

Enable/Disable Tracing:

curl --location 'http://localhost:8080/api/config' \
--header 'Content-Type: application/json' \
--method PUT \
--data '{
    "client_config": {
        "enable_logging": true,
        "disable_content_logging": false,
        "drop_excess_requests": false,
        "initial_pool_size": 300,
         "enforce_auth_on_inference": false,
        "prometheus_labels": [],
        "allowed_origins": []
    }
}'

Check Current Configuration:

curl --location 'http://localhost:8080/api/config'

Response includes tracing status:

{
    "client_config": {
        "enable_logging": true,
        "disable_content_logging": false,
        "drop_excess_requests": false
    },
    "is_db_connected": true,
    "is_cache_connected": true, 
    "is_logs_connected": true
}

In your config.json file, you can enable logging and configure the log store:

{
    "client": {
        "enable_logging": true,
        "disable_content_logging": false,
        "drop_excess_requests": false,
        "initial_pool_size": 300
    },
    "logs_store": {
        "enabled": true,
        "type": "sqlite",
        "config": {
            "path": "./logs.db"
        }
    }
}

enable_logging: Master toggle for request tracing.
disable_content_logging: Disable logging of request/response content, but still log usage metadata (latency, cost, token count, etc.).
logs_store: Check Log Store Options for more details.

When using Bifrost as a Go SDK, initialize the logging plugin manually:

package main

import (
    "context"
    bifrost "github.com/maximhq/bifrost/core"
    "github.com/maximhq/bifrost/core/schemas"
    "github.com/maximhq/bifrost/framework/logstore"
    "github.com/maximhq/bifrost/framework/pricing"
    "github.com/maximhq/bifrost/plugins/logging"
)

func main() {
    ctx := context.Background()
    logger := schemas.NewLogger()
    
    // Initialize log store (SQLite)
    store, err := logstore.NewLogStore(ctx, &logstore.Config{
        Enabled: true,
        Type:    logstore.LogStoreTypeSQLite,
        Config: &logstore.SQLiteConfig{
            Path: "./logs.db",
        },
    }, logger)
    if err != nil {
        panic(err)
    }
    
    // Initialize pricing manager (required for cost calculation)
    pricingManager := pricing.NewPricingManager(logger)
    
    // Initialize logging plugin
    loggingPlugin, err := logging.Init(ctx, logger, store, pricingManager)
    if err != nil {
        panic(err)
    }
    
    // Initialize Bifrost with logging plugin
    client, err := bifrost.Init(ctx, schemas.BifrostConfig{
        Account: &yourAccount,
        LLMPlugins: []schemas.LLMPlugin{loggingPlugin},
    })
    if err != nil {
        panic(err)
    }
    defer client.Shutdown()
    
    // All requests are now logged automatically
}

Accessing & Filtering Logs

Retrieve and analyze logs with powerful filtering capabilities via the UI, API, and WebSockets. Advanced Log Filtering Interface

Web UI

When running the Gateway, access the built-in dashboard at http://localhost:8080. The UI provides:

Real-time log streaming
Advanced filtering and search
Detailed request/response inspection
Token and cost analytics

Grouped view

The Group button in the logs toolbar folds related requests into one row. The row’s label says what it holds:

Label	What it groups	Grouped by
`2 turns`	Requests in the same session, under the first request	`session_id`
`1 fallback`	Fallback attempts of one request, under the original attempt	`parent_request_id`

Batch and video requests show N linked for the settled-cost row that follows them. Click the label to expand the row. Session requests have a blue left border and are numbered turn 2, turn 3 and so on. Fallback attempts have a grey border and read fallback 1, fallback 2. A session request with its own fallbacks gets its own toggle. Hover a label to see which key it groups by. Grouping is off by default. Your choice is kept in the URL (grouped=true). It turns off while you filter to a single session_id or parent_request_id, since that is already one group. On very large log tables it can load slower than the flat view. To group through the API, send roots_only=true, and add group_sessions=true to fold sessions too. Each row returns child_count, session_child_count, session_total_cost and session_total_tokens.

API Endpoints

Query logs programmatically using the GET request.

curl 'http://localhost:8080/api/logs?' \
'providers=openai,anthropic&' \
'models=gpt-4o-mini&' \
'status=success,error&' \
'start_time=2024-01-15T00:00:00Z&' \
'end_time=2024-01-15T23:59:59Z&' \
'min_latency=1000&' \
'max_latency=5000&' \
'min_tokens=10&' \
'max_tokens=1000&' \
'min_cost=0.001&' \
'max_cost=10&' \
'content_search=python&' \
'limit=100&' \
'offset=0'

Available Filters:

Filter	Description	Example
`providers`	Filter by AI providers	`openai,anthropic`
`models`	Filter by specific models	`gpt-4o-mini,claude-3-sonnet`
`status`	Request status	`success,error,processing`
`objects`	Request types	`chat.completion,embedding`
`start_time` / `end_time`	Time range (RFC3339)	`2024-01-15T10:00:00Z`
`min_latency` / `max_latency`	Response time (ms)	`1000` to `5000`
`min_tokens` / `max_tokens`	Token usage range	`10` to `1000`
`min_cost` / `max_cost`	Cost range (USD)	`0.001` to `10`
`content_search`	Search in messages	`"error handling"`
`tool_call_names`	Requests whose response called any of these functions	`get_weather,search`
`request_id`	Exact lookup on a log ID	`018f2c3d-4e5f-4a6b-8c9d-0e1f2a3b4c5d`
`limit` / `offset`	Pagination	`100`, `200`

A log’s ID is its request ID, so request_id is an exact primary-key lookup rather than a text search. It takes precedence over the time range — start_time, end_time, and period are ignored when it is set, so a request is found wherever it falls. roots_only is ignored too, so an ID naming a fallback child returns that child rather than collapsing it into its root. In the dashboard, pasting a request ID into the logs search box switches to this lookup automatically; prefix it with id: for request IDs that aren’t UUIDs (any string supplied via the x-request-id header).

Response Format

{
    "logs": [...],
    "pagination": {
        "limit": 100,
        "offset": 0,
        "sort_by": "timestamp",
        "order": "desc"
    },
    "stats": {
        "total_requests": 1234,
        "success_rate": 0.85,
        "average_latency": 100,
        "total_tokens": 10000,
        "total_cost": 100
    }
}

Perfect for analytics, debugging specific issues, or building custom monitoring dashboards.

WebSocket

Subscribe to real-time log updates for live monitoring:

const ws = new WebSocket('ws://localhost:8080/ws')

ws.onmessage = (event) => {
  const logUpdate = JSON.parse(event.data)
  console.log('New log entry:', logUpdate)
}

Background jobs

Long operations run in the background so you can keep using the page. Find them in the bell icon in the top bar, on the Background jobs tab. The bell always opens on Notifications. Each tab has a badge, showing unread notifications on one and running jobs on the other.

Job	Started by
Recalculating log costs	Recalculate cost on the logs page
Indexing logs for Warp	A Warp log index backfill
Virtual key expiry cleanup	The daily cleanup of expired virtual keys

Each job shows its status and, if it reports progress, a progress bar. Totals are approximate, so a finished job’s count can be slightly off. To stop a running job, click Cancel on its row, then Stop job. A cancelled job does not resume. Dismiss a finished job from its row, or click Clear finished in the panel header to remove them all. On the Notifications tab, that spot holds Mark all read. Only administrators see jobs. Without View on the BackgroundJobs RBAC resource, the tab is hidden. Cancelling needs Update on the same resource. GET /api/sidekiq/jobs returns all pending and running jobs, plus jobs finished in the last 24 hours. Results are newest first, up to 50.

Log Store Options

Choose the right storage backend for your scale and requirements. The logging plugin is automatically enabled in Gateway mode with SQLite storage by default. You can configure it to use PostgreSQL by setting the logs_store configuration in your config.json file.

Current Support

SQLite (Default)
PostgreSQL

Best for: Development, small-medium deployments
Performance: Excellent for read-heavy workloads
Setup: Zero configuration, single file storage
Limits: Single-writer, local filesystem only

{
    "logs_store": {
        "enabled": true,
        "type": "sqlite",
        "config": {
            "path": "./logs.db"
        }
    }
}

Best for: High-volume production deployments
Performance: Excellent concurrent writes and complex queries
Features: Advanced indexing, partitioning, replication
Requirement: PostgreSQL database must be UTF8 encoded (see PostgreSQL UTF8 Requirement)

{
    "logs_store": {
        "enabled": true,
        "type": "postgres",
        "config": {
            "host": "localhost",
            "port": "5432",
            "user": "bifrost",
            "password": "postgres",
            "db_name": "bifrost",
            "ssl_mode": "disable"
        }
    }
}

Planned Support

MySQL: For traditional MySQL environments.
ClickHouse: For large-scale analytics and time-series workloads.

Supported Request Types

The logging plugin captures all Bifrost request types:

Text Completion (streaming and non-streaming)
Chat Completion (streaming and non-streaming)
Responses (streaming and non-streaming)
Embeddings
Speech Generation (streaming and non-streaming)
Transcription (streaming and non-streaming)
Video Generation

Logging Headers

Capture specific HTTP request headers into the metadata field of every LLM and MCP log entry. This enables request tracing, tenant identification, and custom debugging without modifying your application code.

How It Works

There are two ways headers get captured into log metadata: 1. Configured Logging Headers - Define a list of header names in the configuration. The logging plugin looks up each configured header (case-insensitive) and stores its value in the metadata. 2. x-bf-lh-* Prefix (Automatic) - Any request header with the x-bf-lh- prefix is automatically captured into metadata with no configuration needed. The prefix is stripped and the remainder becomes the metadata key.

Request Header	Metadata Key	Metadata Value
`x-bf-lh-tenant-id: acme`	`tenant-id`	`acme`
`x-bf-lh-env: production`	`env`	`production`
`x-bf-lh-region: us-east-1`	`region`	`us-east-1`

Both methods can be used together - configured headers and x-bf-lh-* headers are merged into the same metadata map.

Configuring Logging Headers

Web UI
API
config.json

Navigate to Config > Logging
Ensure Enable Logs is toggled on
Scroll to Logging Headers
Enter a comma-separated list of header names (e.g., X-Tenant-ID, X-Correlation-ID)
Click Save Changes

Changes take effect immediately - no restart required.

Include logging_headers in the client_config when updating the configuration:

curl -X PUT http://localhost:8080/api/config \
  -H "Content-Type: application/json" \
  -d '{
    "client_config": {
      "logging_headers": ["X-Tenant-ID", "X-Correlation-ID"]
    }
  }'

Add logging_headers to the client section:

{
  "client": {
    "enable_logging": true,
    "logging_headers": ["X-Tenant-ID", "X-Correlation-ID"]
  }
}

Field	Type	Required	Description
`logging_headers`	`string[]`	No	List of header names to capture in log metadata. Case-insensitive. No restart required.

Usage Examples

Configured headers:

# Config has: logging_headers: ["X-Tenant-ID", "X-Correlation-ID"]
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-Tenant-ID: tenant-123" \
  -H "X-Correlation-ID: req-abc-456" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'

Log metadata: {"x-tenant-id": "tenant-123", "x-correlation-id": "req-abc-456"} Ad-hoc x-bf-lh-* headers (no config needed):

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-bf-lh-env: production" \
  -H "x-bf-lh-version: v2.1.0" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "Hello"}]}'

Log metadata: {"env": "production", "version": "v2.1.0"}

Viewing Metadata in the UI

Metadata is displayed in the log detail view for both LLM and MCP logs as individual key-value entries alongside other request details.

Combining with Required Headers

Required headers and logging headers serve different purposes and can be used together:

Feature	Purpose	Effect on Request
Required Headers	Enforce header presence	Rejects request if missing (400)
Logging Headers	Capture header values	No effect on request - only logs metadata

A common pattern is to require a header and log it:

{
  "client": {
    "required_headers": ["X-Tenant-ID"],
    "logging_headers": ["X-Tenant-ID"]
  }
}

When to Use