Overview
Every request Bifrost handles splits into two parts:- Upstream is time spent waiting on the provider: the network round trip and the provider’s own compute. Bifrost cannot make this faster.
- Overhead is Bifrost’s own work: parsing the request, converting schemas, running plugins, selecting a key, handing the request between goroutines, and writing the response back.
The breakdown is populated automatically whenever logging is enabled. There is nothing to configure. See Built-in Observability for enabling logging.
How it’s measured
Each phase of the pipeline is wrapped in a span. A bucket’s value is the span’s self-time: its own wall-clock duration minus the duration of its direct children. Because a child’s time is subtracted from its parent, work is counted exactly once no matter how deeply spans nest, and the buckets never double-count. Two categories are residuals: they account for overhead that is not attributed to any single phase (see The two residuals).The categories
The breakdown groups its rows into nine categories. Each table below lists every row in a category by the name shown in the drill-down and what it measures.Serialization
JSON parsing and encoding at the edges of the request.Conversion
Translating between Bifrost’s unified schema and a provider’s native shape.Plugins
Time spent inside each configured plugin’s hooks. One row per plugin, shown by the plugin’s name (for example, Enterprise Governance, Semantic Cache, OpenTelemetry), collapsing that plugin’s individual hook phases (pre-hook, post-hook) into a single row. Any plugin you configure appears here automatically.Middleware
HTTP transport authentication and access control, run before the request enters the core pipeline.Key selection
Choosing which provider API key to use for the request.Key pool and Key selection are merged into a single Key selection row in the drill-down, since both are steps of choosing the key.
Processing
The internal request pipeline: the glue that moves a request through the core, across worker goroutines, and back.Networking
Handling the request between the client, the gateway, and the provider.Client delivery
Streaming egress: sending chunks back to the client over the response socket.Scheduling
The two residuals
Two rows are not tied to a single phase. Each accounts for overhead that does not belong to any one measured step.Provider processing
The Provider processing row is the provider’s own server-side handling of the request, excluding the network round trip to the provider (that counts as upstream, not overhead) and any handling already broken out into a more specific row. It varies by provider, and a larger value simply means more of that provider’s handling is not itemized into finer rows. It is normally small.Scheduling
The Scheduling row is overhead that does not belong to any single measured phase, mostly the time the request spends being passed between the stages of the pipeline. It is normally small.Both residuals appear on unary (non-streaming) requests only. See below for why streaming excludes them.
Streaming differences
A streamed response is accounted for differently, because most of its time is spent waiting between chunks rather than doing Bifrost work. Two consequences:- Provider processing and Scheduling are not shown. For a stream, the time between chunks is off-CPU waiting, not Bifrost work, so these two residuals are left out to avoid mislabeling it.
- Per-chunk work still appears in the usual rows. Decoding each chunk shows up in Response parse (Serialization), converting chunks in Stream convert (inbound) and Stream convert (outbound) (Conversion), and writing chunks back to the client in Client write (Client delivery). A stream’s numbers therefore read like a unary request’s.
Reading the breakdown
- Compare overhead against upstream first. If a request feels slow but overhead is a thin sliver next to upstream, the time is the provider’s, not Bifrost’s.
- Look at the largest category. It tells you where Bifrost spent most of its own time on the request, whether that is serialization, plugins, key selection, or networking.
- Drill into a category with View details to see its member rows.
- Other is a fallback for a row that has no assigned category, which is different from Scheduling: Scheduling is measured overhead that belonged to no single phase, whereas Other is a row that exists but has not been filed under a category. Every row Bifrost emits today maps to one of the nine categories, so Other is normally empty.
Next steps
- Built-in Observability - Enable logging and explore request traces.
- Request flow - How a request moves through the core pipeline that these buckets measure.