Overview

Vertex AI is Google’s unified ML platform providing access to Google’s Gemini models, Anthropic Claude models, and other third-party LLMs through a single API. Bifrost performs conversions including:

Multi-model support - Unified interface for Gemini, Anthropic, and third-party models
OAuth2 authentication - Service account credentials with automatic token refresh
Project and region management - Automatic endpoint construction from GCP project/region
Model routing - Automatic provider detection (Gemini vs Anthropic) based on model name
Request conversion - Conversion to underlying provider format (Gemini or Anthropic)
Embeddings support - Vector generation with task type and truncation options
Model discovery - Paginated model listing with deployment information

Supported Operations

Operation	Non-Streaming	Streaming	Endpoint
Chat Completions	✅	✅	`/generate`
Responses API	✅	✅	`/messages`
Embeddings	✅	-	`/embeddings`
Image Generation	✅	-	`/generateContent` or `/predict` (Imagen)
Image Edit	✅	-	`/generateContent` or `/predict` (Imagen)
Video Generation	✅	-	`/predictLongRunning` (Veo models only)
Image Variation	❌	-	Not supported
List Models	✅	-	`/models`
Text Completions	❌	❌	-
Speech (TTS)	❌	❌	-
Transcriptions (STT)	❌	❌	-
Files	❌	❌	-
Batch	❌	❌	-

Unsupported Operations (❌): Text Completions, Speech, Transcriptions, Files, and Batch are not supported by Vertex AI. These return UnsupportedOperationError.Vertex-specific: Endpoints vary by model type. Responses API available for both Gemini and Anthropic models.

Setup & Configuration

Vertex AI requires Google Cloud project configuration and authentication credentials. Four authentication methods are supported.

The aliases field (mapping model names to fine-tuned model IDs or endpoint identifiers) requires v1.5.0-prerelease2 or later. On v1.4.x, use deployments inside vertex_key_config instead - see the v1.5.0 Migration Guide for details.

1. Service Account JSON (Recommended for Production)

Provide a credential JSON string in auth_credentials. The JSON must contain a type field. Supported types: service_account (most common), impersonated_service_account, authorized_user, external_account, external_account_authorized_user. The document names the token, impersonation and credential-source URLs the Google client dials, so setting, changing or removing it through the API is held to the same rule as a key endpoint: a request admitted without any credential check (dashboard authentication disabled or unconfigured, or a whitelisted route) is refused with 403. A request that passed a credential check may set it: an authenticated admin session, or, while the setup lock is active, a request carrying the configured setup token (see Provider endpoint URLs).

An external_account JSON (a Workload Identity Federation credential config) is accepted here too. File, URL and executable credential_source shapes, for example a projected Kubernetes ServiceAccount token, are handled by Google’s library as-is. The AWS shape ("environment_id": "aws1", produced by gcloud ... create-cred-config --aws) is resolved by Bifrost through the AWS SDK default chain, so it works on EKS with IRSA or Pod Identity exactly like AWS Workload Identity Federation; the structured block is the recommended form because it needs no credential file.

Web UI
API
config.json
Go SDK

Google Vertex AI Service Account (JSON) authentication setup in the Bifrost Web UI showing Project ID, Region, and Auth Credentials fields

Navigate to “Model Providers” → “Configurations” → “Google Vertex”
Click “Add Key” (or edit an existing key)
Under Authentication Method, select “Service Account (JSON)”
Set Project ID: Your Google Cloud project ID
Set Project Number (Required only for fine-tuned models): Your GCP project number; leave blank for standard models
Set Region: e.g., us-central1
Set Auth Credentials: Paste your service account JSON or reference an env var (e.g., env.VERTEX_CREDENTIALS)
Configure Aliases: Map model names to fine-tuned model IDs (if using fine-tuned models)
Save

# Step 1: Create the provider
curl -X POST http://localhost:8080/api/providers \
  -H "Content-Type: application/json" \
  -d '{"provider": "vertex"}'

# Step 2: Create a key (Service Account JSON)
curl -X POST http://localhost:8080/api/providers/vertex/keys \
  -H "Content-Type: application/json" \
  -d '{
    "name": "vertex-sa-key",
    "value": "",
    "models": ["*"],
    "weight": 1.0,
    "vertex_key_config": {
      "project_id": "env.VERTEX_PROJECT_ID",
      "region": "us-central1",
      "auth_credentials": "env.VERTEX_CREDENTIALS"
    }
  }'

On v1.4.x, two differences apply: - Pass keys directly in the POST /api/providers body - there is no separate /api/providers/{provider}/keys endpoint. - Use deployments inside vertex_key_config instead of the top-level aliases field for fine-tuned model mappings.

{
  "providers": {
    "vertex": {
      "keys": [
        {
          "name": "vertex-sa-key",
          "value": "",
          "models": ["*"],
          "weight": 1.0,
          "vertex_key_config": {
            "project_id": "env.VERTEX_PROJECT_ID",
            "region": "us-central1",
            "auth_credentials": "env.VERTEX_CREDENTIALS"
          }
        }
      ]
    }
  }
}

On v1.4.x, use deployments inside vertex_key_config instead of the top-level aliases field for fine-tuned model mappings.

func (a *MyAccount) GetKeysForProvider(ctx *context.Context, provider schemas.ModelProvider) ([]schemas.Key, error) {
    switch provider {
    case schemas.Vertex:
        return []schemas.Key{
            {
                Value:  schemas.SecretVar{}, // Leave empty when using service account credentials
                Models: []string{"*"},
                Weight: 1.0,
                VertexKeyConfig: &schemas.VertexKeyConfig{
                    ProjectID:       *schemas.NewSecretVar("env.VERTEX_PROJECT_ID"),
                    Region:          *schemas.NewSecretVar("us-central1"),
                    AuthCredentials: *schemas.NewSecretVar("env.VERTEX_CREDENTIALS"), // full service account JSON
                },
            },
        }, nil
    }
    return nil, fmt.Errorf("provider %s not supported", provider)
}

2. Application Default Credentials

Leave auth_credentials empty. Bifrost calls google.FindDefaultCredentials() - Google’s ADC library - which resolves credentials in this order:

GOOGLE_APPLICATION_CREDENTIALS env var (path to a JSON credential file)
Application default credential file (~/.config/gcloud/application_default_credentials.json, written by gcloud auth application-default login)
GCE/GKE/Cloud Run/App Engine metadata server (attached service account or Workload Identity)

Web UI
API
config.json
Go SDK

Google Vertex AI Application Default Credentials setup in the Bifrost Web UI showing Project ID and Region fields with no credential inputs

Navigate to “Model Providers” → “Configurations” → “Google Vertex”
Click “Add Key” (or edit an existing key)
Under Authentication Method, select “Service Account (Attached)”
Set Project ID: Your Google Cloud project ID
Set Project Number (Required only for fine-tuned models): Your GCP project number; leave blank for standard models
Set Region: e.g., us-central1
Configure Aliases if needed
Save

Ensure GOOGLE_APPLICATION_CREDENTIALS is set in your environment, or that Workload Identity / gcloud is configured.

# Step 1: Create the provider
curl -X POST http://localhost:8080/api/providers \
  -H "Content-Type: application/json" \
  -d '{"provider": "vertex"}'

# Step 2: Create a key (Application Default Credentials)
curl -X POST http://localhost:8080/api/providers/vertex/keys \
  -H "Content-Type: application/json" \
  -d '{
    "name": "vertex-adc-key",
    "value": "",
    "models": ["*"],
    "weight": 1.0,
    "vertex_key_config": {
      "project_id": "env.VERTEX_PROJECT_ID",
      "region": "us-central1",
      "auth_credentials": ""
    }
  }'

On v1.4.x, pass keys directly in the POST /api/providers body - there is no separate /api/providers/{provider}/keys endpoint.

{
  "providers": {
    "vertex": {
      "keys": [
        {
          "name": "vertex-adc-key",
          "value": "",
          "models": ["*"],
          "weight": 1.0,
          "vertex_key_config": {
            "project_id": "env.VERTEX_PROJECT_ID",
            "region": "us-central1",
            "auth_credentials": ""
          }
        }
      ]
    }
  }
}

func (a *MyAccount) GetKeysForProvider(ctx *context.Context, provider schemas.ModelProvider) ([]schemas.Key, error) {
    switch provider {
    case schemas.Vertex:
        return []schemas.Key{
            {
                Value:  schemas.SecretVar{},
                Models: []string{"*"},
                Weight: 1.0,
                VertexKeyConfig: &schemas.VertexKeyConfig{
                    ProjectID: *schemas.NewSecretVar("env.VERTEX_PROJECT_ID"),
                    Region:    *schemas.NewSecretVar("us-central1"),
                    // Leave AuthCredentials empty - uses Application Default Credentials
                },
            },
        }, nil
    }
    return nil, fmt.Errorf("provider %s not supported", provider)
}

3. API Key (Gemini and Fine-Tuned Models Only)

Set value to your Vertex API key. API key authentication is supported only for Gemini models and fine-tuned Gemini models. For Anthropic models on Vertex, use Service Account or Application Default Credentials.

Web UI
API
config.json
Go SDK

Google Vertex AI API Key authentication setup in the Bifrost Web UI showing API Key, Project ID, Region, and Project Number fields

Navigate to “Model Providers” → “Configurations” → “Google Vertex”
Click “Add Key” (or edit an existing key)
Under Authentication Method, select “API Key”
Set API Key: Your Vertex AI API key
Set Project ID: Your Google Cloud project ID
Set Project Number (Required only for fine-tuned models): Your GCP project number; leave blank for standard models
Set Region: e.g., us-central1
Configure Aliases: Map short names to fine-tuned model IDs (e.g., my-model → 123456789)
Save

# Step 1: Create the provider
curl -X POST http://localhost:8080/api/providers \
  -H "Content-Type: application/json" \
  -d '{"provider": "vertex"}'

# Step 2: Create a key (API Key - Gemini + fine-tuned models)
curl -X POST http://localhost:8080/api/providers/vertex/keys \
  -H "Content-Type: application/json" \
  -d '{
    "name": "vertex-api-key",
    "value": "env.VERTEX_API_KEY",
    "models": ["gemini-pro", "gemini-2.0-flash", "my-fine-tuned-model"],
    "weight": 1.0,
    "aliases": {
      "my-fine-tuned-model": "123456789"
    },
    "vertex_key_config": {
      "project_id": "env.VERTEX_PROJECT_ID",
      "project_number": "env.VERTEX_PROJECT_NUMBER",
      "region": "us-central1"
    }
  }'

On v1.4.x, two differences apply:

Pass keys directly in the POST /api/providers body - there is no separate /api/providers/{provider}/keys endpoint.
Replace the top-level aliases with "deployments" inside vertex_key_config:

"vertex_key_config": {
  "project_id": "env.VERTEX_PROJECT_ID",
  "region": "us-central1",
  "deployments": {
    "my-fine-tuned-model": "123456789"
  }
}

{
  "providers": {
    "vertex": {
      "keys": [
        {
          "name": "vertex-api-key",
          "value": "env.VERTEX_API_KEY",
          "models": ["gemini-pro", "gemini-2.0-flash", "my-fine-tuned-model"],
          "weight": 1.0,
          "aliases": {
            "my-fine-tuned-model": "123456789"
          },
          "vertex_key_config": {
            "project_id": "env.VERTEX_PROJECT_ID",
            "project_number": "env.VERTEX_PROJECT_NUMBER",
            "region": "us-central1"
          }
        }
      ]
    }
  }
}

On v1.4.x, use deployments inside vertex_key_config instead of the top-level aliases field.

func (a *MyAccount) GetKeysForProvider(ctx *context.Context, provider schemas.ModelProvider) ([]schemas.Key, error) {
    switch provider {
    case schemas.Vertex:
        return []schemas.Key{
            {
                Value:  *schemas.NewSecretVar("env.VERTEX_API_KEY"), // only when using Gemini or fine-tuned models
                Models: []string{"gemini-pro", "gemini-2.0-flash", "my-fine-tuned-model"},
                Weight: 1.0,
                Aliases: schemas.KeyAliases{
                    "my-fine-tuned-model": "123456789",
                },
                VertexKeyConfig: &schemas.VertexKeyConfig{
                    ProjectID:     *schemas.NewSecretVar("env.VERTEX_PROJECT_ID"),
                    ProjectNumber: *schemas.NewSecretVar("env.VERTEX_PROJECT_NUMBER"), // required for fine-tuned models
                    Region:        *schemas.NewSecretVar("us-central1"),
                },
            },
        }, nil
    }
    return nil, fmt.Errorf("provider %s not supported", provider)
}

Vertex AI support for fine-tuned models is currently in beta. Requests to non-Gemini fine-tuned models may fail, so please test and report any issues.

vertex_key_config fields:

Field	Required	Description
`project_id`	Yes	Google Cloud project ID
`region`	Yes	GCP region (e.g., `us-central1`, `eu-west1`, `global`)
`auth_credentials`	No	Service account JSON string (leave empty for ADC)
`project_number`	No	GCP project number (required for fine-tuned models)
`force_single_region`	No	When `true`, always call `region` and skip multi-region promotion of multi-region-only models

Key-level fields:

Field	Required	Description
`value`	No	Vertex API key (Gemini and fine-tuned models only; leave empty for Service Account / ADC)
`aliases`	No	Map model names to fine-tuned model IDs or endpoint identifiers (v1.5.0-prerelease2+)
`models`	Yes	Models this key can serve; use `["*"]` to allow all

4. AWS Workload Identity Federation (EKS)

Set aws_workload_identity to federate the AWS identity attached to the Bifrost workload (EKS IRSA or Pod Identity, ECS task role, EC2 instance profile, or AWS_* env vars) into a GCP token through a Workload Identity Pool AWS provider. No GCP service account key and no static AWS key are required. Bifrost resolves AWS credentials with the AWS SDK default chain, signs an STS GetCallerIdentity request with them, and exchanges that signature at the GCP Security Token Service. The default chain consults static credentials first: if AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are present in the workload environment, Bifrost federates as that key rather than as the attached role, so remove them when the attached role is the intended identity. When service_account_email is set the federated token is swapped for a service-account token; otherwise the federated principal calls Vertex AI directly. This block takes precedence over auth_credentials and the two must not be combined. The AWS side and the GCP side are set up in AWS Workload Identity Federation below.

Field	Required	Description
`audience`	Yes	Provider resource name: `//iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/providers/PROVIDER_ID`
`service_account_email`	No	Service account to impersonate. Omit to grant Vertex access to the federated principal directly
`token_lifetime_seconds`	No	Lifetime of the impersonated token (600 to 43200, default 3600). Only used with `service_account_email`. Values above 3600 require the target service account to be allowed by the organization policy constraint `constraints/iam.allowServiceAccountCredentialLifetimeExtension`; otherwise token generation fails
`aws_region`	No	Region for the STS signature. Defaults to `AWS_REGION` / `AWS_DEFAULT_REGION`, then instance metadata
`aws_role_arn`	No	Assume this IAM role with the workload’s credentials before the exchange. Usually unnecessary: map the workload’s own role in the pool provider

PROJECT_NUMBER in the audience is the numeric project number, not the project ID you enter elsewhere in the key. Look it up with:

gcloud projects describe PROJECT_ID --format="value(projectNumber)"

All string fields accept env.VAR_NAME references. GovCloud and China partitions are supported: the STS host is derived from the region.

Web UI
API
config.json
Go SDK

Navigate to “Model Providers” → “Configurations” → “Google Vertex”
Click “Add Key” (or edit an existing key)
Under Authentication Method, select “Workload Identity (AWS)”
Set Project ID: Your Google Cloud project ID
Set Region: e.g., us-central1
Set Workload Identity Provider Audience: the //iam.googleapis.com/... resource name of the AWS provider in your pool
(Optional) Set Service Account to Impersonate, AWS Region and AWS Role ARN to Assume
Save

# Step 1: Create the provider
curl -X POST http://localhost:8080/api/providers \
  -H "Content-Type: application/json" \
  -d '{"provider": "vertex"}'

# Step 2: Create a key (AWS Workload Identity Federation)
curl -X POST http://localhost:8080/api/providers/vertex/keys \
  -H "Content-Type: application/json" \
  -d '{
    "name": "vertex-eks-key",
    "value": "",
    "models": ["*"],
    "weight": 1.0,
    "vertex_key_config": {
      "project_id": "env.VERTEX_PROJECT_ID",
      "region": "us-central1",
      "aws_workload_identity": {
        "audience": "//iam.googleapis.com/projects/123456789012/locations/global/workloadIdentityPools/eks-pool/providers/aws",
        "service_account_email": "bifrost-vertex@my-gcp-project.iam.gserviceaccount.com"
      }
    }
  }'

{
  "providers": {
    "vertex": {
      "keys": [
        {
          "name": "vertex-eks-key",
          "value": "",
          "models": ["*"],
          "weight": 1.0,
          "vertex_key_config": {
            "project_id": "env.VERTEX_PROJECT_ID",
            "region": "us-central1",
            "aws_workload_identity": {
              "audience": "//iam.googleapis.com/projects/123456789012/locations/global/workloadIdentityPools/eks-pool/providers/aws",
              "service_account_email": "bifrost-vertex@my-gcp-project.iam.gserviceaccount.com"
            }
          }
        }
      ]
    }
  }
}

func (a *MyAccount) GetKeysForProvider(ctx *context.Context, provider schemas.ModelProvider) ([]schemas.Key, error) {
    switch provider {
    case schemas.Vertex:
        return []schemas.Key{
            {
                Value:  schemas.SecretVar{},
                Models: []string{"*"},
                Weight: 1.0,
                VertexKeyConfig: &schemas.VertexKeyConfig{
                    ProjectID: *schemas.NewSecretVar("env.VERTEX_PROJECT_ID"),
                    Region:    *schemas.NewSecretVar("us-central1"),
                    AWSWorkloadIdentity: &schemas.VertexAWSWorkloadIdentityConfig{
                        Audience:            *schemas.NewSecretVar("//iam.googleapis.com/projects/123456789012/locations/global/workloadIdentityPools/eks-pool/providers/aws"),
                        ServiceAccountEmail: schemas.NewSecretVar("bifrost-vertex@my-gcp-project.iam.gserviceaccount.com"),
                    },
                },
            },
        }, nil
    }
    return nil, fmt.Errorf("provider %s not supported", provider)
}

GKE Workload Identity Federation

When running Bifrost on GKE, Workload Identity Federation (WIF) lets pods authenticate to Vertex AI without managing service account keys. The pod inherits an IAM identity through the Kubernetes ServiceAccount, and Bifrost picks it up automatically via Application Default Credentials. What you need:

The GCP-side prerequisites below (API enabled, IAM service account, WIF binding)
A Bifrost Vertex key using “Service Account (Attached)” auth - see Application Default Credentials for Web UI, API, config.json, and Go SDK setup. For Helm, see Helm - Google Vertex AI.
The Kubernetes ServiceAccount annotated for WIF:

kubectl annotate serviceaccount KSA_NAME \
  --namespace NAMESPACE \
  iam.gke.io/gcp-service-account=IAM_SA_NAME@PROJECT_ID.iam.gserviceaccount.com

Replace IAM_SA_NAME with the IAM Service Account created in Step 3 below.

GCP Prerequisites

Step 1: Enable the Vertex AI API

The Vertex AI API must be enabled in your project. Search for aiplatform in the API Library or run:

gcloud services enable aiplatform.googleapis.com --project=PROJECT_ID

WIF uses the IAM Credentials API for token exchange. Enable it as well:

gcloud services enable iamcredentials.googleapis.com --project=PROJECT_ID

Step 2: Enable Workload Identity on GKE

Autopilot clusters: WIF is always enabled. Skip this step.Standard clusters: Enable the workload identity pool and GKE metadata server:

# Enable Workload Identity on the cluster
gcloud container clusters update CLUSTER_NAME \
  --location=LOCATION \
  --workload-pool=PROJECT_ID.svc.id.goog

# Enable GKE metadata server on each node pool
gcloud container node-pools update NODEPOOL_NAME \
  --cluster=CLUSTER_NAME \
  --location=LOCATION \
  --workload-metadata=GKE_METADATA

Verify:

gcloud container clusters describe CLUSTER_NAME \
  --location=LOCATION \
  --format="value(workloadIdentityConfig.workloadPool)"
# Expected: PROJECT_ID.svc.id.goog

Step 3: Create an IAM Service Account and grant Vertex access

Create a dedicated IAM Service Account (or use an existing one) and grant it the Vertex AI User role:

# Create the service account
gcloud iam service-accounts create IAM_SA_NAME \
  --display-name="Bifrost Vertex AI" \
  --project=PROJECT_ID

# Grant Vertex AI access
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:IAM_SA_NAME@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"

Step 4: Bind the Kubernetes ServiceAccount to the IAM Service Account

Allow the Kubernetes ServiceAccount to impersonate the IAM Service Account:

gcloud iam service-accounts add-iam-policy-binding \
  IAM_SA_NAME@PROJECT_ID.iam.gserviceaccount.com \
  --role="roles/iam.workloadIdentityUser" \
  --member="serviceAccount:PROJECT_ID.svc.id.goog[NAMESPACE/KSA_NAME]"

Replace NAMESPACE and KSA_NAME with your Bifrost pod’s namespace and Kubernetes ServiceAccount name.Then annotate the Kubernetes ServiceAccount so GKE knows which IAM identity to map:

kubectl annotate serviceaccount KSA_NAME \
  --namespace NAMESPACE \
  iam.gke.io/gcp-service-account=IAM_SA_NAME@PROJECT_ID.iam.gserviceaccount.com

If deploying with the Bifrost Helm chart, set the annotation via serviceAccount.annotations in your values file - see Helm - Google Vertex AI for the full example.

Verify

From inside the Bifrost pod, confirm the GKE metadata server returns a token:

kubectl exec -n NAMESPACE POD_NAME -- \
  wget -qO- --header="Metadata-Flavor: Google" \
  "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token"

Replace NAMESPACE and POD_NAME with your Bifrost namespace and any running Bifrost pod name (e.g., bifrost-0 for a StatefulSet or use kubectl get pods -n NAMESPACE to find it). A JSON response with an access_token field confirms WIF is working. Then send a request through Bifrost to a Vertex model (e.g., vertex/gemini-2.5-flash) to verify end-to-end.

Troubleshooting

Symptom	Likely Cause	Fix
`"could not find default credentials"`	GKE metadata server not enabled, or Kubernetes ServiceAccount missing WIF annotation	Enable GKE metadata server on the node pool (Step 2); verify the `iam.gke.io/gcp-service-account` annotation on the ServiceAccount (Step 4)
`403 Forbidden` from Vertex API	IAM Service Account lacks Vertex permissions	Grant `roles/aiplatform.user` to the IAM Service Account
`403` during token exchange	WIF binding missing	Run the `add-iam-policy-binding` command from Step 4; confirm `roles/iam.workloadIdentityUser` is granted
Wrong project or region errors	Bifrost config mismatch	Check `project_id` and `region` in the Vertex key configuration

AWS Workload Identity Federation

When running Bifrost on AWS (EKS, ECS, EC2), Workload Identity Federation lets the workload’s IAM role authenticate to Vertex AI without a GCP service account key. GCP trusts your AWS account through a Workload Identity Pool AWS provider; Bifrost proves the workload’s identity with a signed STS GetCallerIdentity request and receives a GCP token in return. What you need:

The GCP-side prerequisites below (APIs enabled, pool and AWS provider, IAM grant)
An IAM role attached to the Bifrost workload: an IRSA or EKS Pod Identity role on EKS, an ECS task role, or an EC2 instance profile. It needs no AWS permissions for this flow; sts:GetCallerIdentity is always allowed. Do not also set AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY in the pod: the SDK default chain prefers static credentials over the attached role, and the exchange would then run as that key.
A Bifrost Vertex key using “Workload Identity (AWS)” auth - see AWS Workload Identity Federation (EKS) for Web UI, API, config.json, and Go SDK setup. For Helm, see Helm - Google Vertex AI.

GCP Prerequisites (AWS)

Step 1: Enable the required APIs

gcloud services enable aiplatform.googleapis.com sts.googleapis.com iamcredentials.googleapis.com \
  --project=PROJECT_ID

sts.googleapis.com performs the token exchange. iamcredentials.googleapis.com is only used when you impersonate a service account (service_account_email).

Step 2: Create a Workload Identity Pool and an AWS provider

# Create the pool
gcloud iam workload-identity-pools create POOL_ID \
  --project=PROJECT_ID \
  --location=global \
  --display-name="Bifrost on AWS"

# Trust your AWS account. The attribute mapping exposes the assumed role name as attribute.aws_role,
# and the condition restricts the pool to the Bifrost role only.
gcloud iam workload-identity-pools providers create-aws PROVIDER_ID \
  --project=PROJECT_ID \
  --location=global \
  --workload-identity-pool=POOL_ID \
  --account-id=AWS_ACCOUNT_ID \
  --attribute-mapping="google.subject=assertion.arn,attribute.aws_role=assertion.arn.extract('assumed-role/{role}/')" \
  --attribute-condition="attribute.aws_role == 'BIFROST_ROLE_NAME'"

The audience for the Bifrost key is the provider’s full resource name:

gcloud iam workload-identity-pools providers describe PROVIDER_ID \
  --project=PROJECT_ID --location=global --workload-identity-pool=POOL_ID \
  --format="value(name)"
# //iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/providers/PROVIDER_ID

Prefix the printed projects/... value with //iam.googleapis.com/ if the command omits it.

Step 3: Grant Vertex AI access

Pick one of the two models. The principal in both is the pool-scoped identity of your AWS role:

principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/attribute.aws_role/BIFROST_ROLE_NAME

Option A - direct access (no service account). Leave service_account_email empty in the Bifrost key and grant the role directly:

gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/attribute.aws_role/BIFROST_ROLE_NAME" \
  --role="roles/aiplatform.user"

Option B - impersonate a service account. Set service_account_email in the Bifrost key, give the service account Vertex access, and let the federated principal impersonate it:

gcloud iam service-accounts create IAM_SA_NAME \
  --display-name="Bifrost Vertex AI" --project=PROJECT_ID

gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:IAM_SA_NAME@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"

gcloud iam service-accounts add-iam-policy-binding \
  IAM_SA_NAME@PROJECT_ID.iam.gserviceaccount.com \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/attribute.aws_role/BIFROST_ROLE_NAME"

Step 4: Attach the IAM role to the Bifrost workload (EKS)

With IRSA, annotate the Kubernetes ServiceAccount and let the EKS webhook inject the web identity token:

kubectl annotate serviceaccount KSA_NAME \
  --namespace NAMESPACE \
  eks.amazonaws.com/role-arn=arn:aws:iam::AWS_ACCOUNT_ID:role/BIFROST_ROLE_NAME

With EKS Pod Identity, two prerequisites come first. The cluster needs the Pod Identity Agent add-on, which is what actually hands credentials to the pod:

aws eks create-addon --cluster-name CLUSTER_NAME --addon-name eks-pod-identity-agent

And the IAM role’s trust policy must let the EKS Pod Identity service assume it and tag the session (an IRSA-style OIDC trust policy does not work here):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "pods.eks.amazonaws.com" },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    }
  ]
}

Then create the association:

aws eks create-pod-identity-association \
  --cluster-name CLUSTER_NAME \
  --namespace NAMESPACE \
  --service-account KSA_NAME \
  --role-arn arn:aws:iam::AWS_ACCOUNT_ID:role/BIFROST_ROLE_NAME

Either way the pod receives a credentials source the AWS SDK default chain understands. A region is not guaranteed: IRSA on EKS usually injects AWS_REGION, but a self-managed webhook may not, and Pod Identity does not set one. If neither the environment nor instance metadata provides a region, set aws_region in the key or AWS_REGION in the pod. If deploying with the Bifrost Helm chart, set the annotation via serviceAccount.annotations - see Helm - Google Vertex AI.

Verify (AWS)

From inside the Bifrost pod, confirm the AWS identity is the expected role:

kubectl exec -n NAMESPACE POD_NAME -- aws sts get-caller-identity
# "Arn": "arn:aws:sts::AWS_ACCOUNT_ID:assumed-role/BIFROST_ROLE_NAME/..."

Then send a request through Bifrost to a Vertex model (e.g., vertex/gemini-2.5-flash). In GCP, the Cloud Audit Log entry for the Vertex call identifies the caller under protoPayload.authenticationInfo. With direct access and the google.subject=assertion.arn mapping above, principalSubject is

principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/subject/arn:aws:sts::AWS_ACCOUNT_ID:assumed-role/BIFROST_ROLE_NAME/SESSION

. With impersonation, principalEmail is the service account email and the federated AWS identity appears as principalSubject inside serviceAccountDelegationInfo[]. The principalSet://.../attribute.aws_role/... form from Step 3 is the IAM binding target and does not appear as the logged identity.

Troubleshooting (AWS)

Symptom	Likely Cause	Fix
`aws credentials: ... no EC2 IMDS role found` or `failed to refresh cached credentials`	No IAM role reaches the pod	Check the IRSA annotation or the Pod Identity association (Step 4); `aws sts get-caller-identity` inside the pod must succeed
`aws region could not be determined`	Neither `AWS_REGION` nor instance metadata is available	Set `aws_region` in the key, or `AWS_REGION` in the pod environment
`invalid_grant` / `The audience in the token does not match` during the exchange	Wrong `audience`	Copy the provider resource name exactly (Step 2); it starts with `//iam.googleapis.com/`
`The given credential is rejected by the attribute condition`	The AWS role name does not match the provider condition	Align `--attribute-condition` with the role attached to the pod
`403 Permission 'iam.serviceAccounts.getAccessToken' denied`	Missing impersonation grant	Run the `add-iam-policy-binding ... roles/iam.workloadIdentityUser` command from Step 3, Option B
`403 Forbidden` from Vertex API	The principal or service account lacks Vertex permissions	Grant `roles/aiplatform.user` per Step 3
Requests succeed but as the node role (Bifrost older than this feature)	An `external_account` JSON with `"environment_id": "aws1"` was resolved by Google’s library, which reads instance metadata	Upgrade; current Bifrost resolves that JSON through the AWS SDK chain, or switch the key to `aws_workload_identity`

Beta Headers

For Anthropic models on Vertex AI, Bifrost validates anthropic-beta headers and drops unsupported headers from the request. Supported: computer-use-*, compact-*, context-management-*, interleaved-thinking-*, context-1m-* Not supported: structured-outputs-*, advanced-tool-use-*, mcp-client-*, prompt-caching-scope-*, files-api-*, skills-*, fast-mode-*, redact-thinking-* You can override these defaults per provider via the Beta Headers tab in provider configuration or via beta_header_overrides. See the full support matrix in the Anthropic provider docs.

1. Chat Completions

Request Parameters

Core Parameter Mapping

Parameter	Vertex Handling	Notes
`model`	Maps to Vertex model ID	Region-specific endpoint constructed automatically
All other params	Model-specific conversion	Converted per underlying provider (Gemini/Anthropic)

Key Configuration

The key configuration for Vertex requires Google Cloud credentials:

{
  "vertex_key_config": {
    "project_id": "my-gcp-project",
    "region": "us-central1",
    "auth_credentials": "{service-account-json}"
  }
}

Configuration Details:

project_id - GCP project ID (required)
region - GCP region for API endpoints (required)
- Examples: us-central1, us-west1, eu-west1, global
auth_credentials - Service account JSON credentials (optional if using default credentials)
aws_workload_identity - AWS → GCP Workload Identity Federation block; takes precedence over auth_credentials (see section 4)

Authentication Methods

Service Account JSON (recommended for production)

{ "auth_credentials": "{full-service-account-json}" }

Application Default Credentials (for local development)
- Requires GOOGLE_APPLICATION_CREDENTIALS environment variable
- Leave auth_credentials empty

AWS Workload Identity Federation (Bifrost running on AWS)

{ "aws_workload_identity": { "audience": "//iam.googleapis.com/projects/.../providers/aws" } }

Gemini Models

When using Google’s Gemini models, Bifrost converts requests to Gemini’s API format.

Parameter Mapping for Gemini

All Gemini-compatible parameters are supported. Special handling includes:

System prompts: Converted to Gemini’s system message format
Tool usage: Mapped to Gemini’s function calling format
Streaming: Uses Gemini’s streaming protocol

Refer to Gemini documentation for detailed conversion details.

Anthropic Models (Claude)

When using Anthropic models through Vertex AI, Bifrost converts requests to Anthropic’s message format.

Parameter Mapping for Anthropic

All Anthropic-standard parameters are supported:

Reasoning/Thinking: reasoning parameters converted to thinking structure
System messages: Extracted and placed in separate system field
Tool message grouping: Consecutive tool messages merged
API version: Automatically set to vertex-2023-10-16 for Anthropic models

Refer to Anthropic documentation for detailed conversion details.

Special Notes for Vertex + Anthropic

Responses API uses special /v1/messages endpoint
anthropic_version automatically set to vertex-2023-10-16
Minimum reasoning budget: 1024 tokens
Model field removed from request (Vertex uses different identification)
prompt_cache.auto_inject applies to Claude models only. Vertex serves Claude and Gemini side by side, and only Claude takes a per-block cache marker; Gemini caches through a cachedContent resource instead, so injection is a no-op there. See Prompt caching.

Region Selection

The region determines the API endpoint:

Region	Endpoint	Purpose
`us-central1`	`us-central1-aiplatform.googleapis.com`	US Central
`us-west1`	`us-west1-aiplatform.googleapis.com`	US West
`eu-west1`	`eu-west1-aiplatform.googleapis.com`	Europe West
`global`	`aiplatform.googleapis.com`	Global (no region prefix)

Availability varies by region. Check GCP documentation for model availability.

Multi-region-only models

Some models (e.g. Claude Opus 4.7/4.8) are published only on Google’s multi-region pool endpoints. When such a model is requested with a single US or Europe region, Bifrost automatically promotes the call to the matching pool endpoint (aiplatform.us.rep.googleapis.com / aiplatform.eu.rep.googleapis.com) and rewrites the locations/ path segment to us / eu. Set force_single_region: true on the Vertex key (or per-deployment) to disable this and keep the exact region you configured — for example when you serve these models from a single region via provisioned throughput. Endpoint rewrites are emitted at debug log level.

Streaming

Streaming format depends on model type:

Gemini models: Standard Gemini streaming with server-sent events
Anthropic models: Anthropic message streaming format

2. Responses API

The Responses API is available for both Anthropic (Claude) and Gemini models on Vertex AI.

Request Parameters

Core Parameter Mapping

Parameter	Vertex Handling	Notes
`instructions`	Becomes system message	Model-specific conversion
`input`	Converted to messages	String or array support
`max_output_tokens`	Model-specific field mapping	Gemini vs Anthropic conversion
All other params	Model-specific conversion	Converted per underlying provider

Gemini Models

For Gemini models, conversion follows Gemini’s Responses API format.

Anthropic Models (Claude)

For Anthropic models, conversion follows Anthropic’s message format:

instructions becomes system message
reasoning mapped to thinking structure

Configuration

Gateway
Go SDK

curl -X POST http://localhost:8080/v1/responses \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vertex/claude-3-5-sonnet",
    "input": "What is AI?",
    "instructions": "You are a helpful assistant",
    "project_id": "my-gcp-project",
    "region": "us-central1"
  }' \
  -H "X-Goog-Authorization: Bearer {token}"

resp, err := client.ResponsesRequest(schemas.NewBifrostContext(ctx, schemas.NoDeadline), &schemas.BifrostResponsesRequest{
    Provider: schemas.Vertex,
    Model:    "claude-3-5-sonnet",
    Input:    messages,
    Params: &schemas.ResponsesParameters{
        Instructions: schemas.Ptr("You are a helpful assistant"),
    },
})

Special Handling

Endpoint: /v1/messages (Anthropic format)
anthropic_version set to vertex-2023-10-16 automatically
Model and region fields removed from request
Raw request body passthrough supported

Refer to Anthropic Responses API for parameter details.

3. Embeddings

Embeddings are supported for Gemini and other models that support embedding generation.

Request Parameters

Core Parameters

Parameter	Vertex Mapping	Notes
`input`	`instances[].content`	Text to embed
`dimensions`	`parameters.outputDimensionality`	Optional output size

Advanced Parameters

Use extra_params for embedding-specific options:

Gateway
Go SDK

curl -X POST http://localhost:8080/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{
    "model": "text-embedding-004",
    "input": ["text to embed"],
    "dimensions": 256,
    "task_type": "RETRIEVAL_DOCUMENT",
    "title": "Document title",
    "project_id": "my-gcp-project",
    "region": "us-central1",
    "autoTruncate": true
  }'

resp, err := client.EmbeddingRequest(schemas.NewBifrostContext(ctx, schemas.NoDeadline), &schemas.BifrostEmbeddingRequest{
    Provider: schemas.Vertex,
    Model:    "text-embedding-004",
    Input: []schemas.EmbeddingInputItem{
        {Content: schemas.EmbeddingContent{{Type: schemas.EmbeddingContentPartTypeText, Text: schemas.Ptr("text to embed")}}},
    },
    Params: &schemas.EmbeddingParameters{
        Dimensions: schemas.Ptr(256),
        ExtraParams: map[string]interface{}{
            "task_type": "RETRIEVAL_DOCUMENT",
            "title": "Document title",
            "autoTruncate": true,
        },
    },
})

Embedding Parameters

Parameter	Type	Description
`task_type`	string	Task type hint: `RETRIEVAL_QUERY`, `RETRIEVAL_DOCUMENT`, `SEMANTIC_SIMILARITY`, `CLASSIFICATION`, `CLUSTERING` (optional)
`title`	string	Optional title to help model produce better embeddings (used with task_type)
`autoTruncate`	boolean	Auto-truncate input to max tokens (defaults to true)

Task Type Effects

Different task types optimize embeddings for specific use cases:

RETRIEVAL_DOCUMENT - Optimized for documents in retrieval systems
RETRIEVAL_QUERY - Optimized for queries searching documents
SEMANTIC_SIMILARITY - Optimized for semantic similarity tasks
CLASSIFICATION - For classification tasks
CLUSTERING - For clustering tasks

Response Conversion

Embeddings response includes vectors and truncation information:

{
  "embeddings": [
    {
      "values": [0.1234, -0.5678, ...],
      "statistics": {
        "token_count": 15,
        "truncated": false
      }
    }
  ]
}

Response Fields:

values - Embedding vector as floats
statistics.token_count - Input token count
statistics.truncated - Whether input was truncated due to length

4. Image Generation

Image Generation is supported for Gemini and Imagen on Vertex AI. The provider automatically routes to the appropriate format based on the model type.

Request Parameters

Core Parameter Mapping

Parameter	Vertex Handling	Notes
`model`	Mapped to deployment/model identifier	Model type detected automatically
`prompt`	Model-specific conversion	Converted per underlying provider (Gemini/Imagen)
All other params	Model-specific conversion	Converted per underlying provider

Model Type Detection

Vertex automatically detects the model type and uses the appropriate conversion:

Gemini Models: Uses Gemini format (same as Gemini Image Generation)
Imagen Models: Uses Imagen format (detected via IsImagenModel())

Configuration

Gateway
Go SDK

curl -X POST http://localhost:8080/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vertex/imagen-4.0-generate-001",
    "prompt": "A sunset over the mountains",
    "size": "1024x1024",
    "n": 2,
    "project_id": "my-gcp-project",
    "region": "us-central1"
  }' \
  -H "X-Goog-Authorization: Bearer {token}"

resp, err := client.ImageGenerationRequest(schemas.NewBifrostContext(ctx, schemas.NoDeadline), &schemas.BifrostImageGenerationRequest{
    Provider: schemas.Vertex,
    Model:    "imagen-4.0-generate-001",
    Input: &schemas.ImageGenerationInput{
        Prompt: "A sunset over the mountains",
    },
    Params: &schemas.ImageGenerationParameters{
        Size: schemas.Ptr("1024x1024"),
        N:    schemas.Ptr(2),
    },
})

Request Conversion

Vertex converts requests based on model type:

Gemini Models: Uses gemini.ToGeminiImageGenerationRequest() - same conversion as standard Gemini (see Gemini Image Generation)
Imagen Models: Uses gemini.ToImagenImageGenerationRequest() - Imagen-specific format with size/aspect ratio conversion

All request bodies are converted to map[string]interface{} and the region field is removed before sending to Vertex API.

Response Conversion

Gemini Models: Responses converted using GenerateContentResponse.ToBifrostImageGenerationResponse() - same as standard Gemini
Imagen Models: Responses converted using GeminiImagenResponse.ToBifrostImageGenerationResponse() - Imagen-specific format

Endpoint Selection

The provider automatically selects the endpoint based on model type:

Fine-tuned models: /v1beta1/projects/{projectNumber}/locations/{region}/endpoints/{deployment}:generateContent
Imagen models: /v1/projects/{projectID}/locations/{region}/publishers/google/models/{model}:predict
Gemini models: /v1/projects/{projectID}/locations/{region}/publishers/google/models/{model}:generateContent

Streaming

Image generation streaming is not supported by Vertex AI.

5. Image Edit

Requests use multipart/form-data, not JSON.

Image Edit is supported for Gemini and Imagen models on Vertex AI. The provider automatically routes to the appropriate format based on the model type. Request Parameters

Parameter	Type	Required	Notes
`model`	string	✅	Model identifier (must be Gemini or Imagen model)
`prompt`	string	✅	Text description of the edit
`image[]`	binary	✅	Image file(s) to edit (supports multiple images)
`mask`	binary	❌	Mask image file
`type`	string	❌	Edit type: `"inpainting"`, `"outpainting"`, `"inpaint_removal"`, `"bgswap"` (Imagen only)
`n`	int	❌	Number of images to generate (1-10)
`output_format`	string	❌	Output format: `"png"`, `"webp"`, `"jpeg"`
`output_compression`	int	❌	Compression level (0-100%)
`seed`	int	❌	Seed for reproducibility (via `ExtraParams["seed"]`)
`negative_prompt`	string	❌	Negative prompt (via `ExtraParams["negativePrompt"]`)
`maskMode`	string	❌	Mask mode (via `ExtraParams["maskMode"]`, Imagen only): `"MASK_MODE_USER_PROVIDED"`, `"MASK_MODE_BACKGROUND"`, `"MASK_MODE_FOREGROUND"`, `"MASK_MODE_SEMANTIC"`
`dilation`	float	❌	Mask dilation (via `ExtraParams["dilation"]`, Imagen only): Range [0, 1]
`maskClasses`	int[]	❌	Mask classes (via `ExtraParams["maskClasses"]`, Imagen only): For `MASK_MODE_SEMANTIC`

Request Conversion Vertex uses the same conversion functions as Gemini:

Gemini Models: Uses gemini.ToGeminiImageEditRequest() - same conversion as standard Gemini (see Gemini Image Edit)
Imagen Models: Uses gemini.ToImagenImageEditRequest() - Imagen-specific format with edit mode mapping and mask configuration (see Gemini Image Edit)

Model Validation: Only Gemini and Imagen models are supported. Other models return ConfigurationError. Request Body Processing:

All request bodies are converted to map[string]interface{} for Vertex API compatibility
The region field is removed before sending to Vertex API
For Gemini models, unsupported fields are stripped via stripVertexGeminiUnsupportedFields() (removes id from function_call and function_response)

Response Conversion

Gemini Models: Responses converted using GenerateContentResponse.ToBifrostImageGenerationResponse() - same as standard Gemini
Imagen Models: Responses converted using GeminiImagenResponse.ToBifrostImageGenerationResponse() - Imagen-specific format

Endpoint Selection The provider automatically selects the endpoint based on model type:

Gemini models: /v1/projects/{projectID}/locations/{region}/publishers/google/models/{model}:generateContent
Imagen models: /v1/projects/{projectID}/locations/{region}/publishers/google/models/{model}:predict

Streaming Image edit streaming is not supported by Vertex AI. Image Variation Image variation is not supported by Vertex AI.

6. List Models

Request Parameters

None required. Automatically uses project_id and region from key config.

Response Conversion

Lists models available in the specified project and region with metadata and deployment information:

{
  "models": [
    {
      "name": "projects/{project}/locations/{region}/models/gemini-2.0-flash",
      "display_name": "Gemini 2.0 Flash",
      "description": "Fast multimodal model",
      "version_id": "1",
      "version_aliases": ["latest", "stable"],
      "capabilities": [...],
      "deployed_models": [...]
    }
  ],
  "next_page_token": "..."
}

Custom vs Non-Custom Models

Important: Vertex AI’s List Models API only returns custom fine-tuned models that have been deployed to your project. It does NOT return standard foundation models (Gemini, Claude, etc.).

To provide a complete model listing experience, Bifrost performs multi-pass model discovery:

Three-Pass Model Discovery

First Pass - Custom Models from API Response
- Queries Vertex AI’s List Models API
- Returns only custom fine-tuned models deployed to your project
- Custom models are identified by having deployment values that contain only digits
- Example: "deployment": "1234567890"
Second Pass - Non-Custom Models from Aliases
- Adds standard foundation models from your aliases configuration
- Non-custom models have alphanumeric deployment values (e.g., gemini-pro, claude-3-5-sonnet)
- Filters by the key-level models allowlist, if specified
- Example: "deployment": "gemini-2.0-flash"
Third Pass - Allowed Models Not in Aliases
- Adds models specified in models that weren’t in the aliases map
- Ensures all explicitly allowed models appear in the list
- Uses the model name itself as the deployment value
- Skips digit-only model IDs (reserved for custom models)

Model Filtering Logic

If models is empty and no aliases are configured: No models are returned
If models is empty but aliases are configured: Only aliased models are returned
If models is ["*"]: All models from all three passes are included (unrestricted)
If models is non-empty: Only models/aliases whose request names appear in models are included
Duplicate Prevention: Each model ID is tracked to prevent duplicates across passes

Model Name Formatting

Non-custom models from aliases and allowed models are automatically formatted for display:

gemini-pro → “Gemini Pro”
claude-3-5-sonnet → “Claude 3 5 Sonnet”
gemini_2_flash → “Gemini 2 Flash”

Formatting uses title case and converts hyphens/underscores to spaces.

Example Configuration

With Custom Models Only
With Foundation Models
With Allowed Models Filter

{
  "aliases": {
    "my-gemini-ft": "1234567890",
    "my-claude-ft": "9876543210"
  },
  "vertex_key_config": {
    "project_id": "my-project",
    "region": "us-central1"
  }
}

This returns only your custom fine-tuned models from the API.

{
  "aliases": {
    "gemini-2.0-flash": "gemini-2.0-flash",
    "claude-3-5-sonnet": "claude-3-5-sonnet-v2@20241022"
  },
  "vertex_key_config": {
    "project_id": "my-project",
    "region": "us-central1"
  }
}

This returns both custom models AND foundation models from aliases.

{
  "models": ["gemini-2.0-flash", "claude-3-5-sonnet"],
  "aliases": {
    "gemini-2.0-flash": "gemini-2.0-flash",
    "claude-3-5-sonnet": "claude-3-5-sonnet-v2@20241022",
    "gemini-1.5-pro": "gemini-1.5-pro"
  },
  "vertex_key_config": {
    "project_id": "my-project",
    "region": "us-central1"
  }
}

Only returns gemini-2.0-flash and claude-3-5-sonnet, excluding gemini-1.5-pro.

Pagination

Model listing is paginated automatically. If more than 100 models exist, next_page_token will be present. Bifrost handles pagination internally.

Caveats

Project ID and Region Required

Severity: High Behavior: Both project_id and region required for all operations Impact: Request fails without valid GCP project/region configuration Code: vertex.go:127-138

OAuth2 Token Management

Severity: Medium Behavior: Tokens cached and automatically refreshed when expired Impact: First request slightly slower due to auth; cached for subsequent requests Code: vertex.go:34-55

Anthropic Model Detection

Severity: Medium Behavior: Automatic detection of Anthropic vs Gemini models Impact: Different conversion logic applied transparently Code: vertex.go chat/responses endpoints

Model-Specific Responses API Handling

Severity: Low Behavior: Responses API automatically routes to Anthropic or Gemini implementation based on model Impact: Different conversion logic applied transparently per model Code: vertex.go:836-1080

Anthropic Version Lock

Severity: Low Behavior: anthropic_version always set to vertex-2023-10-16 for Claude Impact: Cannot override Anthropic version for Claude on Vertex Code: utils.go:33, 71

Embeddings Precision Preservation

Severity: Low Behavior: Vertex returns float64 embeddings, and Bifrost preserves that precision in normalized embedding responses Impact: No precision loss in the /v1/embeddings response path Code: embedding.go:84-91

List Models API Returns Only Custom Models

Severity: High Behavior: Vertex AI’s List Models API only returns custom fine-tuned models, NOT foundation models Impact: Bifrost performs three-pass discovery to include foundation models from aliases and the key-level models allowlist Why: This is a Vertex AI API limitation - foundation models must be explicitly configured Code: models.go:76-217

Configuration

HTTP Settings: OAuth2 authentication with automatic token refresh | Region-specific endpoints | Max Connections 5000 | Max Idle 60 seconds Scope: https://www.googleapis.com/auth/cloud-platform Endpoint Format: https://{region}-aiplatform.googleapis.com/v1/projects/{project}/locations/{region}/{resource} Note: For global region, endpoint is https://aiplatform.googleapis.com/v1/projects/{project}/locations/global/{resource}

Video Generation

Vertex AI routes video generation through Gemini’s Veo models using the predictLongRunning endpoint. All parameters are identical to Gemini Video Generation.

Only Veo models are supported (e.g., veo-2.0-generate-001). Passing a non-Veo model name returns a configuration error.

Supported Operations

Operation	Supported	Notes
Generate	✅	`POST /v1/videos`
Retrieve	✅	`GET /v1/videos/{id}`
Download	✅	`GET /v1/videos/{id}/content`
Delete	❌	Not supported
List	❌	Not supported
Remix	❌	Not supported

​Overview

​Supported Operations

​Setup & Configuration

​1. Service Account JSON (Recommended for Production)

​2. Application Default Credentials

​3. API Key (Gemini and Fine-Tuned Models Only)

​4. AWS Workload Identity Federation (EKS)

​GKE Workload Identity Federation

​GCP Prerequisites

​Verify

​Troubleshooting

​AWS Workload Identity Federation

​GCP Prerequisites (AWS)

​Verify (AWS)

​Troubleshooting (AWS)

​Beta Headers

​1. Chat Completions

​Request Parameters

​Core Parameter Mapping

​Key Configuration

​Authentication Methods

​Gemini Models

​Parameter Mapping for Gemini

​Anthropic Models (Claude)

​Parameter Mapping for Anthropic

​Special Notes for Vertex + Anthropic

​Region Selection

​Multi-region-only models

​Streaming

​2. Responses API

​Request Parameters

​Core Parameter Mapping

​Gemini Models

​Anthropic Models (Claude)

​Configuration

​Special Handling

​3. Embeddings

​Request Parameters

​Core Parameters

​Advanced Parameters

​Embedding Parameters

​Task Type Effects

​Response Conversion

​4. Image Generation

​Request Parameters

​Core Parameter Mapping

​Model Type Detection

​Configuration

​Request Conversion

​Response Conversion

​Endpoint Selection

​Streaming

​5. Image Edit

​6. List Models

​Request Parameters

​Response Conversion

​Custom vs Non-Custom Models

​Three-Pass Model Discovery

​Model Filtering Logic

​Model Name Formatting

​Example Configuration

​Pagination

​Caveats

​Configuration

​Video Generation

Overview

Supported Operations

Setup & Configuration

1. Service Account JSON (Recommended for Production)

2. Application Default Credentials

3. API Key (Gemini and Fine-Tuned Models Only)

4. AWS Workload Identity Federation (EKS)

GKE Workload Identity Federation

GCP Prerequisites

Verify

Troubleshooting

AWS Workload Identity Federation

GCP Prerequisites (AWS)

Verify (AWS)

Troubleshooting (AWS)

Beta Headers

1. Chat Completions

Request Parameters

Core Parameter Mapping

Key Configuration

Authentication Methods

Gemini Models

Parameter Mapping for Gemini

Anthropic Models (Claude)

Parameter Mapping for Anthropic

Special Notes for Vertex + Anthropic

Region Selection

Multi-region-only models

Streaming

2. Responses API

Request Parameters

Core Parameter Mapping

Gemini Models

Anthropic Models (Claude)

Configuration

Special Handling

3. Embeddings

Request Parameters

Core Parameters

Advanced Parameters

Embedding Parameters

Task Type Effects

Response Conversion

4. Image Generation

Request Parameters

Core Parameter Mapping

Model Type Detection

Configuration

Request Conversion

Response Conversion

Endpoint Selection

Streaming

5. Image Edit

6. List Models

Request Parameters

Response Conversion

Custom vs Non-Custom Models

Three-Pass Model Discovery

Model Filtering Logic

Model Name Formatting

Example Configuration

Pagination

Caveats

Configuration

Video Generation