Бесключевая аутентификация восходящего потока для шлюзов AI API: шаблон объединения удостоверений рабочей нагрузки

Production gateway workers should not carry long-lived upstream API keys when the provider can authenticate the workload itself. The safer pattern is to exchange an existing cloud, Kubernetes, CI, or SPIFFE identity for a short-lived provider credential, cache that credential correctly, and restrict it to the smallest useful authorization boundary.

The difficult part is not the token exchange. It is handling uneven provider support, concurrent refreshes, identity mapping, control-plane exceptions, outages, and migration without creating a hidden fallback to a broadly privileged shared key.

Start with authentication capabilities, not a global keyless setting

Fact: Authentication support differs by provider and operation. OpenAI and Anthropic support workload identity federation for API workloads. Azure OpenAI supports Microsoft Entra ID and managed identities. Google Cloud supports attached service accounts, Application Default Credentials, and Workload Identity Federation for external workloads. However, runtime inference and administrative APIs do not necessarily accept the same credential types.

For example, OpenAI workload identity tokens currently cannot call Admin API endpoints. A gateway therefore cannot safely represent authentication as one provider-level field such as auth_mode: federated.

Recommendation: Build a capability matrix keyed by provider account, endpoint family, environment, and operation class:

Store this matrix as versioned configuration. Route admission should reject an operation when its configured identity cannot satisfy the required authentication capability. This belongs in an AI API control plane for developer teams, not in undocumented environment variables on individual workers.

Put a credential broker between routing and provider adapters

Provider adapters should not fetch metadata credentials, construct assertions, or manage token refresh independently. Introduce a credential broker with one narrow interface:

Credential acquire(CredentialRequest request)

CredentialRequest {
  provider_account_id
  workload_identity_id
  operation_class
  target_service_account
  audience
  scopes
}

Credential {
  value
  expires_at
  authorization_context
  provenance
}

The credential value is sensitive and must stay in memory or an approved ephemeral cache. The authorization context records the provider project, service account, scopes, roles, or deployment restrictions represented by the token. Provenance identifies how it was acquired, such as oidc_exchange, managed_identity, adc, or break_glass.

This contract keeps routing provider-neutral while preserving details needed for authorization checks and telemetry. It also prevents an multi-model API gateway for teams from treating every bearer token as interchangeable.

Use provider-specific acquisition adapters

Each acquisition adapter should perform only the steps required by its provider:

• Obtain an external assertion from the local identity source, such as a Kubernetes projected service-account token, cloud metadata service, or SPIFFE workload API.
• Request the exact audience expected by the provider or configured federation rule.
• Exchange the assertion when the provider requires token exchange.
• Return the resulting credential and authoritative expiration time.
• Classify failures as local identity, exchange, authorization, configuration, or provider availability errors.

Vertex AI's OpenAI-compatible endpoint is a useful example of separation: an application can obtain a Google access token through Application Default Credentials and supply that token as the OpenAI client's credential. The protocol adapter remains OpenAI-compatible, while credential acquisition remains Google-specific.

Make the token cache authorization-aware

Fact: Federated credentials are short-lived. OpenAI workload identity access tokens last no more than one hour, cannot outlive the external subject token or verified certificate, and are issued without refresh tokens. The workload must repeat the exchange before expiration.

A cache key must include every dimension that can change the resulting authority:

cache_key = hash(
  provider_account_id,
  workload_identity_id,
  target_service_account,
  audience,
  normalized_scopes,
  operation_class
)

Omitting the target account, audience, or scopes can cause a worker to reuse a valid token in the wrong authorization context. That is more dangerous than a cache miss.

Refresh before expiry using a safety margin and randomized jitter. One practical policy is:

refresh_at = expires_at
             - max(configured_margin, token_lifetime * 0.10)
             - random_jitter

The exact margin should account for clock skew, exchange latency, retry time, and the longest expected request. Do not refresh solely on a fixed timer detached from expires_at.

Prevent exchange storms with single-flight refresh

When many requests encounter the same near-expiry token, only one execution should perform the exchange. Other callers should continue using the existing credential while it remains inside the permitted window, or wait briefly for the shared refresh result.

Use a per-cache-key single-flight lock. In a multi-process or multi-node deployment, decide whether each worker may maintain its own cache or whether refresh coordination must be distributed. A local cache is simpler but increases exchange volume. A distributed cache reduces exchanges but expands the sensitive-token boundary and requires encrypted transport, strict access control, and careful eviction.

Recommendation: Prefer in-process caches unless provider limits or fleet size demonstrate the need for a shared token service. Coordinate refreshes within each process first, then measure exchange volume.

Bind identities to exact workload claims

Federation moves trust from a stored key to identity claims. A permissive mapping can therefore be as dangerous as a leaked shared credential.

Validate the issuer and intended audience, then bind the mapping to immutable or tightly controlled claims. For Kubernetes, include the namespace and service account rather than trusting any workload in the cluster. For CI, restrict the repository, workflow, environment, and protected branch or deployment context. For cloud identities, map a specific workload principal rather than an entire tenant or organization.

Create separate upstream service accounts or managed identities for production, staging, background jobs, and administrative automation. Restrict each identity to the models, projects, workspaces, deployments, and API operations it needs.

Fact: OpenAI can restrict workload identity mappings with API permissions. Anthropic associates exchanged credentials with a service account and warns that federation is only as secure as the upstream identity configuration. Managed identities and cloud IAM similarly depend on correctly scoped role assignments.

Separate runtime credentials from administrative credentials

An inference worker should never inherit an organization-admin token merely because some control-plane task still needs one. Split the system into two security boundaries:

• Runtime plane: Handles model requests and receives only federated or managed credentials scoped to inference.
• Control plane: Performs project, workspace, billing, or service-account administration through isolated jobs with separately stored credentials and stronger approval controls.

Administrative credentials should have no path into runtime environment variables, pod specifications, debug bundles, or token caches. Record their use in the same append-only audit system used for configuration changes.

Fail closed without making recovery improvised

Recommendation: Do not automatically replace a failed federation exchange with a shared static key. Automatic fallback can bypass a revoked mapping, conceal an incorrect audience, or restore access that an identity administrator intentionally removed.

Return a classified upstream-authentication error, stop dispatch to the affected provider account, and allow normal routing policy to select another provider only when that alternative is permitted for the tenant and workload. Model fallback and credential fallback are different decisions.

A break-glass key may still be justified for severe identity-provider outages. Keep it disabled or inaccessible in normal operation. Activation should require an authenticated approval, have a fixed expiration, restrict the affected provider account and operations, and alert on every request made with it. Its provenance must appear in request telemetry.

Migrate one routing pool at a time

1. Inventory runtime and administrative credentials. Identify which processes can read each one.
2. Create a dedicated workload identity and narrowly scoped provider service account.
3. Configure exact issuer, audience, subject, and permission mappings.
4. Add a preflight check that acquires a token and calls a low-impact permitted endpoint.
5. Canary a small routing pool using federation only. Do not leave automatic key fallback enabled during the test.
6. Compare authentication failures, exchange latency, token age, and provider authorization results with the existing path.
7. Increase traffic gradually and observe at least one complete token-refresh cycle.
8. Remove the old key from runtime access, then revoke it after the defined observation period.

Repeat this process by provider account and environment. A global cutover makes it difficult to distinguish an identity mapping error from a provider-specific token behavior.

Instrument and test the failure modes

Record token exchange latency, refresh attempts, refresh failures, credential age, issuer, mapped principal, provider account, operation class, and authentication error category. Never record access tokens, external assertions, client secrets, or complete authorization headers.

Before federation becomes the only production path, test:

• Clock skew between workers, identity providers, and provider token services.
• Metadata-server or workload identity endpoint unavailability.
• Expired assertions and incorrect audiences.
• Role removal and federation-rule disablement.
• Token exchange throttling and temporary provider errors.
• Concurrent refresh under peak traffic.
• Signing-key rotation while old tokens remain valid.

For OIDC issuer rotation, publish old and new signing keys with sufficient overlap for verifier caches and issued tokens. OpenAI documents 600-second caching for discovery and remote JWKS data, with a JWKS refresh when a token references an unknown key ID. Do not assume every verifier behaves identically; test rotation with live canaries.

Adopt keyless runtime authentication as a bounded migration

The goal is not to claim that the gateway contains no secrets. It is to remove long-lived provider keys from high-volume runtime workers and confine unavoidable administrative credentials to a smaller control-plane boundary.

Start with the capability matrix, broker credentials through one interface, cache by authorization context, use exact identity mappings, and fail closed on exchange errors. Complete the migration only after observing successful refresh, revocation, outage, and signing-key-rotation tests. That produces a keyless runtime path whose availability and authorization behavior are understood rather than assumed.

Related reading

• verifying provider credentials before routing production traffic