Provider-referentiekluizen voor multi-model AI-gateways: afzonderlijke runtime-, beheerders-, facturerings- en BYOK-toegang

Downstream API-sleutels en upstream-providerreferenties lossen verschillende problemen op. Een door uw gateway uitgegeven ontwikkelaarssleutel identificeert de context van de app, het team, de tenant, het budget en het beleid. Met een upstream providersleutel kan de gateway geld uitgeven en toegang krijgen tot modellen in een provideraccount. Door deze als hetzelfde soort geheim te behandelen, eindigen teams met één onbeperkte sleutel in een gedeeld project, beheerdersreferenties in runtime-services en geen betrouwbare manier om te antwoorden welke tenant welke kosten aan de providerzijde heeft veroorzaakt.

Het praktische patroon is een kluis voor de inloggegevens van de provider: een speciaal controlevlak voor het importeren, classificeren, opslaan, selecteren, roteren en controleren van upstream-referenties. Het moet zich achter de router, het factureringsgrootboek, de beleidsengine en de operationele workflow bevinden, niet in de applicatiecode, modelconfiguratiebestanden, tenantrecords of analysegebeurtenissen.

Het lezersprobleem: upstream-referenties worden een onzichtbare infrastructuur

De meeste implementaties met meerdere modellen beginnen met een eenvoudig doel: stuur één OpenAI-compatibel verzoek door naar de best beschikbare provider. Dan verschijnen er meer accounts: één providerproject voor productie, een ander voor evaluatie, een Anthropic-werkruimte voor een bedrijfseenheid, een Google Cloud-project voor Gemini en verschillende door de klant geleverde sleutels voor BYOK-contracten.

Het risico is niet slechts een geheime lekkage. Het is verlies van autorisatiecontext. Een geldige providersleutel kan technisch gezien mogelijk een eindpunt aanroepen, maar de gateway moet nog steeds weten of die sleutel is toegestaan voor deze tenant, deze modelfamilie, dit gegevensbewaarbeleid, dit budget, deze regio en dit automatiseringspad.

Feit: providerplatforms stellen verschillende accountgrenzen en inloggegevens bloot. OpenAI documenteert projecten en serviceaccounts, en API-sleutelmachtigingen voor serviceaccounts zijn standaard lees- en schrijftoegang voor de API-bronnen van het project. OpenAI stelt ook Admin API-sleutelobjecten afzonderlijk van het gewone project-/runtime-API-gebruik bloot. Anthropic documenteert werkruimten als een organisatorische grens en stelt dat Admin API-eindpunten Admin API-sleutels vereisen die verschillen van standaard API-sleutels; Anthropic merkt ook op dat API-sleutels zijn gekoppeld aan de werkruimte waar ze zijn gemaakt en niet tussen werkruimten kunnen worden verplaatst. Volgens de Gemini API-sleuteldocumentatie van Google is elke Gemini API-sleutel gekoppeld aan een Google Cloud-project en worden API-beperkingen aanbevolen om de schade te beperken als een sleutel in gevaar komt.

Aanbeveling: bouw niet één generiek 'provider_key'-veld en noem het klaar. Bouw een referentie-inventaris op die providerspecifieke grenzen behoudt en tegelijkertijd een genormaliseerd beleidsmodel aan de gateway blootstelt.

Definieer een identificatietaxonomie voordat u sleutels accepteert

Een kluis moet dubbelzinnige inloggegevens weigeren. Tijdens het importeren moet de operator of de automatiseringsworkflow de referentie classificeren. Gebruik minimaal deze categorieën:

• Inloggegevens voor runtime-inferentie: gebruikt door de gateway om eindpunten voor modelinferentie aan te roepen, zoals chat, reacties, insluitingen, moderatie, transcriptie of het genereren van afbeeldingen, afhankelijk van de ondersteuning van de provider.
• Inloggegevens voor beheerdersautomatisering: worden gebruikt om organisaties, werkruimten, projecten, gebruikers, sleutels of administratieve bronnen aan de providerzijde te beheren. Deze mogen zich nooit in het runtime-verzoekpad bevinden.
• Facturerings- en rapportagegegevens: worden gebruikt om gebruiks-, facturen-, kosten- of organisatierapporten op te halen waar providers deze API's ondersteunen. Houd ze gescheiden van de gevolgtrekkingssleutels, zodat rapportagetaken geen modelgebruik kunnen genereren.
• Alleen evaluatiegegevens: gebruikt door benchmark-, QA-, migratie- of faseringsworkflows. Ze moeten lage quota hebben, duidelijke milieulabels en mogen niet in aanmerking komen voor terugval in de productie.
• BYOK-inloggegevens van klant: door de klant geleverde sleutels die zijn gekoppeld aan een specifieke tenant, provideraccount, contract en gegevensbeleid. Ze mogen niet worden samengevoegd in gedeelde routering, tenzij de klant zich daar expliciet voor aanmeldt.

Deze taxonomie is niet alleen documentatie. Het moet toegangscontrole, geschiktheid voor routering, waarschuwingen en rotatieworkflows stimuleren. Als een inloggegevens worden geïmporteerd zonder categorie, eigenaar, provideraccountgrens en toegestaan gebruik, moet deze uitgeschakeld blijven.

Bewaar geheimen in een kluis, niet in productrecords

De kluis zou het enige onderdeel moeten zijn dat upstream-referenties kan ontsleutelen. Andere systemen kunnen referenties, hashes, statusvelden en beleidsmetagegevens opslaan, maar niet de inlogwaarde zelf.

Bewaar geen stroomopwaartse geheimen op deze plaatsen

• Tenantprofielrijen.
• Configuratiebestanden voor modelroutering.
• Promptlogboeken of traceerreeksen.
• Analytics-gebeurtenispayloads.
• CI-variabelen voor ontwikkelaars.
• Support tickets, chat tools, or screenshots.

A usable vault design has two planes. The secret plane stores encrypted credential material and tightly controls decrypt operations. The metadata plane stores non-secret attributes used by routing and governance. The router should usually need only a credential ID and a short-lived in-memory secret retrieval at dispatch time, not broad database access to every provider key.

Protect the vault as high-value infrastructure: envelope encryption or managed KMS, strict service identities, break-glass procedures, backup and restore testing, access review, and alerting on unusual decrypt volume. A central vault simplifies governance, but it also concentrates risk. That is the trade-off.

Attach policy metadata to every credential

The metadata model should be explicit enough that the gateway can decide whether a credential is eligible before it touches a provider endpoint.

A practical credential record includes:

• credential_id: internal immutable identifier.
• provider: OpenAI, Anthropic, Gemini, Azure OpenAI, or another adapter.
• provider_account_boundary: organization, project, workspace, cloud project, subscription, or equivalent.
• credential_class: runtime, admin, billing, evaluation, or BYOK.
• environment: production, staging, development, evaluation, sandbox.
• tenant_binding: shared platform credential, single tenant, tenant group, or customer BYOK tenant.
• allowed_model_families: for example, text generation, embeddings, vision, image, audio, or specific model profiles.
• allowed_endpoints: normalized gateway capabilities mapped to provider endpoints.
• data_policy: allowed retention class, logging class, residency requirement, and feature restrictions.
• budget_scope: cost center, reseller customer, internal department, or contract.
• owner: named team or accountable person.
• created_at, expires_at, rotation_due_at, last_used_at.
• health_status: unknown, healthy, degraded, unauthorized, quota_exhausted, disabled.
• emergency_disable: immediate routing block independent of normal policy state.

Keep this model provider-neutral, but do not erase provider realities. An Anthropic workspace-bound key and a Gemini key tied to a Google Cloud project are not interchangeable just because both can generate text. The gateway needs that provenance for audits, chargeback, and safe failover.

Separate runtime, admin, and billing access

The most important rule is simple: a key used for runtime inference should not manage provider organizations, workspaces, users, projects, or administrative resources.

Runtime traffic is high volume and exposed to the largest operational surface. It passes through request routers, retry logic, streaming handlers, model adapters, and incident workflows. Admin credentials are low frequency and high impact. They should live behind a separate approval path with short TTLs, named human approval where appropriate, strong logging, and no runtime routing eligibility.

Billing credentials also deserve separation. A reporting job that reconciles invoices should not be able to generate completions, and a runtime inference key should not be the only way to retrieve usage reports. When a provider does not offer fine-grained separation, compensate in the gateway: isolate the credential, limit which internal service identity can retrieve it, and log every use.

Recommendation: make credential class a hard authorization boundary, not a label. A runtime dispatcher should be unable to request decryption for an admin credential even if a configuration mistake references its ID.

Build a credential-selection policy engine

Credential selection should happen after the gateway authenticates the downstream caller and before any provider call is attempted. The policy engine should join several inputs:

• Tenant ID and downstream API key scope.
• Requested model profile or provider-specific model ID.
• Endpoint capability: chat, embeddings, image, audio, batch, files, tools, or admin automation.
• Data-retention and residency requirements.
• Budget, credit reservation, and cost center.
• Rate-limit state and quota pressure.
• Credential metadata, health, environment, and tenant binding.

The engine should return one of three results: allow with a selected credential, deny with a policy reason, or require approval. Denials should be precise enough for operations teams to fix the problem without revealing secret material to developers.

Example decision:

{
  "tenant_id": "tenant_42",
  "requested_profile": "fast-text-prod",
  "endpoint": "chat.completions",
  "data_policy": "no_prompt_logging",
  "credential_requirements": {
    "class": "runtime",
    "environment": "production",
    "tenant_binding": "tenant_42",
    "allowed_model_family": "text",
    "health_status": "healthy"
  },
  "decision": "allow",
  "credential_id": "cred_8f2...",
  "audit_reason": "tenant BYOK credential matches runtime text profile and data policy"
}

Do not implement fallback as “try the next key.” Fallback must re-run policy. A shared platform credential may be valid for provider access but invalid for a BYOK-only customer. A credential in another project may have quota, but it may violate cost attribution or retention requirements.

Handle BYOK as tenant-owned access, not spare capacity

BYOK changes the trust model. The customer supplied the credential so their traffic can be charged to, governed by, or isolated within their provider account. That credential should be bound to the customer tenant and provider account provenance.

Recommended BYOK controls:

• One vault record per customer, provider, account boundary, and environment.
• No cross-tenant routing through BYOK credentials.
• No use as shared fallback capacity unless the customer explicitly opts in.
• Customer-visible health state that does not reveal the raw key.
• Separate rotation workflow that allows the customer to add a replacement before the old key is disabled.
• Clear attribution in usage analytics and invoices: gateway tenant, provider account boundary, credential ID, model profile, and request trace ID.

For agencies, resellers, and Partner API automation, BYOK can be more complex because a service may provision tenants and credentials programmatically. The same rule still applies: automation can import and bind credentials, but it should not blur tenant ownership.

Add preflight health checks without leaking prompts

A credential can fail for many reasons: revoked key, wrong workspace, missing model access, disabled billing, quota exhaustion, endpoint restriction, regional policy mismatch, or provider outage. Discovering that only after a production request arrives creates noisy incidents.

Use health checks that validate capability without sending customer prompts. A synthetic check might call a minimal endpoint, list permitted models where appropriate, or send a harmless fixed prompt if that is the only practical option. Keep these checks cheap, rate-limited, and labeled as synthetic traffic in telemetry and billing.

Health checks should run:

• At credential import.
• Before enabling a credential for production routing.
• After provider-side restriction changes.
• During rotation cutover.
• Periodically for credentials with production eligibility.

Trade-off: automated checks catch expired or under-scoped keys early, but poorly designed checks can create unnecessary provider calls, billing noise, or false alarms during provider outages. Store the health result with timestamp, provider error class, endpoint tested, and model family tested. Do not store secret values or sensitive prompts.

Rotate with two slots, not one risky replacement

Credential rotation should not be a delete-and-pray operation. Use a two-slot rotation model:

1. Import replacement credential as inactive, with full metadata and owner.
2. Run synthetic health checks for the intended endpoints, model families, and account boundary.
3. Enable shadow eligibility for a small slice of safe synthetic or low-risk traffic where appropriate.
4. Shift production traffic gradually from old credential to new credential.
5. Monitor errors, latency, quota, and cost attribution by credential ID.
6. Freeze fallback to the old credential once the new credential is stable.
7. Revoke the old credential at the provider and mark the vault record revoked.
8. Verify no decrypts or provider calls occur through the old credential after revocation.

Rotation deadlines should be visible in operations views and alerting. Emergency rotation needs a shorter path: disable credential, block routing, enable approved replacement, and preserve all audit records for incident review.

Restrict provider keys where the provider supports it

Gateway policy is necessary, but provider-side restrictions reduce blast radius if a key is compromised or misused. For Gemini and other cloud-platform API keys, use API/service restrictions and suitable application restrictions where available. For provider projects, workspaces, and service accounts, avoid broad organizational privileges when a project-scoped runtime key is enough.

Recommendation: maintain a provider-side restriction checklist for every credential class. The checklist should be part of import approval and rotation approval, not a separate security task that may be skipped under pressure.

Trade-off: provider-side restrictions add operational overhead. New endpoints, model families, regions, or automation features may require policy and restriction changes. That is preferable to discovering after a leak that one key could access every workload in a shared project.

Keep an append-only credential audit log

An audit trail should answer who imported a credential, what it was allowed to do, which routing decisions selected it, when it failed, and when it was rotated or revoked.

Log these events:

• Credential created or imported.
• Metadata changed, including allowed endpoints, tenant binding, or data policy.
• Health check executed and result recorded.
• Credential selected by routing policy for a request.
• Credential decrypt requested by an internal service identity.
• Provider call failed due to authentication, authorization, quota, or restriction error.
• Rotation started, traffic shifted, old credential revoked.
• Emergency disable enabled or cleared.
• Admin or break-glass credential accessed.

Do not put raw credential values in audit events. Use credential IDs, provider account boundaries, request trace IDs, actor identities, and policy decision reasons. For high-volume runtime traffic, you can sample detailed decrypt telemetry, but routing selection and cost attribution should remain complete enough for billing and incident response.

Implementation checklist

• Create a credential taxonomy and reject unclassified imports.
• Move all provider secrets into a dedicated encrypted vault.
• Store routing metadata separately from secret material.
• Make runtime, admin, billing, evaluation, and BYOK credentials separate authorization classes.
• Bind BYOK credentials to tenant and provider account provenance.
• Require policy-engine approval before selecting any upstream credential.
• Run prompt-safe health checks before production eligibility.
• Use two-slot rotation with gradual traffic shift and provider-side revocation.
• Apply provider-side restrictions wherever available.
• Maintain append-only audit logs for import, use, failures, rotation, and revocation.
• Keep admin credentials behind break-glass controls: short TTL, named approval, strong logging, no runtime use.

Actionable conclusion

Start by inventorying every upstream provider credential currently used by the gateway, scripts, CI jobs, evaluation harnesses, and partner automation. For each one, assign a class, owner, provider account boundary, tenant binding, allowed endpoints, allowed model families, rotation deadline, and emergency-disable status. Anything you cannot classify should be disabled or quarantined until it has a clear purpose.

Then enforce one architectural rule: downstream developers receive gateway-scoped keys; the gateway alone controls upstream provider access. That separation lets you preserve least privilege, tenant attribution, billing accuracy, data-policy routing, and safe automation even as providers, projects, workspaces, and BYOK customers multiply.

Related reading

• team-facing API key management and leak response
• data-retention-aware routing policies
• billing ledger and cost attribution controls