Data-Retention-Aware AI API Routing: Enforce ZDR, Residency, and Logging Policies at the Gateway

Security teams do not only need to know which model is cheapest, fastest, or most capable. They need to know whether a specific request can legally and operationally be sent to a specific provider, endpoint, region, feature, and logging mode.

That is harder than it sounds. A model may be acceptable for ordinary internal chat, but not for customer PII. A provider may offer zero data retention for one API path, while a search-grounding feature stores prompts and outputs for a fixed period. A region may support storage residency, but not the processing mode you expected. Developer-owned logs may be configurable, while provider abuse-monitoring logs follow a different policy.

The practical answer is to move retention decisions out of individual applications and into the AI API gateway. The gateway should classify the request, evaluate it against a provider capability matrix, block incompatible features, route only to approved model profiles, and record a policy decision without storing raw prompts by default.

The reader problem: provider privacy terms are not runtime controls

Most teams start with a spreadsheet or security review that says which AI providers are approved. That is useful, but it is not enough for production routing.

Applications make runtime choices:

• Which model ID should handle this request?
• Should the request use search grounding, file upload, code execution, batch processing, prompt caching, or stored conversations?
• Which region or endpoint should process the request?
• Can the system log the raw prompt for debugging?
• Can fallback routing send the same request to another provider?

Each of those choices can change the retention profile. A request that was compliant in plain chat mode may become non-compliant when the developer turns on grounding or persistent conversation storage. A fallback rule designed for reliability may accidentally route regulated data to a provider path that has not been approved for zero data retention, data residency, or abuse-monitoring controls.

Recommendation: treat retention behavior as a first-class routing constraint, not as documentation attached to a provider account.

Facts to encode before designing policy

The exact terms vary by provider, product, contract, region, endpoint, and feature. Do not rely on memory or a one-time review. Build a source-owned matrix and update it when terms change.

Several current public provider documents illustrate why this is necessary:

• OpenAI: API data residency is documented as project-configured, with regional requests requiring region-specific domain prefixes. OpenAI also distinguishes storage support from processing support by region and notes additional requirements for non-US regions. OpenAI states that non-US API data residency requires approval for abuse-monitoring controls and a Modified Retention amendment.
• Anthropic: Anthropic documents zero data retention for API-related commercial use cases, while noting that some related products or compliance feeds have separate retention models, including longer retention for Activity Feed and remote session transcripts.
• Google Gemini: Gemini API terms distinguish unpaid and paid services. For unpaid services, Google may use submitted content and generated responses to improve products; for paid services, Google says prompts and responses are not used to improve products. Gemini Developer API ZDR documentation says paid-service abuse-monitoring logs normally retain prompts and responses for a limited period, while approved ZDR projects clear user content and identifiable metadata before logging.
• Feature-specific storage: Gemini documentation states that Grounding with Google Search and Grounding with Google Maps store prompts, contextual information, and generated output for 30 days, with no way to disable that storage when those features are used.
• Developer-owned logs: Gemini API logging documentation says developer-owned API logs can be retained up to 55 days by default for billing-enabled projects and that developers can choose shorter windows such as 7, 14, or 28 days.
• Risk management: NIST’s Generative AI Profile recommends monitoring AI-generated content for privacy risks and connecting generative-AI policies to existing data, software, legal, compliance, and risk-management processes.

These are facts to verify against current vendor documentation before rollout. The architectural lesson is stable: retention is not one provider-level Boolean.

Architecture: a gateway policy engine in the request path

A retention-aware gateway has five core components:

1. Request sensitivity classifier: labels the workload before routing.
2. Provider capability matrix: describes provider, model, endpoint, region, retention, logging, and feature behavior.
3. Policy-as-code rules: convert security requirements into runtime allow, deny, or review decisions.
4. Feature gate layer: blocks retention-changing features unless explicitly allowed.
5. Audit and analytics layer: records useful metadata without storing raw prompts by default.

The gateway does not need to understand every legal nuance. It needs to enforce the decisions your legal, security, compliance, and platform teams have approved.

Step 1: classify request sensitivity before selecting a model

Start with a small classification taxonomy. It should be simple enough for developers to use, but expressive enough to drive policy.

Example sensitivity labels:

• public: public documentation, marketing copy, public website content.
• internal: non-public company information with low sensitivity.
• confidential: strategy, contracts, customer context, unreleased product details.
• customer_pii: names, emails, addresses, account identifiers, support transcripts.
• regulated: healthcare, financial, legal, education, or jurisdiction-specific protected data.
• source_code: proprietary code, configuration, architecture files.
• credentials: secrets, tokens, passwords, private keys. In most systems this should be blocked, not routed.

Classification can come from multiple sources:

• An application-supplied header, such as X-Data-Class: customer_pii.
• Tenant policy, where all traffic from a regulated customer is treated as regulated unless downgraded by an approved rule.
• Endpoint policy, where support-ticket summarization defaults to customer_pii.
• Lightweight content scanning for credentials, obvious PII, or policy violations.

Recommendation: do not depend entirely on automatic detection. Require applications to declare the intended data class, then use scanning to catch obvious mismatches or force a safer class.

Step 2: build a provider capability matrix

The capability matrix is the source of truth the router evaluates. It should be versioned, reviewed, and tested like production configuration.

Example fields:

{
  "profile_id": "provider_x.chat.eu.zdr",
  "provider": "provider_x",
  "model": "model-large",
  "api_family": "chat_completions",
  "endpoint": "https://eu.example-provider.com/v1",
  "region": "eu",
  "processing_residency": ["eu"],
  "storage_residency": ["eu"],
  "zdr_eligible": true,
  "zdr_contract_required": true,
  "training_use": "not_used_for_training_on_paid_api",
  "abuse_monitoring": "approved_modified_retention_required",
  "developer_log_retention_days": 0,
  "raw_prompt_logging_allowed": false,
  "supported_features": {
    "plain_chat": true,
    "streaming": true,
    "tool_calls": true,
    "search_grounding": false,
    "maps_grounding": false,
    "file_upload": false,
    "batch": false,
    "stored_conversations": false
  },
  "last_reviewed": "2026-08-01",
  "source_refs": ["security-review-123", "vendor-doc-version-abc"]
}

Use model profiles rather than raw model IDs. A profile combines model, provider, endpoint, region, feature set, and retention posture. Developers request model_profile: compliant_summarization, not just model: fastest-large-model.

Recommendation: include contractual prerequisites in the matrix. A route is not ZDR-approved merely because a vendor offers ZDR somewhere. It is approved only when your account, project, region, and endpoint meet the required conditions.

Step 3: write policy-as-code rules

Policy rules should be explicit, testable, and readable by security and platform teams.

Example rules in pseudocode:

deny if data_class == "credentials"
  reason "credentials_must_not_be_sent_to_model"

allow only if data_class in ["regulated", "customer_pii"]
  and profile.zdr_eligible == true
  and profile.zdr_contract_required_satisfied == true
  reason_on_failure "model_profile_not_zdr_eligible"

deny if residency_required == "eu"
  and "eu" not in profile.processing_residency
  reason "region_processing_not_supported"

deny if data_class in ["confidential", "customer_pii", "regulated"]
  and request.raw_prompt_logging == true
  reason "raw_prompt_logging_not_allowed"

deny if request.features.search_grounding == true
  and policy.requires_zdr == true
  and profile.feature_storage.search_grounding_days > 0
  reason "grounding_requires_retained_content"

deny if fallback_profile.retention_level < primary_profile.retention_level
  reason "fallback_weakens_retention_policy"

These rules should run before provider selection and again before fallback. Fallback routing is a common source of accidental policy drift: the primary route may be compliant, while the fallback route is merely available.

Step 4: treat tools and features as retention-changing capabilities

Do not model retention as a property of the base model alone. Features often change storage, logging, or review behavior.

Give each feature its own policy flags:

• Search grounding: may store prompts, retrieved context, and generated output depending on provider terms.
• Maps or location grounding: may introduce location-specific logs or retention rules.
• File upload: may store files separately from prompts and responses.
• Code execution: may create temporary files, execution logs, or sandbox artifacts.
• Batch jobs: may have different retention, queueing, and result-storage behavior from synchronous API calls.
• Stored conversations: intentionally persist content and should never be hidden behind a generic chat option.
• Evaluation or review dashboards: may create human-review workflows or longer-lived datasets.

Recommendation: make retention-changing features opt-in at the tenant and route level. If a developer enables grounding_search=true, the gateway should re-evaluate the request against feature storage rules before sending it upstream.

Step 5: preserve analytics without storing raw prompts

Retention-aware routing should not blind the platform team. You can keep useful AI usage analytics while minimizing content storage.

Safe default telemetry fields:

• tenant ID and project ID
• hashed or internal API key ID
• model profile ID and provider ID
• request timestamp and region
• input, output, cached, and reasoning token counts when available
• latency, status code, retry count, and fallback decision
• estimated and settled cost
• data classification label
• policy version and policy decision reason
• feature flags requested and feature flags allowed

Avoid storing raw prompts and model outputs by default for confidential traffic. If debugging requires content, use a controlled workflow:

• customer or tenant approval
• narrow time window
• sampling limit
• redaction pass
• separate access control
• short expiration
• audit log of who enabled it and why

This is a trade-off. Blocking raw prompt logs makes debugging, support, quality review, and abuse investigation harder. But storing everything by default creates a larger privacy, breach, and compliance surface.

Step 6: return actionable denial reasons

A generic 403 forbidden frustrates developers and encourages workarounds. Return a stable machine-readable reason and a human-readable explanation.

Example response:

{
  "error": {
    "type": "policy_denied",
    "code": "grounding_requires_30_day_storage",
    "message": "Search grounding is not allowed for workloads marked requires_zdr because this provider feature stores prompt, context, and output content.",
    "request_id": "req_123",
    "policy_version": "retention-policy-2026-08-01",
    "allowed_actions": [
      "disable_search_grounding",
      "choose_profile:zdr_plain_chat",
      "request_exception"
    ]
  }
}

Useful denial codes include:

• model_profile_not_zdr_eligible
• region_processing_not_supported
• storage_residency_not_supported
• raw_prompt_logging_not_allowed
• feature_requires_content_storage
• fallback_weakens_retention_policy
• contract_prerequisite_missing
• credentials_detected

Step 7: add an exception workflow, not a hidden bypass

Some exceptions are legitimate: incident response, customer-approved debugging, migration testing, or a temporary provider limitation. The gateway should support exceptions without turning them into permanent shadow policy.

Each exception should include:

• approver identity
• requesting team or tenant
• ticket or risk-review link
• business justification
• allowed model profiles and features
• data classes covered
• expiration date
• additional logging requirements

Recommendation: make exceptions narrower than ordinary policy. Avoid global switches such as disable_retention_policy=true. Prefer scoped overrides such as “allow debug prompt logging for tenant A, endpoint B, for 24 hours, with redaction and security approval.”

Operational checklist

• Create a versioned provider capability matrix.
• Assign an owner for provider terms, contract prerequisites, and retention reviews.
• Require applications to declare data class, residency requirement, and requested features.
• Default confidential and regulated traffic to no raw prompt logging.
• Represent tools, grounding, file upload, batch, and stored conversations as separate capability flags.
• Run policy checks before primary routing and before fallback routing.
• Log policy version, model profile, data class, feature flags, and denial reason.
• Keep analytics metadata separate from prompt and output content.
• Test representative allow and deny cases in CI.
• Review policy drift whenever a provider changes terms, regions, endpoints, or features.

Trade-offs to make explicit

Strict routing reduces choice. ZDR and residency constraints may prevent use of the newest model, the lowest-cost route, or a feature-rich endpoint.

Regional routing can increase latency or cost. The nearest compliant region may not support the desired processing mode, or may require a different provider path.

Feature gates surprise developers. A developer may think they are only enabling search, but security sees a new retention behavior. Documentation and denial messages reduce friction.

Prompt minimization complicates debugging. Teams need redacted samples, tenant-approved debug windows, and strong metadata to investigate issues without storing everything.

The matrix requires maintenance. Provider terms change. New models launch. Regions expand. Features move from beta to production. A stale matrix is worse than no matrix because it creates false confidence.

What is a recommendation, and what is a prediction?

Recommendations: enforce retention at the gateway, classify requests before routing, build a provider capability matrix, block retention-changing features by policy, avoid raw prompt logging by default, and version every policy decision.

Prediction: AI platform teams will increasingly treat privacy posture as part of model selection. Instead of asking “which model should we use?” applications will ask for a model profile that satisfies capability, cost, latency, residency, and retention constraints.

Prediction: provider-specific privacy features will continue to diverge. Gateways that normalize only request and response formats will not be enough; production teams will need policy normalization as well.

Actionable conclusion

Data-retention-aware routing is not a separate compliance dashboard. It belongs in the request path.

Start with three deliverables: a request sensitivity taxonomy, a versioned provider capability matrix, and a small set of policy-as-code rules for ZDR, residency, raw logging, fallback, and retention-changing features. Then make the gateway return clear denial reasons and preserve analytics without storing raw content by default.

That design centralizes decisions that would otherwise be scattered across SDK options, environment variables, provider consoles, and team-specific conventions. It also gives security and platform teams a practical audit trail: which request was allowed, which policy version applied, which model profile was selected, and why.

Related reading

• safe prompt logging and gateway-level usage analytics
• building a provider capability matrix during an OpenAI-compatible API migration
• governing agent tools with scopes, approvals, and audit trails