Δημιουργήστε ένα βιβλίο αποδεικτικών στοιχείων AI API προτού υποσχεθείτε πιστώσεις SLA στους πελάτες

When an upstream model provider has an incident, your customers do not only ask whether the provider was down. They ask whether their requests failed, which keys and models were affected, whether the outage counts against your SLA, and what credit they should receive. A provider status page helps, but it cannot answer those tenant-specific questions by itself.

An AI API gateway should treat outage evidence as a first-class ledger: provider signals, synthetic probe results, request-level symptoms, tenant impact windows, attribution decisions, and credit adjustments. The goal is not to create a prettier incident timeline. The goal is to make customer-facing outage decisions explainable, repeatable, and resistant to both over-crediting and under-crediting.

What Provider Status Pages Can and Cannot Prove

Fact: Atlassian Statuspage exposes public API endpoints for summary, status, components, unresolved incidents, incidents, and scheduled maintenance. OpenAI and Anthropic publish Statuspage-style JSON endpoints that include component and incident data. Google Cloud documents Service Health APIs and dashboards, and also exposes public health information through feeds. Cloud providers also publish post-incident reviews for some events.

Those feeds are useful inputs. They can tell you that a provider acknowledged an incident, which broad component was affected, and when the incident moved through states such as investigating, identified, monitoring, resolved, or postmortem. They are not enough to decide customer credits for an AI API reseller operations program.

Status pages often lag real customer impact. They may describe a broad component such as chat, responses, embeddings, images, or files without mapping cleanly to your model aliases, provider accounts, regions, deployments, service tiers, or tenant allowlists. They also cannot tell you whether a specific customer’s requests failed because of upstream degradation, gateway throttling, exhausted budget, invalid input, client-side timeout behavior, or a local network problem.

Recommendation: use provider status as one evidence stream, not as the credit trigger. Credits should depend on tenant impact evidence and a reconciliation pass after request logs, usage settlement, and provider incident updates are available.

The Ledger Record You Need

Start with a durable outage record that can represent both provider-wide incidents and tenant-specific impact. The record should be append-only or versioned so later classification changes do not erase the original detection trail.

{
  "incident_id": "inc_2026_05_29_openai_azure_eastus_latency",
  "provider": "azure_openai",
  "provider_account_id": "acct_prod_shared_01",
  "surface": "chat_completions",
  "model_or_deployment": "gpt-4o-prod-eastus",
  "region": "eastus",
  "source_signals": ["provider_status", "synthetic_canary", "request_symptoms"],
  "confidence": "confirmed",
  "provider_started_at": "2026-05-29T11:42:00Z",
  "provider_acknowledged_at": "2026-05-29T12:05:00Z",
  "detected_at": "2026-05-29T11:45:11Z",
  "mitigated_at": "2026-05-29T12:31:44Z",
  "recovered_at": "2026-05-29T12:48:00Z",
  "affected_tenant_ids": ["ten_123", "ten_456"],
  "linked_request_ids": ["req_a", "req_b", "req_c"],
  "credit_policy_version": "sla_2026_04",
  "credit_status": "pending_reconciliation"
}

The provider incident time and the tenant impact time are separate fields. A provider may report a broad incident from 11:30 to 13:00, but one tenant may have been unaffected because its allowed models used another provider. Another tenant may have seen failures only from 11:47 to 12:18 because routing moved it to a fallback model. A third may have experienced high latency but no failed requests. Those differences matter when credits become money.

Ingest Three Evidence Streams

1. Provider Status APIs

Collect provider status data on a schedule and on demand during incidents. Store the raw payload hash, retrieval time, provider component identifiers, incident state, incident URL or provider reference, and any text summary. Do not overwrite old states. If an incident moves from investigating to identified to monitoring, preserve each transition.

Normalize provider components into your own gateway dimensions:

• provider: OpenAI, Anthropic, Azure OpenAI, Google, or another upstream
• surface: chat, responses, embeddings, images, batch, audio, realtime, files, moderation
• model or deployment: upstream model, internal alias, or cloud deployment name
• region: where applicable
• account or credential pool: shared account, tenant-bound account, or BYOK credential
• service tier: standard, priority, provisioned, batch, or internal tier name

Trade-off: provider status ingestion is cheap and authoritative for acknowledgement, but it may be late, broad, or incomplete. It should raise suspicion and help with customer communication, but it should not automatically generate credits.

2. Synthetic Canaries

Synthetic canaries are small, controlled requests sent through the same gateway path as production traffic. They should cover the combinations you sell, not every possible provider feature. For example, a reseller might run canaries for each provider, surface, model tier, and region used by paid plans.

Each canary result should include:

• canary_id and schedule version
• provider, surface, model, region, credential pool, and route profile
• start time, first byte time, completion time, and timeout threshold
• HTTP status, provider error code, gateway error code, and retry count
• latency percentile bucket, not only pass or fail
• request correlation ID and redacted response metadata

Canaries should avoid raw customer prompts. Use fixed, non-sensitive prompts and keep response validation simple. A canary can verify that the endpoint accepts requests, streams complete, and latency stays within a threshold. It does not need to judge model quality for outage credit decisions.

Trade-off: canaries give independent evidence before customers report a problem. They also add cost and can consume quota during provider stress. Throttle canaries during active incidents and avoid aggressive retry behavior that contributes to upstream load.

3. Real Request Symptoms

Request-level evidence is the strongest source for tenant impact. Every gateway request should already have tenant, key, user or end-user identifier, model alias, provider route, region, status, latency, token usage, and billing state. For outage reporting, add classification fields that separate upstream symptoms from customer and gateway causes.

Useful symptom classes include:

• provider_5xx: upstream HTTP 5XX or equivalent provider server error
• provider_timeout: no provider response before gateway timeout
• provider_429_spike: upstream rate limit above normal baseline
• stream_abort_upstream: provider stream ended unexpectedly
• latency_degraded: completed request exceeded policy threshold
• gateway_throttle: gateway denied the request due to tenant limit or fairness policy
• customer_error: invalid request, unsupported model, bad authentication, or malformed payload
• budget_exhausted: tenant spend or credit limit blocked dispatch

Only the first five should normally count as outage symptoms, and even then only after attribution. A spike in upstream 429s may be a provider problem, but it can also be caused by one tenant exceeding its reserved throughput. A timeout may be provider degradation, but it can also be caused by a client disconnect or an application-level timeout that is shorter than your gateway timeout.

This is where a multi-model API gateway for teams needs disciplined event modeling. The route decision, retry attempt, fallback result, final response, and billing settlement should be joinable by correlation ID.

Separate Provider Incident Windows from Tenant Impact Windows

Do not credit customers for the provider’s broad incident duration. Credit them for the portion of the incident that affected their eligible traffic under your policy.

Use at least five window markers:

• detection time: when your system first detected abnormal behavior
• first affected request: first eligible tenant request with an outage symptom
• provider acknowledgement time: when the upstream provider published or confirmed the incident
• mitigation time: when your gateway applied fallback, traffic shaping, route disablement, or another mitigation
• recovery confirmation: when canaries and real traffic returned below thresholds

The tenant impact window can be computed per tenant, per key, per model alias, and per service tier. For example:

tenant_id: ten_123
model_alias: support-fast
eligible_requests: 842
failed_provider_requests: 96
degraded_latency_requests: 211
first_affected_at: 2026-05-29T11:47:08Z
last_affected_at: 2026-05-29T12:22:31Z
mitigation_applied_at: 2026-05-29T12:04:19Z
credit_status: provisional

This separation also protects you from over-crediting. If a tenant had no traffic during the provider incident, there may be no credit. If a tenant routed only to unaffected models, there may be no credit. If all failures were invalid requests or budget denials, the incident report should say so without blaming the provider.

Define Credit Policy Before the Incident

Credit policy should be a table or versioned configuration, not a support judgment improvised under pressure. The policy does not need to be fully automatic, but it should make eligibility and approval rules explicit.

Include these fields:

• eligible symptoms: provider 5XX, upstream timeout, stream abort, severe latency degradation, or other classes
• minimum duration: for example, five consecutive minutes or a threshold number of affected requests
• latency threshold: fixed threshold, percentile baseline, or service-tier-specific target
• excluded causes: customer errors, budget exhaustion, tenant throttle, malformed requests, unsupported model, client disconnects
• retry treatment: whether successful fallback requests count as failed, degraded, or mitigated
• credit formula: percentage of affected usage, fixed wallet adjustment, plan-based SLA credit, or manual review
• approval authority: automatic below a limit, operations approval above it, finance approval for large tenants
• provider credit handling: passed through, shared, retained, or handled separately from customer credits
• evidence retention: how long incident, request, canary, and credit records remain queryable

Recommendation: store the credit policy version on every provisional and final credit decision. If you revise the policy later, historical decisions should still explain which rules were applied at the time.

Reconcile Before Final Credits

During an incident, customers need status updates quickly. Credit decisions can be provisional. The final credit pass should happen after logs are complete, delayed usage events have settled, retries and fallbacks are classified, and provider post-incident information has been reviewed.

A practical reconciliation workflow looks like this:

1. Create or update the incident record from provider status, canary alerts, or request symptom thresholds.
2. Freeze the affected route profile or mark it as degraded so new routing decisions can avoid it where possible.
3. Generate provisional tenant impact windows from request logs and canary timelines.
4. Exclude customer-side and gateway-side causes using classification rules.
5. Join usage and billing records so affected requests are tied to settled cost and customer charge records.
6. Apply the credit policy version and produce provisional credit rows.
7. Review ambiguous cases, especially partial latency degradation and high-value tenants.
8. Post final credit adjustments to the wallet or invoice ledger with links back to the incident record.

This ties outage handling into unified AI API billing instead of leaving credits as manual notes in a support ticket. The adjustment should be auditable: who approved it, which policy applied, which requests were eligible, and whether it is final or reversible.

Expose Customer and Partner API Views

Customers need a clear incident explanation, not your entire internal trace store. Downstream platforms that resell or embed your API may also need to show incident status inside their own portals. A Partner API endpoint can expose a tenant-scoped incident view.

GET /partner/incidents/{incident_id}/tenant-impact?tenant_id=ten_123

{
  "incident_id": "inc_2026_05_29_openai_azure_eastus_latency",
  "status": "final",
  "summary": "Elevated upstream latency and intermittent failures affected selected chat requests.",
  "affected_windows": [
    {
      "model_alias": "support-fast",
      "first_affected_at": "2026-05-29T11:47:08Z",
      "last_affected_at": "2026-05-29T12:22:31Z",
      "eligible_failed_requests": 96,
      "eligible_degraded_requests": 211
    }
  ],
  "affected_keys": ["key_live_abc"],
  "credit_status": "applied",
  "credit_adjustment_id": "adj_789",
  "evidence": {
    "canary_ids": ["canary_chat_eastus_01"],
    "sample_request_ids": ["req_a", "req_b"]
  }
}

Keep this view scoped. Include synthetic canary IDs, request correlation IDs, affected models, affected keys, and final credit state. Avoid raw prompts and full response bodies by default. If a customer needs deeper evidence, provide a controlled export with redaction and authorization checks.

Customer Incident Report Template

A useful customer report can be short if the ledger is complete. It should distinguish confirmed facts from operational interpretation.

Facts

• Provider and surface affected
• Tenant impact window
• Number of eligible failed, timed out, aborted, or degraded requests
• Affected model aliases, regions, and customer keys
• Mitigation applied and recovery confirmation time

Gateway Assessment

• Whether the symptoms were attributed to the provider, gateway, tenant configuration, or customer request behavior
• Whether fallback routing reduced impact
• Whether the incident meets the credit policy threshold
• Whether credit is provisional, approved, applied, or denied

Limits

• Provider incident details may change after postmortem publication
• Latency degradation thresholds are policy-defined, not universal
• Requests excluded as customer errors or budget denials are not counted as provider outage impact

Predictions and Design Bets

Prediction: customer-facing outage reporting will become a normal feature of AI API reseller platforms, especially for SaaS vendors that expose AI features under their own brand. As AI workloads move from experimentation to production, customers will expect the same incident accountability they receive from cloud infrastructure vendors.

Prediction: provider status pages will remain necessary but insufficient. More incidents will involve partial degradation: latency spikes, streaming interruptions, regional capacity issues, quality regressions, tool-call failures, or account-specific quota behavior. Those cases require request-level and tenant-level evidence.

Recommendation: build the ledger before promising strong SLA credits. Retrofitting incident evidence after a disputed outage is expensive and often inconclusive.

Actionable Conclusion

Do not let provider status pages become your only source of truth for customer credits. Build an outage evidence ledger that joins provider incident feeds, synthetic canaries, request symptoms, route decisions, tenant impact windows, billing records, and final credit adjustments.

The minimum useful version is straightforward: ingest provider status JSON, run low-volume canaries for sold model-region tiers, classify request failures by cause, compute tenant impact windows, and apply a versioned credit policy after reconciliation. That gives support, finance, engineering, and downstream partners the same evidence base. It also lets you explain incidents without blindly mirroring upstream status pages or issuing manual goodwill credits whenever a customer complains.

Related reading

• customer-facing usage and adjustment reporting