Δημιουργήστε μια λογιστική καταγραφής χωρητικότητας AI API προτού αποδεχτείτε νέους ενοικιαστές

Do not wait for live traffic to discover that a new customer cannot be served. Runtime throttles answer a narrow question: should this request run right now? They do not answer the commercial question that matters before onboarding: can this provider account, project, workspace, region, model family, and spend envelope support the tenant you are about to sell?

An AI API capacity ledger gives that question a concrete answer. It converts upstream limits into internal supply, subtracts reservations, applies headroom, and blocks plan creation or tenant activation when the promised workload has no credible backing capacity.

What the Capacity Ledger Controls

A capacity ledger is not a replacement for runtime rate limiting, billing, or incident response. It sits earlier in the lifecycle: sales, plan design, tenant provisioning, model profile assignment, and upgrades.

The reader problem is specific: AI API resellers and SaaS teams can sell access faster than upstream providers can approve, expose, or sustain quota. A customer may buy a plan promising a model, region, latency tier, or monthly volume before anyone has checked whether the relevant upstream pool exists. The result is predictable: 429 errors, blocked upgrades, emergency quota requests, forced fallback models, and awkward customer communication.

The ledger should answer questions like:

• Can we activate this tenant on this model family without consuming reserved enterprise capacity?
• Can a reseller create a plan that promises 20 million tokens per month in a specific region?
• Can this customer upgrade from burstable access to committed capacity?
• Which provider pool backs the promise, and when must that assumption be reviewed?
• What happens if the provider limit has not been reconciled recently?

This is especially important for an AI API reseller platform, where downstream portals may sell packaged access to many customers while upstream quota remains concentrated in a small number of provider accounts.

Facts to Build Around

The following are factual constraints from provider documentation and should shape the design.

• OpenAI documents API rate limits at organization and project level. Limits vary by model, may be shared across model families, and can be inspected through response headers.
• OpenAI documents monthly spend controls separately from rate limits. Enforced hard spend limits can return 429 errors, and enforcement may not be instantaneous.
• OpenAI Admin APIs support organization workflows such as project administration, API key management, spend limits, alerts, audit logs, and rate limit operations.
• Anthropic documents an Admin API Rate Limits API for querying configured organization and workspace limits.
• Anthropic API responses include rate-limit headers showing the enforced limit, current usage, and reset timing.
• Azure OpenAI quota is assigned by subscription, region, model, and deployment type in Tokens-per-Minute units. Deployments consume from the available quota pool.
• Azure documentation recommends the Cognitive Services Usages Reader role as minimal access for viewing available quota across a subscription.
• Google Cloud quotas restrict how much of a resource a project can use, can often be adjusted, and are monitored through Cloud Quotas. System limits are fixed.

These facts imply a design requirement: the ledger cannot model capacity as a single global number. It must preserve provider-specific dimensions while exposing a normalized admission decision to your own product and Partner API.

Recommended Ledger Model

Model upstream capacity as a first-class control-plane object. A useful capacity pool record usually needs these fields:

• provider: openai, anthropic, azure_openai, gemini, or another upstream.
• provider_account_id: the organization, subscription, billing account, or tenant that owns the quota.
• scope: organization, project, workspace, cloud project, region, deployment, or service account.
• region: required for Azure and residency-sensitive workloads; optional or global for providers where limits are not regional.
• model_family: the normalized internal family, not only the exact provider model string.
• deployment_type: standard, provisioned, batch, priority, realtime, or another capacity class.
• rpm_limit: requests per minute where available.
• tpm_limit: tokens per minute or equivalent token-rate capacity.
• batch_queue_limit: queued jobs, enqueued tokens, file size, or provider-specific async capacity.
• monthly_spend_cap: the hard or configured upstream budget boundary.
• sellable_fraction: the percentage of measured capacity allowed for commercial commitments.
• effective_from and effective_until: so temporary quota increases do not become permanent assumptions.
• source: API, console evidence, account-manager approval, manual override, or inferred from headers.
• last_reconciled_at: for stale-data warnings and approval gates.

Then add reservation records that consume from those pools:

• tenant_id or reseller_customer_id: the customer or downstream tenant.
• reservation_class: committed, burstable, internal, batch, free_trial, or emergency.
• model_profile_id: the product-facing bundle of models and fallbacks.
• required_rpm and required_tpm: the admission estimate.
• monthly_volume_commitment: expected monthly token volume or spend.
• region_policy: required, preferred, or unrestricted.
• status: pending, active, degraded, expired, rejected, or awaiting_provider_quota.
• approved_by and approval_reason: for commercial exceptions.
• review_at: when capacity must be checked again.

The point is not to perfectly predict every token. The point is to make sales and onboarding decisions explicit enough that they can be audited, challenged, and updated before customers depend on them.

Admission Control Is Different From Runtime Throttling

Runtime throttling is request-level protection. It estimates or measures pressure for a live request and decides whether to dispatch, delay, degrade, or reject. That is necessary, but it is reactive.

Admission control is supply-side planning. It decides whether a tenant, reseller plan, model profile, or upgrade is allowed to exist. A strong implementation connects capacity checks to Partner API automation, plan validation, customer onboarding, and admin approvals.

For example, plan creation should fail validation if the plan promises:

• a model family that has no mapped provider pool;
• a region where no quota exists;
• a latency tier backed only by best-effort capacity;
• a committed token volume larger than sellable headroom;
• a batch tier where queue limits are unknown;
• a spend commitment that would push the provider account near its hard cap.

This is not customer-facing throttling. It is product integrity. If your gateway lets a reseller advertise a plan that cannot be backed by provider quota, runtime code will inherit a promise it cannot keep.

Calculate Sellable Capacity With Headroom

Do not sell 100 percent of the upstream number. Provider limits can drift, model families can share pools, retries consume extra quota, and customer traffic does not arrive evenly. A practical policy exposes only a fraction of measured capacity as sellable.

A simple formula is enough to start:

sellable_tpm = provider_tpm_limit * sellable_fraction
available_tpm = sellable_tpm - active_reserved_tpm - pending_reserved_tpm

If a provider pool exposes 1,000,000 TPM and your policy sells 70 percent, only 700,000 TPM is available for commitments. If active tenants reserve 500,000 TPM and pending approved deals reserve 100,000 TPM, the remaining available TPM is 100,000.

The sellable fraction should vary by pool type:

• Stable high-volume pool: 70-80 percent may be reasonable after reconciliation is mature.
• New provider account: 50-60 percent is safer until real traffic confirms behavior.
• Shared model-family pool: reduce sellable capacity because unrelated model aliases may compete.
• Manual or stale evidence: reduce sellable capacity or block new commitments.
• Emergency fallback pool: do not sell it as normal capacity.

Spend caps need a similar treatment. A provider hard spend limit is a budget backstop, not a product capacity plan. If the upstream hard cap is $50,000 per month, the gateway should reserve a lower internal threshold for planned usage and alert before the hard limit can interrupt production traffic.

Use Reservation Classes Instead of Per-Request Reservations

Do not try to reserve provider capacity for every possible request during admission. That belongs in runtime throttling. Capacity admission works better with reservation classes.

Committed Customers

Committed customers get explicit capacity reservations. Their plans should specify required model families, regions, service tier, token-rate assumptions, and review dates. Idle reservations reduce utilization, but they make promises credible.

Burstable Customers

Burstable customers draw from a shared pool. They can use capacity when available but should not block committed onboarding. Their plan language, API status, and degradation behavior must be clear.

Internal Workloads

Internal evals, support tools, analytics jobs, and administrative automations should have their own reservation class. Otherwise, internal traffic can silently consume capacity that sales believes is available.

Batch Jobs

Batch capacity needs separate modeling because it may be governed by queue size, job count, enqueued tokens, file limits, or turnaround commitments rather than only RPM and TPM.

Free Trials

Free trials should use intentionally constrained pools. A free trial should not consume scarce regional or priority capacity unless that is part of a deliberate go-to-market decision.

Reconcile Provider Capacity on a Schedule

The ledger is only useful if it is fed by reconciliation. Provider control planes change outside your gateway: admins adjust projects, workspaces, deployments, spend caps, keys, regions, and model availability. Build scheduled pulls and event-driven checks after known admin changes.

A practical reconciliation loop looks like this:

1. Fetch provider-visible limits where APIs exist.
2. Normalize dimensions into capacity pool records.
3. Compare new values with the previous ledger snapshot.
4. Mark reservations at risk if capacity decreased or disappeared.
5. Expire temporary increases when their evidence window ends.
6. Raise alerts when stale pools still back active commitments.
7. Record every change in an audit log.

Some capacity evidence will be imperfect. A provider may expose limits through an API, a cloud quota console, response headers, or an account-representative workflow. Support all of these as different evidence types. The ledger should show confidence and freshness, not pretend every provider emits the same control-plane data.

This overlaps with broader AI API quota management, but the capacity ledger has a narrower purpose: deciding what can be safely sold or activated before traffic arrives.

Admission Checks to Put in the Product

Capacity admission should be part of the workflows where promises are created. The most useful checks are usually these.

Plan Creation Check

When an admin creates a reseller plan, validate the advertised model families, service tiers, regional commitments, monthly volume, burst assumptions, and fallback policy. If no provider pool can support the plan with headroom, the plan should remain draft or require an override.

Tenant Activation Check

When a customer is provisioned, reserve the relevant capacity class. The activation response should expose a status such as active, pending quota approval, degraded to fallback, or blocked.

Upgrade Check

Before increasing a tenant’s plan, run the same admission calculation against the target reservation. If capacity is insufficient, the upgrade should become a pending request tied to a provider quota workflow.

Model Profile Check

If model aliases or profiles change, recalculate reservations. A harmless-looking alias promotion can move tenants from one provider pool to another and break the supply model.

Regional Policy Check

For Azure and residency-sensitive workloads, validate region explicitly. Equivalent models in another region do not satisfy a residency promise unless the customer policy allows fallback.

Implementation Detail: Admission Decision Shape

Keep the admission result machine-readable. A Partner API, reseller portal, or internal admin UI should not have to parse human notes.

{
  "decision": "approved",
  "tenant_id": "tenant_123",
  "model_profile_id": "profile_support_fast",
  "reservation_class": "committed",
  "backing_pool_id": "pool_azure_eastus_gpt4_standard",
  "required_tpm": 120000,
  "required_rpm": 600,
  "sellable_tpm_before": 700000,
  "reserved_tpm_before": 520000,
  "available_tpm_after": 60000,
  "headroom_policy": "standard_70_percent",
  "evidence_freshness": "fresh",
  "review_at": "2026-11-01T00:00:00Z"
}

For a rejection, return a reason that a product workflow can act on:

{
  "decision": "blocked",
  "reason_code": "insufficient_regional_tpm",
  "required_region": "eastus",
  "required_tpm": 300000,
  "available_tpm": 90000,
  "recommended_action": "request_provider_quota_increase",
  "fallback_options": [
    "standard_global_pool",
    "lower_volume_plan",
    "pending_activation"
  ]
}

Avoid returning only “no capacity.” The caller needs to know whether the issue is regional quota, model availability, stale evidence, spend cap exposure, missing deployment, or a policy conflict.

Audit the Commercial Decision

Every capacity decision should be auditable. Record:

• who requested the tenant, plan, or upgrade;
• who approved or overrode the decision;
• which provider pool backed the promise;
• which headroom policy was applied;
• which evidence source was used;
• when the reservation expires or must be reviewed;
• what changed if a tenant was degraded or moved to another pool.

This matters during incidents. If a pool exhausts quota, the team needs to know whether the problem came from unexpected traffic, stale provider evidence, an override, a model-profile migration, or a sales plan that should not have been activated.

Trade-Offs

A capacity ledger adds operational overhead. It can also reject revenue until upstream quota is confirmed. That is frustrating, but it is preferable to selling plans that fail under load.

Hard reservations improve predictability for high-value tenants, but they reduce utilization when customers are idle. Soft reservations improve utilization, but they require stronger runtime controls and clearer customer-facing degradation rules.

Regional modeling is necessary for Azure and residency-sensitive workloads, but it complicates fallback routing. A model available in one region may not satisfy a tenant whose data must stay in another region.

Manual evidence is sometimes unavoidable. The risk is not that manual evidence exists; the risk is treating it like fresh API-derived truth. Use stale-data warnings, expiration dates, and approval requirements.

Provider hard spend limits are useful backstops, but they can interrupt production traffic with 429s. The gateway should alert and reserve internally before upstream hard caps become the enforcement mechanism.

What Is a Recommendation, and What Is a Prediction?

Recommendations: model capacity as ledger objects, reconcile provider limits, apply headroom, reserve by tenant class, connect admission checks to plan and tenant workflows, and audit every decision.

Predictions: capacity admission will become more important as reseller portals, agency offerings, and SaaS tenant onboarding become more automated. Customers will expect instant activation, while provider quota approvals and regional capacity will remain uneven. Teams that make supply explicit in the control plane will have fewer emergency escalations than teams that rely only on runtime throttles.

Facts: the provider-specific statements above should be treated as facts from current public documentation, but provider APIs and quota semantics can change. Keep reconciliation code and documentation links under review.

Actionable Conclusion

Start with one high-value workflow: block committed tenant activation unless a fresh provider pool can support the requested model family, region, rate, and spend envelope after headroom. Then add plan validation, upgrade checks, stale-evidence alerts, and Partner API status fields.

The minimum useful version is small: a capacity pool table, a reservation table, a reconciliation job, a headroom policy, and an admission decision API. That is enough to move capacity from tribal knowledge into the product control plane, where sales, support, operations, and downstream reseller portals can see whether a promise is actually backed by supply.

Related reading

• runtime rate-limit shaping for live AI API traffic