Rabaty zbiorcze jako produkt odsprzedawcy: zarządzanie marżą dla poziomów Async AI API

Batch capacity is not just cheaper inference. For an AI API reseller, it is a separate product tier with different economics, customer expectations, wallet behavior, reconciliation rules, and support risk. Treating it as “the same API, just discounted” is how platforms end up with confusing invoices, broken margin assumptions, and customers expecting realtime latency from an asynchronous service.

The practical problem is this: providers may offer materially lower rates for asynchronous batch processing, but your customers buy from you, not from the provider. You need a defensible way to expose the savings while protecting gross margin, reserving enough balance before large jobs run, settling only completed work, and showing customers why the same model can have two different prices.

What Is Fact, Recommendation, and Prediction

Facts

Several major AI providers document asynchronous batch APIs with lower unit pricing than synchronous calls. OpenAI documents Batch API processing with a 24-hour completion window and a 50% discount compared with synchronous APIs. Anthropic documents Message Batches for large volumes of Messages requests with a 50% cost reduction. Google documents Gemini batch mode for large request volumes at 50% of standard cost. Azure OpenAI documents global batch processing with separate enqueued token quota, a 24-hour target turnaround, and 50% lower cost than global standard.

Those facts do not mean every provider supports every endpoint, region, model feature, file format, or operational behavior in the same way. Current limits and supported features should be checked against provider documentation before publication, pricing updates, or contract changes.

Recommendations

Expose batch as an async tier, not as a hidden discount inside realtime inference. Keep realtime and async usage in separate ledger categories. Reserve customer wallet balance before accepting a job, then settle only completed request rows. Show separate invoice lines for realtime and async work, even when both use the same model family. Publish a margin policy that explains whether the provider discount is passed through, retained, or blended into a fixed customer price.

Predictions

As batch APIs mature, more reseller platforms will sell “async AI credits” as a separate budget pool. Some customers will try to route latency-sensitive work through batch tiers to lower their bill, so gateways will need controls that enforce turnaround expectations, job size rules, and workload classification. The platforms that win customer trust will be the ones that make batch savings visible without making invoices harder to audit.

Define Batch as a Customer-Facing SKU

The first product decision is naming. Do not sell it as “discounted GPT,” “cheap Claude,” or “half-price Gemini.” Those names anchor customers on the model and price, not the service behavior. A clearer catalog separates service tiers from model families:

• Realtime: interactive requests where latency matters and responses are expected during the user workflow.
• Priority: higher-cost, lower-latency capacity for urgent application paths.
• Standard: normal synchronous inference with ordinary rate and quota behavior.
• Async batch: queued processing for jobs that can complete later and tolerate partial failure handling.

This distinction matters for AI API margin management because the customer is not buying a lower-quality version of the same product. They are buying a different delivery promise. That delivery promise should say what the tier is for: nightly evaluations, document enrichment, large embedding jobs, moderation backfills, synthetic data review, transcript analysis, or other workloads that do not block an end user.

A good async SKU description includes four fields: supported model families, expected turnaround language, maximum job size policy, and billing behavior. Avoid saying “guaranteed within 24 hours” unless the upstream provider contract actually guarantees it. If the provider says target turnaround, use target turnaround. If the provider says completion window, quote that carefully in customer-facing terms.

Choose a Pricing Model Before You Ship

Batch discounts create a commercial choice. The provider may charge you less, but you decide how much of that discount the customer sees. There are three practical approaches.

Full Pass-Through Discount

In this model, if provider batch capacity is 50% cheaper, the customer price also drops by roughly 50%, plus any explicit platform fee. This is easy to explain and attractive for price-sensitive customers. It works best when your reseller strategy is volume growth, not margin expansion.

The trade-off is margin compression. It may also train customers to avoid realtime tiers even when the workload is not appropriate for batch. If you choose full pass-through, protect the realtime product with explicit use-case guidance, job controls, and different budgets.

Partial Pass-Through Discount

Here the provider discount is shared. For example, a provider’s realtime cost is $10 per unit and batch cost is $5. You might charge the customer $7.50 for async batch. The customer sees a meaningful 25% discount from realtime, and you retain additional margin for queueing, reporting, support, failed-row handling, and reconciliation.

This is often the most defensible reseller model because the platform is doing real work beyond forwarding a provider batch file. The key is transparency: the customer does not need to see your upstream cost, but they should see that async batch has its own unit price and invoice category.

Fixed Async Price Plans

Some resellers prefer fixed prices by model family or package: “Async Standard Models,” “Async Premium Models,” or “Async Embeddings.” This can simplify sales and quoting when upstream provider pricing changes frequently. It also reduces invoice noise for customers who care more about budget predictability than provider-level passthrough.

The risk is price drift. If provider list prices, batch discounts, region modifiers, or model mappings change, a fixed plan can silently become unprofitable. Use a versioned price table with effective dates and contract IDs. A minimum useful schema includes:

pricing_policy_version
customer_contract_id
model_family
service_tier
provider_list_price
provider_batch_price
customer_realtime_price
customer_async_price
discount_pass_through_percent
effective_from
effective_until
currency
rounding_rule

Separate Realtime and Async Ledgers

The ledger is where product clarity becomes operational. If realtime calls and async batch rows land in one blended usage bucket, customers will eventually ask why the same model was billed at different rates. Finance will ask whether margin is calculated correctly. Support will struggle to explain partially completed jobs.

Use separate ledger categories for realtime and async work. At minimum, each settled usage record should include tenant, customer contract, API key or service account, model alias, provider, service tier, job ID, request row ID, input units, output units, provider cost basis, customer price basis, reservation ID, settlement status, and invoice line group.

This separation also helps unified AI API billing stay understandable. A customer invoice can show two lines for the same model family:

The customer does not need to understand the provider’s internal batch object. They need to understand that async work was priced differently because it used a different service tier with different turnaround expectations.

Reserve Wallet Balance Before Accepting Jobs

Batch jobs can be large. If you accept a job without checking funds, you may discover hours later that the customer cannot pay for completed work. The safer pattern is quote, reserve, execute, settle, and release.

1. Quote: Estimate the maximum customer charge from submitted rows, model, service tier, token estimates, tool usage if relevant, and configured async price.
2. Reserve: Hold wallet balance or credit limit before creating the upstream batch job.
3. Execute: Submit the job to the provider and track provider status separately from customer billing status.
4. Settle: Charge only rows that completed and produced billable usage.
5. Release: Return unused reservation for failed, expired, canceled, rejected, or unprocessed rows.

Over-reservation is a real customer experience problem. Conservative estimates protect you, but they can lock a customer’s credits for many hours. A better implementation stores both an estimated maximum and a running provisional estimate. If the provider returns row-level usage during or after completion, update the provisional amount and release excess funds promptly after finalization.

Every reservation movement should be a ledger entry, not a mutable balance edit. A clear sequence might be: reservation_created, reservation_increased, row_settled, row_failed_release, job_expired_release, manual_adjustment, invoice_finalized. This makes disputes auditable.

Settle at Row Level, Not Only Job Level

Batch jobs often have mixed outcomes. Some rows complete, some fail validation, some hit model limits, and some may remain unprocessed if a job expires or is canceled. If your ledger only stores batch-level totals, you cannot cleanly answer which customer rows were billed.

Use durable row-level records with stable IDs. Each row should have a customer-visible row identifier, an internal row ID, provider custom ID if supported, request hash, model alias, estimated cost, final usage, final status, error class, settlement status, and invoice line reference. The request hash helps with idempotency and dispute review without requiring raw prompt retention by default.

Row-level settlement has a higher storage and ingestion cost, but it is the only robust way to handle partial completion. It also supports better customer reporting: “9,842 rows completed, 121 failed validation, 37 expired, $18.42 released.” That is more useful than “batch failed” or “batch partially complete.”

Expose Batch State Through the Partner API

If your customers are agencies, SaaS platforms, or downstream reseller portals, they need to show batch status inside their own product. A useful Partner API should expose job state, row counts, estimated cost, reserved amount, settled amount, released amount, and error summaries. This is part of practical AI API reseller operations, not a nice-to-have dashboard feature.

A compact response shape can look like this:

{
  "job_id": "batch_job_123",
  "customer_id": "cust_456",
  "service_tier": "async_batch",
  "model_alias": "chat-premium",
  "status": "running",
  "submitted_rows": 10000,
  "completed_rows": 8420,
  "failed_rows": 31,
  "expired_rows": 0,
  "estimated_customer_cost": "42.00",
  "reserved_amount": "50.00",
  "settled_amount": "35.84",
  "released_amount": "0.00",
  "currency": "USD",
  "partial_completion_allowed": true
}

Use statuses that map provider details into customer-meaningful states: queued, validating, running, completed, completed_with_errors, failed, canceled, expired, settling, settled. Keep the original provider status in an internal field or diagnostic endpoint for support teams.

Customer Controls That Prevent Misuse

Async tiers need controls because customers will optimize for price. Some will try to send interactive workloads through batch if the discount is large enough. That creates product friction when a user-facing workflow waits for a job that was never designed for immediate completion.

Add controls at customer, key, and job level:

• Allowed service tiers: realtime only, async only, or both.
• Allowed model families: prevent expensive async jobs on models outside the contract.
• Maximum job spend: reject jobs above a customer-configured cap.
• Maximum accepted turnaround: let customers express whether a job is useful after 1 hour, 6 hours, or 24 hours.
• Partial completion policy: require all rows, accept partial completion, or cancel if failure rate crosses a threshold.
• Workload tag: evals, embeddings, enrichment, moderation, backfill, analytics, or other governed categories.

These controls also help support. When a customer complains that an async job did not return immediately, the job record should show that it was submitted under an async tier with a relaxed turnaround policy and accepted controls.

Operational Alerts for Margin and Trust

Batch discounts improve economics only if operations are controlled. Add alerts for stuck jobs, provider-side expiration, abnormal row failure rates, repeated validation errors, reservation age, settlement lag, and customers repeatedly submitting latency-sensitive workloads to async tiers.

Margin alerts are just as important as reliability alerts. Track effective margin by customer, model family, provider, service tier, and pricing policy version. A provider price change or routing change can turn a profitable async plan into a loss-making plan without any visible application error.

Implementation Checklist

• Create explicit customer-facing SKUs for realtime, priority, standard, and async batch.
• Define a versioned margin policy with provider list price, provider batch price, customer price, pass-through percentage, contract ID, and effective dates.
• Store realtime and async usage in separate ledger categories.
• Reserve wallet balance before accepting an async job.
• Settle completed rows individually and release unused reservation for failed, expired, canceled, or unprocessed rows.
• Expose queued, running, completed, failed, expired, settled, and partially billed states through the Partner API.
• Show realtime and async batch as separate invoice lines, even for the same model family.
• Avoid hard SLA wording unless the upstream provider contract supports it.
• Add customer controls for model access, max job spend, turnaround tolerance, and partial completion.
• Alert on stuck batches, high failure rates, old reservations, settlement lag, and margin drift.

Conclusion

The commercial value of batch APIs is not the discount by itself. It is the ability to turn discounted asynchronous capacity into a product that customers understand and finance teams can audit. That requires clear async SKUs, a deliberate pass-through or retained-margin policy, wallet reservation, row-level settlement, invoice separation, and Partner API visibility.

The simplest defensible rule is this: sell batch as a separate service tier with a separate promise. Customers can get a lower unit price when they accept delayed processing and partial-completion mechanics. You can keep margin when your pricing policy, ledger, and invoice make that trade-off explicit.

Related reading

• versioned pricing catalog for provider price drift