Seldon Inference Cloud
Frontier-quality inference,
at one fifth of the price.
We serve open-weight frontier models on sparse mixture-of-experts architectures, speculative decoding, and FP4 kernels, across a GPU fleet sourced continuously from the cheapest available capacity. The quality is comparable. The bill is not.
420.0B tokens served daily140 teams in production99.97% uptime
- Seldon Prime 1
- $2.10
- blended per 1M tokens
- Claude Opus 4.8
- $10.00
- blended per 1M tokens
- Difference
- 4.8x
- lower cost per token
from openai import OpenAI client = OpenAI( base_url="https://api.seldon-ai.org/v1", api_key=SELDON_API_KEY,) response = client.chat.completions.create( model="seldon-prime-1", messages=[{"role": "user", "content": prompt}],)Two lines change. The rest of your application does not.
In production at
Customer stories- Developer tools / Series BAI pair programming across large monorepos421.5B tok/moDedicated capacity
- Data infrastructure / Series ASchema inference and pipeline repair96.4B tok/moShared endpoint
- Legal / Series BContract review and obligation extraction188.9B tok/moCustomer VPC
- Customer support / Series AAutonomous resolution for logistics operators42.7B tok/moShared endpoint
- Healthcare / Series CClinical documentation and coding support134.2B tok/moCustomer VPC
- Fintech / Series BTransaction enrichment and risk narrative generation209.8B tok/moDedicated capacity
- Data infrastructure / Series ARobotics fleet telemetry summarisation63.1B tok/moDedicated capacity
- Customer support / EnterpriseMultilingual product and order assistance311.4B tok/moCustomer VPC
01The problem
Inference is not a line item. It is your cost of goods.
A traditional software company pays for hosting and keeps roughly 80 cents of every dollar. An AI-native company pays a per-token toll on every unit of value it delivers, and that toll scales exactly with usage. This is the structural reason AI companies trade at lower multiples than the software companies they otherwise resemble.
At published frontier rates, a coding assistant at 1,000 developer seats runs a 14% gross margin. That is beneath the floor investors use to identify a healthy AI-native business, and far beneath the 75 to 85 percent that defines a software company. The product is fine. The arithmetic is not.
See the full margin modelThe question is not whether the model is good enough. It is whether you can afford to serve the answer.
AI coding agent, 1,000 developer seats, at published frontier rates.
Against a 75 percent healthy-SaaS benchmark.
02How
Eight published techniques, assembled into one stack
None of this is proprietary research. Sparsity, speculative decoding, FP4 kernels, KV compression, disaggregated scheduling, and routing are all in the literature. The work is making them coexist in a single serving path under production traffic, and then buying the hardware well.
USD per million output tokens
Llama 3.3 70B dense, FP8 weights, one rented H200 at $4.39/GPU-hr, 1K in / 1K out chat, 50 tok/s/user, stock vLLM with continuous batching, PagedAttention and chunked prefill, no prefix cache, 100% utilization.
| Stage | Multiplier | Running cost |
|---|---|---|
| Stock stack | 1.00x | 0.70 |
| Sparsity | 0.52x | 0.36 |
| Speculation | 0.70x | 0.25 |
| Precision | 0.64x | 0.16 |
| KV compression | 0.81x | 0.13 |
| Serving stack | 0.82x | 0.11 |
| Disaggregation | 0.77x | 0.08 |
| Routing | 0.78x | 0.06 |
| Capacity | 0.63x | 0.04 |
| Seldon | 0.058x | 0.04 |
Factors are discounted for overlap between techniques. The undiscounted product would be 83.9x, which would not be honest.
research/tech-batching-economics.md section 6.1; RunPod live pricing 2026-07-17; SemiAnalysis InferenceMAX H200 throughput
03The tradeoff
Up and to the left
Cheap models that cannot do the work are not cheap. The only comparison worth making plots capability against price, and the only claim worth defending is a position on that frontier.
Composite quality index against blended price per million tokens
Log price axis. The quality index is banded in steps of 10 because the 2026 model generation shares no common public benchmark, so finer resolution would imply a precision that does not exist.
- Seldon
- Closed frontier APIs
- Open-weight, other hosts
| Model | Category | Blended $/1M tokens | Quality index |
|---|---|---|---|
| Seldon Prime 1 | Seldon | $2.10 | 80 |
| Seldon Core 1 | Seldon | $1.80 | 80 |
| Seldon Vector 1 | Seldon | $0.21 | 50 |
| Seldon Horizon 1 | Seldon | $1.29 | 80 |
| Claude Fable 5 | Closed frontier API | $20.00 | 90 |
| Claude Opus 4.8 | Closed frontier API | $10.00 | 90 |
| Claude Sonnet 5 | Closed frontier API | $4.00 | 80 |
| Claude Haiku 4.5 | Closed frontier API | $2.00 | 60 |
| gpt-5.6-sol | Closed frontier API | $11.25 | 90 |
| gpt-5.6-terra | Closed frontier API | $5.63 | 80 |
| gpt-5.6-luna | Closed frontier API | $2.25 | 60 |
| Gemini 3.1 Pro | Closed frontier API | $4.50 | 90 |
| Gemini 3.5 Flash | Closed frontier API | $3.38 | 70 |
| Grok 4.5 | Closed frontier API | $3.00 | 80 |
| DeepSeek V4-Pro | Open-weight, other host | $1.63 | 80 |
| DeepSeek V4-Flash | Open-weight, other host | $0.11 | 60 |
| GLM-5.2 | Open-weight, other host | $1.45 | 80 |
| Kimi K2.7-Code | Open-weight, other host | $1.43 | 70 |
| Qwen3.5-397B-A17B | Open-weight, other host | $1.09 | 80 |
We publish the derivation rather than a single headline score, because a composite index is only as good as its method. If you disagree with the weighting, every underlying figure is cited.
Pricing from vendor rate cards, July 2026. Quality index derivation is documented in the source tree.
04What it changes
The same product, on the right side of the margin line
Identical token volume, identical output quality requirements, identical everything except the rate card. This is what moving inference COGS does to a profit and loss statement.
Monthly inference spend by application archetype
Token volumes and cache hit rates are modelled per archetype with every assumption published. The frontier comparator is Claude Sonnet 5 at introductory pricing.
- Seldon
- Closed frontier
| Category | Workload | Seldon | Closed frontier |
|---|---|---|---|
| AI coding agent | 1,000 developer seats | $106K | $156K |
| Customer support agent | 100,000 conversations per month | $8.3K | $12K |
| Document and RAG analysis product | 10,000 monthly active users processing 200,000 documents per month | $21K | $36K |
| High-volume classification and extraction pipeline | 50,000,000 items per month | $48K | $68K |
Sonnet 5's introductory rate expires 31 August 2026, after which the same comparison uses $3 input and $15 output, widening the gap.
Assumptions and derivation in the economics module
AI coding agent, moving from a closed frontier model to Seldon Prime 1. From 14% to 39% gross margin.
This does not help everyone equally. For a support agent, the same switch cuts the inference bill 9.6x and buys only 5.7% of margin, because inference was 6.3% of revenue to begin with. That was the entire prize. Inference was never the binding constraint there, and we tell you if you are in that case before you switch.
Read the counterweightWhat changed
The same argument, from the other side of it
Every figure quoted below is computed, not asserted: each is the same cost function the calculators run, applied to that customer's published workload shape, for the specific pairing named beside it. All four assume a move from gpt-5.6-luna to Seldon Vector 1. The margin calculator fixes its Seldon leg to Seldon Prime 1, so it will not return these figures directly, and your own volumes on your own tier are the number that actually matters.
Inference was 40 percent of revenue and it was the only line that scaled with every seat we sold. Moving the completion path over took an afternoon because the API is wire-compatible. The margin change was the difference between raising a bridge and not needing one.
We were paying a long-context surcharge on every contract we ingested. Seldon does not charge one, and the prefix cache holds across a document set, so a hundred-clause review costs what the first clause used to.
The routing layer is the part that surprised us. Roughly seventy percent of our classification traffic never needed a frontier model, and nobody had been separating those requests out. We did not change our quality bar to get the saving.
They told us upfront that inference was only six percent of our revenue and that switching would not fix our margins, which is not what a vendor normally says in a sales call. We moved anyway for the latency, and the honesty is why we expanded.
05Capacity
The identical GPU trades at a wide spread
An H100 is an H100. The hourly price for one, on non-interruptible on-demand terms, varies by 6.2x between the cheapest and the dearest major provider. Even inside a single hyperscaler, the spread across regions and commitment terms reaches 5.4x.
That dispersion persists because capacity is sold on contract structures rather than spot commodity terms. Reservations, committed-use discounts, and interruptible tiers all price the same silicon differently depending on who carries the utilization risk. We buy across all of it continuously and carry that risk ourselves, which is a large part of why our floor sits where it does.
How the fleet is sourcedCheapest to dearest published rate for an H100 on like-for-like terms.
Both legs normalised to identical terms by an independent index.
Same provider, same GPU, different region and commitment term.
Per million output tokens after the full stack, on sourced capacity.
Runs on
- NVIDIA H100
- NVIDIA H200
- NVIDIA B200
- NVIDIA GB200 NVL72
- AMD MI355X
- InfiniBand NDR
Accelerators the serving stack is built and tuned for. Kernel work targets each generation directly rather than relying on a portability layer.
Drop-in for
- OpenAI API format
- Anthropic Messages format
- LangChain
- LlamaIndex
- Vercel AI SDK
- OpenTelemetry
Wire-compatible surfaces. Existing application code moves over by changing a base URL and a key, with no SDK rewrite.
Deploys into
- AWS
- Google Cloud
- Azure
- Customer VPC
- On-premises
- Air-gapped
Target environments for dedicated and private deployments, including customer-controlled infrastructure.
06Trust
We publish what we run, not a wall of badges
Seldon holds no certifications. Rather than imply otherwise, our trust center lists every framework we are working toward with a real status and a real date, alongside the controls that are implemented today.
A badge tells a technical buyer nothing. TLS 1.3 in transit, AES-256 at rest, tenant isolation, zero-retention inference with its carve-outs stated plainly, SSO and SCIM, and audit logging tell them something they can check.
Read the trust centerNo item on this list is presented as achieved. Each report goes to customers under NDA once it exists, and not before.
Run the numbers on your own workload.
Send us a week of traffic shape, input and output token mix, cache hit rate, and latency targets. We will return a modelled cost comparison against your current provider, with the assumptions written out so you can check them.