Vector 1

Demo · AI infrastructure

Self-host the model, or pay per token?

A rented GPU costs the same whether it's busy or idle, so the cost of each token you serve depends mostly on how busy you keep it. Plug in your hardware, throughput and expected load to see your cost per million tokens and the utilization where self-hosting beats an API.

Your deployment

The alternative

Defaults are the illustrative assumptions from Vector 1's self-hosting analysis: replace them with your own benchmarks and quotes. Throughput depends heavily on model size, batch size and serving stack.

Cost per million tokens

The formula

cost per million = GPU $/hour × GPUs × (1 + overhead) ÷ (tokens/second × 3,600 × utilization) × 1,000,000

break-even utilization = GPU $/hour × GPUs × (1 + overhead) ÷ (tokens/second × 3,600 × API price ÷ 1,000,000)

Utilization is the lever most teams underestimate. A replica serving one department's assistant sits idle nights and weekends; a pool of organizations with different peak hours (tax season, open enrollment, overnight batch jobs) can keep the same hardware busy. The model ignores price differences in model quality, latency guarantees and data-control requirements, which often decide the question on their own.