GPU-Hours Explained: The Unit Behind AI Costs

GPU-hours are the fundamental unit for measuring and pricing AI compute resources, calculated simply as the number of GPUs multiplied by runtime hours. This metric serves as the primary currency for compute budgeting in machine learning, allowing researchers and businesses to estimate costs per experiment and compare resources across different hardware configurations. Understanding GPU-hours is essential for managing AI infrastructure expenses, which can range from hundreds of dollars for smaller models to millions for frontier AI systems.

Not all GPU-hours are created equal due to significant throughput differences between hardware classes. NVIDIA's H100 GPUs deliver up to 6x faster performance than previous-generation A100s for AI workloads, while AMD's MI300X offers competitive throughput with 304 compute units and 5.3 TB/s memory bandwidth. These performance gaps directly impact cost per experiment—training the same model on high-end H100s might require 1,000 GPU-hours, while older hardware could need 3,000+ GPU-hours, dramatically affecting project budgets despite identical computational outcomes.

When comparing costs across cloud providers, many normalize pricing using "H100-equivalent" units to enable apples-to-apples comparisons, accounting for the raw performance differences between GPU generations and vendors.

For example, to calculate Ornn's H100 index, we employ a sophisticated matrix-weighted regional methodology that uses market-cap based weighting balanced by volume traded dynamics, as volume traded serves as a heuristic for market efficiency and therefore price efficiency. The index represents not the price of any given provider's H100s, but rather a benchmark of aggregate prices for H100 compute with a baseline standard of performance across multiple geographic regions and cloud providers. This approach prioritizes price signals from providers with substantial market presence and higher transaction volumes, which create more liquid markets with superior price discovery mechanisms compared to providers with minimal regional presence.

Example Calculation

1. 1.Setup: 64 H100 GPUs training for 10 hours
2. 2.Calculation: 64 GPUs × 10 hours = 640 GPU-hours
3. 3.Cost: 640 GPU-hours × $4.50/hour = $2,880 total compute cost

Mini-Glossary

• •Throughput: The computational performance rate of a GPU, measured in operations per second
• •Efficiency: The ratio of useful computational work completed per GPU-hour consumed