NVIDIA H100 GPU: 80GB HBM3 Specs, Pricing & Rental. Rent H100 GPU from $2.20/hr
80GB HBM3, 3.35 TB/s bandwidth, FP8 Transformer Engine, 400 Gb/s InfiniBand. Per-minute billing on H100 GPU rentals, live in under 2 minutes.
- GPU Architecture
- NVIDIA Hopper
- VRAM
- 80 GB HBM3
- Memory Bandwidth
- 3.35 TB/s
No long contracts · Per-minute billing · Deploy in ~2 minutes
At a glance
Renting an NVIDIA H100 on Spheron AI starts at $2.20/hr per GPU on spot capacity, which can be reclaimed at any time. Dedicated capacity is $3.96/hr per GPU, bookable right now rather than listed and unavailable. Billing is per minute after a 20-minute minimum runtime, and an instance is usually live in under 2 minutes. The H100 has 80GB of HBM3 and 3.35 TB/s of memory bandwidth, which is enough headroom to fine-tune Llama 3 70B in 4-bit on a single card or serve a 70B model at production latency. For multi-node training, every H100 node ships with 400 Gb/s InfiniBand and GPUDirect RDMA. On-demand H100 pricing on AWS, GCP, and Azure currently sits between about $3 and $7 per GPU per hour.
NVIDIA H100 pricing
| Provider | Price/hr | vs Spheron AI | Availability |
|---|---|---|---|
Spheron AIYour price | $3.96/hrDEDICATED$2.20/hrSpot | - | Bookable now |
Google Cloud | $3.00/hr | 36% more than spot | Listed rate |
Runpod | $3.29/hr | 50% more than spot | Low capacity |
Latitude.sh | $3.37/hr | 53% more than spot | Listed rate |
AWS | $3.90/hr | 77% more than spot | Listed rate |
Lambda Labs | $4.29/hr | 1.1x more expensive | Bookable |
CoreWeave | $6.16/hr | 1.6x more expensive | Reserved only |
Azure | $6.98/hr | 1.8x more expensive | Listed rate |
Spheron AI rates are live marketplace prices and move with partner supply, so what is shown is what you can book right now. Competitor figures are published on-demand rates taken from each provider's own pricing page. Each is measured against whichever of our own rates it actually sits against: our on-demand rate when it lands at or above it, and our spot rate when it falls below, because spot is preemptible and not the same product. A listed rate is not the same thing as available capacity: check before you plan a run. Where a competitor row carries a capacity note, that is what that provider's own console reported on 24 Aug 2026. It is a dated observation, not a live feed, and their supply may have changed since. Any row marked reserved only is the exception to that on-demand comparison: that provider sells no on-demand tier for this GPU, so its figure is a committed or reservation rate, which is normally the cheaper of the two ways to buy the same card.
NVIDIA H100 specifications
NVIDIA Hopper
80 GB HBM3
3.35 TB/s
4th Generation
16,896
34 TFLOPS
67 TFLOPS
989 TFLOPS
1,979 TFLOPS
3,958 TOPS
116 GB DDR4
26 vCPUs
2.4 TB NVMe SSD
400 Gb/s InfiniBand
700W
Need More H100 Than
What's Listed?
We'll Source It.
On-demand by default, reserved pricing when you commit. Tell us the cluster size, hardware and term, and we match you with the right data center partner.
Reserved Capacity
Custom Clusters
Supplier Matchmaking
How It Works
- 01
Submit your requirements
Tell us your GPU needs, timeline, and scale
- 02
Spheron AI sources from certified data centers
We match, negotiate pricing, and handle setup
- 03
Reserved capacity confirmed
Guaranteed availability, ready to deploy
Typical turnaround: 24–48 hours for custom sourcing
When to pick the H100
Pick the H100 if
You are training or fine-tuning a 30B+ parameter model, serving a 70B model in production, or running multi-node distributed training that depends on InfiniBand and GPUDirect RDMA. The H100 is also the right call for FP8 inference workloads where the Transformer Engine pays back the price difference vs older silicon.
Pick the A100 instead if
Your model fits in 80GB and you do not need FP8. The A100 runs roughly half the price of the H100 and handles BERT-class training, sub-30B fine-tunes, and most production inference without breaking a sweat. Spend the savings on more GPUs.
Pick the H200 instead if
You are bottlenecked by memory, not compute. The H200 keeps the H100 architecture but doubles to 141GB of HBM3e at 4.8 TB/s. Long-context inference, KV-cache-heavy serving, and fitting 70B models without sharding are where the H200 wins.
Pick the B200 instead if
You are training trillion-parameter models or pushing the absolute frontier of throughput per node. The B200 has roughly 2.3x the FP8 dense TFLOPS of H100, 192GB HBM3e at 8 TB/s, and FP4 support. Real-world inference speedups vary by model and stack. The catch is availability and price. Most teams do not need it yet.
NVIDIA H100 use cases
LLM training and fine-tuning
Train transformer architectures from scratch or fine-tune frontier-class models with FP8 mixed precision. The H100's Transformer Engine cuts memory and time per step versus FP16.
Production LLM inference
Serve large models at low latency with vLLM, TensorRT-LLM, or SGLang. The H100 handles 70B class models at production traffic without sharding overhead.
Diffusion and video generation
Run image and video diffusion pipelines that need high VRAM and bandwidth. The H100 chews through batch sizes that the RTX 4090 can not touch.
HPC and scientific computing
FP64 throughput on the H100 is 34 TFLOPS, three times the A100. That matters for simulation work where double precision is non-negotiable.
NVIDIA H100 benchmarks
~12,500 tok/s
single H100 SXM
~460 tok/s
single H100 SXM
up to 3.1x faster
vs A100 80GB
up to 4x faster
vs A100 80GB
~2.4x faster
vs A100 80GB
up to 1.6x faster
Transformer Engine on
Launch vLLM on an H100 in under 2 minutes
Spin up a Spheron AI H100 instance, pull the vLLM image, and serve Llama 3 70B with an OpenAI-compatible API. Drop the same client into your existing OpenAI SDK call and point the base URL at the new endpoint.
# 1. Provision an H100 from the Spheron AI CLI (or use the dashboard)spheron deploy --gpu h100 --image vllm/vllm-openai:latest # 2. Inside the instance, serve Llama 3 70B with FP8python -m vllm.entrypoints.openai.api_server \ --model meta-llama/Meta-Llama-3-70B-Instruct \ --quantization fp8 \ --max-model-len 8192 \ --gpu-memory-utilization 0.92 \ --port 8000 # 3. Hit the endpoint from any OpenAI-compatible clientcurl http://<instance-ip>:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "meta-llama/Meta-Llama-3-70B-Instruct", "messages": [{"role": "user", "content": "Explain FP8 quantization."}] }'Need multi-GPU or multi-node? Add --tensor-parallel-size 2 for 2x H100, or contact us for InfiniBand-connected clusters.
Interconnect fabric
InfiniBand for multi-node H100 training
Every H100 node on Spheron AI ships with 400 Gb/s InfiniBand. Multi-node training jobs synchronize gradients over RDMA at sub-microsecond latency, so distributed training scales close to linear with node count instead of stalling on the network.
400 Gb/s InfiniBand connectivity per GPU
NVIDIA ConnectX-7 network adapters
GPUDirect RDMA for zero-copy GPU-to-GPU transfers
Optimized for NCCL collective operations
Sub-microsecond GPU-to-GPU latency
Bare-metal clusters up to 80x H100 (10 nodes)
Tested with PyTorch DDP, DeepSpeed ZeRO, and Megatron-LM
NVLink and NVSwitch within each node
H100 vs alternatives
Same compute, 1.76x more memory and 1.4x more bandwidth on H200. If you keep hitting OOM on long context or large KV caches, the H200 fixes that.
H100 vs B200B200 is the next generation: ~2.3x H100's FP8 dense TFLOPS, 192GB HBM3e at 8 TB/s, and FP4 support. Real-world inference speedups depend on model and stack maturity. Worth the jump if you can secure capacity.
H100 vs RTX 4090The 4090 has 24GB and no NVLink. Fine for sub-13B inference and hobbyist training, not for serving 70B models or any serious distributed work.
01Technical Brief
NVIDIA H100 Release Date and Cloud Availability
The NVIDIA H100 Tensor Core GPU was announced at GTC March 2022 and started shipping in Q3 2022 inside the NVIDIA DGX H100 systems. Broad cloud availability followed: AWS launched the p5 instance family (H100 SXM5) in July 2023, Google Cloud A3 went GA in September 2023, and Microsoft Azure ND H100 v5 reached general availability in October 2023. CoreWeave, Lambda, and other GPU-focused clouds had H100 capacity by mid-2023.
On Spheron AI the H100 is available in both SXM5 and PCIe form factors with per-minute billing and no contract. Capacity is sourced from data center partners across North America, Europe, and Asia. Spot pricing is meaningfully below the dedicated rate for fault-tolerant workloads. Live availability and pricing is on the pricing page. The H100 successor, the H200 (same Hopper architecture with 141GB HBM3e), launched in 2024 and is also available on Spheron AI; the next-generation Blackwell B200 ships native FP4 support and 192GB HBM3e.
02Technical Brief
H100 VRAM and Memory Bandwidth: 80GB HBM3 at 3.35 TB/s
The H100 ships with 80GB of HBM3 memory at 3.35 TB/s of bandwidth. That is roughly 1.6x the bandwidth of the A100 80GB (2.0 TB/s HBM2e) and the single biggest reason H100 delivers a step-change in inference throughput on memory-bandwidth-bound workloads. For autoregressive LLM decode at batch size 1, the H100's bandwidth ceiling for a 70B FP16 model is about 24 tokens per second versus 14 tokens per second on the A100.
Where the 80GB VRAM matters: a 70B model in FP8 quantization fits with roughly 10GB of KV cache headroom, enough for moderate-batch production serving. A 30B-class model fits in BF16. For 70B in FP16 or 100B+ models you need the H200 (141GB HBM3e at 4.8 TB/s) or B200 (192GB HBM3e at 8 TB/s). The H100's Transformer Engine adds native FP8 support that the A100 does not have, roughly doubling tensor-core throughput on Transformer attention and feed-forward layers.
NVIDIA H100 FAQ
Specs, pricing and availability questions teams ask before renting the NVIDIA H100.
On Spheron AI the H100 starts at $2.20/hr per GPU on spot capacity, which can be reclaimed at any time. Dedicated capacity is $3.96/hr per GPU, bookable right now rather than listed and unavailable. Billing is per minute after a 20-minute minimum runtime, so a one-hour test costs you one hour. For comparison, AWS p5 H100 instances run around $3.90/hr per GPU on demand after the mid-2025 price cut, Google Cloud A3-high is about $3.00/hr, and Azure ND H100 v5 sits near $7/hr.
Spot instances on Spheron AI are the cheapest path, often 50 to 70 percent below the dedicated rate. Both are on-demand tiers with per-minute billing. The trade-off is that spot instances can be reclaimed when demand spikes, so checkpoint your training state every 15 to 30 minutes and treat spot as a fit for fault-tolerant workloads. For long uninterrupted runs, stay on dedicated (99.99% SLA, non-interruptible).
Yes. Spheron AI bills per minute after a 20-minute minimum runtime. You can rent an H100 for a single hour to benchmark a workload, or keep it running for months. There are no contracts, reserved-instance lock-ins, or commit fees on dedicated or spot.
Most H100 instances are live in under 2 minutes. The hardware is pre-warmed, so provisioning is closer to a container start than a hyperscaler VM boot. If you have a Docker image ready, you can be running training inside two minutes of clicking deploy.
If you are doing FP8 inference, training models above 30B parameters, or running anything that benefits from the Transformer Engine, the H100 is the right choice. It is roughly 3 to 4x faster on those workloads. For sub-30B training and most inference at scale, the A100 80GB delivers better dollars per token.
Yes. Spheron AI supports up to 8x H100 per node with NVLink, and bare-metal clusters up to 80x H100 across 10 nodes connected by 400 Gb/s InfiniBand. Every cluster is tested with PyTorch DDP, DeepSpeed ZeRO-3, and Megatron-LM. For larger configurations, contact us.
PyTorch, TensorFlow, JAX, and the major serving stacks (vLLM, TensorRT-LLM, SGLang, Triton). Containers ship with CUDA 12.4+, cuDNN, NCCL, and the standard NVIDIA AI Enterprise libraries. You can also bring your own Docker image.
Run your own image as a GPU containerH100 capacity is online across North America, Europe, and Asia, sourced from data center partners. Specific availability shifts with demand. The dashboard shows current capacity per region in real time.
SXM5 is the higher-power variant (700W) with NVLink connectivity between GPUs in a node, which matters for multi-GPU training. PCIe is air-cooled, lower power (350W), and easier to mix with existing servers. Spheron AI offers both. Pick SXM for distributed training, PCIe for single-GPU inference.
A QLoRA fine-tune of Llama 3 70B on a 50k-sample dataset takes roughly 8 to 12 hours on a single H100. Full-parameter fine-tunes need 4 to 8 H100s and run in the 24 to 48 hour range depending on dataset size and sequence length. We have a fine-tuning case study with the exact numbers.
Yes, with checkpointing. Spot instances on Spheron AI are reclaimed when demand rises, so unsaved state is at risk. The safe pattern is: checkpoint every 15 to 30 minutes to a persistent volume, restart from the latest checkpoint on preemption, and reserve spot for training, batch jobs, and fault-tolerant inference. Use dedicated instances (99.99% SLA) for production serving.
Yes. The Transformer Engine on Hopper supports FP8 mixed precision out of the box. In practice this is a roughly 1.7x speedup over FP16 with no measurable accuracy loss for most LLM workloads, plus halved memory pressure on activations. vLLM, TensorRT-LLM, and PyTorch all support FP8 paths.
For 100+ GPU deployments and production-critical workloads, Spheron AI offers live chat support in the app, sourcing assistance for capacity, and SLA-backed instances. Smaller deployments are self-serve through the dashboard.
Talk to our teamFor the same H100 hardware, Spheron AI is meaningfully cheaper than AWS p5, Azure ND H100, and GCP A3 on-demand. As of April 2026 hyperscaler on-demand H100 pricing runs roughly $3.00 per GPU per hour on GCP A3-high, ~$3.90/hr on AWS p5, and ~$7/hr on Azure ND H100 v5. Spheron AI starts at $3.96/hr. Same chip, different pricing model.
For most teams, renting wins. An H100 costs roughly $25,000 to $40,000 to buy, plus a host server, power, and cooling, and it depreciates as newer GPUs ship. Renting on Spheron AI starts at $3.96/hr per GPU with per-minute billing and no commitment, so you only pay while a job runs and you can move to H200 or B200 the day they help. Buy only if you keep an H100 near 24/7 utilization for more than a year and already have the data-center overhead to support it.
On Spheron AI the NVIDIA H100 is $3.96/hr on-demand and $2.20/hr on spot, per GPU, billed per minute after a 20-minute minimum runtime, with no contract. Compared against published rates elsewhere, all per GPU: Google Cloud lists $3.00/hr (36% more than spot); Runpod lists $3.29/hr (low capacity, 50% more than spot); Latitude.sh lists $3.37/hr (53% more than spot); AWS lists $3.90/hr (77% more than spot); Lambda Labs lists $4.29/hr (bookable, 1.1x more expensive); CoreWeave lists $6.16/hr (reserved only, no on-demand tier, 1.6x more expensive); and Azure lists $6.98/hr (1.8x more expensive). Each comparison above is against whichever of our rates the competitor actually sits against: our on-demand rate when it lands at or above it, and our spot rate when it falls below, because spot is preemptible and not the same product. Competitor rates are published on-demand figures from each provider's own pricing page, and the capacity notes are what their own consoles reported on 24 Aug 2026. A listed rate is not the same thing as available capacity.