CUDA Out of Memory — Solutions
Fix PyTorch CUDA out of memory errors. Calculate GPU memory requirements for your model, find the right batch size, and apply memory optimization techniques.
Built by Michael Lip
Frequently Asked Questions
Why am I getting CUDA out of memory?
Your model, optimizer states, gradients, and intermediate activations exceed GPU VRAM. Training requires roughly 4x the model size (1x params + 1x gradients + 2x Adam states) plus activation memory which scales with batch size.
How do I reduce GPU memory usage?
1) Reduce batch size. 2) Use mixed precision (torch.cuda.amp). 3) Use gradient checkpointing (model.gradient_checkpointing_enable()). 4) Use gradient accumulation. 5) Clear cache with torch.cuda.empty_cache(). 6) Use a smaller model.
How much GPU memory does my model need?
For training with Adam: memory ≈ (num_params * 4 bytes * 4) + activation_memory. The 4x comes from: parameters (4B), gradients (4B), Adam m state (4B), Adam v state (4B). Activations depend on batch size and model architecture. Use HeyTensor's Memory Calculator for an estimate.
What does 'Tried to allocate X MiB' mean?
PyTorch is telling you how much memory the current operation needs. The error also shows total GPU memory and current allocation. If allocated + requested > total, you're out of memory. The solution is to reduce the requested amount (smaller batch) or free existing allocations.
About This Tool
This tool is part of HeyTensor, a free suite of PyTorch and deep learning utilities. All calculations run entirely in your browser — no data is sent to any server. The source code is open on GitHub.
Contact
HeyTensor is built and maintained by Michael Lip. For questions or feedback, email [email protected].