PyTorch Memory Calculator

Estimate GPU VRAM usage for training — model, gradient, optimizer, activations
Off On
Off On
Model memory—
Gradient memory—
Optimizer state—
Activation memory—
Total VRAM—

GPU compatibility

* Estimates assume typical transformer architecture. Activation memory depends on hidden size, layers, attention heads. This calculator uses a simplified heuristic (hidden=4096, layers=32, heads=32).

This tool provides approximate VRAM estimates for PyTorch training. Actual usage may vary. Use for initial sizing.