LayerNorm Shape Calculator
Calculate PyTorch LayerNorm output shapes and parameter counts. LayerNorm preserves input shape. Enter normalized_shape to see dimensions and trainable params.
Built by Michael Lip
Frequently Asked Questions
Does LayerNorm change the output shape?
No. LayerNorm always outputs the same shape as the input. It normalizes across the dimensions specified by normalized_shape but does not change any dimensions.
What is the difference between LayerNorm and BatchNorm?
BatchNorm normalizes across the batch dimension (each feature independently across samples). LayerNorm normalizes across feature dimensions (each sample independently). LayerNorm works with any batch size and is the standard in Transformers. BatchNorm is standard in CNNs.
What should normalized_shape be?
normalized_shape should match the last N dimensions of your input tensor. For a Transformer with input [batch, seq_len, d_model], set normalized_shape=[d_model]. For normalizing over both seq and features, use [seq_len, d_model].
How many parameters does LayerNorm have?
With elementwise_affine=True (default), LayerNorm has 2 × product(normalized_shape) trainable parameters. For normalized_shape=[512], that is 1,024 parameters. With elementwise_affine=False, it has zero.
Is this tool free?
Yes. All HeyTensor tools are free, run in your browser, and require no signup.
About This Tool
This tool is part of HeyTensor, a free suite of PyTorch utilities. All calculations run in your browser. Source code is open on GitHub.
Contact
HeyTensor is built by Michael Lip. Email [email protected] for feedback.
TENSOR PREFLIGHT · DOWNLOADABLE PYTORCH TOOLKIT
Take the next step in your own PyTorch model.
Trace a local forward pass, inspect tensor shapes in an HTML report, and turn a repair into a regression check. Includes eight broken-and-fixed workflows, source code and setup instructions.
Python 3.9+ · PyTorch 2.8+ · Runs locally