How to Fix "expected 2D or 3D input (got 4D input)" in BatchNorm1d

A 4-dimensional tensor reached nn.BatchNorm1d, which accepts only (N, C) or (N, C, L). A (N, C, H, W) tensor is the output of a 2D convolution, so the layer you want is nn.BatchNorm2d(C). If the 1D layer has to stay, fold the spatial dimensions first with x.flatten(2).

Check your own shape

Pick the layer, type the num_features you built it with, the shape of the tensor you feed it, and whether the model is in train or eval mode. The tool names the exact exception PyTorch raises, or reports the output shape when the call is valid. Every rule was checked against torch 2.8.0 on CPU and the strings are copied from the real exceptions.

The error

ValueError: expected 2D or 3D input (got 4D input)

This is a ValueError, not a RuntimeError. It is raised in Python, inside the _check_input_dim method of the module, before any tensor arithmetic runs. The three BatchNorm classes differ only in that method, and each one prints its own accepted ranks:

nn.BatchNorm1d(8)(torch.randn(2, 8, 4, 4))
# ValueError: expected 2D or 3D input (got 4D input)

nn.BatchNorm2d(8)(torch.randn(2, 8, 4))
# ValueError: expected 4D input (got 3D input)

nn.BatchNorm3d(8)(torch.randn(2, 8, 4, 4))
# ValueError: expected 5D input (got 4D input)

The number in parentheses is the rank of the tensor you passed. The message tells you which layer you called and how far off you are, which is enough to pick the fix from the list below.

The rule behind all three layers

BatchNorm normalizes each channel over every other dimension. To do that it needs to know which dimension holds the channels, and PyTorch fixes that at dimension 1. The batch is dimension 0. Anything after dimension 1 is treated as spatial and averaged over. That leaves one degree of freedom, the number of spatial dimensions, and the class name pins it:

• nn.BatchNorm1d(C) takes (N, C) or (N, C, L). Zero or one spatial dimension.
• nn.BatchNorm2d(C) takes (N, C, H, W). Two spatial dimensions, no exceptions.
• nn.BatchNorm3d(C) takes (N, C, D, H, W). Three spatial dimensions.

The documented contract for the 1D layer lives at torch.nn.BatchNorm1d, which states the input as (N, C) or (N, C, L) with N the batch size, C the number of features or channels, and L the sequence length. The output has the same shape as the input. The error strings themselves are not documented, so everything quoted on this page is observed behaviour on torch 2.8.0.

Cause 1, a conv feature map goes into BatchNorm1d

This is the case behind the search query. A Conv2d returns (N, C, H, W), and a nn.BatchNorm1d was pasted after it from a fully connected block or a 1D model.

x = conv(x)                    # (2, 8, 4, 4)
x = nn.BatchNorm1d(8)(x)
# ValueError: expected 2D or 3D input (got 4D input)

x = nn.BatchNorm2d(8)(x)       # (2, 8, 4, 4)  correct

Swap the layer. BatchNorm2d(8) has the same parameters, the same running_mean of 8 entries, and the same math. Only the accepted rank changes. If the module is shared with a 1D path and cannot be swapped, fold the spatial dimensions into one:

N, C, H, W = x.shape
y = bn1d(x.flatten(2))         # (2, 8, 16)
y = y.view(N, C, H, W)         # back to (2, 8, 4, 4)

# x.view(N, C, -1) does the same fold

The statistics come out identical either way, because both layers average over every element that is not the channel.

Cause 2, a single sample with no batch dimension

The mirror image of the 4D error. A tensor of shape (8) goes into BatchNorm1d(8), usually during inference on one example pulled out of a dataset.

nn.BatchNorm1d(8)(torch.randn(8))
# ValueError: expected 2D or 3D input (got 1D input)

x = x.unsqueeze(0)             # (1, 8)

Adding the batch dimension clears the rank check and lands you on the next one. In training mode a (1, 8) input raises:

ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 8])

The same (1, 8) tensor passes in eval mode and returns (1, 8). The check counts N times every spatial dimension, so (1, 8, 4) passes even in training because each channel sees 4 values, while (1, 8, 1) fails. Call model.eval() for inference. During training, drop the last incomplete batch with DataLoader(..., drop_last=True).

Cause 3, the channel count does not match

Once the rank is right, PyTorch compares dimension 1 with num_features. The message is a RuntimeError and the order of the two numbers trips people up:

nn.BatchNorm1d(8)(torch.randn(2, 4))
# RuntimeError: running_mean should contain 4 elements not 8

The first number, 4, is what your input has at dimension 1. The second number, 8, is what the layer was built with. Read it as "the input wants a running_mean of 4 but the layer holds 8".

The most common way to hit this with sequence data is a (N, L, C) layout, the one Transformers and most tokenizers produce. BatchNorm reads the sequence length as the channel count:

x = torch.randn(2, 16, 8)      # (N, L, C)
nn.BatchNorm1d(8)(x)
# RuntimeError: running_mean should contain 16 elements not 8

y = bn(x.transpose(1, 2))      # (2, 8, 16), valid
y = y.transpose(1, 2)          # back to (2, 16, 8)

If you built the layer with track_running_stats=False the same mismatch is reported against the weight instead: RuntimeError: weight should contain 4 elements not 8. Same cause, a different parameter is checked first.

Cause 4, LayerNorm on a channels-first tensor

People reach for nn.LayerNorm as a way around the batch size limit and hit a different contract. LayerNorm normalizes over the trailing dimensions named in normalized_shape, so the channel count must be last, not at dimension 1.

nn.LayerNorm(8)(torch.randn(2, 8, 4, 4))
# RuntimeError: Given normalized_shape=[8], expected input with shape [*, 8], but got input of size[2, 8, 4, 4]

The missing space before the bracket is in the real string. Three fixes work on torch 2.8.0. Move the channels last and back with ln(x.permute(0, 2, 3, 1)).permute(0, 3, 1, 2). Or normalize over all three trailing dims with nn.LayerNorm([8, 4, 4]), which ties the layer to a fixed spatial size. Or use nn.GroupNorm(1, 8), which accepts (N, C, *) directly and has no batch size requirement. LayerNorm and GroupNorm ignore train and eval mode, so the batch of 1 problem from Cause 2 disappears with either.

What PyTorch 2.8.0 actually raises

Every row below was produced by running the shape through the layer on torch 2.8.0, CPU, with default arguments, and copying the exception. The tool at the top of this page reproduces all of them.

Note the check order. The (1, 4) input raises the batch size error in training and the channel error in eval, because the batch check runs first and only in training mode. Fixing one message can reveal the next, which is why the tool takes the mode as an input.

Quick reference

The three BatchNorm rows are the whole story for the 4D error. Count the dimensions of the tensor, subtract 2, and that is the digit in the class name you need.

Built by Michael Lip. Behaviour verified against torch 2.8.0 on 6 September 2026.