How to Fix "Cannot Broadcast Tensors" in PyTorch
Broadcasting requires dimensions to match from the right. Use .unsqueeze() to add dimensions. Example: (3,4) + (4,) works, but (3,4) + (3,) fails because the rightmost dimensions 4 and 3 don't match.
Broadcasting Rules
Two dimensions are compatible when:
- They are equal, or
- One of them is 1
Dimensions are compared from right to left:
# Works: rightmost dims match
(3, 4) + (4,) -> (3, 4) ✓ 4==4
(3, 4) + (1, 4) -> (3, 4) ✓ 4==4, 1 broadcasts to 3
(2, 3, 4) + (4,) -> (2, 3, 4) ✓ 4==4
# Fails: rightmost dims don't match
(3, 4) + (3,) -> ERROR ✗ 4!=3
(2, 3) + (2,) -> ERROR ✗ 3!=2Fix: Use unsqueeze()
a = torch.randn(3, 4) # shape (3, 4)
b = torch.randn(3) # shape (3,)
# a + b # ERROR: (3,4) + (3,) — 4 != 3
# Fix: add a dimension to make (3,) -> (3, 1)
b = b.unsqueeze(-1) # shape (3, 1)
result = a + b # (3, 4) + (3, 1) -> (3, 4) ✓Common Patterns
# Add bias per channel to image: (batch, C, H, W) + (C,)
bias = torch.randn(64) # (64,)
bias = bias.unsqueeze(0).unsqueeze(-1).unsqueeze(-1) # (1, 64, 1, 1)
# or more concisely:
bias = bias.view(1, -1, 1, 1) # (1, 64, 1, 1)
output = features + bias # broadcasts ✓
# Scale per sample: (batch, features) * (batch,)
weights = torch.randn(32) # (32,)
weights = weights.unsqueeze(-1) # (32, 1)
result = data * weights # (32, 128) * (32, 1) -> (32, 128) ✓