How Many Parameters Does Conv2d(256, 512, 3) Have?
Conv2d(256, 512, 3) has 1,180,160 trainable parameters. This includes 1,179,648 weights and 512 bias terms.
Formula Breakdown
For a Conv2d layer, the parameter count is:
parameters = in_channels * out_channels * kernel_size^2 + out_channels (bias)
parameters = 256 * 512 * 3 * 3 + 512
parameters = 256 * 512 * 9 + 512
parameters = 1,179,648 + 512
parameters = 1,180,160Each of the 512 output filters is a 3D kernel of shape (256, 3, 3). That gives 512 × 256 × 3 × 3 = 1,179,648 weights, plus 512 bias terms. Total: 1,180,160 trainable parameters.
Memory Usage
In float32, this layer uses 4.50 MB of memory for weights alone. During training with Adam optimizer, multiply by 3 = 13.51 MB.
Architecture Context
This layer configuration is found in the 256-to-512 channel expansion in VGG-16/19 and ResNet layer4. Understanding parameter counts helps you estimate model size, memory requirements, and the risk of overfitting. Layers with more parameters need more training data and compute to train effectively.
Convolutional layers are parameter-efficient compared to fully connected layers because weights are shared across spatial positions. A Conv2d(256, 512, 3) processes any input spatial size with the same 1,180,160 parameters.
PyTorch Code to Verify
import torch.nn as nn
layer = nn.Conv2d(256, 512, kernel_size=3)
# Count parameters
total = sum(p.numel() for p in layer.parameters())
print(f"Total parameters: {total}") # 1,180,160
# Break it down
print(f"Weight shape: {layer.weight.shape}") # (512, 256, 3, 3)
print(f"Weight params: {layer.weight.numel()}") # 1,179,648
print(f"Bias shape: {layer.bias.shape}") # (512,)
print(f"Bias params: {layer.bias.numel()}") # 512
# Without bias (common in batch-normalized networks)
layer_no_bias = nn.Conv2d(256, 512, kernel_size=3, bias=False)
print(f"Without bias: {sum(p.numel() for p in layer_no_bias.parameters())}") # 1,179,648Comparison: With vs. Without Bias
| Configuration | Parameters |
|---|---|
| Conv2d(256, 512, 3) (with bias) | 1,180,160 |
| Conv2d(256, 512, 3, bias=False) | 1,179,648 |
When using BatchNorm after a convolutional layer, the bias is redundant because BatchNorm has its own bias term. Setting bias=False saves 512 parameters per layer.