What Is the Dimension of BERT-base pooler_output?

Shape at a Glance

last_hidden_state:  (batch_size, seq_len, 768)   # one vector per token
pooler_output:      (batch_size, 768)            # one vector per batch item

The sequence dimension is the key difference. last_hidden_state keeps every token position, so its length varies with the input. pooler_output collapses that dimension to a single 768-vector per example, which is convenient for classification heads that expect a fixed-size representation.

How It Is Computed

The pooler is a single dense layer followed by tanh. It reads only the hidden state at position 0, the [CLS] token:

pooled = dense(last_hidden_state[:, 0, :])   # shape (batch, 768)
pooler_output = tanh(pooled)                  # shape (batch, 768)

With hidden size 768, the dense layer has 768×768 weights plus a 768-vector bias:

weights: 768 * 768  = 589,824
bias:               =     768
total:              = 590,592 parameters

Why It Is Always 768

In BERT-base the hidden size H is 768, and every transformer output vector lives in that space. The pooler is a linear map from 768 → 768, so its output is always 768-dimensional. The number never changes with the number of layers, attention heads, or tokens; it is pinned to the model dimension.

Hidden size (H):  768
Intermediate:     3072   (4 * H)
Attention heads:  12
Layers:           12
pooler_output:    (batch, 768)

Reading the Output Dict

BertModel.forward() returns a BaseModelOutputWithPoolingAndCrossAttentions. The fields you normally use are last_hidden_state, pooler_output, and hidden_states. Only the first element of pooler_output is ever used by the model’s prediction heads:

outputs = model(input_ids)
cls_vec  = outputs.pooler_output        # (batch, 768)
# or the underlying tensor, identical to:
cls_vec  = outputs.last_hidden_state[:, 0]  # then densed + tanh

pooler_output vs .pooler.dense

When you build the model with config.output_hidden_states=True or access submodules directly, you may see model.pooler and model.pooler.dense. These are the same layer that produces pooler_output. The named module is a BertPooler whose single dense member holds the 768×768 + 768 parameters; pooler_output is just its output applied through tanh. Accessing the tensor directly bypasses the tanh:

pre_tanh = model.pooler.dense(last_hidden_state[:, 0])   # no tanh
pooled   = model.pooler(last_hidden_state)               # tanh applied

Disabling the Pooler

If you do not need a sentence-level vector, you can turn the pooler off entirely. Passing add_pooling_layer=False to BertModel removes it, so pooler_output comes back as None and model.pooler is absent:

model = BertModel.from_pretrained("bert-base-uncased",
                                  add_pooling_layer=False)
outputs = model(input_ids)
outputs.pooler_output   # None
outputs.last_hidden_state  # (batch, seq, 768)

With the pooler disabled, build your own pooled vector when you need one, for example a mean over tokens:

# mask-aware mean pooling over last_hidden_state
mask = attention_mask.unsqueeze(-1)          # (batch, seq, 1)
sums = (last_hidden_state * mask).sum(dim=1)
counts = mask.sum(dim=1).clamp(min=1e-9)
mean_pooled = sums / counts                  # (batch, 768)

Contrast With Other BERT Models

Model          | Hidden | pooler_output dim
------------------------------|--------------
BERT-tiny      |  128   | (batch, 128)
BERT-mini      |  256   | (batch, 256)
BERT-small     |  512   | (batch, 512)
BERT-base      |  768   | (batch, 768)
BERT-large     | 1024   | (batch, 1024)

In every BERT variant the pooler output dimension equals the hidden size. BERT-large uses 1024, so its pooler_output is (batch_size, 1024) with a 1024×1024 dense layer and 1,049,600 total pooler parameters.