How to Fix "Input type (double) and bias type (float) should be the same"

Your input tensor is float64 and the layer's parameters are float32. The float64 almost always came from NumPy: np.random.rand and np.array([1.5]) produce float64, and torch.from_numpy keeps that dtype, while nn.Conv2d and nn.Linear are created in float32. Fix it with x = x.float() before the forward call, or convert at the source with a.astype(np.float32).

Check your combination

Pick the dtype of your input, how you built it, the layer it goes into, the dtype of that layer's weights and the device. The tool prints the exact exception torch 2.8.0 raises, or says Valid. Every combination it can output was run on torch 2.8.0, CPU and Apple MPS, and the strings are copied from the real exceptions.

The error and its two siblings

RuntimeError: Input type (double) and bias type (float) should be the same

One cause, three message families. Which one you see depends on which kernel catches the mismatch first.

• Convolution. nn.Conv1d, nn.Conv2d and nn.Conv3d check the input against the bias before doing any work, and name both in the message. Drop the bias with bias=False and the check disappears, so the CPU kernel raises instead with expected scalar type Double but found Float.
• Matrix multiply. nn.Linear, F.linear and x @ w land in addmm or mm. With a bias present the message is mat1 and mat2 must have the same dtype, but got Double and Float. Without a bias it is expected m1 and m2 to have the same dtype, but got: double != float. Same check, different spelling.
• Normalization and loss weights. nn.LayerNorm and nn.BatchNorm1d with float64 affine parameters raise mixed dtype (CPU): expect parameter to have scalar type of Float. A float64 class weight passed to F.cross_entropy raises expected scalar type Float but found Double.

Elementwise operations do not raise at all. float64_tensor * float32_tensor promotes to float64 and carries on. That silence is why the mismatch travels through a preprocessing pipeline unnoticed and only surfaces at the first layer with weights.

Where the float64 comes from

Run these four lines and the whole page makes sense.

np.array([1.5]).dtype                       # float64
torch.from_numpy(np.random.rand(2)).dtype   # torch.float64
torch.tensor(1.5).dtype                     # torch.float32
torch.get_default_dtype()                   # torch.float32

The PyTorch reference for torch.from_numpy says the returned tensor and the array share the same memory, and it lists numpy.float64 first among the accepted dtypes. Sharing memory means there is no conversion step where a dtype could change. A float64 array becomes a float64 tensor, full stop. torch.tensor(np_array) copies instead of sharing, but it also reads the dtype from the array, so it produces float64 too.

The trap then compounds. A TensorDataset built from torch.from_numpy arrays yields float64 batches through a DataLoader, because collation stacks tensors without touching their dtype. Your training loop receives float64, your model was created in float32, and the first nn.Conv2d raises. On torch 2.8.0 the batch from such a loader reports torch.float64 for the features and torch.int64 for integer labels.

Other sources of float64 tensors: pandas columns via .values, scipy outputs, image arrays after a division by 255 in NumPy, and np.mean or np.linspace results. All of them go through the same door.

Fix 1, cast the tensor

x = torch.from_numpy(a)        # torch.float64
out = model(x.float())         # works, output float32

.float() is shorthand for .to(torch.float32). Put it where the tensor is created rather than at every call site, or add it inside the Dataset.__getitem__ so batches leave the loader in the right dtype. If your model can run in more than one precision, the safest spelling is x.to(model.weight.dtype) or x.to(next(model.parameters()).dtype), which follows the weights wherever they go.

Fix 2, convert the array at the source

a = np.random.rand(1, 3, 8, 8).astype(np.float32)
x = torch.from_numpy(a)        # torch.float32, still shares memory
out = model(x)                 # works

This is the better fix for data pipelines because it halves the memory the arrays occupy and keeps the zero-copy property of torch.from_numpy. torch.tensor(a, dtype=torch.float32) and torch.as_tensor(a, dtype=torch.float32) also produce float32 and were verified to run through nn.Linear without error.

Fix 3, run the model in float64

model = model.double()
out = model(torch.from_numpy(a))   # works, output float64

Choose this only when you need double precision for numerical reasons, for example in scientific code that compares against a NumPy reference. Float64 doubles the memory of every parameter, activation and gradient, and consumer GPUs execute float64 arithmetic far slower than float32. Apple MPS does not execute it at all, which the last section covers.

Fix 4, torch.set_default_dtype does less than you think

The torch.set_default_dtype reference says the default dtype is used to infer the dtype of tensors constructed from Python floats, and that set_default_dtype(torch.float64) exists to facilitate NumPy-like type inference. Read that literally. It governs what PyTorch creates. It says nothing about arrays that NumPy already created.

Measured on torch 2.8.0 after torch.set_default_dtype(torch.float64):

So the call does make a float64 model match a float64 NumPy array, but only because the model moved, not the array. Flip the array to float32 and the same error returns with the words swapped. It is a global switch that changes every tensor you create afterwards, so treat it as a last resort.

Messages you searched for that torch 2.8.0 does not raise

Two phrasings bring people to this page and neither occurs verbatim in torch 2.8.0.

"expected torch.FloatTensor but got numpy.float64". The closest real message is raised by the legacy constructor when handed a NumPy scalar: torch.FloatTensor(np.float64(1.5)) gives TypeError: new(): data must be a sequence (got numpy.float64). Handed a float64 array instead, torch.FloatTensor(np.random.rand(2)) does not raise and returns float32. If you have the FloatTensor wording in an old traceback, the modern equivalent is the conv or matmul message above.

"expected dtype torch.float32 for the 'weight' argument but got dtype torch.float64". Also absent from 2.8.0. A float64 weight given to F.cross_entropy raises expected scalar type Float but found Double, and a float64 affine weight in layer_norm raises the mixed dtype message. The fix is the same in every case: weight = weight.float().

What torch 2.8.0 actually raises

Every row was produced on torch 2.8.0, CPU, Python 3.9.6, arm64, with NumPy 1.26.4, and the exception text was copied without edits.

The int64 rows matter for anyone feeding integer pixel arrays or token ids straight into a conv or linear layer. PyTorch reports the C type name, long long, in the conv message and the scalar name, Long, in the matmul message. Both mean torch.int64, and x.float() fixes both, unless the layer you wanted was nn.Embedding, which takes int64 by design.

Apple MPS behaves differently

On an Apple Silicon machine the mismatch usually never reaches the layer, because the MPS backend has no float64 at all. Moving the tensor raises first:

TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead.

The same TypeError fires on model.double().to("mps"), so Fix 3 is unavailable on MPS. Fix 1 and Fix 2 both work.

Three more MPS results from the same run, all with float32 and float16 tensors that did reach the device:

• nn.Conv2d with a bias prints the same message as CPU: Input type (float) and bias type (c10::Half) should be the same. With bias=False the wording changes to Input type (MPSFloatType) and weight type (MPSHalfType) should be the same.
• nn.Linear and F.linear with a float32 input and float16 weight do not raise a Python exception. The process aborts with a Metal assertion, Destination NDArray and Accumulator NDArray cannot have different datatype in MPSNDArrayMatrixMultiplication, and the interpreter dies with signal 6. No try block catches it. An int64 input raises a normal RuntimeError: MPS device does not support linear for non-float inputs.
• nn.LayerNorm on MPS accepted every float32, float16 and bfloat16 pairing without error. An int64 input raises RuntimeError: Failed to create function state object for: layer_norm_single_row_long.

The abort is the one to remember. If a training script on a Mac exits with no traceback right after you enabled half precision, this is the likely cause, and the fix is still to make the input dtype match the weight dtype.

Built by Michael Lip. Behaviour verified against torch 2.8.0 on 6 September 2026.