Fixing "expected dtype torch.float32 for the weight argument but got dtype torch.float64"

By Michael Lip · September 18, 2026 · 6 min read

What the Error Means

PyTorch implements elementwise and matrix operations with a strict same-dtype rule. If you multiply a float32 tensor by a float64 tensor, you do not get a silent promotion the way numpy would give you one. You get a RuntimeError. The library refuses to guess which precision you wanted and stops before it burns memory on an accidental upcast.

The message names two things. the weight argument is the parameter the operator is holding — for nn.Linear that is self.weight, a tensor of shape (out_features, in_features); for nn.Conv2d it is the filter bank of shape (out_channels, in_channels/groups, kH, kW). got dtype torch.float64 is the tensor you supplied. So the error reads: this module's weights are single precision, your input is double precision, and the two cannot be combined.

The diagnosis is one line of code. Print the dtypes of both sides:

print(layer.weight.dtype)  # torch.float32  <- what the module holds
print(x.dtype)         # torch.float64  <- what you passed

Once those two lines disagree, the fix is to make them agree — in whichever direction your project actually needs. For the neighboring family of type errors, where the offending value is not a dtype but a scalar, see expected scalar type Float but found Double.

Why the Mismatch Happens

Two defaults collide. numpy's floating-point default is float64, and PyTorch's floating-point default is float32. Neither library is wrong; they simply disagree, and every path that moves an array from one to the other preserves the wrong half of the agreement.

The trap is the fifth row. torch.from_numpy is zero-copy: it wraps the existing numpy buffer instead of reading the values out. That is exactly why it is fast, and exactly why it carries float64 straight into your model. If the array came from pandas, DataFrame.to_numpy() and df.values are also float64 for the same reason. So the chain is usually CSV → pandas → numpy float64 → torch.from_numpy → nn.Linear, with the dtype never once questioned.

There is a second, rarer source: an explicit .double() somewhere in the codebase. If a loader or a test helper calls x.double(), or a module was moved with model.to(torch.float64), then that side is double while a freshly constructed submodule is still float32. Grep for .double(), float64, and set_default_dtype before assuming the data is at fault.

Reproduce It in Four Lines

The smallest failing program is four lines. Run it and you will get the error verbatim:

import numpy as np
import torch, torch.nn as nn

W = np.random.randn(4, 3)   # numpy default: float64
layer = nn.Linear(3, 4)    # weight: float32

layer(torch.from_numpy(W))
# RuntimeError: expected dtype torch.float32 for the weight argument
# but got dtype torch.float64

This is also the reason the error often shows up in a unit test rather than in training. Tests routinely build inputs with np.ones, np.zeros, or hand-written lists, none of which go through the training-time data pipeline where a .float() cast already exists.

Fix 1 — Cast the Data to float32

This is the fix for the overwhelming majority of cases. Convert once, at the boundary where numpy ends and torch begins.

# Option A: cast on the numpy side, then wrap
W32 = W.astype(np.float32)      # new array, float32
x = torch.from_numpy(W32)      # dtype inherited: torch.float32

# Option B: cast on the torch side (equivalent result)
x = torch.from_numpy(W).float()   # .float() == .to(torch.float32)
x = torch.from_numpy(W).to(torch.float32) # explicit form

# Option C: go through torch.tensor(), which copies and casts
x = torch.tensor(W, dtype=torch.float32)

Where you put the cast matters more than which spelling you use. Casting inside forward() is correct but wasteful: you pay a full copy of the batch on every step. Cast once in the dataset or the loader, and every later forward() is clean.

# Better: fix it once at the data boundary
class CSVTorchDataset(torch.utils.data.Dataset):
  def __init__(self, path):
    self.raw = np.loadtxt(path, delimiter=",") # float64

  def __getitem__(self, i):
    # cast here, once per sample, never inside forward()
    return torch.from_numpy(self.raw[i].astype(np.float32))

# Then a collate/batching step keeps float32 all the way through
loader = torch.utils.data.DataLoader(ds, batch_size=32)

Fix 2 — Switch the Whole Model to float64

If your project genuinely needs double precision — iterative solvers, financial or scientific modeling, gradient checks, or numerical work where float32 accumulation loses too many bits — then the correct direction is to move the model to the data, not the data to the model. model.double() is the switch.

model = MyNet()
model.double()   # every parameter and buffer -> torch.float64
# equivalent to: model.to(torch.float64)

x = x.double()   # the INPUT must move too
x = x.to(torch.float64)  # explicit form
y = model(x)     # now both sides are float64

Two things trip people up here. First, model.double() changes the parameters in place and returns self, so it must be called before you hand model.parameters() to an optimizer that captured the old tensors. Second, and this is the one that produces the mirror-image error, double() does not touch your input. A model in float64 fed a float32 tensor raises expected dtype torch.float64 for the weight argument but got dtype torch.float32 — the same error with the operands swapped.

# Common follow-up bug: labels don't follow the data
model.double()
x = x.double()
y = y.long()              # keep integer labels as long, never double
# For regression targets, cast them too:
# y = y.double()

Fix 3 — Set the Global Default Dtype

torch.set_default_dtype(torch.float64) changes what torch.get_default_dtype() returns, which changes the dtype of newly created floating-point tensors and of module parameters initialized after the call. Used as a one-liner at the top of a script, it makes torch agree with numpy and the mismatch stops appearing.

import torch
torch.set_default_dtype(torch.float64)

torch.tensor([1.0, 2.0]).dtype   # torch.float64 now
nn.Linear(3, 4).weight.dtype     # torch.float64 now

# Note: set_default_dtype only accepts floating-point dtypes.
# torch.set_default_dtype(torch.int64) raises a RuntimeError.

Use this with care. It is process-global mutable state: it affects every tensor created anywhere in the program afterward, including inside third-party libraries that were written against a float32 norm. Weight initialization also assumes sqrt(1/fan_in) style bounds scaled for float32; RNG values themselves are not affected, but the behavior of pretrained checkpoints is, because a checkpoint saved in float32 loaded into a float64 module is a dtype mismatch again. In a large codebase the mismatch that set_default_dtype fixes in one file reappears in another. Prefer it for self-contained scripts and numerical experiments; prefer Fix 1 for libraries and training pipelines.

The Variant With torch.FloatTensor and numpy.float64

A second wording of the same failure appears when the mismatch is caught at the tensor type level rather than the dtype level:

Two historical spellings, one cause. In PyTorch 0.4 and earlier, tensors carried a distinct type per dtype, so the message named types: torch.FloatTensor was float32 and torch.DoubleTensor was float64. The Expected object of scalar type Float but got scalar type Double form persisted into the 1.x series and still shows up in older Stack Overflow answers and in torch.nn.functional helpers that pass a numpy array straight through. The expected torch.FloatTensor but got numpy.float64 wording comes from a wrapper layer — typically custom code or a thin binding — that receives the raw numpy object before it has been wrapped by torch.from_numpy at all.

The mapping is one-to-one, so treat all three messages as the same defect:

In the numpy.float64 case the object is not even a tensor yet. Something handed a numpy scalar or array to an operator expecting a tensor, and the type check fired before the dtype check could. The repair is the same torch.as_tensor(x, dtype=torch.float32) you would write for the modern message.

Before and After

# ---------------- BEFORE (raises RuntimeError) ----------------
import numpy as np
import torch
import torch.nn as nn

class Net(nn.Module):
  def __init__(self):
    super().__init__()
    self.fc1 = nn.Linear(784, 128)   # weight: float32
    self.fc2 = nn.Linear(128, 10)    # weight: float32

  def forward(self, x):
    return self.fc2(torch.relu(self.fc1(x)))

model = Net()

# data arrives from numpy with numpy's default dtype
X = np.loadtxt("train.csv", delimiter=",")  # float64!
xb = torch.from_numpy(X[:32])          # still float64 (zero-copy)
model(xb)
# RuntimeError: expected dtype torch.float32 for the weight
# argument but got dtype torch.float64

# ---------------- AFTER (runs) ----------------
import numpy as np
import torch
import torch.nn as nn

class Net(nn.Module):
  def __init__(self):
    super().__init__()
    self.fc1 = nn.Linear(784, 128)
    self.fc2 = nn.Linear(128, 10)

  def forward(self, x):
    return self.fc2(torch.relu(self.fc1(x)))

model = Net()

X = np.loadtxt("train.csv", delimiter=",")
xb = torch.from_numpy(X[:32]).float()   # <-- float64 -> float32
y = model(xb)

assert xb.dtype == model.fc1.weight.dtype == torch.float32
print(y.dtype)   # torch.float32

# Keep it fixed for good: cast at the data boundary, not per batch.
def to_tensor(batch):
  return torch.as_tensor(batch, dtype=torch.float32)

Which Fix to Choose

If the corrupted dtype has already propagated and you are triaging a wider set of failures, the percentage breakdown in most common PyTorch errors is a useful map, and PyTorch shape errors explained covers the adjacent shape family that frequently appears in the same traceback.

Prevention Checklist

• Cast at the data boundary: Dataset or collate, never inside forward().
• Prefer torch.as_tensor(x, dtype=torch.float32) over bare torch.from_numpy(x).
• Never call .double() on inputs without also calling model.double().
• Grep the codebase for set_default_dtype and float64 before adding a cast — you may be fixing a symptom.
• Keep integer labels as torch.long; only floats are affected by this class of error.
• Add assert x.dtype == model.fc1.weight.dtype in a test, so the mismatch fails loudly and early.

Check Your Own Dtype Pair

Type the dtype on each side of the error, separated by a pipe, to see whether the check passes and which cast to use.

Frequently Asked Questions