How to Fix "Trying to backward through the graph a second time"

Your first .backward() freed the tensors autograd saved during the forward pass, and a second backward walked into the same graph and found them gone. The fix depends on why there was a second walk. Sum the losses and call backward once, detach any state you carry between steps, or set retain_graph=True when you truly need two passes.

Check your training loop

Answer four questions about the loop that fails. The tool reports whether torch 2.8.0 raises, prints the exact exception text, and shows the fix that matches your case. Every path was run on CPU and the strings are copied from the real exceptions.

The error

RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

The message is long because it describes two triggers. During the forward pass every differentiable op saves what its backward formula needs, such as the output of exp or the inputs of mul. The first backward consumes those saved tensors and, unless told otherwise, releases them. The graph structure survives, so a second backward starts fine and fails the moment it needs a released value.

The PyTorch reference for torch.Tensor.backward defines the switch: with retain_graph=False "the graph used to compute the grads will be freed", with True "it will be retained", and the default None means the value "is inferred from create_graph". The same page adds that "in nearly all cases setting this option to True is not needed and often can be worked around in a much more efficient way". The four causes below are those cases.

Cause 1, backward called twice on the same loss

The plain version. Two calls, one graph, no retain.

x = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
loss = (x * x).sum()
loss.backward()
loss.backward()   # raises

loss.backward(retain_graph=True)
loss.backward()   # runs, x.grad is tensor([ 4.,  8., 12.])

Notice the number. Gradients accumulate, so the second pass doubled x.grad from [2, 4, 6]. If you see this pattern in your own code it usually means a helper function calls backward and the caller does too. Remove one of them.

Cause 2, a hidden state carried across steps without detach

This is the RNN version and the one that surprises people, because each step calls backward exactly once. The hidden state returned by step one still points at step one's graph. Step two multiplies it again, so step two's loss depends on step one's saved tensors, which step one's backward freed.

w = torch.tensor(0.5, requires_grad=True)
h = torch.tensor(1.0)
for step in range(4):
    h = h * w
    loss = (h - 0.1) ** 2
    loss.backward()        # raises on step 2
    h = h.detach()         # add this line and it runs

The detach is the correct fix. Truncated backpropagation through time is exactly this pattern, and it keeps every step's graph the same size. In the reproduction the graph depth measured from the loss was 4 on every step after detaching. With retain_graph=True and no detach the loop also runs, but the depth climbed 4, 5, 6, 7, one node per step, because each backward walks the entire history. Recomputing the forward pass from scratch on every step is a third working option when the sequence is short.

Cause 3, two losses that share a forward pass

An auxiliary head, a regulariser on features, a contrastive term. All of them reuse a tensor from the main forward pass. Backpropagating them one after the other fails on the second call.

feat = x.exp()
l1 = feat.sum()
l2 = (feat * feat).sum()

l1.backward()
l2.backward()          # raises, feat's graph is gone

(l1 + l2).backward()   # x.grad tensor([ 17.4964, 116.5854, 826.9431])

Summing before backward gives the same gradient as l1.backward(retain_graph=True) followed by l2.backward(), which the reproduction confirmed to four decimals, and it walks the graph once instead of twice. Weighted sums work the same way. Sum the losses, then call backward once. Calling backward twice on the summed loss puts you back at cause 1.

Cause 4, the GAN pattern

The discriminator loss is computed on the generator's output. When that output is not detached, the discriminator's backward runs all the way into the generator and frees its saved tensors. The generator loss then needs them.

fake = G(z)
d_loss = criterion(D(fake), zeros)          # should be D(fake.detach())
d_loss.backward()
g_loss = criterion(D(fake), ones)
g_loss.backward()                            # raises

Using D(fake.detach()) for the discriminator loss cuts the graph at the generator boundary. The generator's saved tensors survive, g_loss.backward() runs, and G.weight.grad is populated. That is also the intended behaviour, because the discriminator update should not push gradient into the generator.

The "directly access saved tensors" variant

The parenthetical in the message is real and was checked. After backward, reading a saved tensor through the private grad_fn attributes raises the identical text, with the same "Trying to backward through the graph a second time" prefix even though no backward was attempted.

y = x.exp()
loss = y.sum()
loss.backward()
y.grad_fn._saved_result     # raises the same RuntimeError

The same happened for _saved_self on a multiplication, both when the saved operand was a leaf and when it was an intermediate. Before backward the read works and returns the tensor. You will hit this path from custom autograd tooling, from debugging hooks, or from a library that inspects the graph after an optimizer step.

torch.autograd.grad behaves the same

torch.autograd.grad(loss, x) also frees the graph. Calling it twice raises the same error, and so does autograd.grad followed by loss.backward(). Two settings make the second call succeed: retain_graph=True, or create_graph=True, which retains the graph implicitly because higher order derivatives need it. Both returned tensor([ 3., 12., 27.]) on the second call for a cubic loss.

What retain_graph=True actually keeps

The docs call it inefficient. The reproduction put a number on it. A forward pass of two chained exp calls on a 2048 by 2048 float32 tensor makes autograd save two intermediates of 16 MB each. A pack hook attached a weak reference to every saved tensor, and after backward the count of survivors was read.

retain_graph=False: autograd saved 2 tensors (32 MB total); after backward 0 still alive holding 0 MB
retain_graph=True:  autograd saved 2 tensors (32 MB total); after backward 2 still alive holding 32 MB

So on this graph the flag holds 32 MB until the loss tensor itself is dropped, and on a real network it holds every activation the backward formulas need. Process RSS was also sampled, and it did not shrink after either backward on macOS because the allocator keeps freed pages, so the weak reference count is the measurement to trust. On CUDA the retained tensors show up directly in torch.cuda.memory_allocated(), which is why this flag so often precedes an out of memory error in a loop.

What torch 2.8.0 actually raises

Every row below was produced by one reproduction script on torch 2.8.0, CPU, and the exception text was copied without edits. The tool at the top of the page reproduces the rows that a training loop can reach.

One string covers every failing row. Nothing in the message tells you which of the four causes you hit, which is why the traceback line matters more than the text. Look at which backward call raised, then look at what tensor it shares with the previous one.

How common this is

This error sits at number 10 of 25 in the HeyTensor ranking of PyTorch errors, tagged as roughly a fifth of all gradient errors. The ranking's methodology combines Stack Overflow question frequency, view counts and votes collected through the SO API in April 2026, weighted 50, 30 and 20 percent. It is the only entry in the top ten that has nothing to do with shapes, dtypes or devices. It is purely a control flow mistake, which is why a decision tool fits it better than a shape calculator.

Built by Michael Lip. Behaviour verified against torch 2.8.0 on 6 September 2026.