RuntimeError: Trying to backward through the graph a second time" in PyTorchThis error means the computation graph backward() needs has already had its intermediate buffers freed, usually by an earlier backward() call that already ran through the same graph. It rarely shows up as literally calling loss.backward() twice in a row on purpose; it's almost always one call reusing a piece of a graph that a different, earlier call already consumed. This article reproduces the three real ways that happens and covers the fix for each, since they're not the same fix.
The simplest way to trigger it is to call backward() twice on the same loss tensor:
import torch
x = torch.tensor([2.0], requires_grad=True)
y = x ** 2
loss = y.sum()
loss.backward()
print(x.grad) # tensor([4.]) -- d/dx of x^2 at x=2
loss.backward()
# RuntimeError: Trying to backward through the graph a second time
# (or directly access saved tensors after they have already been freed).The first call computes gradients and, by default, frees the intermediate tensors the graph cached during the forward pass, since keeping them for every step would grow memory usage without bound across a long training run. The second call has nothing left to differentiate through. This exact two-line reproduction is rare in real code, but the underlying pattern, a graph getting consumed once and then reached for again, shows up in two much more common and less obvious forms below.
If a graph genuinely needs a second backward pass on purpose, telling PyTorch to keep the buffers around fixes it directly:
x = torch.tensor([2.0], requires_grad=True)
y = x ** 2
loss = y.sum()
loss.backward(retain_graph=True)
loss.backward()
print(x.grad) # tensor([8.]) -- gradients accumulate: 4 + 4backward() call, or a custom training loop that intentionally computes a gradient penalty by differentiating a quantity that itself depends on gradients already computed once. In both cases, two passes through the same forward computation are the actual intended behavior, not a bug.
retain_graph=True makes the error go away without fixing the underlying problem, and the visible cost is memory that keeps growing across training steps instead of being freed, since every step's now-retained graph stays alive. The two triggers below are exactly this case, and detaching the reused tensor is the correct fix, not retain_graph=True.
A recurrent model's hidden state carried from one training step into the next is the most common real-world source of this error, and it has nothing to do with calling backward() twice explicitly:
rnn = torch.nn.RNN(input_size=3, hidden_size=4, batch_first=True)
h = torch.zeros(1, 1, 4)
opt = torch.optim.SGD(rnn.parameters(), lr=0.01)
for step in range(2):
inp = torch.randn(1, 1, 3)
out, h = rnn(inp, h) # h still carries the previous step's graph
loss = out.sum()
opt.zero_grad()
loss.backward() # fails on step 2
opt.step()
# RuntimeError: Trying to backward through the graph a second timeStep one runs cleanly. Step two's rnn(inp, h) call builds a new graph, but that graph's first operation depends on h, and h is still the literal output tensor from step one's forward pass, still wired into step one's graph. Step one's backward() already freed that graph's buffers. Step two's backward() tries to walk back through it anyway, because h never stopped pointing at it.
Calling .detach() on the hidden state before it re-enters the model breaks the link back to the previous step's graph while keeping the actual values:
rnn = torch.nn.RNN(input_size=3, hidden_size=4, batch_first=True)
h = torch.zeros(1, 1, 4)
opt = torch.optim.SGD(rnn.parameters(), lr=0.01)
for step in range(3):
inp = torch.randn(1, 1, 3)
out, h = rnn(inp, h)
h = h.detach() # break the link to the previous step's graph
loss = out.sum()
opt.zero_grad()
loss.backward()
opt.step()
print("all 3 steps ran cleanly")detach() returns a new tensor with the same numbers but no autograd history, which is exactly what a training loop that intentionally carries state forward (as opposed to backpropagating through the entire sequence history every step, which is what truncated backpropagation through time is for) actually wants. This is also why the error tends to appear on the second iteration of a loop specifically, not the first: nothing is wrong until a tensor from a completed, already-backward()'d graph gets fed into a new forward pass.
A third, easy-to-miss version happens with intermediate values accumulated into a list or a running variable, then reused after the fact:
x = torch.tensor([1.0], requires_grad=True)
losses = []
running = x.clone()
for i in range(3):
running = running * 1.5
losses.append(running)
total = sum(losses)
total.backward()
print(x.grad) # works fine the first time
total.backward() # e.g. a notebook cell re-run, or a logging call that
# accidentally calls backward() again on the same object
# RuntimeError: Trying to backward through the graph a second timeNothing here looks like calling backward() twice in the same block of code. The two calls can be in different cells of a notebook, different iterations of an outer loop that reuses a variable name, or a metrics/logging function that unintentionally triggers another backward() on an object still referencing an already-consumed graph. The fix is the same principle as the RNN case: either don't reuse total for a second backward() call at all (recompute the forward pass instead), or, if a genuine second pass through the exact same computation is required, use retain_graph=True on the first call.
Situation Fix
───────────────────────────────────────────────── ─────────────────────────────────────────
Two backward() calls needed through the same graph retain_graph=True on the first call
by design (shared encoder, two loss heads, etc.)
RNN/LSTM hidden state carried into the next step h = h.detach() before the next forward()
Accumulated loss reused for a second backward() call recompute the forward pass instead of
by accident (notebook re-run, stray logging call) reusing the stored tensor, or detach it
Unsure which applies ask: was a second pass through the exact
same forward computation intended?