Fix PyTorch RuntimeError: Expected Scalar Type Long but Found Float

By EducatBild · PyTorch · Error fixes · Deep learning

Training a classification model, and suddenly the loop crashes with RuntimeError: expected scalar type Long but found Float. Almost every time, the cause is the same: your loss function wants integer class indices and got floating-point values instead. Below is why PyTorch draws that line so strictly, and where the mismatch tends to sneak in beyond the obvious case.

1. The Exact Error Message (Both Variants)

Depending on your PyTorch version, you will see one of these two messages in your traceback:

Modern PyTorch (≥1.8)

RuntimeError: expected scalar type Long but found Float

Older PyTorch (<1.8)

RuntimeError: Expected object of scalar type Long but got scalar type Float
for argument #2 'target' in call to _thnn_nll_loss_forward

Both messages point to the same problem. A full traceback typically looks like this:

Traceback (most recent call last):
  File "train.py", line 34, in <training loop>
    loss = criterion(outputs, labels)
  File ".../torch/nn/modules/loss.py", line 1179, in forward
    return F.cross_entropy(input, target, weight=self.weight,
  File ".../torch/nn/functional.py", line 3029, in cross_entropy
    return torch._C._nn.cross_entropy_loss(input, target, ...)
RuntimeError: expected scalar type Long but found Float

The key line is loss = criterion(outputs, labels). PyTorch is telling you that labels arrived as a torch.float32 tensor when the loss function required torch.int64 (also called torch.long).

2. Root Cause: CrossEntropyLoss Expects Class Indices as torch.long

nn.CrossEntropyLoss computes the negative log-likelihood over a set of class probabilities. Internally it calls torch.nll_loss, which uses each element of the target tensor as an integer index into the output dimension, and integer indexing on the C++ backend only accepts a 64-bit integer type: torch.int64, which PyTorch exposes as torch.long.

The most common reason your labels end up as float32 instead of int64:

• You loaded them from a CSV or NumPy array without specifying a dtype, so they defaulted to float64 or float32.
• You used torch.tensor([0, 1, 2]) on a Python list that happened to be inferred as float.
• You applied a normalisation step (e.g., dividing by the number of classes) that converted the integer tensor to float.
• Your Dataset.__getitem__ returns np.float32 arrays for both inputs and targets, so the DataLoader converts everything to float tensors.
• You used label_binarize or one-hot encoding and forgot to convert back to class indices.

3. The One-Line Fix: labels.long()

Call .long() on your target tensor before passing it to the loss function, and the crash is gone.

import torch
import torch.nn as nn

criterion = nn.CrossEntropyLoss()

# Suppose labels arrived as float (common when loaded from CSV/numpy)
labels = torch.tensor([0, 2, 1, 3], dtype=torch.float32)
print(labels.dtype)  # torch.float32  ← wrong for CrossEntropyLoss

# One-line fix
labels = labels.long()
print(labels.dtype)  # torch.int64  ← correct

outputs = torch.randn(4, 4)  # batch_size=4, num_classes=4
loss = criterion(outputs, labels)  # works without error
print(loss.item())

You can also apply the cast directly in your training loop to keep things concise:

for inputs, labels in dataloader:
    inputs  = inputs.to(device)
    labels  = labels.long().to(device)   # <-- cast here, before loss

    optimizer.zero_grad()
    outputs = model(inputs)
    loss    = criterion(outputs, labels)
    loss.backward()
    optimizer.step()

Or fix it at the source, inside your Dataset, so it never comes up again:

import numpy as np
import torch
from torch.utils.data import Dataset

class MyDataset(Dataset):
    def __init__(self, X, y):
        self.X = torch.tensor(X, dtype=torch.float32)
        self.y = torch.tensor(y, dtype=torch.long)   # enforce long here

    def __len__(self):
        return len(self.y)

    def __getitem__(self, idx):
        return self.X[idx], self.y[idx]

4. BCELoss vs CrossEntropyLoss Dtype Requirements (Common Confusion)

A frequent source of confusion is switching between binary and multi-class classification. The two loss functions have opposite requirements for the target tensor:

• CrossEntropyLoss: targets must be torch.long (integer class indices, shape [batch_size])
• BCELoss / BCEWithLogitsLoss: targets must be torch.float32 (probabilities or binary 0/1 values, same shape as the output)

import torch
import torch.nn as nn

batch_size = 8
num_classes = 5

# ── Multi-class: CrossEntropyLoss ──────────────────────────────
logits_ce = torch.randn(batch_size, num_classes)
targets_ce = torch.randint(0, num_classes, (batch_size,))  # dtype=torch.long automatically
criterion_ce = nn.CrossEntropyLoss()
loss_ce = criterion_ce(logits_ce, targets_ce)  # OK

# ── Binary: BCEWithLogitsLoss ──────────────────────────────────
logits_bce = torch.randn(batch_size, 1)
targets_bce = torch.randint(0, 2, (batch_size, 1)).float()  # must cast to float!
criterion_bce = nn.BCEWithLogitsLoss()
loss_bce = criterion_bce(logits_bce, targets_bce)  # OK

# ── Common mistake: using float labels with CrossEntropyLoss ──
bad_targets = torch.randint(0, num_classes, (batch_size,)).float()  # accidentally float
# criterion_ce(logits_ce, bad_targets)  # <-- RuntimeError!
criterion_ce(logits_ce, bad_targets.long())  # fixed with .long()

If you are migrating a binary classifier to multi-class (or vice versa), double-check which loss function you are using and flip the target dtype accordingly.

5. Embedding Layer Index Type (Same Error, Different Context)

The exact same RuntimeError: expected scalar type Long but found Float shows up with nn.Embedding too, for the same underlying reason: an embedding layer maps integer token indices to dense vectors, and its input has to be torch.long.

import torch
import torch.nn as nn

vocab_size = 1000
embed_dim  = 64
embedding  = nn.Embedding(vocab_size, embed_dim)

# Token indices should be long
indices_wrong = torch.tensor([4, 17, 42, 0], dtype=torch.float32)
# embedding(indices_wrong)  # RuntimeError: expected scalar type Long but found Float

indices_correct = indices_wrong.long()
output = embedding(indices_correct)   # works: shape [4, 64]
print(output.shape)

The fix is identical: call .long() on the index tensor before passing it to the embedding layer. This is especially easy to forget when token ids come from a tokeniser that returns NumPy arrays, since those default to int32 on some platforms. That's still the wrong type; PyTorch requires int64.

import numpy as np

# numpy default integer is int64 on Linux/macOS but int32 on some Windows builds
token_ids_np = np.array([4, 17, 42, 0])
token_ids    = torch.from_numpy(token_ids_np).long()  # safe on all platforms
output = embedding(token_ids)

6. Full Dtype Table for Common PyTorch Loss Functions

Use this table as a quick reference whenever you are not sure which dtype to use for inputs and targets.

If you lose track of which row applies, the table splits cleanly by what the loss function is counting: CrossEntropyLoss and NLLLoss index into discrete classes, so they need long. Everything else in the table (regression, binary, probability outputs) is comparing continuous values, so it stays in float32.

7. Debugging Snippet: Print Dtype of Every Tensor Before the Forward Pass

When you are not sure which tensor has the wrong dtype, add a quick diagnostic block before the forward pass. This prints the name, shape, and dtype of every tensor you are about to use and makes the culprit obvious.

def debug_dtypes(**tensors):
    """Print name, shape, and dtype for each tensor. Call before forward pass."""
    print("=" * 55)
    for name, t in tensors.items():
        if hasattr(t, "dtype"):
            print(f"  {name:20s}  shape={str(tuple(t.shape)):20s}  dtype={t.dtype}")
        else:
            print(f"  {name:20s}  (not a tensor, type={type(t).__name__})")
    print("=" * 55)


# Usage inside your training loop:
for inputs, labels in dataloader:
    debug_dtypes(inputs=inputs, labels=labels)

    # After confirming dtypes, comment out the line above to speed up training
    labels = labels.long()
    outputs = model(inputs)
    loss = criterion(outputs, labels)

Example output when labels are accidentally float:

=======================================================
  inputs               shape=(32, 3, 224, 224)      dtype=torch.float32
  labels               shape=(32,)                  dtype=torch.float32  <-- wrong!
=======================================================

After applying labels.long():

=======================================================
  inputs               shape=(32, 3, 224, 224)      dtype=torch.float32
  labels               shape=(32,)                  dtype=torch.int64   <-- correct
=======================================================

You can also check dtypes inline without a helper function using PyTorch's built-in .dtype attribute and assertions:

assert inputs.dtype == torch.float32, f"inputs dtype mismatch: {inputs.dtype}"
assert labels.dtype == torch.long,    f"labels dtype mismatch: {labels.dtype}"

Add assertions like these at the top of your training loop while you're debugging, then pull them once the model is stable. It's a cheap form of data validation that catches dtype bugs the moment they appear instead of after an hour of confused staring at a traceback.

Other common dtype-related gotchas to watch for

• Mixed precision (torch.cuda.amp): When using automatic mixed precision, model outputs may be float16. The loss function will upcast them internally, but make sure your loss criterion itself remains in float32 (it does by default when you use GradScaler).
• Double precision models: If you call model.double(), your input tensors also need to be float64. Labels still stay long.
• Segmentation masks: Pixel-wise segmentation uses CrossEntropyLoss with 2D targets of shape [N, H, W]. Each pixel value is a class index and must be long, not float.
• One-hot targets: CrossEntropyLoss expects class indices (a 1D tensor of integers), not one-hot encoded vectors. If you pass a one-hot float tensor, you get a shape error on top of the dtype error. Convert back to class indices with labels = one_hot_labels.argmax(dim=1).long().

8. Related Fixes

Dtype errors and device errors tend to show up back to back in the same training run, since both come from PyTorch refusing to silently coerce a tensor into something it isn't. Two others worth bookmarking:

• Fix PyTorch CUDA Out of Memory Error: reduce batch size, gradient checkpointing, and memory profiling
• Fix PyTorch RuntimeError: Expected all tensors to be on the same device
• Fix CUDA device-side assert triggered: find the host-side error behind the cryptic GPU crash

Found a mistake or have a question? The examples on this page were tested on PyTorch 2.x. The dtype behaviour described here has been consistent since PyTorch 1.6.