ValueError: setting an array element with a sequence" (Ragged Nested Sequence) in NumPyThis error shows up the moment you try to build a NumPy array out of nested lists that don't all have the same length, and the message itself doesn't mention "ragged" or "nested" anywhere, which is exactly why it's hard to search for the first time you hit it. This article reproduces the base case, the real data-pipeline trigger (batching variable-length tokenized sequences), and the two working fixes with a real tradeoff between them.
Building an array from a list of rows with different lengths raises immediately in current NumPy:
import numpy as np
np.array([[1, 2, 3], [4, 5]])
# ValueError: setting an array element with a sequence. The requested array
# has an inhomogeneous shape after 1 dimensions. The detected shape was
# (2,) + inhomogeneous part."Inhomogeneous" is NumPy's word for what's normally called a ragged array: the outer list has 2 elements, but those elements (the inner lists) don't all have the same length, so there's no single rectangular shape that describes the whole thing. A regular NumPy array with a numeric dtype has to be a fixed-shape block of memory, so NumPy refuses rather than guessing at a shape. This isn't limited to lists of lists, either -- mixing a scalar with a list at the same nesting level trips the identical error:
np.array([1, [2, 3]])
# ValueError: setting an array element with a sequence. The requested array
# has an inhomogeneous shape after 1 dimensions. The detected shape was
# (2,) + inhomogeneous part.Uniform nested lists, by contrast, build a normal 2D array with no error at all, which is why this often surprises people the first time one row in a much larger dataset happens to be a different length than the rest:
np.array([[1, 2], [3, 4], [5, 6]])
# array([[1, 2],
# [3, 4],
# [5, 6]]) -- shape (3, 2), int64, no errorThe single most common way this actually happens in an ML pipeline isn't a typo, it's tokenized text. Different sentences tokenize to different numbers of tokens, and stacking them directly into an array without padding first reproduces the exact same error on realistic-looking data:
tokenized = [
[101, 2054, 2003, 102], # "what is" -- 4 tokens
[101, 2129, 102], # "how" -- 3 tokens
[101, 2079, 2017, 2113, 102], # "do you know" -- 5 tokens
]
np.array(tokenized)
# ValueError: setting an array element with a sequence. The requested array
# has an inhomogeneous shape after 1 dimensions. The detected shape was
# (3,) + inhomogeneous part.This is the version worth actually recognizing, because the fix is not "make the lengths match by accident" the way a small hand-written example might suggest -- it's a real design decision about how to handle variable-length input, covered below.
Passing dtype=object explicitly tells NumPy not to try to build a rectangular numeric array, and it accepts the ragged input immediately:
arr = np.array([[1, 2, 3], [4, 5]], dtype=object)
print(arr) # [list([1, 2, 3]) list([4, 5])]
print(arr.shape) # (2,) -- not (2, 3), each element is still a Python list
print(arr.dtype) # objectThis stops the error and is genuinely useful for temporarily holding heterogeneous data, but it's a much weaker object than a normal NumPy array: arr.shape is (2,), not (2, 3), because each element really is just a reference to a Python list, not a row of native numbers. Vectorized math, broadcasting, and any operation that assumes a fixed inner dimension either fail outright or silently fall back to slow, element-by-element Python loops. Reach for this only as a holding pattern before deciding how to actually make the data rectangular, not as the final shape you feed into a model.
For the tokenized-sequence case, and for variable-length ML input generally, the real fix is padding every sequence out to the same length before building the array, using a designated pad value that the model is told to ignore:
PAD_TOKEN = 0
maxlen = max(len(seq) for seq in tokenized)
padded = np.array([seq + [PAD_TOKEN] * (maxlen - len(seq)) for seq in tokenized])
print(padded)
# [[ 101 2054 2003 102 0]
# [ 101 2129 102 0 0]
# [ 101 2079 2017 2113 102]]
print(padded.shape) # (3, 5) -- a real, uniform, numeric arrayThis gives back a normal numeric array with full vectorization and broadcasting support, at the cost of also needing an attention mask (or equivalent) downstream so the model doesn't treat the padding as real content -- which is exactly what tokenizer libraries like Hugging Face's transformers do automatically when you pass padding=True, rather than something you'd usually hand-roll in production. The point of reproducing it manually here is to make visible what that automatic padding is actually doing under the hood, since this same shape-mismatch error is what it exists to prevent.
Code that built ragged arrays without an explicit dtype=object used to run with only a VisibleDeprecationWarning, silently falling back to an object array anyway. That fallback was removed in favor of a hard error, which means old code that happened to construct ragged arrays by accident -- and never looked at its warnings -- can start failing today on an input it accepted for years, with no other code change involved. If you hit this error on a codebase that "used to work," the honest read is that the ragged construction was already a latent bug; a NumPy upgrade just turned the silent version into a loud one.
Situation Fix
───────────────────────────────────────────── ─────────────────────────────────────────
Genuinely heterogeneous data, held temporarily dtype=object -- accept the tradeoff,
before deciding how to reshape it don't feed it straight into math ops
Variable-length sequences going into a model Pad to a uniform length (+ an
(tokenized text, time series, etc.) attention mask / ignore-index if needed)
One row is the wrong length by mistake Fix the data source -- this is a real
(typo, bad parse, malformed record) bug, not a shape to work around
Old code suddenly fails after a NumPy upgrade The ragged construction was already
a latent bug; add dtype=object or
padding explicitly, don't pin NumPy back