Fix "numpy.linalg.LinAlgError: Singular matrix" in NumPy

This error means the matrix you handed to np.linalg.inv() has no inverse at all, not that NumPy failed to compute one. It shows up most often not in a raw matrix-algebra example but buried inside ordinary linear regression code, where a design matrix built from real features turns out to be singular for reasons that aren't obvious from the DataFrame it came from. This article reproduces the two most common real-world triggers and covers three different fixes, each appropriate for a different situation.

Reproducing the Error

The textbook trigger is a matrix where one row is a multiple of another:

import numpy as np

A = np.array([[1., 2.],
              [2., 4.]])   # row 2 is exactly 2x row 1

np.linalg.inv(A)
# numpy.linalg.LinAlgError: Singular matrix

The determinant of A is 1*4 - 2*2 = 0. Any matrix with a zero determinant has no inverse, and NumPy raises rather than returning something misleading. That much is easy to spot when the matrix is small and printed on screen. It's much less obvious inside a 20-feature design matrix built from a real dataset, which is where this error actually costs debugging time.

Real Trigger 1: Collinear Regression Features

The normal-equations form of linear regression solves for coefficients directly: beta = inverse(X^T X) @ X^T @ y. If two input features are exact linear combinations of each other, not just correlated but mathematically redundant, X^T X becomes singular:

np.random.seed(0)
n = 100
x1 = np.random.randn(n)
x2 = 2 * x1                      # x2 is just x1, scaled -- perfectly redundant
X = np.column_stack([np.ones(n), x1, x2])
y = 3 + 2 * x1 + np.random.randn(n) * 0.1

beta = np.linalg.inv(X.T @ X) @ X.T @ y
# numpy.linalg.LinAlgError: Singular matrix

Nothing about x1 and x2 looks wrong individually, no NaNs, no duplicate column names, no constant values. The problem only exists in the relationship between them: x2 carries zero information beyond what x1 already provides, so the regression has no unique way to split credit between two coefficients that are mathematically tied together. Engineered features (a total that's the sum of two others already in the matrix, a ratio built from two existing columns) are a common real-world source of this, not just accidental duplication.

Real Trigger 2: The Dummy Variable Trap

One-hot encoding a categorical column without dropping a reference level, alongside an intercept term, produces a subtler version of the same redundancy:

import pandas as pd

cat = np.random.choice(['A', 'B', 'C'], size=60)
dummies = pd.get_dummies(pd.Series(cat), dtype=float)   # columns: A, B, C
X = np.column_stack([np.ones(60), dummies.values])       # intercept + all 3 dummies
y = np.random.randn(60)

np.linalg.inv(X.T @ X) @ X.T @ y
# numpy.linalg.LinAlgError: Singular matrix

Here the redundancy is that the three dummy columns always sum to exactly 1 for every row, which is precisely what the intercept column already is. Four columns, three degrees of freedom. np.linalg.matrix_rank(X) confirms it directly: 3, not 4, for a 4-column matrix. This is a well-known statistics pitfall with a one-line fix, dropping one category as the reference level:

dummies_fixed = pd.get_dummies(pd.Series(cat), drop_first=True, dtype=float)
X_fixed = np.column_stack([np.ones(60), dummies_fixed.values])   # intercept + 2 dummies

beta = np.linalg.inv(X_fixed.T @ X_fixed) @ X_fixed.T @ y
print(beta)   # works -- coefficients are now relative to the dropped category

Fix: The Pseudo-Inverse (pinv)

When you can't or don't want to fix the underlying redundancy, np.linalg.pinv() computes the Moore-Penrose pseudo-inverse, which exists for every matrix, singular or not:

beta = np.linalg.pinv(X.T @ X) @ X.T @ y
print(beta)
# [3.00751531 0.40229397 0.80458794]  -- runs without error

Better Fix: lstsq, Which Also Flags the Real Problem

np.linalg.lstsq() solves the same regression problem without ever forming X^T X or calling inv() at all, and it returns the matrix's rank alongside the solution:

beta, residuals, rank, singular_values = np.linalg.lstsq(X, y, rcond=None)
print(beta)
# [3.00751531 0.40229397 0.80458794]  -- same answer as pinv
print(rank, X.shape[1])
# 2 3   -- rank below column count is the actual signal something's redundant

Comparing rank to the number of columns is what pinv() never tells you: if they don't match, the design matrix has redundant features worth investigating, even though lstsq() still returns a usable answer either way. This is the version worth reaching for by default in regression code, not just as a debugging step.

Fix for a Matrix That Must Be Invertible: Ridge Regularization

Some code genuinely needs an invertible matrix back, not just a least-squares solution, for example a covariance matrix used downstream in a formula that specifically calls for its inverse. Adding a small value to the diagonal (ridge regularization) makes a singular or near-singular matrix invertible with a negligible change to the result:

lam = 1e-3
XtX = X.T @ X
beta_ridge = np.linalg.inv(XtX + lam * np.eye(XtX.shape[0])) @ X.T @ y
print(beta_ridge)
# [3.00748536 0.40229353 0.80458706]  -- nearly identical to the pinv/lstsq answer

This is the same mathematical idea behind ridge regression as a modeling technique, applied here purely as a numerical-stability fix. A larger lam shifts the result further from the unregularized answer, so it's worth starting small (1e-6 to 1e-3) and checking that the output doesn't move much as you increase it, which confirms the fix isn't quietly changing what the computation means.

Summary: Which Fix Applies

Situation                                         Fix
────────────────────────────────────────────    ─────────────────────────────────────────
Regression via normal equations, X^T X sing.     np.linalg.lstsq(X, y) -- also reports rank
One-hot encoding + intercept (dummy trap)         pd.get_dummies(..., drop_first=True)
Need a coefficient answer quickly, no diagnosis   np.linalg.pinv(A)
Genuinely need an invertible matrix returned      A + lam * np.eye(n)  (ridge/Tikhonov)
Unsure which applies                              Start with lstsq; check rank vs. columns