Why does scikit-learn require every feature to be numeric?

Because the actual math underneath every scikit-learn estimator, whether it's computing a dot product for linear regression or a split threshold for a decision tree, operates on arrays of numbers. There is no general, model-agnostic way to interpret a string like 'Santiago' numerically, so scikit-learn refuses to guess and raises this error instead of silently doing something arbitrary.

Is pd.get_dummies() or OneHotEncoder the better fix?

For a one-off script or notebook, pd.get_dummies() is faster to write. For anything that will see new data later, training a model, saving it, and calling predict() on a different DataFrame, OneHotEncoder inside a ColumnTransformer is the safer choice, because it remembers the exact columns it was fitted on and applies them consistently, instead of silently regenerating a different column set from whatever categories happen to appear in the new data.

When is LabelEncoder actually the right choice for a feature column?

Only when the categories have a genuine, meaningful order, like 'low', 'medium', 'high', and you want that order preserved as increasing integers. For that case, OrdinalEncoder (with an explicit categories argument) is the correct tool. LabelEncoder itself is documented for encoding the target variable y in classification, not for encoding input features, and using it on an unordered feature column like a city name introduces a fake numeric ordering that a linear model will treat as real.

Fix "ValueError: could not convert string to float" in sklearn

Fix "`ValueError: could not convert string to float`" in sklearn

This error shows up the moment a scikit-learn estimator's fit() or predict() receives a column it can't treat as a number, almost always a categorical or text column left unencoded in a DataFrame that otherwise looks numeric. This article reproduces the error directly, covers the two common encoding fixes and where each one breaks, and the specific predict-time bug that a naive fix introduces without you noticing at training time.

Contents

Reproducing the error
Quick fix: pd.get_dummies()
The hidden bug: get_dummies() at predict time
The LabelEncoder trap
The robust fix: ColumnTransformer + OneHotEncoder
Summary: which fix applies
Related posts

Reproducing the Error

Any DataFrame with a mix of numeric and string columns triggers this the moment it's passed to fit() without preprocessing:

import pandas as pd
from sklearn.linear_model import LinearRegression

df = pd.DataFrame({
    'sqft': [1000, 1500, 2000, 1200],
    'city': ['Santiago', 'Vina', 'Concepcion', 'Santiago'],
    'price': [100000, 150000, 200000, 110000]
})

X = df[['sqft', 'city']]
y = df['price']

LinearRegression().fit(X, y)
# ValueError: could not convert string to float: 'Santiago'

The traceback points at whichever internal NumPy conversion step first tries to cast the column, not necessarily the line that looks most suspicious, so on a wider DataFrame it's easy to lose time scanning the wrong column. The actual cause is always the same: at least one column scikit-learn is being asked to treat as a numeric feature contains strings.

Note: This is not sklearn-specific plumbing, it's NumPy underneath. np.asarray(df[['sqft', 'city']], dtype=float) raises the identical error on its own, confirming the DataFrame's mixed dtypes are the real cause, not anything about the specific estimator.

Quick Fix: pd.get_dummies()

For a one-off script, one-hot encoding the categorical column with pandas directly is the fastest path to something that runs:

X_dummies = pd.get_dummies(X, columns=['city'])
print(X_dummies.columns.tolist())
# ['sqft', 'city_Concepcion', 'city_Santiago', 'city_Vina']

model = LinearRegression().fit(X_dummies, y)  # works

This resolves the immediate error. It also introduces a second, quieter bug that only surfaces later, at predict time, on different data.

The Hidden Bug: get_dummies() at Predict Time

pd.get_dummies() only creates columns for the categories actually present in whatever DataFrame you pass it. Training data and new data rarely contain the exact same set of categories, so calling it separately on each produces two DataFrames with different columns:

X_new = pd.DataFrame({'sqft': [1300], 'city': ['Vina']})
X_new_dummies = pd.get_dummies(X_new, columns=['city'])
print(X_new_dummies.columns.tolist())
# ['sqft', 'city_Vina']   -- only one city column, because only one city appears here

model.predict(X_new_dummies)
# ValueError: The feature names should match those that were passed during fit.
# Feature names seen at fit time, yet now missing:
# - city_Concepcion
# - city_Santiago

Training used three city columns because the training data contained three cities; this single-row prediction request only contains one, so get_dummies() only generates one. Scikit-learn correctly refuses to guess what the missing columns should contain and raises, rather than silently treating them as zero.

Why this is worse than it looks: in a notebook, training and evaluation data are often built from the same one-time get_dummies() call, so this bug never appears during development. It shows up for the first time in a serving path, days or weeks later, on a single new row with a narrower category set than the training data, which is exactly the kind of environment where it's hardest to debug quickly.

The LabelEncoder Trap

Reaching for LabelEncoder instead of one-hot encoding avoids the column-mismatch problem entirely, since it always produces exactly one integer column. That's also what makes it the wrong tool here:

from sklearn.preprocessing import LabelEncoder

le = LabelEncoder()
codes = le.fit_transform(['Santiago', 'Vina', 'Concepcion', 'Santiago'])
print(codes, list(le.classes_))
# [1 2 0 1] ['Concepcion', 'Santiago', 'Vina']

This fits without error and predicts without error. The problem is what it encodes: alphabetically-assigned integers with no real relationship to each other. A linear model fitted on this column will treat Vina (2) as numerically "twice" Santiago (1), an artifact of encoding order, not a fact about the data. Tree-based models are less affected since they only ever compare against a threshold, but for anything based on distance or a linear coefficient, this silently corrupts the feature, with no error or warning at any point to flag it. LabelEncoder's own documentation states it's intended for encoding the target variable y, not input features.

The Robust Fix: ColumnTransformer + OneHotEncoder

The version that avoids both problems above, fake ordering and predict-time column mismatches, wraps OneHotEncoder in a ColumnTransformer inside a Pipeline. The encoder remembers the exact categories seen at fit time and applies that same column layout to any future input, including handling a category it has never seen before:

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder
from sklearn.linear_model import LinearRegression

preprocessor = ColumnTransformer(
    [('city_ohe', OneHotEncoder(handle_unknown='ignore'), ['city'])],
    remainder='passthrough'
)

pipe = Pipeline([
    ('prep', preprocessor),
    ('model', LinearRegression())
])

pipe.fit(X, y)

# Same single-row prediction that broke get_dummies() above
pipe.predict(pd.DataFrame({'sqft': [1300], 'city': ['Vina']}))
# works, produces the correct column layout automatically

# A city never seen during training
pipe.predict(pd.DataFrame({'sqft': [1300], 'city': ['Valparaiso']}))
# also works, because handle_unknown='ignore' encodes it as all-zeros
# instead of raising

handle_unknown='ignore' is doing the real work here: without it, an unseen category at predict time raises instead of silently zeroing, which is usually the safer failure mode for anything running unattended, since you'd rather see an explicit error the first time it happens than a silent all-zeros row.

Summary: Which Fix Applies

Situation                                          Fix
────────────────────────────────────────────      ─────────────────────────────────────────────
One-off script or notebook, no future predict()     pd.get_dummies(), fine as-is
Model will call predict() on new data later          ColumnTransformer + OneHotEncoder in a Pipeline
Categories have a genuine order (low/medium/high)    OrdinalEncoder with an explicit categories= list
Encoding the target y for classification             LabelEncoder (its actual intended use)
Encoding an unordered input feature column            Never LabelEncoder; see ColumnTransformer fix above