Skip to content

ML & Dev Glossary

Plain-language definitions, each linked to the guides and tools that actually use the term.

Activation Function

A non-linear function (ReLU, sigmoid, tanh) applied after each neural network layer. Without it, stacking layers would collapse into a single linear transformation — it’s what lets networks learn complex patterns.

Backpropagation

The algorithm that computes how much each weight in a neural network contributed to the error, working backward from the output layer, so gradient descent knows which direction to adjust each one.

How many training examples are processed together before the model’s weights are updated once. Larger batches use more VRAM but often train faster and more stably.

Confusion Matrix

A table breaking a classifier’s predictions into true positives, false positives, true negatives, and false negatives — the raw counts that precision, recall, and F1 are all computed from.

The point where training stops meaningfully improving — the loss has flattened out. A model that fails to converge is one of the most common warnings/errors in scikit-learn.

Cosine Similarity

A measure of how similar two vectors are, based on the angle between them rather than their magnitude. The standard way to compare embeddings for search and recommendation.

NVIDIA’s platform for running general-purpose computation on the GPU instead of the CPU — what PyTorch/TensorFlow use under the hood to train and run models fast. Most "out of memory" and device-mismatch errors are CUDA-level.

A fixed-length vector of numbers that represents a piece of data (a word, sentence, image, or document) such that similar items end up close together in vector space. The basis for search, recommendation, and retrieval-augmented generation.

One full pass through the entire training dataset. A model is typically trained for multiple epochs, with loss re-evaluated after each one.

Gradient Descent

The core optimization algorithm behind most model training: repeatedly nudge the model’s parameters in the direction that most reduces error, a small step at a time.

A setting chosen before training (learning rate, batch size, number of layers) as opposed to a parameter the model learns from data. Tuned by experimentation, not gradient descent.

How big a step gradient descent takes on each update. Too high and training diverges or oscillates; too low and training is stable but painfully slow — usually the first hyperparameter worth tuning.

One-Hot Encoding

Turning a categorical value into a vector of 0s with a single 1 marking which category it is — the standard way to feed non-numeric categories into a model that only understands numbers.

When a model learns the training data’s noise and quirks instead of the general pattern — it looks great on training metrics and performs worse on new, unseen data.

Of everything a model predicted positive, the fraction that was actually positive (TP / (TP + FP)). High precision means few false alarms — the model rarely flags something it shouldn’t.

Storing model weights at lower numeric precision (e.g. 8-bit or 4-bit instead of 32-bit floats) to shrink memory use and speed up inference, at some cost to accuracy.

Of everything that was actually positive, the fraction the model correctly caught (TP / (TP + FN)). High recall means few misses — the model rarely lets a real positive slip through.

Any technique that discourages a model from fitting noise in the training data too closely (penalizing large weights, dropping units at random, stopping training early) — the general defense against overfitting.

Splitting text into the smaller units (tokens — often sub-words, not whole words) that a language model actually operates on. What you’re charged for by most LLM APIs, and a common source of padding/length errors.

The dedicated memory on a GPU that holds model weights, activations, and data during training or inference. Running out of it is the single most common reason a model training/inference job crashes.