A machine learning model doesn't get told the rules. It gets shown enough examples that it infers them on its own. That's the entire difference from traditional software, where a programmer writes the logic explicitly. A spam filter isn't running a list of "if email contains X, mark as spam" rules a person wrote, it learned what spam tends to look like from millions of examples of spam and not-spam. Same underlying idea behind a streaming service's recommendations, or a voice assistant parsing what was just said.
Most of what shows up in practice falls into one of three buckets, and they're not interchangeable, each solves a structurally different problem.
Supervised learning is the most common by far: you give the algorithm labeled examples (this email is spam, this one isn't) and it learns to predict the label for new, unseen data. This requires labeled data to exist in the first place, which is often the actual bottleneck in a real project, not the modeling.
Unsupervised learning skips the labels entirely. The algorithm looks for structure in the data on its own, which is how customer segmentation or anomaly detection happens without anyone pre-tagging what "normal" looks like. The trade-off: without labels, there's no single objectively correct answer to check the output against, evaluating whether the clusters or patterns it found are actually useful requires human judgment in a way supervised learning doesn't.
Reinforcement learning is the odd one out: instead of a fixed dataset, an agent takes actions in an environment and learns from reward signals, adjusting its behavior to maximize cumulative reward over time. This is the approach behind most game-playing AI and a lot of robotics, and it's notably more data- and compute-hungry than the other two, since the agent has to actually try things (including bad ones) to learn what works.
Linear regression (y = mx + c, just with more dimensions) is usually the first thing anyone implements, and it's still the right tool whenever the relationship between inputs and output is roughly linear. Resist the urge to reach for a neural network before checking if the boring option already works, a linear model that fits well is also far easier to explain to a stakeholder or auditor than a deep network.
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
model = LinearRegression()
model.fit(X_train, y_train)
print(f"R^2 on test set: {model.score(X_test, y_test):.3f}")
Decision trees split data on a sequence of yes/no questions and are worth using specifically because the logic is actually readable afterward, unlike most deep learning models, which matters a lot when a decision needs to be explained to someone outside the modeling team. Neural networks (layers of connected nodes, loosely inspired by how neurons fire) are what you reach for once the pattern is too complex for the simpler methods to capture, at the cost of needing more data and losing that direct interpretability. Support vector machines find the cleanest boundary between classes and tend to hold up well on smaller, cleaner datasets where deep learning is overkill and the extra data a neural network would need simply isn't available.
| Method | Reach for it when | Main downside |
|---|---|---|
| Linear regression | Relationship looks roughly linear; explainability matters | Underfits genuinely non-linear patterns |
| Decision trees | A readable, auditable decision path is required | Prone to overfitting without pruning or ensembling |
| Neural networks | Pattern is too complex for simpler methods, enough data exists | Data-hungry, hard to interpret, easy to overfit on small datasets |
| Support vector machines | Smaller, cleaner dataset with a clear class boundary | Scales poorly to very large datasets |
The practical risks aren't abstract. A model trained on biased historical data reproduces that bias at scale, faster than any individual human decision-maker could manage. A lending model trained on past approval data can encode decades of discriminatory lending patterns without anyone writing a single explicitly discriminatory rule, because the bias lives in the training data's outcomes, not in any line of code. This is the same failure mode covered in more depth in explainable AI for regulated industries, worth reading if this is a live concern for a specific deployment.
Data privacy is the other recurring problem: training data has to come from somewhere, and that somewhere is often personal information that wasn't collected with model training specifically in mind. Neither of these risks gets solved by better code alone, they require deliberate choices about what data gets used and how model outputs get evaluated across different subgroups, not just in aggregate.
Andrew Ng's Coursera course is still the standard starting point for the theory, and it's held up well despite its age. Once the concepts click, Kaggle is where the theory turns into actual skill, competing on a real dataset teaches more in a weekend than another few chapters of lecture notes. Stack Overflow is where most people end up anyway the first time scikit-learn throws an error they don't recognize.
Disclaimer: this is an educational overview, not a substitute for a full course or textbook treatment of any of the methods above.