How Machine Learning Works: Data, Models and Learning from Examples

Machine learning builds programs whose behavior comes from data instead of hand-written rules. Rather than a programmer writing every decision, a learning algorithm adjusts a set of numbers called parameters until outputs match example answers well enough. This article explains the three parts behind almost every machine learning system: data, a model, and a training procedure that reduces error.

What a model actually is

A model is a function with adjustable parameters. Given an input x, it produces a prediction y-hat. The simplest useful example is a straight line, y-hat = w * x + b, where w (the weight) and b (the bias) are the parameters. Before training, w and b hold arbitrary starting values, so predictions are poor. Training finds values that make predictions good.

Modern models follow the same idea at a much larger scale. A large language model may have billions of parameters, and an image classifier may have tens of millions. The arithmetic is the same: inputs flow through functions that multiply by weights and apply simple nonlinear operations.

Parameters versus hyperparameters

Parameters are learned from data during training. Hyperparameters are choices made before training starts, such as the learning rate, the number of layers, or the batch size. Engineers usually tune hyperparameters by trying several settings and comparing results on data held back from training, which the overfitting article in this section covers in detail.

Data is the teacher

Supervised learning uses pairs of inputs and correct outputs, called labels. A spam filter might see thousands of emails, each marked spam or not spam. Unsupervised learning finds structure in inputs without labels, for example by grouping customers with similar purchase patterns. Reinforcement learning learns from reward signals gathered by acting in an environment, as in game-playing systems.

Data quality usually matters more than cleverness in the algorithm. Wrong labels, unrepresentative samples, and duplicated rows all push the learned model in the wrong direction. A model can only learn patterns that exist in its training data, and it learns the flaws in that data just as faithfully as the signal.

The loss function measures error

To improve a model, we need a number that says how wrong it is. This number is the loss function. For regression, where the output is a real number, a common choice is mean squared error: the average of the squared differences between predictions and targets. For classification, cross-entropy loss is standard because it rewards confident correct probabilities and punishes confident wrong ones heavily.

The loss turns a vague goal, "make good predictions," into a precise problem, "minimize this function." Choosing the loss is a modeling decision because it defines what counts as a good answer. A loss that ignores rare but important cases produces a model that ignores them too.

Task	Typical output	Common loss	Example
Regression	Real number	Mean squared error	Predicting house prices
Binary classification	Probability of one class	Binary cross-entropy	Spam or not spam
Multi-class classification	Probability per class	Cross-entropy	Recognizing digits 0 to 9
Ranking	Ordered scores	Pairwise ranking loss	Ordering search results

Gradient descent takes the downhill step

Once we have a loss, we need a way to reduce it. Gradient descent computes the gradient, which is the slope of the loss with respect to each parameter, and then moves every parameter a small step in the direction that lowers the loss. The update rule is simple: new parameter equals old parameter minus learning rate times gradient.

The learning rate sets the step size. A rate that is too large makes the loss bounce around or diverge. A rate that is too small makes training slow and sometimes stuck. Most modern training uses variants such as stochastic gradient descent, which estimates the gradient from a small random batch of examples, and adaptive methods such as Adam, which scale each parameter's step using running estimates of its gradient size.

Why small batches help

Computing the gradient over the full dataset is expensive. A mini-batch of, for example, 32 or 256 examples gives a noisy but fast estimate. The noise is not only a cost. It can help the optimizer move past flat or sharp regions of the loss surface, which is one reason nearly all deep learning uses mini-batches.

Training is a loop

Putting the pieces together gives a loop that repeats until results stop improving. The model predicts for a batch, the loss measures the error, the gradient shows how each parameter contributed, and the optimizer updates the parameters. One full pass over the training data is called an epoch. Training may run for a few epochs or for millions of steps, depending on the task and the size of the model.

Simple linear regression is a special case where the best parameters can be computed directly with linear algebra. Most useful models have no such shortcut, so iterative optimization is the general tool.

Where learning can go wrong

A low training loss does not guarantee a useful model. The model may have memorized the examples rather than the pattern, which is overfitting. It may also be underfitting, meaning it is too simple to capture the pattern at all. Both failures show up only when performance is measured on data the model did not train on.

Bias in the data is another failure. If a hiring dataset reflects past discrimination, a model trained to imitate it will reproduce that pattern with high accuracy. Optimization cannot tell that the target itself is unfair. Humans must check both the data and the goal.

Features are the inputs the model sees

Models do not see the world directly. They see features, which are measurable properties of each example such as pixel intensities, word counts, or sensor readings. Choosing features is a form of modeling. A house price model given square footage and neighborhood learns different relationships than one given only the number of bedrooms. Deep learning reduced the need for hand-crafted features on images and text, because networks learn useful features from raw inputs. Even so, the representation still determines what is easy to learn.

Normalization is the most common preparation step. If one feature ranges from 0 to 1 and another from 0 to one million, the gradient for the large-scale weight dominates, and a learning rate that suits one feature is wrong for the other. Subtracting the training mean and dividing by the training standard deviation puts features on comparable scales. Those statistics must come from the training set alone, which connects this step to the leakage problem discussed in the evaluation article.

Key takeaways

A machine learning model is a function with parameters. Training adjusts those parameters so that a loss function, which measures error on examples, becomes small.

Gradient descent is the engine that performs this adjustment. It moves each parameter a small step opposite to its gradient, repeated over many batches of data.

Data and the choice of loss define what the model learns. Errors in either will be learned faithfully, so checking data quality matters as much as checking the algorithm.

Honest results must be measured on data the model has not seen. The next article in this section explains how to make that measurement reliable and why it matters so much.