How Convolutional Neural Networks Work

A convolutional neural network, usually shortened to CNN, is the type of model that made modern computer vision practical. Instead of treating an image as one long list of numbers, a CNN slides small learned filters across the picture and looks for local patterns such as edges, corners and textures. Those patterns are then combined into shapes, parts and whole objects.

This post explains the core operations, how they stack, and why this design fits images so well.

What a digital image looks like to a computer

A computer does not see a cat or a street sign. It sees a grid of numbers. A grayscale image is a two-dimensional array where each value is a brightness level, often from 0 (black) to 255 (white). A color image adds a third dimension with three channels, one each for red, green and blue. A 224 by 224 pixel color image therefore holds 224 × 224 × 3 = 150,528 numbers.

Feeding all of those values into a fully connected network would need a huge number of weights, and the network would have no built-in knowledge that nearby pixels are related. A pixel and its neighbor usually belong to the same surface or edge. A pixel on the far side of the image usually does not. CNNs encode this fact directly into their structure.

The convolution operation

A convolution takes a small grid of weights, called a kernel or filter, and slides it across the input. At each position it multiplies every kernel weight by the pixel value underneath, adds the products together, and writes one number into an output grid. That output grid is called a feature map.

A common kernel size is 3 by 3, which means nine weights per input channel. The same nine weights are reused at every position in the image. This reuse is called weight sharing, and it means a pattern detector learned in one corner of the picture works everywhere else too.

Kernels and feature maps

Each kernel learns to respond to a particular pattern. One kernel might light up on vertical edges, another on diagonal lines, another on a specific color contrast. A single convolution layer usually has many kernels, for example 64 or 128, and each one produces its own feature map. The result is a stack of maps, one per filter, that describes the image in a richer way than the raw pixels did.

The network does not start with hand-picked edge detectors. The kernel weights begin as small random values and are adjusted during training until the filters become useful. Researchers have repeatedly found that the first layer of a trained CNN learns edge-like and color-blob filters, which is a good sign that the learned features match what a human would expect.

Stride and padding

Two settings control how the kernel moves and how big the output becomes. Stride is the number of pixels the kernel jumps between positions. A stride of 1 visits every position, while a stride of 2 skips every other one and roughly halves the output size. Padding adds a border of zeros around the input so the kernel can reach the edge pixels.

The output width of a convolution follows a simple formula: output = (input − kernel + 2 × padding) ÷ stride + 1. A 3 by 3 kernel with stride 1 and padding 1 keeps a 224 by 224 image at 224 by 224. The first layer of AlexNet, a famous 2012 network, used an 11 by 11 kernel with stride 4 on a 227 by 227 input and produced a 55 by 55 output, which matches the formula exactly.

Stacking layers into a hierarchy

One convolution layer only sees small local patches. The power of a CNN comes from stacking many layers. Early layers detect edges and color transitions. Middle layers combine those edges into textures, corners and simple shapes such as circles or grids. Deeper layers respond to larger structures like eyes, wheels or leaves. Because each layer's receptive field, the area of the original image that influences one output value, grows with depth, later layers can reason about big regions.

Between convolution layers, networks usually apply a nonlinear activation function, most often ReLU, which outputs zero for negative inputs and the input itself otherwise. Without this nonlinearity, a stack of convolutions would collapse into one big linear operation and could not learn complex patterns.

Pooling and downsampling

Pooling layers shrink feature maps. Max pooling with a 2 by 2 window and stride 2 keeps the largest value in each block of four, which halves both height and width. This reduces computation and makes the network less sensitive to small shifts. If a cat moves a few pixels to the left, the strongest response in a pooled region often stays the same.

Strided convolutions can do the same job. Many modern networks skip explicit pooling and let a stride of 2 handle the reduction, which keeps more information in the learned weights.

A quick comparison of common layer types

Layer type	Main job	Typical effect on shape
Convolution (3 × 3, stride 1)	Detect local patterns	Keeps height and width
Convolution (stride 2)	Detect patterns and downsample	Halves height and width
Max pooling (2 × 2)	Keep strongest response	Halves height and width
ReLU	Add nonlinearity	No shape change
Fully connected	Combine features into classes	Output is one score per class

Why AlexNet changed the field

Before 2012, computer vision systems relied heavily on hand-designed features such as SIFT and HOG. In the ImageNet Large Scale Visual Recognition Challenge of 2012, AlexNet, a CNN trained on two GPUs, reduced the top-5 error rate to about 15.3 percent. The second-best entry, which used traditional methods, scored about 26.2 percent. That gap was large enough to convince most of the field to switch to learned features.

Later networks such as VGG, GoogLeNet and ResNet kept the same basic idea but went deeper and added better training tricks. ResNet, introduced in 2015, used skip connections that let gradients flow through very deep stacks, and it reached a top-5 error near 3.6 percent on the same benchmark.

Training a CNN with backpropagation

A CNN learns through the same process as other neural networks. A batch of labeled images passes forward through the layers and produces predicted class scores. A loss function measures how far those scores are from the correct labels. Backpropagation then computes how each kernel weight contributed to the error, and an optimizer such as stochastic gradient descent nudges every weight a small step in the direction that reduces the loss.

Because kernel weights are shared across the image, each weight receives gradient signal from many positions at once. This is one reason CNNs train efficiently on large image datasets.

Key takeaways

A CNN works because its structure matches the structure of images. Small kernels capture local patterns, weight sharing makes those detectors useful everywhere, and stacked layers build from edges to objects. Stride, padding and pooling control how the spatial size changes, and the output formula lets you predict those sizes before running the model.

The learned filters are not designed by hand. They emerge from training on labeled data, which is why the quality and variety of the training images matter as much as the architecture. Understanding the convolution operation is the foundation for almost every later vision model, including detectors and image generators.