Skip to content
ContentLora

    Tip: press / anywhere to search.

    Explainer

    How neural networks work: neurons, layers, weights and backpropagation

    An artificial neural network is a machine learning model loosely inspired by the brain: nodes stand in for neurons, and weighted connections between them can be made stronger or weaker during training.[1] Each neuron adds up its weighted inputs plus a bias and passes the result through a nonlinear activation function, and stacking layers of neurons lets a network model very complicated relationships.[2][3] Networks are trained with backpropagation and gradient descent, and today's deep networks can have more than a trillion parameters.[4][5]

    Editor reviewedUpdated Explainer21 claims · 100% primaryArtificial intelligence, Computing
    Reading level: 1 passage differs; the rest is the same for everyone.

    This page assumes the basic ideas of training and testing covered in how machine learning works.

    A model inspired by the brain

    Artificial neural networks were originally inspired by the structure of the brain.[1] Each neuron is represented by a node with a value, and nodes influence one another through connections, likened to synapses, that can be made stronger or weaker.[1] Training adjusts the strength of those connections.[1]

    In practice, a trained model is a complex collection of numbers learned from data.[6] The next sections describe what it actually computes.

    Neurons, weights and biases

    Picture a row of input numbers, such as a house’s size and age, feeding into a layer of “neurons”. Each neuron multiplies every input by its own weight, adds the results together, adds one more number called a bias, and passes the total on.[2] The layers between the inputs and the final answer are called hidden layers.[2] The weights and biases are the network’s parameters, the numbers it learns during training.[6]

    A neuron in a hidden layer computes the sum of the products of its inputs (the previous layer’s values) and a unique weight per input, plus a bias.[2] Parameters add up quickly: in Google’s worked example, three inputs, one hidden layer of four neurons and a single output already give 21 parameters, 16 for the hidden layer and 5 for the output.[7] Google’s glossary defines a deep model as a network with more than one hidden layer.[8]

    Why nonlinearity matters

    A network made only of weighted sums would be limited, because linear operations performed on linear operations are still linear.[3] Networks therefore pass each neuron’s output through an activation function, a nonlinear transformation, before sending it to the next layer.[3] Stacking these nonlinearities is what lets a network model very complicated relationships between inputs and outputs.[3]

    Three activation functions are common: sigmoid, tanh and the rectified linear unit (ReLU).[9] ReLU outputs zero for negative inputs and passes positive inputs through unchanged, and it often works better than smooth functions because it is less prone to the vanishing gradient problem and easier to compute.[9]

    How a network learns: backpropagation

    Training a neural network follows the same loop as other machine learning: predict, measure the loss, adjust the parameters to reduce it.[10] Backpropagation is the most common algorithm for doing this; it makes gradient descent feasible for networks with many layers, and libraries such as Keras handle it automatically.[4]

    Deep networks bring their own problems. The gradients for layers close to the input can become very small, so those layers train very slowly or not at all, a failure known as vanishing gradients.[11] Networks can also overfit their training data; one common remedy, dropout, randomly switches off some neurons’ outputs during each training step.[12][13]

    Different shapes for different data

    Engineers arrange layers differently depending on the task. Convolutional neural networks contain at least one convolutional layer and have been especially successful at problems such as image recognition.[14] Recurrent neural networks feed hidden-layer values from one run into the next, which makes them useful for evaluating sequences.[14] The Transformer, proposed in 2017 and built on attention rather than recurrence, is now the most widely used architecture for building large language models; see how transformer models work.[15][16]

    A short history

    Researchers began reasoning about the mathematics of networks of neurons and synapses in the 1940s, but by the end of the 1960s discouraging theoretical results led many to suspect such networks would never be useful.[17] Interest revived in the 1980s and faded again in the 1990s,[18] before the machine learning revolution that the Nobel Committee dates to around 2010.[19]

    The 2024 Nobel Prize in Physics went to John Hopfield and Geoffrey Hinton for foundational discoveries and inventions that enable machine learning with artificial neural networks.[20] Hopfield created an associative memory that can store and reconstruct patterns such as images, and Hinton built on it with the Boltzmann machine, which can learn to recognise characteristic elements in data.[21]

    Today’s networks are often enormous and built from many layers, which is why the field is called deep learning, and large language models can contain more than one trillion parameters.[5] For research into what those parameters represent inside a trained network, see how mechanistic interpretability works.

    Developing in AI & Computing

    Questions readers ask

    What is a neuron in an artificial neural network?

    A node in a hidden layer. Its value is the sum of each input multiplied by its own weight, plus a bias, usually passed through an activation function before going to the next layer.[2][3]

    Why do neural networks need activation functions?

    Stacking linear operations only produces another linear operation. A nonlinear activation function, such as sigmoid, tanh or ReLU, lets stacked layers model very complicated relationships between inputs and outputs.[3][9]

    What is backpropagation?

    The most common training algorithm for neural networks. It makes gradient descent feasible for networks with many layers, and machine learning libraries usually handle it automatically.[4]

    What makes a neural network "deep"?

    Google's glossary defines a deep neural network as one with more than one hidden layer. Today's networks are often enormous and built from many layers; training them is called deep learning.[8][5]

    Why did neural network research win a Nobel Prize in Physics?

    The 2024 prize went to John Hopfield and Geoffrey Hinton for foundational discoveries and inventions that enable machine learning with artificial neural networks. Hopfield created an associative memory network that can store and reconstruct patterns, and Hinton built on it with the Boltzmann machine.[20][21]

    Sources

    · 21 claims from 3 publishers

    Each numbered claim is a statement we checked against the sources listed with it. Tap a number in the text to see its sources in place.

    1. [1]

      Artificial neural networks were originally inspired by the brain; neurons are represented by nodes with values, and the nodes influence each other through connections, likened to synapses, that can be made stronger or weaker. confirmedas of 2024-10-08

    2. [2]

      Layers between a network's input layer and output layer are called hidden layers, and their nodes are called neurons; each neuron's value is the sum of each input multiplied by its own weight, plus a bias. confirmedas of 2026-11-07

    3. [3]

      Stacking linear operations only produces another linear operation, so networks apply a nonlinear activation function to each neuron's output; stacking nonlinearities lets a network model very complicated relationships between inputs and outputs. confirmedas of 2026-11-07

    4. [4]

      Backpropagation is the most common training algorithm for neural networks; it makes gradient descent feasible for networks with many layers, and machine learning libraries usually handle it automatically. confirmedas of 2026-11-07

    5. [5]

      Today's artificial neural networks are often enormous and built from many layers; these are called deep neural networks, and training them is called deep learning. Large language models can contain more than one trillion parameters. confirmedas of 2024-10-08

    6. [6]

      In supervised learning, a model is a complex collection of numbers that defines the mathematical relationship between input features and output labels; these learned numbers, the weights and biases, are called parameters. confirmedas of 2026-11-07

    7. [7]

      In Google's worked example, a network with three inputs, one hidden layer of four neurons and one output has 21 parameters, 16 for the hidden layer and 5 for the output. confirmedas of 2026-11-07

    8. [8]

      Google's glossary defines a deep model, or deep neural network, as a neural network with more than one hidden layer. confirmedas of 2026-11-07

    9. [9]

      Sigmoid, tanh and ReLU are three commonly used activation functions; ReLU returns zero for negative inputs and the input itself otherwise, and it often works better because it is less prone to the vanishing gradient problem and easier to compute. confirmedas of 2026-11-07

    10. [10]

      During training, a model compares its prediction with the actual label; the difference is called the loss, and the model gradually updates itself to reduce it. confirmedas of 2026-11-07

    11. [11]

      In deep networks the gradients for layers near the input can become very small, so those layers train very slowly or not at all; this is called the vanishing gradient problem. confirmedas of 2026-11-07

    12. [12]

      Dropout regularization helps prevent overfitting in neural networks by randomly dropping out unit activations for a single gradient step. confirmedas of 2026-11-07

    13. [13]

      Overfitting means building a model that matches, or memorizes, its training data so closely that it fails to make correct predictions on new data; Google calls it a common problem in machine learning. confirmedas of 2026-11-07

    14. [14]

      Convolutional neural networks contain at least one convolutional layer and have been especially successful at problems such as image recognition, while recurrent neural networks feed hidden-layer values from one run into the next, making them useful for sequences. confirmedas of 2026-11-07

      • Machine Learning Glossary · Google for Developers · Primary source · Glossary, convolutional neural network (retrieved 2026-11-07)
      • Machine Learning Glossary · Google for Developers · Primary source · Glossary, recurrent neural network (retrieved 2026-11-07)
    15. [15]

      The 2017 paper "Attention Is All You Need" proposed the Transformer, an architecture based solely on attention mechanisms that dispenses with recurrence and convolutions. confirmedas of 2017-06-12

    16. [16]

      Google's machine learning course calls the Transformer the most successful and widely used architecture for building large language models. confirmedas of 2026-11-07

    17. [17]

      Researchers began reasoning about the mathematics of the brain's network of neurons and synapses in the 1940s; at the end of the 1960s, discouraging theoretical results led many to suspect such networks would never be useful. confirmedas of 2024-10-08

    18. [18]

      Interest in artificial neural networks revived in the 1980s, but many researchers lost interest again during the 1990s. confirmedas of 2024-10-08

    19. [19]

      According to the Nobel Committee, the machine learning revolution that began around 2010 was made possible by access to vast amounts of training data and an enormous increase in computing power. confirmedas of 2024-10-08

    20. [20]

      The 2024 Nobel Prize in Physics was awarded to John Hopfield and Geoffrey Hinton for foundational discoveries and inventions that enable machine learning with artificial neural networks. confirmedas of 2024-10-08

    21. [21]

      John Hopfield created an associative memory network that can store and reconstruct patterns such as images; Geoffrey Hinton built on it with the Boltzmann machine, which can learn to recognise characteristic elements in a type of data. confirmedas of 2024-10-08

    Revision history (1)
    1. Page created.

    Created Nov 7, 2026. Last reviewed by an editor on Nov 8, 2026. Next scheduled review: May 7, 2027.

    Cite this page

    "How neural networks work: neurons, layers, weights and backpropagation." ContentLora, updated Nov 7, 2026. https://contentlora.com/explain/how-neural-networks-work

    Spotted an error? Suggest a correction or emailcorrections@contentlora.com.