Geoffrey Hinton: the scientist who kept faith in neural networks

Thinking about the brain

Hinton was born in London in 1947. He studied experimental psychology and artificial intelligence in England and Scotland, and the University of Toronto calls Cambridge University his alma mater. He earned his PhD from the University of Edinburgh in 1978.

From the start he was interested in how real brains learn. He has said that because his father was a biologist, he thought in biological terms. In the 1980s the dominant view in AI was that intelligence meant processing symbols, such as words and numbers. Hinton was not convinced, and worked instead on neural networks, programs made of simple connected units whose connection strengths can change, so the network can learn.

“My father was a biologist, so I was thinking in biological terms.”

— Geoffrey Hinton, via MIT Technology Review

Backpropagation and the Boltzmann machine

After his PhD, Hinton became a postdoctoral fellow at the University of California, San Diego, where he worked with David Rumelhart and Ronald Williams. According to the Computer History Museum, they rediscovered the backpropagation algorithm, and in 1986 they published papers showing that it let neural networks learn several layers of features. Backpropagation compares the network's answer with the right answer and adjusts the connections layer by layer, from the output backwards. It still underpins almost all neural networks today.

While at Carnegie Mellon University, Hinton and his colleague Terrence Sejnowski built on John Hopfield's network using ideas from statistical physics. The result, published in 1985, was the Boltzmann machine, a network that can learn the typical features of a type of data and even create new examples. In 1987 Hinton joined the University of Toronto.

Toronto became his base for the rest of his career. One of his postdoctoral students there was Yann LeCun, who showed that backpropagation in convolutional networks was very good at reading handwritten numbers. Hinton was among the first researchers supported by the Canadian Institute for Advanced Research, and in 2006 the university named him a University Professor, its highest academic rank. He later co-founded the Vector Institute in Toronto and became its chief scientific adviser.

The long wait, and AlexNet

During the 1990s many researchers lost interest in neural networks. The Nobel committee notes that Hinton was one of those who kept working on them. In 2006 he and colleagues found a way to pretrain a network layer by layer, which gave it a better starting point for learning to recognise things in pictures.

The breakthrough came in 2012. Hinton's graduate students Alex Krizhevsky and Ilya Sutskever trained a deep network called AlexNet on ImageNet, a huge collection of labelled photos, using graphics cards for the heavy computing, with Hinton as their supervisor. The Computer History Museum says that before AlexNet almost none of the leading computer vision papers used neural networks, and after it almost all of them did. Google later acquired the company the three had formed, and Hinton worked at Google for about a decade.

By the late 2000s his students were already using graphics processors to train networks, and their first successes came in speech recognition before images.

Honours and a new focus

Hinton shared the 2018 Turing Award, often called the highest honour in computing, with Yann LeCun and Yoshua Bengio for their work on deep learning. In 2024 he shared the Nobel Prize in Physics with John Hopfield for discoveries that enable machine learning with artificial neural networks.

In early 2023 he left Google so that he could speak freely about the risks of AI. He has said AI could bring great benefits, for example in health care, and has also urged that the possible dangers be taken seriously and that young researchers work on AI safety.

The skills and habits that made the difference

The first was persistence. Hinton worked on neural networks for around forty years, including long stretches when few others did. The ideas were sound; they were waiting for enough data and computing power.

The second was borrowing from other fields. Psychology, biology and physics all shaped his work. The Boltzmann machine came straight from statistical physics.

The third was working with students. Many of his best-known results, including AlexNet, came from teams of graduate students he supervised and encouraged.