1 From physics to AI: Hopfield and Hinton revolutionized artificial neural networks by Yue Wang, Rundong Zhao, Rui-Qin Zhang

With the rapid increase in computing technology and the widespread application of big data, the development of artificial intelligence (AI) is changing the way we live and work at an unprecedented pace. From smart assistants to self-driving cars and from medical diagnosis to personalized recommendations, AI technology is permeating various industries and affecting multiple aspects of our daily lives. This is just the beginning. It is expected that AI will profoundly impact the progress of human society in the future.

The 2024 Nobel Prize in Physics was awarded to two pioneers in this field, John Hopfield and Geoffrey Hinton, in recognition of their creative use of physics tools and methods that contributed to the foundation of neural network technology.

Inspired by the structure and function of human brain neurons, artificial neural networks have emerged as a prominent area of AI research since the 1980s. Researchers have developed various models based on different connection methods to mimic the brain’s information processing capabilities.

Hopfield proposed Hopfield networks that can store patterns and reconstruct them. In his foundational papers published in 1982 and 1984, he introduced a feedback interconnection network and defined an energy function based on neuron states and connection weights [1, 2]. This network is capable of solving problems related to associative memory and optimization. When a trained Hopfield network is given a distorted or incomplete image, it can find the most resembling image pattern stored in its network.

Hopfield was inspired by his previous studies on spin magnetism. He created the Hopfield network consisting of nodes with different values and connections with varying strengths. The network’s energy, analogous to spin systems in physics, is determined by the node values and connection strengths. When an image is stored in the Hopfield network, values are assigned to the nodes, and connection strengths are adjusted to minimize the network’s energy for that image. When a new image is fed into the network, it evaluates each node to see if changing its value reduces the system’s energy. This process repeats until no further energy reduction is possible, at which point the images are stored in the network.

figure a

When a distorted image is input, the network will find and output the most similar stored pattern, following the principle of minimum energy. This functionality allows it to replicate distorted or missing information, useful in signal processing, noise elimination, and data compression. Initially, node values were assigned as 1 or 0, representing black and white. Later, with the introduction of pixels, node values became more diversified, enabling Hopfield networks to store more complex images and information [3,4,5].

In Hopfield’s work, the network stores specific image patterns during training. Building on Hopfield’s foundation, Hinton developed methods that can independently discover data properties. For example, if the network is fed with images of cats, it recognizes and saves their characteristics as patterns. This process is akin to human learning, advancing from simple memory retrieval to a deeper level of pattern recognition, which plays an important role in the development of artificial neural networks.

Hinton employed ideas from statistical physics to develop such networks, focusing on the collective behavior of complex systems. In statistical physics, the state of the whole system is analyzed, and the probability of each state is determined by its energy. In the nineteenth century, Boltzmann first proposed the famous Boltzmann equation when studying the collective motion of gas molecules. Hinton adapted this equation to describe his network, which he termed the Boltzmann machine.

Boltzmann machines consist of visible and hidden nodes. The values stored on these nodes and the strengths of their connections influence the system's energy and probability. When sample images are input through the visible nodes, the network adjusts the node values to reflect the input pattern, increasing the likelihood of that pattern’s occurrence. With increased training, the network effectively learns to store the characteristics of these patterns. When a new image is input, the trained Boltzmann machine identifies the closest stored pattern, enabling it to distinguish the input category [6, 7].

The original Boltzmann machine was inefficient, but through Hinton and others’ efforts, its performance has improved significantly. Today, Boltzmann machines are often integrated into large neural networks, providing identification and recommendations based on public preferences.

In 1986, D. E. Rumelhart and Hinton introduced the famous error back-propagation algorithm (BP algorithm) [8], which remains foundational in self-supervised learning today. This algorithm significantly improved the training process of neural networks, enabling them to learn from data more effectively. In 2012, Hinton, along with his students Alex Krizhevsky and Ilya Sutskever, developed AlexNet, which is widely regarded as a breakthrough in the field of computer vision. This model successfully integrated deep neural networks with large datasets and GPUs, further advancing deep learning and igniting the third wave of AI research. Hinton is often referred to as the “Godfather of AI.”

Hopfield and Hinton’s groundwork paved the way for artificial neural networks. The emulation of human learning forms the basis for many AI and machine learning tools across various fields. In scientific research, AI aids drug discovery by analyzing biological data and predicting molecular behavior. It also contributes to the discovery of novel material states by analyzing phase transitions and complex systems, as well as enhancing climate modeling and prediction through the analysis of vast datasets. In daily life, AI powers large language models, smart home devices, facial recognition systems, financial and health care advice, and so forth. All of these are foreseeable to deeply influence the behavior and progress of human society.

Although there is some debate in the academic community regarding the awarding of the Nobel Prize in Physics to achievements in computer science, this decision highlights the importance of interdisciplinary thinking and underscores that physics serves as a foundational pillar for many other fields. The research backgrounds of the two laureates, both rooted in physics, exemplify how principles from this discipline can drive innovations in artificial intelligence and beyond.

The Nobel Prize awarded to neural networks emphasizes the value of employing physics methods to address interdisciplinary problems and promote scientific, technological, and social advancements. As Michael Faraday once said, no one can define what the good is of a newborn baby. The progress of fundamental physics research holds immense potential, likely leading to unexpected developments in diverse fields. In the future, we may continually witness physics methods influencing non-traditional areas, providing ideas, solutions, and opportunities for interdisciplinary challenges.