The analysis

Hopfield and Hinton: the Nobel Prize that unites physics and artificial intelligence

The Nobel Prize in Physics awarded to Hopfield and Hinton for their fundamental discoveries in machine learning of artificial neural networks and their connection to physics. Underlying this is pioneering work carried out around the middle of the last century which, based on the neurophysiology of the brain, proposed increasingly complex mathematical models

by Luca Mari

8 October 2024

John Hopfield e Geoffrey Hinton, nella foto, sono stati insigniti del Premio Nobel per la Fisica 2024, annunciato nel corso di una conferenza stampa presso l’Accademia Reale Svedese delle Scienze a Stoccolma, Svezia, martedì 8 ottobre 2024. (Christine Olsson/TT News Agency via AP)

3' min read

3' min read

The Nobel Prize in Physics was awarded this year to John Hopfield and Geoffrey Hinton "for foundational discoveries and inventions that enable machine learning in artificial neural networks". This motivation is not so explanatory, and this is not so much because, as has happened on other occasions, in order to understand the meaning of the award winners' contributions one needs specific knowledge in their field of work, but because it is not clear what machine learning - the usual Italian translation of 'machine learning' - has to do with physics. The following explanation is still insufficient: 'Hopfield created a structure capable of storing and reconstructing information. Hinton invented a method that can discover properties in data sets and that has become important for the large neural networks that are widespread today."

At the event where the award was announced, it was also said that "the two award-winners have used concepts from fundamental physics to design artificial neural networks, which have been used for the advancement of research in various fields of physics, as well as in applications of everyday life." Of course, but the connection to physics continues to appear a little tenuous.

Obviously, the available information is not yet sufficient to dispel these doubts, but we can make some assumptions, and with that try to hint at some possible fundamental relationships between the technology we are calling 'artificial intelligence' and physics.

From physics to artificial intelligence

Underlying all of this is pioneering work carried out around the middle of the last century, which, on the basis of the knowledge then available on the neurophysiology of the brain, proposed mathematical models, first of the behaviour of individual neurons and then, progressively, of networks of neurons, increasingly complex and capable of ever more sophisticated behaviour. As electronic technologies improved, these models were soon implemented, initially in the form of electronic circuits and then as software systems, thus simulating the behaviour of biological neural networks. The challenge was, and is, obvious: if each individual neuron is a relatively simple system with non-intelligent behaviour, and yet a network of such elementary processing units can exhibit intelligent behaviour, what are the conditions that make this transformation possible?

The founding work of Hopfield and Hinton

In the foundational work of Hopfield and Hinton, we can identify three basic and complementary elements for an answer.

First. In the 1980s, Hopfield proposed an artificial neural network structure, inspired by a physical particle system, capable of modifying its state in such a way as to construct complete information from partial or noise-affected data. Such a system is therefore able to operate, in principle, by patterns and, for example, correctly recognise an object it has never seen from similar examples it has previously been presented with. It is therefore a system that operates not by memorisation but by learning: in embryo what we now call 'generalisation capacity' and which is in action in every chatbot when we see that it knows how to answer questions that, quite plausibly, no one had ever asked it before. It is a system that operates not by memorisation but by learning.

Second. In Hopfield and Hinton's work, the idea that a complex entity such as a neural network can be interpreted as a system that evolves by trying to minimise a 'cost' function, which could be energy in the case of physical systems and error, i.e. the difference between the response that is believed to be correct and the response given by the system, in the case of artificial neural networks, plays a decisive role. Indeed, the idea of optimising an error function is the basis of the training of today's artificial neural networks, such as the one that enables ChatGPT. The optimisation of a network's connections, which model the synapses between neurons, through the algorithm known as back-propagation is a form of minimisation, and Hinton made a decisive contribution to the formulation of this algorithm with his 1986 paper.

Nobel Fisica 2024 a Hopfield e Hinton per le reti neurali

Finally, the neural networks that Hopfield and Hinton helped design forty years ago are dynamic systems, and therefore have a memory that can be modified not only in their training, but also during their operation. Until a few years ago, various artificial neural network architectures were studied, developed and used, and some of them (such as recurrent neural networks, RNNs) retained the characteristic of being dynamic systems. In a famous 2017 article, Attention is all you need, Transformers were introduced, a new architecture that at least for now has become dominant due to its obvious effectiveness, but which, somewhat surprisingly, contains no dynamic components in operation. Could it be that this idea of Hopfield and Hinton will also be taken up in a 'next generation' of artificial neural networks, perhaps also with the aim of reducing the large amount of energy required in their operation?