"Roads? Where we are going we don't need roads"

"Artificial intelligence has explored a narrow region of the space of possible architectures, one built on dense, feedforward, energy-intensive computation that separates training from deployment and ignores the physical world."

It is an interesting pastime to think about the things that we take for granted today that tomorrow will be obsolete. Things and habits that today are standard practice but, in the future, will disappear because something better substitutes them or they were found to be pernicious, or simply because of another reason that we cannot foresee right now. What habits or foods are we taking now that tomorrow will be known to be prejudicial? Let’s imagine a time traveler coming to our present time from a previous era. Depending on when they come from, they would be surprised with stuff we take for granted; for instance, antibiotics, fridges, the internet, computers, mobile phones, airplanes, cars, heating and cooling systems, vaccines, drinking water and food availability, electricity, satellites, and naturally AI, among many, many others.

Let’s play another game. We are the visitors from the past, and we arrive at a future time. What things would be surprised by? As a neuroscientist, I would like to be surprised by the fact that in the future many neurological diseases have been eradicated, and that people with severe paralyses have a possibility to regain mobility once again, among others. As an AI practitioner and researcher, I would like to know that rather than having achieved superhuman AI, we achieved efficient AI. Efficient in the sense that it does not impose a large toll on the environment: neither from sourcing its hardware’s raw materials nor in the way its data centers must be powered and cooled. Efficient also in the sense of how models are trained and deployed.

I believe that future generations of machine learning practitioners and scientists will look at the past with awe thinking that we -at present time- divided learning and inference into 2 different stages instead of having a system that continuously learns from its environment. Why have we normalized this, and even developed strategies around it instead of building the next paradigm? The problem of today is that we are reaching the limits of our current computing paradigm, to wit, the Von Neumann architecture. The next generation of AI will emerge with advances from different fronts, from which I stress out neuroscience and material science. Let’s not forget that the computer revolution came from the discovery that semiconductors could replace vacuum tubes as the building blocks for computation. Neuroscience will also play an important role in the upcoming AI generation. Let’s remember that the brain -any brain, the insect brain, the fish brain, the octopus brain, the mammalian brain- is the only thing in nature that so far exhibits continual learning, body regulation and control, fault and error tolerance, and in some cases intelligence, reasoning, and creativity. All in a very low energetic budget. Why shouldn’t we then study its computing principles and replicate them?

For efficient AI to be a thing in the near-future, we need to go past current paradigms. Both conceptually and in the implementation. How can we expect to solve intelligence, i.e. to replicate it, by using only the 3 main paradigms of machine learning, namely, supervised learning, unsupervised learning, and reinforcement learning? We must be humble with Nature! There are neural and cognitive phenomena that cannot be explained by those paradigms. So, no: reward is not enough[2] and attention is not all you need[3].

As I said before, and keep repeating ad nauseam, the Von Neumann paradigm for computation is not the right one for true AI¹. It is great for spread sheets, word processors, the internet, and hey! Up to some extent, it is good for generative AI². However, it is not the right medium to achieve a system that is able to learn new behaviors and concepts from few examples, learns from imitation and mistakes, is able to control its body, and is able to plan and execute decisions that align with our values. Such a system might not be created with the materials and methods from the silica age. So, I stress again the importance of material science in the future of AI with the return of analog computing and the development of the memristor as promising directions of research.

Finally, the title of this post comes from Back to the Future’s last scene³, where Doc Brown surprises Marty with news from the future, where traffic roads are a thing of the past. In a similar way, we could imagine a scene in which we are interacting with an AI practitioner from the future, and after training a machine learning model in a GPU cluster, we are concerned about what strategy should we follow to quantize the model such as it can be deployed on the edge. Our friend from the future might ask us with awe: “Training and inference in different platforms and stages? Quantization? In the future, we don’t need to do that!”.