From Natural Intelligence to Physical AI

The tl;dr

• Physical AI is the next big wave of research and development in the field of artificial intelligence. Its proponents claim that Physical AI holds the promise of revolutionizing industry.

  
  

• But state of the art AI will not deliver on these promises, because it is not capable of understanding the structure, variability, and complexity of the physical world that we inhabit. 

• Noumenal Labs is a newly formed deep tech company that is laser focused on building digital brains for Physical AI — so it can be deployed profitably, efficiently, safely, and at scale.

• We are using our unique, proprietary macroscopic physics discovery technology to build object centered world models that will power the brains of autonomous systems — unlocking machines that can act in intelligent and situationally appropriate ways in the real world, and that can adapt to a changing world in real time. 

• Driven by key insights from statistical physics and cognitive science, in particular by Karl Friston’s active inference framework, the approach pioneered at Noumenal Labs unlocks the capability of machines to represent the physical world in the same way we do, enabling them to act safely and in alignment with human values — and thereby, to deliver on the promise of Physical AI.

Physical AI: What it is — and why it promises to change everything

There’s been a lot of buzz about Physical AI in the last year — especially after NVIDIA’s GTC conference in March. One of the more impactful events in the AI space, GTC is always an eventful conference. But what really wowed audiences this year was the level of attention and interest paid to what is now being called “Physical AI.” Attendees were struck by the attention-grabbing embodied intelligence displayed by robots and drones being controlled by state of the art AI systems. 

But what is Physical AI, exactly? And why are so many key players in the field convinced that it will revolutionize industry and human existence more generally?  

Physical AI refers to the application of artificial intelligence to control autonomous systems in real time, out in the real world. In short, the term denotes efforts in industry to automate autonomy. The kinds of autonomous systems we’re talking about are machines that interact with complex changing environments, often with direct impacts on human beings, like robots, drones, and unmanned vehicles. What these use cases have in common is the need for a system to be able to select between different courses of actions, in order to act safely and in situationally appropriate ways. They also have in common the need for AI systems to adapt to new information and a changing environment in real time. 

Physical AI is a new category of AI that extends far beyond the narrow use cases that we’re used to seeing today. Physical AI represents a new generation of intelligent machines that can radically improve the quality of life of humans. Think robots that can do chores around the house, care for the elderly and sick, or accomplish complex assembly tasks in a factory, or novel tasks in an unknown environment — with little or no human supervision. 

Physical AI thus holds the promise of a huge leap forward toward the science fiction future we’ve been promised for decades. While chatbots and automated data analysts are an impressive outcome of the current phase of AI research, and have already been world-changing, the true promise and potential of AI lies in getting embodied AI solutions out into the real physical world. 

What do we need for genuine Physical AI? 

This all sounds amazing. But in our view, state of the art AI approaches will not deliver on the promise of Physical AI — at least not on its current trajectory. This is because research and development in AI has focused on rote prediction (next word or next image prediction), rather than on the generation of descriptions of the physical world that can generate actionable insights. As such, this approach will not yield machines that truly understand the dynamics of the real world and are, therefore, not able to act appropriately and safely within it.

Leading companies and research labs working on Physical AI, such as Figure, Tesla, and DeepMind are repurposing the same set of technologies that power current generative applications of AI in text, image, and video generation: diffusion models or feedforward transformer architectures, which are trained on expert trajectories. These techniques apply standard machine learning to massive, expensive datasets. While learning from expert trajectories works in narrow task settings, it does not generalize to cases outside the training data and struggles to handle edge cases or novel situations. Massive datasets coupled with deep learning will never overcome the challenges posed by a dynamic ever changing world. Many are talking about the “data problem” in Physical AI, which follows the line of reasoning that there simply is not enough data to train systems to behave appropriately using contemporary techniques. Embodiment is treated as a big data problem, but it is not. The real problem is the lack of models that are able to encode the structure of the real world as it actually is. In other words, they are not embodied like we are, because they lack a grounded world model that represents their ecological niche — an understanding of reality that is grounded in the world that we inhabit.  

In addition, mainstream architectures trained using deep learning are not capable of continuous learning. Once a model is learned from an enormous dataset, that’s it. If new capabilities are required, the user needs to add new, carefully curated and annotated data into a training set — and must train all over again. This is time consuming and is easily the most expensive and labor intensive step in model training. This inability to learn in real time leads to the lack of adaptability — which is problematic, since real-time adaptation must be a critical feature of Physical AI if there is any chance of success. 

From the natural intelligence to Physical AI

To deliver on the promise of Physical AI at scale, we need a new AI architecture — one that requires less resources, not more, and one that allows for the kind of real-time adaptation and learning that is necessary for skillful embodied action. At Noumenal Labs, we are drawing on nature and its study to design machine intelligence for Physical AI that understands the world around us the same way that we do. To do so, we are equipping machine intelligence with models that are grounded in the physical world and are structured according to our intuitive understanding of physical laws, which was refined by scientific experimentation.

Our argument is premised on the embodied intelligence thesis. We have learned from decades of methodical study of natural intelligence that all intelligence implies a model of the world — an explicit representation of the objects and relationships that matter to us. In other words, creatures evolved intelligence to solve the specific problems that matter to them in their ecological niche, and learned to combine the solutions that it acquired through evolution and experience to solve more complex problems. And thus, living creatures, even very simple ones, effectively model the physical laws of their environment. This was the central insight of the active inference approach developed originally by one of Noumenal’s Scientific Advisors, Karl Friston. Living creatures discover the macroscopic rules that govern the interactions between objects, which we can understand as a generalization of physical laws to mesoscale objects. 

The hallmark of higher order cognitive processes is the ability to compose these models on the fly. As life evolved from single to multicellular organisms, it transitioned from modeling the chemistry of their environment to modeling the object centered macroscopic physics of their environment. In its evolutionary history, when new situations arose, the brain repurposed these simple models, by combining them in situationally specific ways, to describe these new more complex phenomena. This ability to redeploy models in new situations and assemble them into hybrid models powers emergent intelligence and analogical reasoning in humans. Indeed, our mental models are often and intuitively redeployed in new situations and new configurations. This is why, when we offer up explanations for things, we often do so by offering up analogies. For instance, we say that spacetime curvature is like a heavy ball on a rubber mat. That is the critical design feature of the brain as an organ of thought, and the critical insight that will unlock super-intelligence.  

But where would these models come from? This is where Noumenal’s unique macroscopic physics discovery technology really shines. Noumenal Labs is developing unique macroscopic physics discovery technology. This technology allows us to build compute efficient world models for autonomous systems that learn the interactions between objects in an environment directly from data in an unsupervised way, just like the brain does. The ensuing approach yields models and network architectures — and models of model generation — that can be used in advanced data processing to make sense of the kind of rich, multi-dimensional, and multi-modal time series data that matter to humans in the real world. It can also be used as the basis for the digital brains of artificial agents that vastly enhance and empower our decision making, and even act in the world on our behalf.