Human brain’s ‘temporal scaffolding’ inspires new AI approaches

Applying a recent hypothesis about how the brain operates during sleep could improve the lifelong learning abilities of artificial intelligence.

Scientists believe a recent hypothesis about how the human brain uses sleep and awake periods to learn over time could be the key to overcoming artificial intelligence’s limitations with lifelong learning. Christopher Kanan, an associate professor at University of Rochester’s Department of Computer Science, is part of a transdisciplinary team that received $2 million in funding from the National Science Foundation (NSF) to use the “temporal scaffolding” hypothesis to produce AI that rapidly learns, adapts, and operates in uncertain conditions.

Current forms of AI struggle with learning new tasks continuously throughout life and perform poorly in resource-constrained environments. Kanan says that while AI models have made progress in sequence learning—applying lessons learned in previous encounters to decide what to do next—they struggle to learn as efficiently as humans. He notes that while conventional methods of reinforcement learning led to AI models that can beat professional players at games such as StarCraft 2 and Dota 2, the AI models needed enormous amounts of experience to acquire those skills.

“OpenAI’s Dota 2 AI required the equivalent of 45,000 human years’ worth of gameplay experience to beat the world’s best players, but the best players only had been playing the game a few years at the time,” says Kanan. “This indicates something is fundamentally wrong with how reinforcement learning works. Instead, we are proposing to use an alternative paradigm to hopefully get much more efficient learning.”

The temporal scaffolding hypothesis proposes that the brain reactivates wake experiences during sleep in an accelerated manner, enabling the brain to detect important patterns within those experiences. Mimicking this process, the team will develop deep-learning networks with the ability to swiftly adapt and operate under resource constraints, much like the human brain does. The researchers envision applying their new approach in critical areas such as health care, autonomous systems, and national security.

Kanan will develop deep-learning models and algorithms based on temporal scaffolding and test the models across standardized benchmarking tasks. Principal investigator Dhireesha Kudithipudi of the University of Texas at San Antonio (UTSA) and co-PI Garret Rose of the University of Tennessee, Knoxville, will both try to implement some of the algorithms Kanan’s lab develops in hardware. UTSA neuroscientist Itamar Lerner developed the temporal scaffolding hypothesis that serves as the inspiration for the algorithms, and Northeastern University’s John Basl will serve as the senior personnel responsible for AI ethics.

Delving deeper into deep learning

The project marks Kanan’s latest endeavor in the rapidly changing field of artificial intelligence and deep learning.

Kanan is part of a Department of Energy–funded project to develop artificial intelligence to help scientists take the next step toward creating fusion energy sources. Using the University of Rochester Laboratory for Laser Energetics’ OMEGA experimental database as training data, that team aims to use machine learning to design higher-performing implosions and better understand the complexity of the underlying nonlinear physics of fusion.

In 2021, Kanan earned an NSF Faculty Early Career Development (CAREER) award to create deep neural networks that excel in a broad set of circumstances, are capable of learning from new data over time, and are robust to dataset bias.

He is also helping his colleagues explore and understand how AI is changing higher education. As part of a loose collection of Rochester AI and pedagogy experts, Kanan and others have been providing workshops to help faculty, staff, and students understand the capabilities and limitations of generative AI tools like ChatGPT. He predicts they will be a major disruptor in higher education and many other settings.

“The one thing I’m sure of is ChatGPT and others like it are here to stay,” says Kanan. “And we, as educators, will just have to deal with that.”

Read more