Topic: Efficient Continual Deep Learning

Next in our fall seminar series, we welcome Dr. Christopher Kanan from the University of Rochester.

Abstract:
Continual learning methods enable progressive knowledge accumulation in deep neural networks, and it has the potential to make training much more efficient and has many exciting applications such as on-device learning. Whereas conventional deep learning can suffer from catastrophic forgetting of previously acquired abilities when updating a network, over the past five years, continual learning has largely overcome this limitation. Despite this achievement, I argue the vast majority of continual learning algorithms have little practical value. In this talk, I show that methods are highly inefficient in terms of compute, memory, and storage. Some methods even require more computation than re-training from scratch as a dataset grows! For continual learning to have real-world applicability, the research community cannot ignore the resources used by these algorithms. Subsequently, I demonstrate that three new neuroscience-inspired algorithms created by my lab can vastly improve computational efficiency on large-scale vision and text understanding datasets. Our work has the potential to make continual learning more practical and widely adopted as well as advancing the goals of GreenAI.

You can see the presentation live at UTSA San Pedro I Yotta room 430.

Those who cannot attend in-person are welcome to tune in virtually through Zoom: https://utsa.zoom.us/j/91049981814.