MATRIX Spring Seminar Series – Nicholas Soures

TACOS: Task Agnostic Continual Learning in Spiking Neural Networks

Nicholas Soures

Rochester Institute of Technology

 https://utsa.zoom.us/j/92387759081

Friday, March 25, 2022

11 AM – 12 PM CST

Catastrophic interference, the loss of previously learned information when learning new information, remains a major challenge in machine learning. Since living organisms do not seem to suffer from this problem, researchers have taken inspiration from biology to improve memory retention in artificial intelligence systems. However, previous attempts to use bio-inspired mechanisms have typically resulted in systems that rely on task boundary information during training and/or explicit task identification during inference, information that is not available in real-world scenarios. Here, we show that neuro-inspired mechanisms such as synaptic consolidation and metaplasticity can mitigate catastrophic interference in a spiking neural network, using only synapse-local information, with no need for task awareness, and with a fixed memory size that does not need to be increased when training on new tasks. Our model, TACOS, combines neuromodulation with complex synaptic dynamics to enable new learning while protecting previous information. We evaluate TACOS on sequential image recognition tasks and demonstrate its effectiveness in reducing catastrophic interference. Our results show that TACOS outperforms existing regularization techniques in domain-incremental learning scenarios. We also report the results of an ablation study to elucidate the contribution of each neuro-inspired mechanism separately.

MATRIX Spring Seminar Series – Dr. Roland Green

Training of Giant Neural Networks with Weight Streaming

Roland Green

Cerebras Systems

 https://utsa.zoom.us/j/92387759081

Friday, March 4, 2022

11 AM – 12 PM CST

State-of-the-art language models have grown in parameter count by three orders of magnitude over the last two years. This growth has presented challenges for training both in terms of storage and compute requirements. In this talk, we survey existing approaches used to scale training to clusters of compute units and explore the limitations of each in the face of giant models. One thing that all these approaches have in common is the storage of model parameters on the compute units, which we find to be a primary driver of complexity and communication overhead. We present a new paradigm for giant model training, called weight streaming, based on the disaggregation of model storage and compute, and describe an implementation of this paradigm using Cerebras wafer-scale systems. Our weight streaming architecture enables the training of models two orders of magnitude larger than the current state-of-the-art, with a simple scaling model. Combined with built-in support for weight sparsity, our solution can make training giant networks tractable for the first time.

LINK

MATRIX Spring Seminar Series – Dr. Isabel Muzzio

Regaining our bearings: Neural representations and circuits underlying spatial reorientation.

Isabel Muzzio

University of Texas San Antonio

https://utsa.zoom.us/j/92387759081

Friday, February 18, 2022

11 AM – 12 PM CST

Reorientation – regaining one’s bearings after becoming lost – is a fundamental navigation process wherein the internal sense of direction becomes unreliable, forcing disoriented navigators to reorient only using external cues. Across species the geometry of the surrounding space plays a dominant role in reorientation, even when other directionally informative cues, such as landmarks, are available. Consequently, the modular theory has proposed that reorientation is driven by a mechanism that is sensitive to geometry and impermeant to non-geometric landmarks. In support of this theory, we have shown that the hippocampal map aligns to the geometry of a layout during reorientation. However, the modular theory’s exclusive reliance on geometry fails in situations of contextual ambiguity (identity of a context is unclear) as well as following overtraining, after animals learn the directional value of landmarks and use these cue to minimize errors in geometrically equivalent locations. In these situations, disoriented animals must identify the environment in which they are lost (context recognition) and recover facing direction within the context (heading retrieval). Disoriented animals initially use non-geometric landmarks to perform context recognition, but exclusively rely on the shape of the layout to compute heading retrieval; however, over time they incorporate features to recover heading in order to maximize reward. The neural mechanisms associated with these processes exhibit unique patterns of activity and dynamics. Following disorientation, heading retrieval cells rapidly align to environmental geometry, while context recognition cells align to local landmarks over days, as animals learn the value of these cues. We further demonstarte that the retrosplenial cortex (RSC), a region involved in scene perception containing neurons responsive to location, borders, and head-direction, is required for the use of geometry during reorientation. These inputs are modulated by a long-range GABAergic projection from CA1 to RSC, which inhibits the use of geometry, thereby enhancing the use of landmarks, as animals learn their directional value. This research uncovers important aspects of navigation with critical applications to the fields of robotics and health.

LINK

MATRIX Spring Seminar Series – Dr. Ehsan Hoque

Advancing Health Equity and Access Using AI

Ehsan Hoque

University of Rochester

https://utsa.zoom.us/j/92387759081

Friday, February 11, 2022

11 AM – 12 PM CST

Did you know using 43 muscles of our face, we can create 10,000 unique expressions? Imagine combining it with language, prosody, body language, physiology, and varying the conversational context (e.g., face to face, computer-mediated, social network). The complexity of modeling such dynamics is still an unsolved problem in computing, making it a grand challenge.

In this talk, I will demonstrate how we can computationally tackle some aspects of the grand challenge with the explicit goal of improving health equity and access. I will provide examples of how AI’s ability to quantify human behavior enables compelling health applications in movement disorders, PTSD, Autism, aging, and terminal illness. I will describe how we can create high-quality and population-representative input data outside of the lab and produce accurate, generalizable, and unbiased results. I will reflect on our experience of engaging the stakeholders early in the process, paving the way for AI into clinical care needs, resulting in practical solutions that end-users, clinicians, and patients find value in.

LINK

MATRIX Spring Seminar Series – Dr. Ayse Coskun

AI for Efficient and Resilient Computing

Ayse Coskun

Boston University

 https://utsa.zoom.us/j/92387759081

Friday, February 4, 2022

11 AM – 12 PM CST

Today’s computers, ranging from small-scale embedded to large-scale systems, face challenges in delivering predictable performance, while maintaining efficiency, resilience, and security. Much of computer system management has traditionally relied on (manual) expert analysis and policies that rely on heuristics derived based on such analysis. This talk will demonstrate a new approach on designing “automated analytics” methods for computing systems, leading the path towards a longer-term vision where computing systems are able to self-manage and improve. Specifically, the talk will first cover how to systematically diagnose root causes of performance variations (or performance “anomalies”), which cause substantial efficiency losses and higher cost. Second, it will discuss how to identify applications running on computing systems and discuss how such discoveries can help reduce vulnerabilities and avoid unwanted applications. The talk will highlight how to apply machine learning to help understand and improve complex systems and show several pathways and challenges in real-world deployment of AI-based methods.

LINK

MATRIX Spring Seminar Series – Dr. Ethan Ahn

Hardware Implementation of Machine Learning by Emerging Nanodevices

Ethan C. Ahn

Assistant Professor of Electrical & Computer Engineering

The University of Texas at San Antonio

 https://utsa.zoom.us/j/92387759081

Friday, January 28, 2022

11 AM – 12 PM CST

The conventional scaling of today’s silicon-CMOS technology cannot satisfy the performance and energy-efficiency needs of abundant-data applications, as they require real-time analytics on enormous quantities of user data. Therefore, research is essential on novel computing hardware to tackle the fundamental limitation of the conventional von Neumann computing (“memory bottleneck”) and further enable emerging applications in e-commerce transactions, financial or business decisions, information security, defense, and forecasting of retail trends and demands. Neuromorphic (bio- or brain-inspired) computing is an emerging computing paradigm that takes inspiration from the way the brain computes. While computing systems based on the classical von Neumann architecture have one or more central processing units physically separated from memory, biological information processing features a co-localized memory and logic system where memory is distributed with the processing. Because the neural networks are extremely efficient at certain tasks that require massive amounts of data to be processed (e.g., recognition, classification, and prediction), research on neuromorphic computing is strongly driven by the need to invent a new computing topology that can lead to great improvements in computing efficiency in these machine learning tasks. In this talk, cutting-edge research on emerging nanoscale memory devices such as resistive random access memory (RRAM), phase-change memory (PCM), and spin-transfer-torque magnetic random access memory (STT-MRAM) is introduced in the context of hardware implementation of machine learning. Future perspectives on these emerging device technologies will also be discussed.

LINK