Creating new Opportunities for AI with Ultra Low-bit Precision Neural Networks

Davis Sawyer

Co-Founder of Deeplite Inc.

https://utsa.zoom.us/j/92387759081

Friday, April 8, 2022

11 AM – 12 PM CST

Abstract: Edge technologies have emerged as a major computing paradigm in recent years. From automated optical inspection to autonomous driving, the possibilities for edge AI to benefit daily life are growing faster than ever. Namely, the exponential growth of Deep Neural Networks (DNNs), maturing IoT deployment architectures, as well as economic and environmental benefits of AI inference at the point of data capture have driven tremendous academic and commercial interest in the field. However, the majority of high-value tasks like Natural Language Processing (NLP) and Computer Vision (CV) still require power-hungry or purpose-built hardware to achieve acceptable inference performance. This significantly limits the potential to bring AI to the billions of connected devices around us. To make edge AI more accessible and affordable, we introduce Deeplite Runtime (DeepliteRT), a novel software approach to quantize and run ultra low-bit DNNs on low-power, low-cost processors. In this talk, we will discuss the latest in training-aware quantization and share experimental results for DeepliteRT, where our ultra-low bit models achieved near-GPU level latency using a simple ARM Cortex-A CPU. We will conclude with a discussion on remaining challenges and future research directions.