Speech Inversion based on Articulatory Phonology

Carol Espy-Wilson, Ph.D.
Professor
University of Maryland
College Park, MD

Date:
03/01/2024

Time:
11:00 am – 12:00 pm CST

Zoom: https://utsa.zoom.us/j/94807623288

Abstract:
Speech articulation is a complex activity requiring finely timed coordination across the articulators: lips, jaw, tongue, glottis and soft palate. This systematic coordination of the articulators differs between individuals, and across languages and dialects of the same language, accounting for many aspects of foreign accent, speech disorders and speaking style. Visualization of articulatory movements has required expensive and specialized equipment. However, we have developed a Speech Inversion (SI) system based on Articulatory Phonology that can accurately recover articulatory movements directly from the acoustic speech signal using machine learning techniques. In this talk, we will discuss the development of the SI system and how we are using the generated articulatory information for mental health assessment. I will also discuss an autoencoder architecture (a Mirror network) that we developed to explore sensorimotor learning. We show that the Mirror network can be used to control an articulatory synthesizer that takes as input articulatory parameters to generate continuous, natural sounding speech for unseen speakers. Further, we show that the Mirror network can learn in an unsupervised way meaningful articulatory representations with comparable accuracy to the SI system that is trained in a completely supervised fashion.