Topic: Speech Inversion based on Articulatory Phonology

Next in our spring seminar series, we welcome Dr. Carol Espy-Wilson, a professor from the University of Maryland.

Abstract:
Speech articulation is a complex activity requiring finely timed coordination across the articulators: lips, jaw, tongue, glottis and soft palate. This systematic coordination of the articulators differs between individuals, and across languages and dialects of the same language, accounting for many aspects of foreign accent, speech disorders and speaking style. Visualization of articulatory movements has required expensive and specialized equipment. However, we have developed a Speech Inversion (SI) system based on Articulatory Phonology that can accurately recover articulatory movements directly from the acoustic speech signal using machine learning techniques. In this talk, we will discuss the development of the SI system and how we are using the generated articulatory information for mental health assessment. I will also discuss an autoencoder architecture (a Mirror network) that we developed to explore sensorimotor learning. We show that the Mirror network can be used to control an articulatory synthesizer that takes as input articulatory parameters to generate continuous, natural sounding speech for unseen speakers. Further, we show that the Mirror network can learn in an unsupervised way meaningful articulatory representations with comparable accuracy to the SI system that is trained in a completely supervised fashion.

You can see the presentation live at the UTSA Main Campus in the Student Union Mesquite room (2.01.24).

Those who cannot attend in-person are welcome to tune in virtually through Zoom: https://utsa.zoom.us/j/94807623288.