Professor Ross Maddox receives grant for collaborative project with ECE & CS professors
Professor Ross Maddox (BME and Neuroscience) and collaborators Zhiyao Duan (ECE), Chenliang Xu (CS) have received a pilot grant from the University of Rochester Arts Sciences and Engineering & the Center for Emerging and Innovative Sciences. Their project is titled, "Real-time synthesis of a virtual talking face from acoustic speech."
Project description:
It's easier to understand someone speaking to you in a noisy room when you can see their face. This is especially true for people who are hard of hearing. The goal of this project is to develop an augmented virtual reality system for generating a realistic speaking face from real-time speech audio. We will accomplish this by pushing the state of the art in deep neural networks for audio and video processing. Our techonology has the potential to help people listen when no face is available--such as when talking to tech support on the phone or listening to a podcast, and may also help in a variety of educational contexts.