Course Calendar (Subject to Change)

Week Day Date Topic Speaker Required Reading Extended Reading Assigned Due
1 Tue Aug 26 Introduction Zhiyao Duan Lyon: Machine Hearing: An Emerging Field Course Project;
Paper Reviews;
HW0
1 Thu Aug 28 Auditory Scene Analysis Zhiyao Duan Bregman: ASA Book Chapter 1 Wang & Brown: CASA Book Chapter 1 HW1
2 Tue Sep 2 Signal Processing Review Zhiyao Duan Mueller: Fundamentals of Music Processing Book Chapter 2
2 Thu Sep 4 Python Programming for Audio\
Slides		 Xingjian Du librosa;
Audio Input Representations		 MIR with Python;
Python for Scientific Audio
3 Tue Sep 9 Single Pitch Detection Zhiyao Duan Cheveigne: CASA Book Chapter 2.1-2.3 Cheveigne & Kawahara: YIN;
Kim et al: CREPE;
Gfeller et al: SPICE;
Riou et al: PESTO		 HW2 HW1
3 Thu Sep 11 Human Auditory Sensation Zhiyao Duan Yost: Hearing Book Chapter 11 Patterson: Auditory Images;
Lyon et al: Sparse Auditory Representations		 Paper Review 1 (due Sun)
4 Tue Sep 16 Human Auditory Sensation Zhiyao Duan Yost: Hearing Book Chapter 13 Shamma: Encoding Sound Timbre in the Auditory System
Wang & Shamma: Spectral Shape Analysis
4 Thu Sep 18 Rhythm Analysis Zhiyao Duan Mueller: Fundamentals of Music Processing Book Chapter 6
Ellis: Beat Tracking by Dynamic Programming		 Klapuri et al: Meter Analysis
Heydari et al: BeatNet
Zhao et al: Beat Transformer
Foscarin et al: Beat This!
Desblancs et al.: Zero Note Mamba
Chang & Su: Beast		 HW3 HW2
5 Tue Sep 23 Timbre Representation Huiran Herrera-Boyer et al: Signal Processing Methods for Music Transcription Book Chapter 6 Childers et al.: The Cepstrum;
Davis & Mermelstein: MFCC
Huang et al: Music Timbre Style Transfer
5 Thu Sep 25 Timbre Representation Huiran Tzanetakis: Music Data Mining Book Chapter 2 Hermansky: PLP;
Hermansky & Morgan: RASTA
Wu et al: Transplayer
6 Tue Sep 30 NMF Audio Modeling Zhiyao Duan Smaragdis & Brown: NMF Polyphonic Music Transcription Lee & Seung: NMF HW4 HW3
6 Thu Oct 2 More on NMF Zhiyao Duan Smaragdis et al.: PLCA Virtanen: Monaural Sound Source Separation Paper Review 2 (due Fri)
7 Tue Oct 7 HMM Audio Modeling Zhiyao Duan Rabiner: HMM Mysore: PhD Thesis Chapter 2
7 Thu Oct 9 Deep Learning for Audio
CIRC Intro; Bluehive Cheat Sheet		 Zhiyao Duan Goodfellow et al.: Deep Learning Book Chapter 6 Hinton et al.: DNN for Speech Recognition; HW5 HW4
8 Tue Oct 14 NO CLASS: Fall Break How to write a paper?
How to give a talk?
How to make a poster?
8 Thu Oct 16 Deep Learning Implementation
PyTorch 101 		TBD Goodfellow et al.: Deep Learning Book Chapter 9
DNN for Speech Separation;
Huang et al: Singing Voice Separation by RNN		 Project Proposal
9 Tue Oct 21 BP derivation Zhiyao Duan Goodfellow et al.: Deep Learning Book Chapter 14 Schluter & Bock: Onset Detection by CNN;
Hamel & Eck: Music Feature Learning with DBN;
9 Thu Oct 23 Speech Technology Zhiyao Duan Ravanelli et al: SpeechBrain, Park et al: Review of Speaker Diarization
ASVSpoof2019		 Extended Reading
Kassis & Hengartner: Breaking Voice Authentication		 HW6 HW5
10 Tue Oct 28 Voice Conversion Zhiyao Duan Sisman et al.: Overview Qian et al.: AutoVC; Sun et al: PPG; Li et al.: StarGANv2-VC Paper Review 3
10 Thu Oct 30 Multi-pitch Analysis Zhiyao Duan Cheveigne: CASA Book Chapter 2 Klapuri: Harmonicity and Spectral Smoothness
Duan et al: Peak and Non-peak Region
11 Tue Nov 4 Multi-pitch Analysis Zhiyao Duan Duan et al: Multi-pitch Streaming Poliner & Ellis: Discriminative Model;
Sigtia et al.: Neural Network for Piano Transcription		 HW6
11 Thu Nov 6 Self Supervised Learning for Music Understanding Frank Cwitkowitz
12 Tue Nov 11 Score-Informed Source Separation Zhiyao Duan Dannenberg & Raphael: Alignment and Accompaniment;
Ewert et al: SISS Overview		 Ewert & Muller: Score-informed NMF;
Duan et al: Soundprism
12 Thu Nov 13 Audio-Visual Scene Understanding Zhiyao Duan Arandjelovic & Zisserman: Objects that Sound; Owens & Efros: AV Scene Analysis Arandjelovic & Zisserman: Look Listen and Learn; Zhao et al.: Sounds of Pixels
13 Tue Nov 18 Project Status Update in Zhiyao's Office Students Check Google Doc for Schedule Project Status Update
13 Thu Nov 20 Room Acoustics and Spatial Audio Zhiyao Duan Machine Learning in Acoustics
Neural IIR Filter Field 		Novel-View Acoustic Synthesis
HRTF Estimation in the Wild
14 Tue Nov 25 Multi-channel Source Localization and Separation Zhiyao Duan Stern et al: CASA Book Chapter 5;
Yilmaz & Rickard: DUET		 Woodruff & Wang: Binaural Localization Reverberant Noisy
14 Thu Nov 27 NO CLASS: Happy Thanksgiving!
15 Tue Dec 2 Interactive Music Systems Zhiyao Duan Gifford et al.: Computational Systems for Music Improvisation Tatar & Pasquier: Music Agents
15 Thu Dec 4 Music Generation TBD Benetatos et al.: BachDuet; Dhawiwal et al.: Jukebox Hadjeres et al.: DeepBach; Roberts et al.: Hierarchical Latent Vector Model; Jaques et al.: Generating Music with Reinforcement Learning;
16 TBD TBD Project Oral Presentations Students TBD Project Report Final;
Slides Final