Team
- McCormack Chew
- Alex Kim
- Jessica Luo
- Jameson (Max) Morris
Mentor
Michael Heilemann, Dan Phinney, Sarah Smith, Tre DiPassio
Abstract
An Electronic Wind Instrument, or EWI, is a type of wind controller that takes in breath input and outputs audio via both Musical Instrument Digital Interface (MIDI) and synthesized audio presets. Some commercial products on the market have designs that can have uncomfortable key layouts or are non-intuitive for musicians when transferring from their original instruments. Also, commercial models that are easily transferable for musicians have higher costs. Both of these can be barriers to entry. With this in mind, our EWI is designed and constructed with priorities in ergonomics and low costs. This EWI is 3D printed with plastic material and programmed with the Teensy 4.1 with the following features: MIDI out and audio out, customizable breath sensitivity and vibrato, and various fingering systems including flute, saxophone, recorder, and a basic beginner finger system.
Note Determination
The instrument features four different fingering configurations, allowing it to be used by multiple kinds of instrumentalists. Users may select saxophone, flute, or recorder fingerings, as well as a simplified set of fingerings which aims to allow new musicians and children to play music without prior knowledge. While no note on the beginner system requires more than two buttons to be pressed down at once, the other instruments involve more complicated button combinations. We utilize a logic tree to analyze the buttons pressed and determine what note the user intends on playing in a way that is computationally efficient.
The logic tree is accessed any time there is a change in the keys being pressed. At each stage of the tree, we check for a note or a combination of notes that when pressed, typically cuts the list of possible intended notes in half. This strategy minimizes the calculations required to determine which note to output, which lengthens battery life and reduces latency. Notably, users may press keys that are not necessarily part of a certain fingering without changing the output, and in these cases, our instrument behaves in the same way that the acoustic instruments would if the same extraneous keys were pressed.
Partial Logic Tree Used to Determine Output Note (Saxophone Configuration).
Synthesis
Additive Synthesis:
Our instrument uses additive synthesis to generate sound which can be outputted through a ¼ inch jack. This means that a series of sine waves are added together to produce a more complex sound such as that of a flute or saxophone. To determine the amplitudes of each sine wave, audio samples of individual notes without embellishments were analyzed using MATLAB’s Fourier functionality. As demonstrated in the figures below, the relative amplitude of the sine waves is not constant throughout the full register of each instrument. This means the timbre of the instrument changes as notes get higher or lower. To compensate for this, we measured the harmonic amplitudes at three different registers and interpolated the values between these registers.
Vibrato:
The EWI further seeks to replicate the sounds of a flute and saxophone by adding vibrato to the synthesized sound. While vibrato on the acoustic versions of these instruments is mostly controlled by embouchure, users of our EWI can control the prominence of the vibrato by using their lips to apply pressure to a sensor located on the mouthpiece. When outputting audio synthesized by the Teensy, increasing the pressure on the mouthpiece will increase the amplitude of a sine wave which is constantly modulating the output frequency of the instrument. When outputting MIDI, the pressure is correlated with the rate at which the modulation wheel oscillates.
ADSR:
An ADSR (Attack, Decay, Sustain and Release) envelope with values based on those of acoustic saxophones and flutes is applied to the synthesized signal to add to the realism of the sound and avoid any clipping of the output.
Frequency Spectrum of Saxophone Samples Obtained Using MATLAB.
Recorded Saxophone Note Sample.
Synthesized Recreation.
Teensy Integration
Button Layout and Connections:
Along with the Teensy, we are using the compatible Audio Shield for audio output. Due to the requirement of 18 buttons plus multiple analog inputs and outputs, we needed to make sure all Teensy pins are available, even the ones that are connected to the Audio Shield. Thus the Main Board PCB on the left was built to have pins for all I/O not used by the Audio Shield and to physically separate the Teensy and AudioShield. This board connects to the different buttons, as well as power, audio/MIDI jacks, a rotary encoder, and breath sensor.
Breath and Lip Sensor:
We specifically chose our breath sensor due to its ability to handle a similar pressure range to reed/wind instrument (1-10kPa1). The Teensy is able to read the voltage out of the breath sensor into bits4, which are then put into a logarithmic scaling function to create a sensitivity curve (which can be seen in the bottom picture). This scaling function is controlled by the user, allowing them to change how sensitive (variable b) the breath sensor is, and the required threshold to trigger a sound. Similarly, we used a lip pressure sensor that was used for testing wind players lip pressure for vibrato3.
Latency:
Since an EWI is used in live music performance, there cannot be a significant delay between when a button is pressed and a sound is produced. We found that a delay less than 15 ms is not noticeable by humans, however a latency of 7ms starts to mess with performer’s ability to play music2. To test the latency of our code we used Teensy’s built in timing functions to find how long the code takes to run. We found that most loops take about 2 ms, and the slowest loop was 3 ms, meaning the latency won’t be noticeable.
Teensy Pinout Map.
Main Board with Button Set.
Graph of Breath Sensitivity Curve with Threshold.
3D Design and Modeling
The EWI is 3D-printed with plastic material with a focus on an ergonomic key layout and comfortable holding positions. Button caps were individually designed to mimic flute and saxophone key physicalities and were placed in positions based on flute and saxophone key placements. Slight adjustments were made for comfortability based on test prints, going through multiple iterations.
The instrument is made up of 26 pieces: 7 making up the general body of the instrument including the mouthpiece, 18 corresponding to the button inputs, and 1 outer hook for thumb placement for holding up the instrument.
Considerations when designing the pieces include: button sizing to fit snugly over the switches, PCB mounts for 7 different-sized PCBs, 3D printer bed size, and durability.
Isometric Views of the EWI.
General Dimensions of the Instrument.
Demos
Evaluation and Discussion
Our EWI has undergone multiple design iterations to ensure an ergonomic feel, considering everything from the button sizing and placement to the sensitivity of the breath sensor. On top of this, many steps were taken to make this instrument accessible for all users of various levels of experience with the implementation of four fingering systems ranging from a basic chart to standard wind instrument fingerings as well as having the production costs under $500. Future improvements could be done via user testing of both button placement and sensitivity. This was done during the design process with a small sample size, so increasing the amount of user testing would help it be more user-friendly. The focus of these tests would be on the synthesis sounds, button feel, latency, and sensitivity for vibrato and breath pressure.
References
1“Air speed and blowing pressure in woodwind and brass instruments: how important are they?” The University New South Wales: Music Acoustics. https://www.phys.unsw.edu.au/jw/air-speed.html#:~:text=Measured%20blowing%20pressures%20usually%20range,higher%20pressures%20are%20sometimes%20recorded
2A. Swanson. “Latency and Its Effect on Performers.” Church Production. https://www.churchproduction.com/education/latency-and-its-affect-on-performers/
3M. Pais Clemente, J. Mendes, J. Cerqueira, A. Moreira, M. Vasconcelos, A. Pinhão Ferreira, JM Amarante, “Integrating piezoresitive sensors on the embouchure analysis of the lower lip in single reed instrumentalists: implementation of the lip pressure appliance (LPA),” Clin Exp Dent Res. Sep, 2019. doi:10.1002/cre2.214. [Online]. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6820570/
4“Teensy 4.1 Development Board.” PJRC Electronic Projects. https://www.pjrc.com/store/teensy41.html