An Intelligent Music Stand
Advisor: David Liben-Nowell
Final Results
I. Background
There is, in Liben family lore, an oft-repeated story about an unintentionally hilarious classical music concert. A pianist, evening gowns, tuxedos. A darkened hall. Anticipation. Applause. The beautiful strains of Mozart fill the hall. The page turner flips the page. The pianist flips the page back. A few moments pass. The pianist nods. The pianist nods again. The pianist nods again, annoyed. Hastily, the pianist reaches up and turns the page. A few moments pass. The page turner flips the page; the pianist, scowling now, flips it back. And so forth. This occurs frequently throughout the first half of the performance. The page turner does not return to the stage after the intermission. The pianist turns her own pages.
In this project, you will develop an algorithmic alternative to this the hapless page-turner.
II. The Project
In this project, you will build an intelligent music stand. Such an intelligent music stand will have the capacity to display a stored piece of music, listen (via microphone) to music being played and identify what notes are being played when, match that played music to a stored score, and in some way usefully display a representation of the point in the score that is currently being played.
You will design and implement:
- a mechanism for reading written music in some form.
- a mechanism for listening to and processing played music and identifying the notes being played.
- a mechanism for matching two pieces of music (the one currently being played and the one stored in advance). You will have to handle wrong notes, variations in tempo, extraeneous noise from the microphone, etc.
- a mechanism for displaying the next bars of the stored music in an manner helpful to the musician.
- a mechanism for testing all of the above (both for music-matching issues and for user-interface issues regarding task 4).
To give you a sense of the possibilities and challenges, here is a simple example: using the Python audiolab package and some simple signal processing to convert/analyze this recording of a scale (played twice) (recorded as a wav file by Philippe Guillaume) into a spectrogram and then tracing the frequency of the highest intensity at each point in time.
III. References
Steven W. Smith, The Scientist and Engineer's Guide to Digital Signal Processing.
Wikipedia entry on the Fast Fourier Transform
Open Courseware notes
from MIT's 6.011 Introduction to Communication, Control, and Signal Processing
HTK (HMM Tool Kit, usually used for speech processing) http://htk.eng.cam.ac.uk
MusicXML (http://www.musicxml.org)
LilyPond (http://lilypond.org)
PortAudio (http://portaudio.com)
PyAudio (http://people.csail.mit.edu/hubert/pyaudio)
Python audiolab package