FFT Analysis: Harmonic Content Of Musical Instruments
at the
Detroit Science & Engineering Fair
International Science & Engineering Fair
Over the years in science and research I've recieved many awards and honors. This page highlight one of my first, an experience I had in high school -- the Science Fair. The text is from the backboard back in then.
|
|
HARMONIC WAVE ANALYSIS FFT Analysis: Harmonic Content Of Musical Instruments at the Detroit Science & Engineering Fair International Science & Engineering Fair |
This project involved writing computer programs that allowed for the analysis of the harmonic content of sound. In order to do this, research was done which involved learning the physics of sound and the mathematics of sound wave analysis. After combining these areas of knowledge with that of computer programming in "C/C++" a computer oscilloscope was written. Blending this oscilloscope technology with Fast Fourier Transform (FFT) routines allowed for the creation of a real-time 128 Band FFT Program. This allowed one to view the sound spectrum for short periods of time as one spoke into a microphone or played a CD.
This program however was inadequate. It did not allow for accurate determination of the sound frequency because in order to get real-time response, the number of band points was limited to 128. Removing the real-time constraint allowed for a much more accurate program to be created.
This modified version of the program allowed for a 2048 band FFT to be performed. However, since it was not real-time, a method of saving the sound to be analyzed was needed. Having researched several different methods, the WAVE file format (the same format Windows uses) was chosen because of the number of existing WAVE files and the ease of recording a WAVE file.
Once the WAVE file was being read properly and the FFT routines from the original program were working on the WAVE file, the Harmonic Analysis routines were developed. The most intense frequency is found through the use of the FFT process. The approximate frequency of the fundamental is calculated to be accurate within 5 Hz. This frequency value is then used to calculate the position of the harmonics, which are simple multiples of the fundamental.
The FFT data is then displayed on the screen in bar graph form with the harmonic data displayed over it. After the FFT was performed on the entire sound sample, the running average was displayed, and its harmonics were found and displayed just like the normal data. This running average is the Harmonic Wave Analysis for that sample. It shows the fundamental frequency of the sound produced and the relative quantity of its harmonic overtones.
To design and write computer programs in "C/C++" that would allow for easy, fast, and accurate analysis of the harmonic wave content of sound, in particular that of different musical instruments, both prerecorded and in real-time, without specialized hardware (other than what is standard on today's multimedia home computers). Following the program's completion, analysis will be done to compare the harmonics of synthesizer instrument samples to those of the corresponding real instruments.
This project involves the use of two high level mathematical methods, the Fourier Transform and one averaging method I developed.
The first mathematical method, the Fast Fourier Transform (FFT), is one of the keys to this project. It is a variation on the Discrete Fourier Transform (DFT) which was developed by Jean Baptiste Joseph Fourier (1768-1830). Jean Fourier was a French mathematician who studied periodic functions and developed this method to assist him in his study. The FFT makes use of the reoccurring multiplications and additions found in the DFT. However, the FFT method adds a constraint depending on the variety used. The most common is the Radix 2 version which requires the number of samples to be in a power of 2. The FFT can be thought of as a way for finding which frequency range is most occurring. This process is used on things other then sound but the periodic repeating sound wave is perfect for the algorithm.
The second mathematical method was designed by me to help implement the averaging of the FFT's output. It is a form of running average. In essence, it keeps an average of all data as the program is running. This average is then shown on the final display. The pseudo-code follows:
n = 0 where n is the counter number A = 0 where A is the average for 1 to finish calculate q where q is one of the number in the average n = n +1 A(n) = ((n-1)/n)*A(n-1))+((1/n)*q) end loop
The manner in which this method works is illustrated in chart below.
# To Avg.(q)nA(n-1)FormulaA(n)Normal Avg. 11((0/1)1)+(1/1)111 321((1/2)1)+(1/2)32(1+3)/2 = 2 532((2/3)2)+(1/3)53(1+3+5)/3 = 3 643((3/4)3)+(1/4)63.75(1+3+5+6)/4 = 3.75 453.75((4/5)3.75)+(1/5)43.8(1+3+5+6+4)/5 = 3.8 263.8((5/6)3.8)+(1/6)23.5(1+3+5+6+4+2)/6 = 3.5
As stated this method yields the same results as averaging the first column of numbers. This method will work when the commonly used "sum and divide" will not because it could overflow the computers integer limit.
The project also involves the physics of sound waves. When multiple sound waves are flowing together they create a complex sound wave. Since sound waves can be represented with the sine function the explanation that follows uses that function.
| Harmonic & sine func. | WavesHarmonic Display |
|
Fundamental (1st) y =sin x | |
|
2nd Harmonic y =sin 2x | |
|
3rd Harmonic y =sin 3x | |
|
1st + 2nd y =sin x + .5 sin 2x | |
|
1st + 3rd y =sin x + .5 sin 3x | |
|
1st + 7th y =sin x + .5 sin 7x | |
| 1st + 3rd + 7th y =sin x + .5 sin 3x + .5 sin 7x |
This project has many varied applications. By creating a program that analyzes the harmonic spectrum of sound, the creation of harmonically accurate digital musical instruments is possible. This is beneficial because current synthetic instruments do not resemble the sound of the real instruments. But real-time music analysis and reconstruction is by far the greatest application that can be foreseen.
Today, when a musician wants to record music into the computer, he or she must use a MIDI interface instrument. This instrument is usually in the form of a keyboard. If another instrument is desired, top dollar must be paid for a MIDI compatible version.
However, within 5 years computer technology will have increased to allow a computer using my discovery to do an accurate analysis of the sound wave in real-time, thus yielding a fundamental frequency. This fundamental frequency could then be compared to a frequency table showing the desired musical note. This would allow musicians to play any instrument and have the computer score the music for them.[*] It would also allow singers to sing music and have it scored.
Not only would this be a great help to the music community but also to the home user. For instance, persons just beginning to learn music often compose music in their head. They often can whistle this music, but cannot change it into musical form. This kind of program would allow them to whistle their melody and have the computer score it for them, easing composition for young Mozarts.
This technology would have monumental impact in the computer game and music industry. A musician could play a piece and it would instantly be saved to a MIDI file. This technology could also be used to help the hearing impaired. If someone is losing their hearing in the upper range, this technology could allow a hearing aid to increase the volume in that portion of the spectrum. In the same sense, this could be a great help to Karaoke technology. By removing everything but the instruments and their harmonics, the Karaoke companies would create an instrumental playback of a soundtrack that does not have a current instrumental release.
This project involved sound and instrument analysis. The original intention was to write a program that would allow for accurate real-time analysis, but since accuracy must be sacrificed for speed, the real-time constraint was removed. After analyzing both synthesizer instrument samples and the corresponding real instrument, it is clear that there are differences. Although the synthesizer instrument samples may sound like the real instrument, they are harmonically different. This may be because the synthesizer manufacturer took an average of all types of that instrument where my sample only had one real instrument. This makes it apparent that synthesizer technology can still be improved significantly.
Despite the success of the Harmonic Wave Analysis program there is still room for improvement. The program takes about 30 seconds to analyze a 5 second sound sample on a 486 66mhz. Fortunately, with today's advancing computer technology, it will merely be a matter of months before the technology allows for quicker analysis. In future research, I intend to re-code all the floating point math into 16 or 32 bit fixed point math. This integer based math would decrease accuracy by an insignificant percentage, but would allow for a great speed increase.
This kind of increase would allow for instruments to be played and analyzed simultaneously. This lets a musician or vocalist play or sing and instantly receive the frequency of the notes. As a result, they could tell if they were sharp or flat. It would also allow musicians to train their ears and vocalists to train their voices more accurately through a graphic display of the frequency and harmonics.
Overall, this project was a success and progressed well beyond the original scope of the purpose.
This project improved my career as an engineer both personally and professionally even before college. At the Cobo Fair (locally) my project earned the following awards:
After That I Went To Tucson, AZ for the International Science And Engineering Fair. My project earned two more awards:
I continue to work on and research this problem in my free time. The inital vision of being able to generate MIDI files from music being played in real-time having been met for a single instrument ten years ago. The issue that has been holding up progress lately is that isolating the harmonics and fundamental for a single instrument is so easy (so easy in fact a high school student can do it) but what is more tricky (besides the constent demands of work and school) is isolating the fundamental of multiple instruments with multiple note playing at the same time. And continually porting the software to new OS and hardware.
Here's a few links to some other ISEF 96 people.
Copyright © 1996-2008 Ryan A Dibble. All Rights Reserved.