Compose sounds from frequencies and visualize them

• Download source files - 31.4 Kb

Introduction

Have you ever wondered why the same note sounds differently when played on different instruments? Do you want to know how sound can become visible? Welcome to the beginner's wave explorer!

With this little application you can compose sounds from scratch by adding many frequencies. You can listen to your sound and add sine waves until you get the desired result. In addition to a common wave display, you can watch the sound change in synaesthesia mode. This alternative visualization creates one bitmap from the whole wave, which you can scale and color to see the sound move and grow.

You might wonder why I wrote the WaveMixer. There are two good reasons: Fun and curiosity. For another sound application, I needed a disturbing noise including a specific frequency. When I realized that my little knowledge about PCM wave was too superficial to generate natural sounding noises, I sacrificed a weekend and made up this nice tool. It is a comfortable start into the wonderful world of pulse code modulated waves.

The sounds are played using the code from A full-duplex audio player in C# using the waveIn/waveOut APIs.

The features and how they work

Your first experience with waves

Before we start composing a sound, we need an empty wave. WaveMixer starts with a fresh and clean wave of five seconds. If you want a shorter or longer sound, enter the length in seconds and create it:

Nothing has happened until now; the new wave has only been initialized. These four lines did that:

//define sampling rate, sample size and number of channels
WaveFormat format = new WaveFormat(44100, 16, 2);
//initialize the samples
short[] samples = 
  new short[(int)(format.Channels * format.SamplesPerSec * numNewWave.Value)];
//load the empty wave into the display control
WaveSound waveSound = new WaveSound(format, samples);
waveControl.WaveSound = waveSound;

Let's create an A! a' is defined as 440Hz, so this will be the first frequency to add. Enter the frequency of a', a maximum amplitude for the sine wave between 1 and 32767, and the time at which the wave shall be added:

The result looks boring and sounds just the same. Left and right channels contain the same beeeeep:

The synaesthesia view is not much better at this point, but you might try a few different image sizes to see how the result changes:

PCM stands for Pulse Code Modulation and means that there are no wave functions or exact descriptions (as you probably know from the MIDI standard), instead samples are taken from the actual wave. A .wav file contains loads of snapshots, the WaveOut API sends them so the speakers so that you can hear something similar to the original wave. You have just created such a set of wave samples from scratch, and here is the code behind the buttons. It calculates the indexes of the first and last affected samples and then mixes the new sample values into the existing ones.

/// <summary>Adds a sine sound of a specific frequency.</summary>
/// <param name="waveSound">The sound that's being edited</param>
/// <param name="frequencyHz">Frequency of the new wave in Hertz.
/// </param>
/// <param name="offsetSeconds">Starting second of the new wave.
/// </param>
/// <param name="lengthSeconds">Length of the new wave in seconds.
/// </param>
/// <param name="amplitude">Maximum amplitude of the new wave.
/// </param>
public void AddWave(WaveSound waveSound,
    float frequencyHz, float offsetSeconds,
    float lengthSeconds, int amplitude)
{
    //get the existing wave samples
    short[] samples = waveSound.Samples;

    //interval for 1 Hz
    double xStep = (2 * Math.PI) / waveSound.Format.SamplesPerSec;

    //interval for the requested frequency = 1Hz * frequencyHz
    xStep = xStep * frequencyHz;

    long lastSample;
    double xValue = 0;
    short yValue;
    short channelSample;

    long offsetSamples = (long)(
        waveSound.Format.Channels
        * waveSound.Format.SamplesPerSec
        * offsetSeconds);

    //if the beginning sample exists
    if (offsetSamples < samples.Length)
    {
        lastSample = (long)(
            waveSound.Format.Channels
            * waveSound.Format.SamplesPerSec
            * (offsetSeconds + lengthSeconds) );

        if (lastSample > samples.Length)
        { //last sample does not exist - shorten the new wave
          lastSample = 
              samples.Length - waveSound.Format.Channels + 1;
        }

        //for all affected samples
        for (long n = offsetSamples; n < lastSample; 
                               n += waveSound.Format.Channels)
        {
            //calculated the next snapshot from the sine wave
            xValue += xStep;
            yValue = (short)(Math.Sin(xValue) * amplitude);

            //mix the value into every channel
            for (int channelIndex = 0; 
               channelIndex < waveSound.Format.Channels; channelIndex++)
            {
                channelSample = samples[n + channelIndex];
                channelSample = (short)((channelSample + yValue) / 2);
                samples[n + channelIndex] = channelSample;
            }
        }
    }
}

Before I explain the visualization, let us make the wave sound a bit more natural. Add 220 Hz (a), 880 Hz (a'') and other octaves to the wave; remember that an octave is frequency*2 or frequency/2:

The result looks more interesting and sounds a little better. Anyway, one note is not yet musical; we need something like an elementary melody. Add another wave of 440 Hz, but let it begin at second 1 and last for only half a second. Look and hear how the sound changes when you add short waves here and there:

Staring at a black and white graph all the time is boring, you can as well observe your wave's bitmap:

Now you'll find that the sound is quite harmonic, but still not natural. What do we need? Dust and dirt for our samples, random waves in varying frequencies! Instead of the exact frequencies, try adding a group of waves. The frequencies should be inside certain limits to keep them close to a' of 440 Hz.

After a few hundred random waves close to 440 Hz, I recommend you to add one exact wave from second 0 to the end, with a high amplitude of at least 30000. This will make the strange noise clearer again. The result sounds fascinating and looks amazing:

In the colorful view you can see how the last, exact wave dominates the sound and leaves only little space for the disturbances that we added before:

The synaesthesia mode

For a long time I've wondered about the colours noises really have. Everybody sees the same sound in different colours, most people even see nothing at all, and they say they can only hear it. The direct translation of pulse codes into pixels does not work, because there are three colours (red, green and blue), but usually only two channels (perception of left and right ears). Which colour belongs to which channel? I'm afraid there's no general answer...

I decided to leave the colour decision to the user. Instead of mixing both channels into one pixel (e.g. red for left, green for right) the WaveMixer paints them next to each other. That means, every sample gets represented by two dots: Left pixel and right pixel. Just as the samples are played one after another, WaveMixer paints the pixels pair by pair. Where will the next row begin is defined by the dimensions of the bitmap. I zoomed the wave picture close enough, so that you could focus every single pixel-pair and try to "hear with your eyes". ;-)

If red, green and/or blue is used for a channel, it can be configured in the user interface. You already know those checkboxes. The actual values of the chosen colour components result from the sample's value. Sadly, the Int16 wave samples had to be reduced to unsigned byte values:

//get the factor that reduces the highest sample to 127
float scale = 127f / maximumSampleValueInTheWholeWave;

//scale one sample
byte scaledSampleValue = (byte)(sampleValue * scale);

//mix the sample's colour
pixelColor = Color.FromArgb(
    channelColors[channelIndex].IsRed ? scaledSampleValue : 0,
    channelColors[channelIndex].IsGreen ? scaledSampleValue : 0,
    channelColors[channelIndex].IsBlue ? scaledSampleValue : 0);

Of course, the picture does not have to be as large as the sound. If you choose dimensions that contain less pixel-pairs than there are samples to visualize, the samples are packed into blocks, and the loudest sample of every block defines the colour:

int samplesPerPixel = 1;
if (countPixels < countSamples)
{
    samplesPerPixel = Math.Ceiling(countSamples / countPixels);
}

How this block size is applied to squeeze many samples into one pixel is explained in the method WaveUtility.PaintWave in the source file. But before you dive into the code, let me show you a cool example.

Fun with the Windows system sounds

Most Windows editions contain a lot of tiny wave files in [installDrive]\[installPath]\Media, usually the path is c:\windows\media and one of the files is notify.wav. It looks funny even in the wave view:

Now, switch over to synaesthesia view and enter a picture width/height of both 250. Alternatively, width=100 and height=200 are also not bad.

Isn't that amazing? Notify.wav looks really cool for such a well known, boring sound:

By the way...

Before you go on playing with the weird waves, have a close look at the frequencies of these notes:

and remember that a C-Dur accord is C-E-G. You could mix this accord by adding the following waves:

1. 264 Hz, Amplitude 30000
2. 330 Hz, Amplitude 20000
3. 396 Hz, Amplitude 10000

Wait and see ... or hear, it's all the same!