Converting audio file to create an Image, and converting audio file into a format usable by class Picture

Question

I have to write a program in Java that uses StdAudio and Picture to create a two-dimensional color visualization of a sound file while it is playing but I'm not really sure how to.

Can someone tell me everything that I need or tell me what I need to do to "convert" the sound file so that it's readable by Picture?

I could grab the samples from the sound file and return them as array of doubles, but then how would that even create an image? How could those values even sync with the image?

I have been playing around in eclipse just trying to figure out how this could possibly even work but my code just ends up being a whole mess.

private final static int SAMPLE_RATE = 44100;
private static int WIDTH = 500;
private static int HEIGHT = 100;


private static JFrame frame;
private static Picture pic;


public static void main(String[] args) throws IOException
{
    pic = new Picture(WIDTH, HEIGHT); // <- blank black image
    String audioFile = "SampleTest2.wav";   
    double[] audio = StdAudio.read(audioFile);


    frame = new JFrame();
    frame.setContentPane(pic.getJLabel());
    frame.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);
    frame.setTitle("Sound Visualization"); 
    frame.setResizable(false); 
    frame.pack();
    frame.setVisible(true);

    for (int k = 0; k < audio.length; k++)
        StdAudio.play(audio[k]);

    for (int i = 0; i < pic.width(); i ++)
    {
        for (int j = 0; j < pic.height(); j++)
        {
            pic.set(i, j, toColor(audio[SAMPLE_RATE + i]));
            //frame.setContentPane(pic.getJLabel());
        }   
        frame.repaint();
    }
}

private static Color toColor(double colVal)
{
    int r = (int) (((colVal + 1) / 2) * 255);
    int g = (int) (((colVal + 1) / 2) * 255);
    int b = (int) (((colVal + 1) / 2) * 255);
    return new Color(r, g, b);
}

Answer 1

To use StdAudio you need wav file with sampling rate of 44100. It means every second of this sound consists of 44100 values(samples). When you load such file with duration of 1 second using method double[] read(String filename) you will obtain an array with 44100 elements. Javadoc of that method tells us the values will be between -1.0 and +1.0. We can iterate over every sample, map values from -1..1 range to 0..255 range (because colors need values from 0 to 255) and paint each pixel with this color. For better effect let's not paint single pixel but a column of 100 pixels.
I'll create image of 500x100. It will display only 500 samples so it will represent 500/44100 = only 0,01 of a second. To create empty picture of that size use:

Picture p = new Picture(500, 100);

To paint separate pixels along the image use:

for (int i = 0; i < 500; i++) {
    p.set(i, 0, color);
}

and to display this picture use:

    p.show();

Next, to create a color we need 3 values: red, green and blue components. Here we have only one value so the resulting image will be a greyscale image because saturation of every component will be the same value new Color(value, value, value). To quickly convert a range from -1..1 to 0..255 use such formula: (int) (((d + 1) / 2) * 255)

I used the first sound file from this site: http://www.music.helsinki.fi/tmt/opetus/uusmedia/esim/index-e.html and the image I obtained is:

The code I used is:

import java.awt.Color;
import java.io.IOException;

public class StackOverflow58899141 {

    private static int IMAGE_WIDTH = 500;
    private static int IMAGE_HEIGHT = 100;

    static String filename = "O:\\1.wav";

    public static void main(final String[] args) throws IOException {
        // reading sound file to samples
        double[] samples = StdAudio.read(filename);
        // creating empty image
        Picture p = new Picture(IMAGE_WIDTH, IMAGE_HEIGHT);
        // filling image from left to right
        for (int i = 0; i < IMAGE_WIDTH; i++) {
            // filling image from top to bottom
            for (int j = 0; j < IMAGE_HEIGHT; j++) {
                // adding 44100 to skip 1s of silence at the beginning
                p.set(i, j, doubleToColor(samples[44100 + i]));
            }
        }
        p.show();
    }

    // convert number from range -1.0..1.0 to 0..255
    private static Color doubleToColor(double d) {
        int val = (int) (((d + 1) / 2) * 255);
        return new Color(val, val, val);
    }
}

Now you have a solid start to understand how it works. Although Picture class allows easy saving of an image it doesn't allow animating. To achieve that you'd need to create own JFrame and draw image and delay drawing each column of pixels to get the animation effect.