I’m trying to figure out how to write a software oscilloscope audio visualization. It’s made more frustrating by the knowledge that I am certain that I have accomplished this task before.
In this context, the oscilloscope is used to draw the time-domain samples of an audio wave form. I have written such a plugin as part of the xine project. However, for that project, I didn’t have to write the full playback pipeline– my plugin was just handed some PCM data and drew some graphical data in response. Now I’m trying to write the entire engine in a standalone program and I’m wondering how to get it just right.
This is an SDL-based oscilloscope visualizer and audio player for Game Music Emu library. My approach is to have an audio buffer that holds a second of audio (44100 stereo 16-bit samples). The player updates the visualization at 30 frames per second. The o-scope is 512 pixels wide. So, at every 1/30th second interval, the player dips into the audio buffer at position ((frame_number % 30) * 44100 / 30) and takes the first 512 stereo frames for plotting on the graph.
It seems to be working okay, I guess. The only problem is that the A/V sync seems to be slightly misaligned. I am just wondering if this is the correct approach. Perhaps the player should be performing some slightly more complicated calculation over those (44100/30) audio frames during each update in order to obtain a more accurate graph? I described my process to an electrical engineer friend of mine and he insisted that I needed to apply something called hysteresis to the output or I would never get accurate A/V sync in this scenario.
Further, I know that some schools of thought on these matters require that the dots in those graphs be connected, that the scattered points simply won’t do. I guess it’s a stylistic choice.
Still, I think I have a reasonable, workable approach here. I might just be starting the visualization 1/30th of a second too late.
I am not sure what your purpose is, but nobody lets an oscilloscope just display a run-on data stream. Instead you’d have a trigger to decide when to start “redrawing”.
In the case of audio you can even go further: use auto-correlation like in those speed adjustment filters (e.g. MPlayer’s scaletempo) to replace the data from the current 1/30 s with the most similar one from around the next 1/30th.
What concerns A-V sync I would expect the delay from the audio driver/hardware to more than compensate for starting to draw 1/30th second too late.
You can show just 512 of the 1470 samples, or you could graph all 1470 and scale it to fit in 512. I’ve seen it done both ways, it just changes the look a little. Most jukeboxes only give partial data to the viz plug-ins so they can’t be used to store a copy of the song.
The A/V sync could be off because the audio and video engines are running one ahead of the other (due to different processing delays). If you don’t have presentation timestamps, you could play a sync test (one frame of “beep” audio with a flash frame of video, then some frames of silent black, in a loop), and use a fast video camera (60 fields per second at least) to shoot the result and calculate your offset, then build that into your display algorithm.
If my memory is correct, pulse is able to give you the delay it introduces.
If you store the PCM data in a ring buffer, you can then use currentPosition-pulseDelay as input position for the oscilloscope, it will then be in sync with the audio
Don’t know if this will be of any help to you but here is the function I wrote for a similar project that used SDL. Wasn’t perfect but it synced up with the start/end of song correctly.
(81 test points)
void MusicState::Update(unsigned int TimeSince)
{
int DataLoc = 22050 * 2 * 2/ 1000 * (SDL_GetTicks() – StartTime);
if(SDL_GetTicks() > TimeLast + 100)
{
Color = 0;
for(int a = 0; a = song->alen)
{
cout << "Trigger" <abuf)[DataLoc + ((44100 / 81) * a)];
}
}
Color = Color % 81;
cout << Color << endl;
TimeLast = SDL_GetTicks();
}
};