Audio Syncing / SimpleAudioEngine Buffer Position

I am creating a rhythm-like game and have been having trouble syncing the background music with whats being drawn.

What I have been doing so far is saving the clock time immediately after starting playing the music, and then on each update I get the current clock time and compare it to the saved clock time when the music was started, giving me the time since the music was started.

This appears to be fairly accurate at the start, but as time goes by it gets slightly out of sync. EDIT: on a 9minute song it was out of sync by about 2 seconds, on a 2min song it was off by about 0.5sec. and the graphics were finishing before the music…

What I would really like is to be able to get the audio buffer position from the SimpleAudioEngine (the current sample being played), but this feature is not available and I don’t know how hard it would be to implement.

Anyone have any suggestions?

Hi Red, I’m also building a rhythm game and I found that SimpleAudioEngine was not adequate for my needs. You might find this post on my blog helpful and also this page where I found the basic idea.

I’m using Apple’s Audio Queue Services since I’m releasing for iOS first and may not release for android at all, but if you need a cross platform solution you can try the same ideas with OpenAL queue buffers — if you do I’d be interested to hear how it turns out.

Good luck!

I’m not very familar with rhytm games, can you please explain which API changes you wish? Do you need seamless audio only or also dynamic changes for some attributes like playback rate?

I think that in 3.x CocosDenshion can be moved to separate thread, and in this case it can load audio or use streaming (with alSourceQueueBuffers) in own thread. Look like in this case SimpleAudioEngine can just provide API to create queue of audio files, and decode them as one.

It’d be great to have a queueing API in CocosDenshion! :slight_smile: The things I’d be looking for would be

  1. queueing files for gapless playback (of course)
  2. clearing the queue (without stopping playback of the current track)
  3. querying the position of the play head from any thread (either from the beginning of the whole queue or from the beginning of the current track)
  4. extra credit: stop playing the current track at future time t and play the next track in the queue without a gap

If there’s someone from the cocos2d-x reading, and since other people are writing down their wishlist for CocosDenshion… :stuck_out_tongue:

Is it possible to stop a song and get the microsecond or position when it stopped? And then use that data to start another song from that position?
Or, if none of that is possible, to run several songs at the same time (so I can mute all tracks except one, and switch them according to game logic)?

Thanks!

@Pablo: typically that’s not possible to do without some delay, since the audio data needs to be loaded into the hardware buffers and that takes time. Basically gapless audio is only possible if you queue up the tracks ahead of time. Also, playing multiple audio streams simultaneously is not ideal if you’re using a compressed audio format since most mobile devices only have one hardware decoder.

That said, getting the microsecond position of an audio stream (stopped or not) would be nice.

Mike Plotz wrote:

@Pablo: typically that’s not possible to do without some delay, since the audio data needs to be loaded into the hardware buffers and that takes time. Basically gapless audio is only possible if you queue up the tracks ahead of time. Also, playing multiple audio streams simultaneously is not ideal if you’re using a compressed audio format since most mobile devices only have one hardware decoder.
>
That said, getting the microsecond position of an audio stream (stopped or not) would be nice.

Thanks for your answer. :slight_smile:
Would the hardware decoder issue prevent multiple tracks from running at the same time, even though all except one would have volume (all the rest would be muted)?

Thanks! :slight_smile:

The hardware issue doesn’t prevent you from playing multiple tracks at the same time, but depending on your target platform you might lose performance because of the extra cpu load. Also, if you need to line up the tracks so they’re all at the same play position that might be difficult.

Mike Plotz wrote:

The hardware issue doesn’t prevent you from playing multiple tracks at the same time, but depending on your target platform you might lose performance because of the extra cpu load. Also, if you need to line up the tracks so they’re all at the same play position that might be difficult.

I get it. Yes, my goal is to have a couple of tracks aligned.
Our team managed to do it in Flash for a game a couple of years ago, but I wanted to see if it was possible in cocos2d-x. What we did was play 5 tracks at the same time, with 4 of them on 0 volume. So shifting from one track to the other was a matter of changing the volume of the tracks (either immediately or progressively, creating a nice effect).

Mike Plotz,

I read your blog post and it looks very interesting. I will definitely need a cross-platform solution but I don’t have any openAL experience and I still have miles to go in my game before the syncing becomes a major issue. Furthermore I’m developing on a Windows machine and the 3.0pre-alpha build is still using MCI instead of openAL, which was kindly refactored by Christian Cristea according to this post: http://www.cocos2d-x.org/boards/6/topics/29949

So I think my best bet is to wait a bit and hopefully the cocos2dx team will find time to update the SimpleAudioEngine and maybe add some of these very desirable features :slight_smile:

I want to make a rythm game. what are the concepts I need to know?:slight_smile: