Soundmanager2, playing songs from a different start point on mobile not working

Hi Guys,

Please can you help I also asked this question on stack overflow http://stackoverflow.com/questions/20... where i have copied and pasted the code, but I wanted to ask here as well. The following is a copy from the stack overflow post.

I was wondering if anybody could give me some advice to play from the middle of a song on mobile with soundmanager2. I need it to work on android 4+ and IOS6 and IOS7. I have a solution, but it's not perfect.

I have found that soundmanager2 has unexpected results with playing music from a different start point of the song, and the behaviour varies with different devices.

I have also tried the same solution with pure HTML5 audio using .currentTime, however I have exactly the same problem with HTML5 audio, and as soundmanager2 uses html5 I have a feeling the problem is related.

My solution so far:

I have found that the best way to make sure that the song plays from the correct start point is to change the position after the song starts playing and not before the song song starts playing. However the undesired effect of this is that the song briefly plays before the change of position kicks in . I try to set the volume to 0 and to 100 to 'mask' this but some mobile devices ignore this. The solution works perfectly on desktop, but that's not good enough.

In soundManger the onplay event does not trigger when the song is playing, as it is triggered immediately when I call .play(); . For this to work, I need to use soundManager.onPosition('scAudio', 500, function .. ) and I have found 500 to be adequate to be picked up in most places. If I make it less than that, it wont be picked up in IOS6,

Can anybody assist with a better solution that doesn't mean the song plays before the position change kicks in?

Please see the code snippet at http://stackoverflow.com/questions/20...
1 person has
this question
+1
Reply
  • Pardon the late follow-up.

    Playing a sound with a position offset can be a bit tricky, mostly when mobile devices are included. They have limitations on auto-play and similar behaviours, which basically mean you cannot start audio unless it's done directly and immediately from within a touch or click event handler - no setTimeout() calls, etc.

    When using Flash, audio cannot start from arbitrary offsets - you must load "at least that far into the sound" in order to start playing it at the given offset. SM2 simplifies this by waiting for onload() to fire first, and then play() fires at the desired position.

    Under HTML5, SM2 calls load() on the sound and then when the onload event fires, it plays with the offset. Keep in mind that "onload" in HTML5 simply means that the sound has loaded enough metadata to know the duration etc., and at that point it can jump to the position in question without having to load all of the sound data up until that point.

    Mobile devices may also ignore volume controls - iOS for example, simply does not support the "volume" attribute via JS. Volume is instead controlled by the physical buttons on the device, and plays according to the present global volume setting.

    In any event, my recommendation to start playing at a given offset:
    mySound.play({
    position: 5000
    });
    As needed, SM2 will "preload" the sound by waiting for onload() as previously mentioned, and then will call play() with the relevant position offset. This should work under iOS, because a click or touch event -> load() -> play() should not be blocked by the device's auto-play "security restriction" due to native events being used.

    Under HTML5, a sound cannot have a position offset assigned unless the HTML5 "canplay" event has fired (which is what SM2 listens to and uses for the onload event.)

    Let me know if you find playback issues with the technique described above. SM2 should give console debug output showing its preloading work in this case, which may help in troubleshooting if you find issues.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • Thanks Scott for your reply,

    Unfortunately this does not work. If I console.log the readystate on iOS, it doesn't move from 1 to 3 for a while. Have a look at this iterator, this ensures that it plays it in iOS7, but i'm still trying to figure it out in iOS6 an some android devices.


    this.beginPlayingAudioFile = function(){

    scAudio.play({
    position: startTime * 1000
    });

    if(initialIteratorCheck) {

    var checkReadyState = setInterval(function() {

    if(scAudio.readyState === 3) {

    scAudio.setPosition(startTime * 1000);

    if( (scAudio.position + 1) >= (startTime * 1000) ) {

    scAudio.play();
    scAudio.setVolume(100);
    initialIteratorCheck = false;
    clearInterval(checkReadyState);
    }

    } else {

    scAudio.pause();

    }

    },100)

    }

    }



    Basically what I am doing is, if the song isn't ready to move to the position, PAUSE, otherwise when readystate === 3 , the position will work and it should play from the position.

    But I don't like doing this, it's hacky.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • Ah-ha. Testing on a real iOS device, it looks like SM2 is not waiting for the "canplay" (i.e., onload) event before attempting to seek to the given position and begin playback.

    Interestingly, this problem didn't happen on the iOS simulator.

    I should be able to fix this - will follow up with progress.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • Try this commit and see if it helps.

    From what I've found in testing, it looks like iOS sometimes fails to start playback when setting currentTime on a native HTML5 object after "canplay" fires. Lame.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • Thanks Scott,

    I have a range of devices to test on, so I will let you know the results. By the way, do you expect this fix to work for android?

    Cheers
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • I don't have a physical Android device to test on, but this should work on devices that support Audio() under HTML5 - and if Flash support is used, then behaviour should be fairly consistent there as well.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • Does the commit referenced earlier work on iOS? An app I'm developing was running into issues where it wouldn't consistently play from a specific position, until a duration was set. Very similar to the issue described by adamlutz in this ticket: https://getsatisfaction.com/schillman...

    I've grabbed the source from the commit and now it looks like it is seeking, but it doesn't actually start playback.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • Scott (Official Rep) January 04, 2014 23:46
    I am following the HTML5 spec in loading a sound, waiting for the "canplay" event and then setting currentTime in order to begin playback (via play()) at the desired position. This should behave correctly under iOS, but then I also believe iOS is quirky and inconsistent at best when it comes to HTML5 audio playback. :/

    It may be worth reviewing serving to HTML5 clients and making sure you've got a proper MIME (content-type) and content-length in your HTTP responses, in addition to byte serving enabled particularly for HTML5 clients. For more details, see how clients download audio.

    If you're finding that iOS won't play from a given offset because it thinks the sound duration is 0/undefined etc., let me know. By the time the HTML5 "canplay" event fires, SM2 should have the duration provided via HTML5 and it should be reflected in the sound object's duration property.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. kidding, amused, unsure, silly indifferent, undecided, unconcerned sad, anxious, confused, frustrated happy, confident, thankful, excited

  • MIME and Content Length are being set properly.


    HTTP/1.1 200 OK
    Content-Type: audio/mpeg
    Accept-Ranges: bytes
    ETag: "4110315309"
    Last-Modified: Mon, 13 Jan 2014 13:05:22 GMT
    Content-Length: 56641722
    Date: Mon, 13 Jan 2014 17:17:35 GMT
    Server: lighttpd/1.4.28


    Here is a test case:

    https://gist.github.com/will-in-wi/29...

    Tapping back and forth between the audio examples will occasionally result in a duration of 0 in the canplay event.

    Is this what you are looking for?
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • I just added more event logging. It looks like sometimes the canplay event will fire, but duration is 0. When that happens, duration will not appear until canplaythrough fires.

    This may only happen on real devices, it seems like the emulator might have different behaviour.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • I created another test case which repeatedly runs this test. On simulators, it has 100% success. On an iPod Touch running iOS 7, it fails about 5% of the time. On a iPad running iOS 7, it fails about 48% of the time. On an iPad running iOS 5, it has 100% success.

    Not sure if this helps at all. I'm trying to trace down the issue. It looks like a workaround might be to look at the duration value on the 'canplay' event and reload the audio if duration is 0. Not sure how this interacts with live audio, or other devices...

    https://gist.github.com/will-in-wi/8c...
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • I ran the test on my wife's iPhone 5 and my personal iPad 2, both running iOS 7, and they both have a 4% failure rate.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • It looks like this only happens when Accept-Ranges is allowed.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • Incidentally, from a first look, it looks like canplay firing while duration is 0 is standard for Android. At least on my Nexus 5...
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • It looks like in most cases on iOS, you have a duration when the canplay event fires. In a small number of cases, you don't get a duration other than 0 until the canplaythrough event fires. In this latter case, you don't get a durationchanged event when you finally get a duration.

    On Android Kit Kat (my only test Android right now), the audio always has a duration of 0 when canplay fires. it is 1-2 seconds later that you actually get a duration. When this happens, durationchanged fires.

    So between these two platforms, you would have to start that audio loading and then listen for canplay, durationchanged, and canplaythrough in order to detect when you actually have a duration and start playing at some point other than 0.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • Scott (Official Rep) January 18, 2014 18:08
    Ah-ha, interesting - I had overlooked the "durationchanged" event.

    It may be wise to modify or update the sound's duration attribute at "canplaythrough" as well, in the event some devices may not have the duration until then. Overall, it sounds like "durationchanged" is the best way to capture updates on that property.

    I've just added this to the current dev branch, see this commit.

    Good devices should have duration by the time "canplay" fires, ideally.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • I started an attempt to reliably fire a callback when I know the audio is seekable: https://github.com/APMG/SoundManager2...

    This has some issues on iOS 6/7, yet but the basic logic seems accurate.

    I've also discovered that iOS 5 doesn't allow seeking until canplaythrough, even though duration is always set by canplay.

    Android seems to allow seeking as soon as durationchange is fired with a duration > 0.

    There is an issue on some platforms where durationchange is fired with a >0 duration, however you can't seek until canplay fires.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • Ultimately, I'm looking to create and play a sound with position set such that I start the sound playing at a point which is not 0. And I want this to work cross-platform.

    Right now, position being set in the sound settings doesn't do anything on iOS or Android.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • Scott (Official Rep) January 18, 2014 22:42
    From what I've seen in quick testing, durationchange fires before canplay ("onload" in SM2 terms) under Firefox, Chrome and Safari.

    In the case where you call play({ position: 5000 }) for example, SM2 will load the sound and wait for canplay/onload before attempting to begin playing the sound with a position offset. By design, an HTML5 audio object will throw a non-seekable-type error if you try to set position before canplay has fired.

    This may conflict with iOS' "auto-play" limitations, where load and play are blocked unless called immediately from a touch/click event. load() -> canplay -> play() should work on iOS because I think they've accounted for that case, but it doesn't seem to be very reliable.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • The issue is that play({ position: 5000 }) almost never works on a mobile device.

    On iOS 6/7 it occasionally will not be able to seek until canplaythrough, although it is usually able to seek oncanplay. Incidentally, this doesn't happen in an emulator, it only misbehaves on real devices.

    On iOS = 0, usually happening right after canplay, but not at canplay.

    Ideally, if position is set in the play() arguments, SM2 would wait to begin playback until it could seek, and then start there.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • https://gist.github.com/will-in-wi/85...

    Here is a test for what I'm trying to do. In all cases, on all devices, I would want this to start playing 10 seconds into the file. This doesn't work on iOS or Android.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • Scott (Official Rep) January 29, 2014 05:07
    Interesting. I wonder if it makes any difference if play({ position: 10000 }) is called from within onclick(), as opposed to position being specified to createSound(). My guess is it probably doesn't matter.

    I'm not sure what to advise here, unfortunately. Native HTML5 audio support on iOS is quirky at best.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • Do you have any interest in trying to work around the iOS/Android quirks in SM2? It looks like it would be possible to, when a position is passed in the sound options, pause playback until you can change currentTime successfully.

    I've built working prototypes for this which I can share. I just don't understand the SM2 codebase with enough depth to do it correctly. I've spent about 6-7 hours trying to do it in SM2 thus far.

    Basically, I know how to fix it, I just don't know how to fix it in SM2.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • Pardon the late follow-up, I'm usually faster at these. ;)

    I'll have to do some more testing on playback with position arguments. This one is tough because position can't be changed until "canplay" has fired in HTML5.

    I suspect these are primarily limitations of mobile, and/or buggy implementations affecting play() -> seek and so forth. Play being restricted to a touch/click is the first limitation; being able to successfully seek is the second, which I suspect is less-reliable.
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned

  • I've been having some luck with the onPosition event to do start position on iOS.


    var mobileAudioLoaded = false;
    if (!mobileAudioLoaded) {
    soundPlayer.play();
    soundPlayer.onPosition(0, function () {
    if (!mobileAudioLoaded) {
    this.setPosition(50000); // start at 50 seconds
    mobileAudioLoaded = true;
    }
    });
    }
  • (some HTML allowed)
    How does this make you feel?
    Add Image
    I'm

    e.g. happy, confident, thankful, excited sad, anxious, confused, frustrated kidding, amused, unsure, silly indifferent, undecided, unconcerned