iOS4 Background Audio Revisited

The other day I went ahead and revisited streaming audio apps for iOS4. I blogged previously about how it is the biggest pain in the ass to get working….properly. And it still is.  Background buffering was a pain to do, but I finally worked out how to apply it to AudioStreamer (a github project by mattgallagher). The solution in this StackOverflow post is incredibly useful.

So, I decided I’d checkout AudioStreamer from git, take a look at other people’s forks, try to merge all the fixes and features then add my own. That’s what I did the other day.

My current github fork of AudioStreamer is here: (and it’s pretty damn stable, just sayin’).

Basically, my fork cherry-picks some commits from other users, notably:

- Shoutcast metadata [jfricker]
- MIME type detection [andybee]
- HE-AACv2 [idevsoftware]
- Level Metering [idevsoftware]
- NSThread memory leak [mattgallagher]

Then I set out to try and fix the bugs I had encountered…and fix up the sample app so that it worked fully with iOS4 multitasking:

- Fixed interruption crashes (crashes when you use iPod app then re-use the sample app, or receive a phone call)
- Background buffering (!!! AudioStreamer now buffers properly in the background)
- Play/pause from iPod controls (The sample app now works properly with the iPod controls)
- Stop all UI updating and timers while backgrounded (The sample app now correctly follows guidelines to stop all UI updating and unnecessary timers to save battery life)

- Retina display example (The sample app now has a retina display example…just to be iPhone 4 compatible)

- Support for Pause button in UI (Pausing now works properly, even for streams that don’t support it)
- Local Notifications (outside app) on error (Notification popup if there is a lack of connectivity when playing music outside the application)

There’s just one more bug for me to squash, and frankly I can’t figure it out. Notifications are sent from AudioStreamer to the UI  in case of an error, the UI then handles the notification with a UIAlertView. Sometimes the alerts refuse to show up, or instead, multiple show up, or…..sometimes it just works as intended (i.e. one alert popup). I’m not sure if it’s an iOS bug or something in my code. It’s rather odd as it isn’t easily reproducible and often a reboot of the device will fix the issue. Very very odd.

, , ,

iOS4 Multitasking: Developer’s Hell

I finally got around to installing iOS4, so I decided I’d go ahead and investigate what it takes to update the Fresh app (streaming radio) to support multitasking. Currently, my conclusion is that implementing ANY of the multitasking APIs is 110% hell unless you’re starting from scratch.

First of all, your application has to play nice while it’s sitting in the background. This means that you’re basically required to implement a whole lot of “if backgrounded” checks, unless you want to soak up the battery. This involves stopping unnecessary timers, updating the UI when the application comes back to the foreground (rather than per a specific event), and saving non-critical alerts for when the user pops back into the application. This basically requires a crapton of refactoring…and we all love how that introduces bugs. If you have any sort of complicated app, you’ll be having to manage state saving so that fast app switching works nicely. Oh, and not to mention that iOS never guarantees that your application stays backgrounded. It can be killed depending on resource usage, available memory or even just idle time. Every task (other than a few specific exceptions) is time limited, so you are constantly having to check and request more time for your task.

Now, this particular application is meant to take advantage of the background audio multitasking API. Now, as an aside, Apple does not provide any “simple” API to stream audio. What they provide is a primitive and generic audio buffer/queue that suits both local files and network streams. Basically, this means that if you want to stream audio over the network, you have to implement the protocol, manually send the requests to get data (via HTTP for example), then feed that data into a buffer (where you have to handle the size of incoming data vs. the size of the buffer), THEN…play the audio from the buffer or handle any errors that occurred along the way. Now, in any half decent modern API, surely, you’d have a perfected version of the algorithm where you could simply plug in a URL and instruct a magical black box to play it. But alas no, each developer is to implement the over complicated, primitive audio queue… just for a streaming radio application. Now combine that with the following multitasking API, and you’re in developer hell.

One of the official pages on backgrounding is nearly impossible to find on the Apple site, so here it is: Now, if you look closely, you’ll see that most sections are detailed nicely….except background audio, fantastic. So, taking a quick look at that section reveals that if you simply “flag” your application as one that streams audio, you’re good to go. Hooray right? Well, sort of, not really, no.

Hurdle 1: Hooking into iPod controls

Surprisingly, this is the EASIEST part of implementing background audio. It seems to “just work” as long as you have fairly simple start/stop/pause methods for your radio streamer. One gotcha is that state changes can now take place from both your play/stop button and the iPod controls… so, basically you should be using some sort of event system for state changes and to reupdate the UI whenever your app hits the foregound again.

Hurdle 2: Interruptions

Interruptions have existed since OS2. They cause your music to stop, usually when someone rings. However, now your application has to act more like the iPod application…except you can’t! Unlike the iPod application, which automatically restarts after you finish your call, the only way your application can automatically restart it’s stream is if the user brings it back to the foreground. Not only that, I’ve found that interruption events are very inconsistent. Sometimes they fire simultaneously with the interruption, sometimes they won’t fire until you bring the application to the foreground. On the times that they don’t fire with the interruption, I find that some sort of crazy auto deallocation of the streamer occurs which I can only fix with a terrible workaround. I am yet to find the actual cause of this.

Hurdle 3: iPhone Simulator

It sucks. Testing background audio with it is impossible! Even in the GM build, there is a bug that mutes all sound from your application when it is backgrounded. This makes it rather difficult to tell whether or not your application IS ACTUALLY SITTING IN THE BACKGROUND.

Hurdle 4: You’re free to do whatever you want…except when you need do something

So, as explained before, if you flag your application for streaming audio, your application can go ahead and play audio and do what it needs to do without any particular time limit or suspended state. Cool! Sort of like a free pass to do whatever the hell I want, right? Wrong. There’s a HUGE exception here: “if your application stops playing audio while in the background, your application is suspended.” Wow, that’s fantastic. So let me get this right…if my STREAMING AUDIO APPLICATION happens to need to BUFFER (i.e. STOP PLAYING AUDIO) while on a dodgy cellular connection, my application is essentially…fucked. Yes, that’s right. If you have a little bit of a network hiccup, and need to buffer, the user will need to jump back into the application to restart the audio stream! Not even the iPod controls (which are meant for external use) allow you restart the stream. I haven’t yet found a fix for this yet, but I’m assuming I’m going to have to go back to that mindfuck of a primitive audio queue to fill it with some sort of dummy audio while it buffers, so that iOS doesn’t fucking suspend my application in the middle of playing music. Doesn’t this found like fun?

Hurdle 5:Backwards Compatibility

Finally, as always, you need to aim for backwards compatibility. Apple claims to have completely culled OS2.x, but let’s face it, there are still 1st Generation iPod Touch users that DIDN’T pay the iOS3/4 upgrade tax, that want to use apps. You have to, at a minimum, ensure your application works on OS3 devices, which means making sure that some method calls, that are iOS4 specific, actually exist. Then using the simulator to test………. /facepalm.


, , ,