January 10th, 2016
(written by lawrence, however indented passages are often quotes)
Amazon is relatively new to the job of running an app store, and some of their mistakes are either hilarious or infuriating, depending on how much money you’ve lost trying to deal with them.
Let’s start with the silliness. Amazon has an app store for Alexa (the personality that powers its voice service, and which can also be reached through the Amazon Echo). The apps are called “skills”. The name change is wise, since “app” suggests a binary file of software, like what you get in the iPhone app store and the Android app store. An Alexa skill is a webservice. If you are a developer of an Alexa skill, to get into the Alexa app store really just means registering the URL of your web service, so that people can connect to it from their Echos.
And this is where we run into the first bit of craziness. Amazon decided that they should model the Alexa app store after the iPhone app store. So there is a certification process to get your app into the store. But think about the difference: you are not uploading a binary file to the Alexa app store, you are simply registering an URL. So Amazon has no real control over your software. You could get an app approved, and then you could swap out the app for any other app, and the Certification team at Amazon would never know. They don’t control your code. Your code is not in their store, so they have no control over what you do. And yet they modeled this process after the iPhone store, where Apple does have control over your app.
Let’s be clear about the difference. In the iPhone app store, developers upload a specific bit of binary code, and Apple has control over it. To change that binary code, a developer has to get approval from the team that overseas certification at the iPhone app store. But in the Alexa app store, since the only thing registered is a URL, the developer can change their code whenever they want. The URL will continue to point to a server that the developer controls, and Amazon will never know what changes are taking place.
Beyond the silliness, there is the outrageous brokenness of the certification system itself. Consider the complaints of the developers, as they write on the Amazon developer forum:
So, this is sort of a last ditch effort, but my team and I are about ready to give up on developing for Alexa, because of the terrible certification process that is unreasonably preventing our code from being released. Has anyone found a way to actively get in contact with the certification team?
Over the break, our two skills were rejected for the second time. The problem is that the only changes between our first rejection and second rejection were changes to implement the requirements of the first rejection. They rejected us for doing exactly what they said – and there seems to be absolutely no way that we can have a rational conversation with the certification team because they live behind some impenetrable iron curtain rather than being available for us to work with.
Joseph Jaquinta replied:
Yup. Our experience exactly.
I just finished yesterday the first draft report on just this problem for my CEO. It’s intended to be the input by which she will decide if we consider pursuing Alexa or divesting. (Drop me a line at email@example.com and I’ll send you a draft. And you can see how closely our experiences compare.)
For me, it boils down to these three problems:
1) Many “requirements” are subjective.
2) Each time you submit, you get a different person.
3) There is no way to engage with Amazon and discuss/get clarity on these subjective judgements.
These three together break certification. A different person each time makes different subjective decisions which cannot be explained, or discussed. There is no way to iterate towards the correct decision because the goalposts move each time you submit. The only skills that can get through unimpeded are the simple ones, which stifles innovation.
Overall it is absurd. The majority of things they object to you can change the minute after they are certified. I just did a bunch of changes last night to once skill with a switch in the code. For everything they insisted I change that I disagreed with, I can flip the switch to do it their way. Once we’re past certification I’ll flip the switch and go back to doing it my way. You can’t do this for everything, but you can do it for many things.
However, the Amazon system is so broken that it potentially offers a fix for itself. When the Certification Team rejects your app, you don’t need to change the app. You do not need to respond to their requests. You do not need to make any of the changes that they demand. Since a new person reviews each submission, and since there is no limit on submissions, one way to get through the certification process is to simply roll the dice and spam the certification team. Submit an app 10 times, or 20 times, or 30 times. At some point you will probably get lucky, and someone will approve your app. And since re-submission only requires clicking a button, re-submitting is much easier than actually responding to any of the criticism that the Certification team gave you.Source