TornadoGuard
by Jeff
Image text: The bug report was marked 'could not reproduce'.
This is a comic with a take on a mobile application store for iPhones or Android devices. (This may be an Android store because the creator of this app is "DroidCoder2187". Both stores take all the reviews and average the ratings for the overall star rating. In this comic, we see why this is a bad idea.
In this case, there are three 5 star reviews about the running, UI and location setting in the app, however, the one that says it actually does not work is given the same weight in the review.
The image text is funny because it is a note from the developer's bug report, which they could not reproduce the error. However, they only would have to have a tornado coming towards their area for it to work correctly, which is a fairly rare situation and especially rare in certain areas of the world.
August 12th, 2011
It was pretty windy.
August 12th, 2011
I think the image text is also a reference to an earlier post, where the nerd girl attempts to fix a bug by having children.
August 12th, 2011
Mm, I think it’s actually a reference to what IT people are ALWAYS saying when bug reports are submitted to them. Like because they can’t reproduce the problem, there must not be a problem. As Jeff pointed out, of course they can’t reproduce the problem, because they would have to experience an actual tornado to know whether it worked properly or not.
August 12th, 2011
“Like because they can’t reproduce the problem, there must not be a problem.”
No, they’re not saying there isn’t a problem, they’re saying they can’t track it down and fix it unless they can reproduce it. This is a genuine issue. You have a complex program, and when you try what the user says, it works perfectly. So how do you track down the bug? You don’t have a starting point to find the cause.
There is a cause, no doubt, but it could be anything. It’s very, very difficult to track down unless you can see the problem happening in front of you.
When you can reproduce the error, then you can look at the program with a debugger, run it step by step and see what’s happening. Then you can fix it.
August 12th, 2011
While I totally agree that that’s what “could not reproduce” should mean, I have definitely seen IT people declare something non-reproducible as an excuse to not fix it.
It’s particularly egregious when I can reproduce it for them consistently at my desk, but they can’t figure out how to reproduce it at their own workstations. Therefore they declare it non-reproducible.
August 13th, 2011
Exactly. This happens ALL the time with the IT people at my work. They are forever closing tickets by simply saying “could not reproduce.”
August 17th, 2011
There are certainly IT people who don’t take the time to try to figure out a problem before they close a ticket, and in busier IT departments when problems are continuing to come there’s often a time and priority issue — spend too much time on one problem, and five more pile up, possibly more critical than the “un-reproducible” one. Sometimes a decision has to be made that a problem is not severe enough to warrant the time it would take to figure it out. Those are real decisions to be made. Declaring something to be “un-reproducible” after a single quick attempt to reproduce it is certainly not right — however if the IT person tries all sorts of things and can’t reproduce it, what can he do?
In your example of *you* being able to reproduce it, a good IT person could/would take that as their starting point. Of course, that means kicking you off your computer for an undetermined amount of time, taking time that might be better spent on other problems, and it’s of course likely harder because your workstation doesn’t have the tools the IT person’s workstation may have, so even if you can reproduce it consistently, the problem may still be marked as “won’t fix”. Not because the problem is not important to fix, but yours is not a “show stopper” and other problems are more critical to the productivity of the company.
Without the constraints of time and problem prioritization, a true computer geek would like nothing better than to figure out how to solve a problem… if they can get any kind of a handle on the problem at all. But a truly non-reproducible one may not give any kind of handle to begin with.
August 13th, 2011
“you can look at the program with a debugger…”
Yes you can. Unfortunately, running a debugger can often change the behaviour enough that the problem goes away.
August 12th, 2011
You are correct. It is a reference to this comic: https://www.xkcd.com/583/
August 12th, 2011
The image text isn’t referencing the earlier comic. The earlier comic and this one are simply referring to the same thing, ie bug reports and programmers being unable to reproduce the error.
In the earlier comic, however, the ‘could not reproduce’ line is used in a double entendre. Whereas this comic is using it straight, the humor being that programmers sometimes can’t reproduce the error reported, because they aren’t using the program in the same environment their users are, especially if they only allot a small window of time to look at each bug and try to reproduce the error.
August 12th, 2011
The problem it’s mainly identifying is that people can review an app without actually knowing whether it works when it’s supposed to. Rather than the app store highlighting reviews of people who have the most important and relevant experience with the app, it just averages everyone’s ratings all together.
August 12th, 2011
That’s one thing I like about Amazon’s review system — you can rate the reviews themselves as useful or not, so an incredibly useful review like the 4th one here would be easier to spot.
August 12th, 2011
I agree. A secondary point of humour is the author’s choice of app which either contemplates, or at least leads me to ponder the ever-troubling dilemma that I’ve experienced and I’m sure others have in which you have a system (be it an app, a contraption you build, or whatever) to accomplish an important task but having no way to test it. The dilemma is whether or not to test it during the actual scenario; because that’s the only way you will know if it works, but if it fails, you’re screwed.
I’ve had this experience with, for example, “my iPhone is the only alarm clock I have available; but will my iPhone battery last until morning so that my alarm will wake me up?” I won’t know unless I try, but if I try and it fails, I won’t wake up and I’m screwed.
In this strip, I can imagine someone purchasing such an app would be faced with a similar concern – “I’m spending money on this app but I have no idea if it will actually work if the time should come that a tornado is approaching. At which point it’s too late.” It may perhaps be inferred that the first three reviewers are ignorant of this concern, and simply assume the app will do what it is designed to do when the time comes. It hasn’t happened to most users yet, as a tornado is a rarity, but they assume it will work, and in the meantime, they like this app’s features a lot, so they rate it [especially true of apps that have the popup “rate this app now!” that many apps include.
The only reviewer who has actually been put in the tornado situation has unfortunately discovered that the app, while designed wonderfully from a UI/features POV, doesn’t actually properly perform its only function.
The irony of the strip, as mentioned by Jessica is that the three reviewers who have not even realized that the app is useless, have been given three times the weight in the average rating of the reviewer whose rating would be of the most importance to a prospective purchaser.
This reminds me a bit of the con artist who sells raffle tickets for $1 each to 1000 people with some grand prize that the con artist doesn’t have. at the end of the day, the con artist placates the ‘winner’ by refunding his $1, and runs off with the other $999 (as 999 other people don’t relize the raffle was a con). There’s no evidence in the strip that the developer intentionally is selling an app that doesn’t work (or even that the app costs money), but it’s the same situatuion that of the 1000 people who bought the app, only one will ever be in a position to realize the app doesn’t work.
August 15th, 2011
Indeed, like “prayer”.
August 12th, 2011
The developer is a Star Wars fan; 2187 is the cell Princess Leia is held in A New Hope.
August 12th, 2011
The ratings theme probably refers to the “bobcat” strip http://xkcd.com/325/ — I believe the alt text refers to average star ratings you can juggle by only packing a bobcat in the packages you ship 1% of the time. (to my memory – can’t see alt text on this browser.)
August 12th, 2011
Close! You can send a bobcat to one in thirty people and still have a 97% positive feedback.
August 13th, 2011
http://xkcd.com/859/
August 15th, 2011
Just FYI, there appear to be 2 Toms here… The “Tom” who posted the previous comment (I) did not post this one. I could choose a different nickname, if that would help?
August 13th, 2011
The alt text is a reference to this comic http://xkcd.com/583/ where they have a problem, but can not reproduce because they can not reproduce.
August 14th, 2011
I did a small spoof on this today, if you are interested.
http://goo.gl/X8dQ1
August 15th, 2011
Maybe phrase ‘did a small copy of this’ would be closer to the truth..
spoof (spf)
n.
…
3. A gentle satirical imitation; a light parody.
…
August 15th, 2011
Yeah, “spoof” was the wrong word. I debated that, but obviously not long enough! Thanx.
August 15th, 2011
PS — I changed the post. Thanx again.
August 16th, 2011
Well, either a) the 1 star get’s too much say, or 2) the 3 reviews were on the wrong aspect of the app.
If I downloaded an app to warn me about tornados, it would be far more relevant whether it were actually able to warn me than how the UI looked.
I recognize, that as a developer, I tend to assume that the 1-star rater probably blamed the app for something that was out of the developers control (i.e. no tornado warning were issued).
But as a user, I have to admit, it’s a pretty useless app if it doesn’t actually warn me about tornados – regardless of whether the developers are able to test it in real-life-scenarios.
So… the culprit isn’t that it “only” gets 4 stars, it’s that it gets 4 stars even though it doesn’t work.
August 16th, 2011
Darn, my point was already made by TH – just ignore it