Listening to users
In the software community, people hold strong opinions on the subject of listening to users. Some feel that users are an essential source of information for making successful products, as evidenced in the customer development methodology, and seek to involve users deeply in product development. Others believe that users don’t know what they want, invoking the quote attributed to Henry Ford, “If I’d asked customers what they wanted, they would have said ‘a faster horse'”. Some say that user needs are unknowable except through the lens of a marketplace, where people choose in aggregate which products suit them best, and customers “vote with their wallets” (anything else is “anecdata”).
Regular readers will not be surprised that I believe they are all right, but only in certain contexts. The right strategy for involving users in product decisions will depend on factors related to the product itself, the market, and the product development method being used.
One of the most important is the life cycle stage of the product: is it a new and rapidly evolving concept, or a mature commodity, or somewhere in between? Simon Wardley explains this well over on his blog, so I won’t rehash his points here, but will add a few of my own.
If what we’re looking for is inspiration for a new product, it’s here that Henry Ford was right: users generally won’t hand you a complete product vision on a silver platter. They’ll frame their input in terms of what they know, and the choices already available to them. However, this doesn’t mean that users don’t have a role to play in this instance: watching users can be a great source of inspiration. It’s the combination of domain knowledge and passionate imagination which triggers the creative spark. Henry Ford applied his engineer’s interests to a problem which was evident all around him.
If our goal is to test whether a new product is a good fit for its users, there is no substitute for user feedback. We can guess at whether there is a fit, and our intuition may be good, but users are the ultimate judges, and we don’t know if we’re right or wrong until users evaluate it. So ask them! By engaging in dialogue with individual users, we can learn unexpected things which will help to refine the idea. If we don’t find what they think until our new product is released, we risk making something that no one wants. Why wait until it’s too late? It can be challenging to extract useful feedback for a product which doesn’t yet exist, but this effort is well worth it to avoid wasting much more effort in software engineering.
When our objective is to incrementally improve an existing product, individual anecdotes can mislead us. A given change may be an improvement for one user, but a disaster for another. What we want to know is whether the new version is better for the population as a whole, and in this case, we do well to rely on data. There are pitfalls here as well, of course. We need to choose our questions carefully, and realize that users will often resist any change: not because they’re stodgy by nature, but because they have to invest effort in adapting to the change. I think of incremental improvement as a joint investment made between product developers and their users, to improve the whole system of people and technology for the better.
By choosing the right tool for the job, we can make better decisions, improve faster, and ultimately solve the right problem for our users.
I think that a good way to describe the relationship between the developer and the user is to say that the developer rather than “listen to the user” should “speak for the user”. Of course, that entails first listening to users to learn what are they needs, but it emphasizes the fact that the developer is in a better position to define the solutions for the user’s needs and champion them. It also strongly implies the commitment developers should have to their users, rather than place them in a position opposing the users.
Tom Berger
March 7, 2011 at 13:53
I think I understand what you mean, but “speak for the user” could carry the (problematic) connotation that the developer is more authoritative than the user. As you say, it’s important to listen, and it’s good to be an advocate for users, but saying it this way could give the wrong message.
For most developers, the listening is the hard part, so I prefer to emphasize that. :-)
Matt Zimmerman
March 7, 2011 at 13:56
Right, “advocate” is probably a better verb to use in this case.
Tom Berger
March 7, 2011 at 13:59
There are those who believe that you should develop for just yourself, ignoring what you *think* you know about your users’ needs. That way, you take a lot of the guesswork out of the process and have a better chance of providing a consistent product.
It’s also way easier to know when you’re happy with a product yourself than it is to guess when “your users” are happy with it. “Your users” won’t all agree with each other anyway, and there’s a pretty good chance that if you like it yourself, someone else will, too.
Soren Hansen
March 7, 2011 at 14:40
I think there is something to this. My only disagreement is that if you are developing an application for a field you don’t work in, it’s important to know the use-case, the processes and the pain points. If it were me, I would write an application with workflows that supported testing the application because that’s where my pain point is.
Liam Johnstone
March 7, 2011 at 16:06
The more similar your users’ needs are to your own, the more you can trust your intuition about what’s right.
It’s a bit of a shortcut, which I think is useful in smaller scale projects. It has its pitfalls, though, like optimizing too highly for your individual needs, or stopping when it’s “good enough for me”, or finding that other people can’t figure out how to use it. I think a lot of free software ends up in each of those situations.
I think it’s commendable to make something which is useful to others, even though they’re different from you. This transforms a software project into something greater, which is about working with people, not just code.
The worst case would be making software for a hypothetical user, different from you, who turns out to actually not exist. :-)
Matt Zimmerman
March 8, 2011 at 15:02
The approach certainly has its limitations. People working as programmers for companies that build software for a completely different industry don’t have the luxury of the same sort of a priori insight into the problem space and even if they did, there’s not the same vested interest in a convenient user experience. In that sort of scenario, you’ll have to rely on user stories, etc. and a lot of guesswork.
In free software, though, where you typically scratch your own itch, I think it’s very, very useful. Solve your own problems and solve them well. Doing it well is important. Cutting corners by only providing an arcane UI for instance will certainly deter users. Doing it well is a powerful way to convince people that your way of attacking the particular problem is good, and failing that, doing it well will often mean that it’s more approachable for people wanting to bring it in a slightly different direction. The old saying “write your code as though it will have to be maintained by an axe wielding maniac who knows where you live” is very applicable :)
I agree that making your software more generally useful is a good thing. I just believe that making it specifically useful and then branching out is the way to reach that goal. With luck, someone with different opinions and preferences than yourself will come along and do those parts for you, and I think this is more likely to happen if what you start out with is consistent.
Soren Hansen
March 10, 2011 at 21:53
I have definitely taken this tactic in my software projects — two are out there, but I’m the only user as far as I know. If somebody else finds it useful, and can help me, then that’s excellent, but I’m not going to design for anything but my own needs. These are scientific applications, however, and my thoughts might be different if I were designing something more generally used, like a media player. Someone out there might want to use a protocol that I don’t use today.
trashbird1240
March 8, 2011 at 17:16
Please do not use bold text for emphasis. My eyes will thank you.
Bill Johnson
March 7, 2011 at 15:13
What do you suggest instead? Blogs are often scanned, rather than read end-to-end, and I want my posts to be easy to scan.
Matt Zimmerman
March 7, 2011 at 15:15
The HTML “em” semantic tag normally italicizes text; in the days of manual type-setting from hand-written or type-written originals a printer would set underlined (i.e. emphasized) text in italics. Keeping the font weight the same aids readability.
Andrew Johnson
March 8, 2011 at 16:35
True, but italicized text doesn’t jump out of a paragraph, which means it doesn’t help with skimming. I’d love a compromise which aided skimming but didn’t interfere with readability.
Matt Zimmerman
March 16, 2011 at 21:57
I myself don’t have an issue with the bold text, but you could use “em”, and define a light background color while keeping the font weight the same. Gives it sort of a highlighter effect.
Maybe a very pale yellow or green.
seth
August 7, 2012 at 12:59
[…] il favore di nuovi utenti. Nel suo blog Matt Zimmerman ha scritto un post sull’argomento (Listening to users), scrive: “If what we’re looking for is inspiration for a new product, it’s here that […]
[mp] Perche’, come e quando ascoltare gli utenti « Idl3's Blog
March 7, 2011 at 16:20
unfortunately canonical is not linstening users opinion. I use ubuntu for above 6 years and i was an active member of ubuntu greek comminity. The last two years i have see ubuntu to trasform from a community distro to a one person decision distro.
vista killer
March 8, 2011 at 12:02
I think you’re right that we aren’t doing a good enough job of listening and responding to feedback from users, certainly on some topics. I’m sorry to hear that you’ve been disappointed by this.
In this post, I’m sharing my personal views on a principle of product development, not speaking on behalf of Canonical on this or any other topic. The question of how a community and a for-profit company can work together, using Canonical as a case study, would be a good subject for a future post.
Matt Zimmerman
March 8, 2011 at 14:45
[…] was reading over the ever wise Matt Zimmerman and his blog post about Listening to Users; in it he argues that user involvement is a nuanced subject and which approach the developer takes […]
DoctorMo's Blog » Blog Archive » Freely Doing What Users Want
March 9, 2011 at 04:05
You know I would say that just responding to users makes a huge difference. Even if you just say “thanks” because now they feel more personally invested and feel like they matter more. I use Boxee and read their blog and I must say they respond to at least 80% of their posts when a new blog goes up and they get hundreds of responses.
As far as Ubuntu is concerned I think I’ll jump ship to Elementary OS when it is released (the 2nd version is what I’m really looking forward to Jupiter+1). I don’t want to knock Ubuntu until I try it, but Ubuntu will change stuff and not polish it up (for example, I can’t go online via the MeMenu which is frustrating). Ubuntu needs to simplify a lot more. For example, multiple desktop is for power users and not normal users and thus should be hidden by default (power users can figure out how to enable it). Kill the MenuBar (i.e. File, Edit, View, etc).
In any case, I have a lot of suggestions, but I feel like Elementary OS is really getting it right. Now people will disagree, of course, but the perfect desktop is in the variety of desktops and not one single desktop, so choose which one is right for you.
Also, Ubuntu Brainstorm is a joke and I stopped using it. Wouldn’t it be nice if there was a release where they only added things that were on Ubuntu Brainstorm?
Lastly, I think Canonical needs to work more on Applications and not just things that tie into the OS. Look at Elementary OS which is working on a new mail app, contacts, dictionary, and more… these things alone can really “sell the complete package.” In any case, always like your blogs… thanks.
Eric Pritchett
March 9, 2011 at 18:33
I should note I’ve been using Ubuntu since 5.04.
Eric Pritchett
March 9, 2011 at 18:40
You’ve said more than I can comfortably respond to in a blog comment, but a few things:
We are trying to do much better in responding to input via Brainstorm, see https://mdzlog.alcor.net/2010/12/10/ubuntu-brainstorm-top-10-for-december-2010/
Many people in the Ubuntu community do a great job responding to comments, forum posts, and so on regarding what’s happening in Ubuntu, which I’m very grateful for.
There is certainly a need for more and better free software applications, so if Elementary produces a new wave of killer apps, I’m sure we’ll end up amplifying their work by distributing the apps to Ubuntu users as well.
There are complex tradeoffs involved in how Canonical invests, because it’s important to us that Ubuntu remain free. It’s only an “investment” if you can earn a return on it somehow!
Matt Zimmerman
March 9, 2011 at 18:47
I understand completely, but what I don’t understand is how what used to be small startup like Dropbox can come up with an awesome product and how Canonical can’t seem to develop Ubuntu One right where it’s easy for the user. I mean just copy exactly what Dropbox does first and go from there (it seems like it’s been years since U1 came out and I hate to say, but personally I much prefer dropbox WAY more).
Here’s another investment idea, copy (first and then make better) what Tonido, Amahi, and Pogoplug are doing which is using the Sheeva plug (takes 3watts on idle) tie it in to ubuntu one, let people remote into their systems via vnc, and offer a ddns forwarding service as well as many many other services). I would imagine Canonical could make a good amount of money doing that. More info at:
http://www.amahi.org
http://www.tonidoplug.com
$99 Sheeva Plug Dev Kit at http://www.globalscaletechnologies.com/p-22-sheevaplug-dev-kit-us.aspx
If you let me know who to email within Canonical I’ll let them know about this, but I think something like this could really take off.
Eric Pritchett
March 9, 2011 at 19:24
Re; listening to users…is a lot more complex than this sound bite, which is pretty good.
http://bigthink.com/users/guykawasaki?utm_source=Big+Think+Weekly+Newsletter+Subscribers&utm_campaign=14c779c0a5-Wednesday_Newsletter_Guy_Kawasaki_March_9_2011&utm_medium=email
GregK
March 10, 2011 at 19:48
Great minds think alike! Thanks for the link.
I don’t worship at the church of Jobs, and I think Apple’s success is attributable to more than one person’s personality traits, but nonetheless there is wisdom in what Guy says.
Matt Zimmerman
March 10, 2011 at 21:28
As a blind Linux user, I wanted to point out that, so far, the 2 main developpers of Orca, the screen reader for Gnome, have been sighted people.
Yes, sometimes it has shown, closing your eyes to test the software just isn’t the same thing, but mostly a very deep involvement with the user community has worked very well.
André
March 16, 2011 at 11:57
Matt it’s great to read about your enthusiasm for watching & listening to users. As a UX Researcher I can assure you that qualitative data (what you refer to here as individual anecdotes) can be quite helpful. I’ve written about it here:
http://userlens.com/post/29015352888/why-you-should-believe-user-tests-that-involve-only-7-9
userlens
August 8, 2012 at 19:07
I think there’s a huge difference between anecdotes (which typically reflect the experiences of a single user) and user tests based on the experiences of 7-9 users. So I think we agree here. :-)
Matt Zimmerman
August 9, 2012 at 00:17