AllThingsD has an interesting little story about some recent Forrester research on internet usage. The gist is that Forrester asked people how much time they spend online and the number went down from 2011 instead of up. This would be shocking if it were true, but it’s not, and presents an interesting case of the problem with self-reporting in research. The article sums it up like this:
“Despite the fact that they always have connected devices and are always online, they don’t really realize they’re online,” said Forrester analyst Gina Sverdlov. “They’re using Google Maps or checking in on Facebook, but that’s not considered online because it has become such a part of everyday life.”
It’s actually amazing to me we don’t spend more time talking about the issues with self-reporting because this stuff happens all the time. People classically overreport the good things about themselves and underreport the bad stuff (if you compared the number of New Yorkers who say they read the times with the traffic/newspaper sales something doesn’t add up). On top of that, though, we ask seemingly innocuous questions that actually turn out to be confusing for regular folks (I don’t know how I’d answer how much time I spend “using” the internet either).
There is something important here as the web continues to shift to an ambient medium that is just tied into our lives. I think for those of us that have been living online for awhile now it’s no surprise, but it’s interesting to see others catching up to that way of thinking.
Nobody "Goes Online" Anymore - Liz Gannes - News - AllThingsD
Hi, I'm 
The best self-reporting thing I remember is from the NY Post during the 2008 primary, where something like 75% of people said they would vote for a woman for President; but they believed only like 25% of their neighbors would.
@Ian: Excellent point. Over 50% of people think they’re above average as well.
What’s your theory on Google’s revenue going down as a result of people clicking ads less? To your point, are we just changing the way we use the Internet?
@Lewis: I think that’s pretty simple: Google’s ad products aren’t optimized for mobile. The use case is still very different for search on the go and on the desktop and they’ll need to adapt the way they think about things to get those back in order. I”m not worried about them.
Your raise a good point. In fact, in the past year I’ve spoken with two actual Forrester research VPs — off the record — about their concern with their firm’s reliance on self-reported data. They both acknowledged their trade needs to balance self-reported survey data with hard, observed behavioral data. I think balance is the key because self-reported data still has its place, both with understanding facts as well as attitudes. If we had no faith in the ability of humans to accurately and truthfully report things they observe, then things like our democracy and courts system would altogether fail. It’s balance — or checks and balances — in data collection that matters. I believe the market research industry is moving from one of contrived artificial laboratory to one consisting of triangulation of digital breadcrumbs in the real world. Instead of getting data to conform to the scientist, the scientist must conform to the data.