Wednesday, December 27, 2006Google "disappears" sex blogs? UPDATED
UPDATE (Dec. 28, 12PM ET): Readers have been writing in to say that whatever happened here, the issue appears to be resolved for now -- at least for the specific sex blogs cited in this post. I haven't verified this, but if it's true, great. Still, the bloggers' concerns still remain: we don't know exactly what happened or how or why, or who or what "fixed" it, or how or why. When all of us rely on one single service to access so much of the information we need each day, and the company behind that service doesn't have to be transparent to its users, problems like this are inevitable. And they can have real-world consequences for "little guy" businesses on the 'net, and for anyone seeking information online.
A number of bloggers who cover topics related to human sexuality say they've suddenly disappeared (or at least been deeply demoted) in Google search results. Popular "indie" blogs that deal with nonfiction sex ed or indie alt-erotica fare (Tiny Nibbles, Comstock Films, ErosBlog, and others) have suddenly vanished to much lower ranking in relevant search results -- even when you have Google's adult filter turned off. And in practical effect, being buried is just as bad as being filtered out entirely.
The drop appears to coincide with changes Google recently made to their keyword ad program, AdSense.
But here's what's so crazy: now, when you search for the missing indie sexblogs, the top results are icky porn spam blogs (aka "porn splogs").
What's disturbing to me (besides the harm it's done to small businesses over the holidays) is that Google's snafu seems to have dropped more sex-positive businesses (that focus on accurate sex ed) than big-gun, mainstream adult businesses (that sell unsafe sex toys and skanky product). To me, this also shows the huge problem with having a monoculture wherin a single business is depended on to provide a communication service. They screw one thing up, and an essential feature (like access to accurate search result information) disappears. (...)Link to full text, and here's more on Fleshbot (includes non-worksafe image): Link.
It used to be that if you searched for Good Vibes, Comstock Films, Tiny Nibbles and Violet Blue, you'd get each of these sites in the top rankings or on the first page (SafeSearch off, natural results). No more. However, if you search for Adam and Eve or Vivid, you get the mainstream sex toy and porn sites on the first page.
So, as Tony Comstock explains eloquently in his post about how his indy film business has been seriously affected, if people read about any of these entities in a magazine and then go to search for them, searchers don't find what they're looking for. Unsafe sex toys and "interracial" porn from huge companies, no problem, but books and toys from women-owned sex-positive healthy sex businesses? No.
And Valleywag writes:
Some word Violet wrote probably triggered a Google ban, inadvertently, but the search engine's rules are opaque, as is the procedure for an appeal against deletion. You think there are other search engines, so that's okay? There are no other search engines.Related BB posts:
• Wireless hacker pleads guilty, Google searches evidence
• Google and NASA sign partnership agreement
You've probably heard this from a million people already, and the Boing Boing post touches on it tangentially ("you'd get each of these sites in the top rankings or on the first page "), but just in case: tinynibbles.com isn't *gone* from the google search, it's just somehow been demoted to around number 50.
This doesn't change the effects on the sites, and I agree that what's happened is a bad thing, but just for precision's sake, it can't be an out-and-out ban. I don't know enough to speak authoritatively about the math of it -- and I'd guess you are surrounded by people who can -- but the first thing that comes to mind is a change in the ranking algorithm.
Damon Haidary says,
Funny how they censor query results but porn pops up in Google Image Labeler...Renato Silveira says,
When I was tasked with tagging this image (JPEG Link, made worksafeish by blurring) my partner failed to match the 8 explicit labels I chose. This begs the question of how accurate tags are if the tagger has ethics.. but I'll leave that to another post.
I own a small-friend-comunity-Brazillian-forum and i've been noticing a heavy simultanous browsing through all, or most, of the threads at the same time by the IP 220.127.116.11. Really eating bandwidth like crazy. So, I looked up the IP, got the GOOGLE MEGA RANGERS CORPS info. So, I banned it. Happy bandwidth, unbanned and all hell broke lose again. I was wondering what the hell google is trying to find, because it just keep visiting threads over and over, couldn't be just indexing. That just makes sense, they are parsing, dividing, really doing some work around it. What for? No Idea.Nebe Barnett says,
So, if you want to wait and see what the hell is going on, you should ban google, like I did.
It looks like Google may have cut rankings for blogs in general, not just sex-oriented ones.Matthew Frederick says (12-28):
That's a fairly well-known site which is hosted on Google blog system. The front page is gone but some individual posts from the site come up in the top 10 results.
I'd be curious to see if there is a common thread (besides sex) that links any of these derankings, like what blog software/features they use.
My guess would be they're working on filtering spam blogs and turned it up too high.
Problem resolved? At least for me, the Google searches for:Frank Scheetz says (12-28),
all return the appropriate sites as the first link. The search for
good vibes doesn't return the blog in question, but that could be Google's standard re-ranking, nothing necessarily nefarious about it.
I was reading your story about Google search. Lately if you do a search for serial numbers or cracks you get the same results,however when you click on a result Google does not allow you to go to the site. Instead it warns you not to go to the site you requested. I use Firefox and I do not have any filters running.Karl Elvis says (12-28)
Just a note on the 'Google "disappears" sex blogs' things; it may be fixed for some sex blogs but for example Chelsea Girl at prettydumbthings is still seeing the issue; google for pretty dumb things and you get links to her blog, but not her blog. So while google may be hand-fixing high-visibility blogs they're not fixing the root cause. Here's CG's entry on it: Link.
- - - - - - - - - - - -
UPDATE, 344PM ET
Danny Sullivan, Editor in Chief at Search Engine Land says,
Some people feel Google's not transparent. Sadly, many of these people seem unaware that Google provides heaps and tons of information on their site about ranking. Is there an issue with a site? Like it's no longer showing up?And Violet Blue responds:
I'm willing to bet Violet and the other bloggers involved didn't make use of thinkg like Google Webmaster Central, http://www.google.com/webmasters/, where they can discover if there's a site penalty that might have happened or gotten in touch with Google. Google's unique in this, by the way. Yahoo doesn't offer such services. Microsoft doesn't. Ask doesn't. Google's added them based on webmaster demand and interest. But no one writes about that.
Instead, what typically happens in these cases is that someone (especially a prominent blogger) just screams loudly for help. And that works, but let's not pretend that any "little guys" were involved here. These are all well known sex blogs, right? All well connected sex blogs able to tap into the blogosphere for the immediate attention that the real little guys can't get.
Now if you really want to talk little guys, head over here: Link. It covers how in particular, you've got one little webmaster in Denmark who at one points declares that ALL of Denmark sites have been wiped out by Google, which doesn't care. Then you've got two Googlers on Christmas Day trying to get the person advice and assistance.
This is all happening in a group Google monitors specifically to help the "little guy." But it's hardly noticed or even that known.
I definitely appreciate the issue that little sites face, and I recently covered one idea on perhaps how Google might be able to provide even faster support, here: Link.
I'm also looking much further into the situtation to see if I can understand more about what happened. But the short story is, lots of sites have lots of glitches that can hit them for a short period of time. This sounds fairly typical and generally gets resolved quickly on its own.
The longer story is Google's done a huge amount of work over this past year to provide support for little webmasters, a story that no one in the general media seems to care to report when I get the usual call to talk about some story in the works on how Google doesn't care or is out to screw the world.
first, I'll say that it is indeed excellent that Google has webmaster tools, and that no, other search engines don't have these options available for webmasters. I'm beyond ectsatic that they fixed my ranking; it was totally unexpected. (Chelsea Girl emailed me, and Pretty Dumb Things is still as it was, dropped in rank).Andrew Ferguson says,
I really don't think that Google had any ill intent when this snafu happened; I honestly believe that Google changed something that affected a number of a *certain kind of blog/website* that caused a number of us to be dropped from our regular rankings. it wasn't just me. my thoughtful guess is that Google was trying to implement something to prevent spam exploitation of their algorithims and /or raking system; this seemed like something that might be done to circumvent certain kinds of attacks on weaknesses. but it seemed to me that it backfired; it dropped the kind of sites that Google would want to keep and pushed the splogs (and people syndicating the original content) to the top in natural results.
it's a simple, though incorrect argument to say that I wrote something naughty that got me penalized. I didn't write, post, or change anything on my site any different than usual, nor did the other webmasters who had the exact same problem at the same time. this points to the entity in control of the rankings, and not the individual(s) causing the change. Google's behavior changed, not ours.
as for the "little guys" comments -- okay, I'll admit that I'm not one of the little guys so much: I'm certainly notorious in the blogosphere and with over a million page views a month it puts me into a bigger league, but once I'm out of the blogosphere, I'm not a big guy at all. and the others affected by this, even less so. and again, Fleshbot -- who is certainly one of the "big guys" when it comes to blogs and sites in general -- was not affected by this. and how do we know if sites outside of this community (who have similar problems with spammers using their keywords) were affected?
it's good to find out about groups like seroundtable, and while I already knew about Google's webmaster tools, I'll bet that lots of others don't. the upside here is that more people will find out about that, but in my case (and the case of all the other sites affected this time) it made no difference. the heart of the issue here is a need for transparency from a system that has great power over an individual. Google could announce when they make changes that affect us. Google could also have public Q/A. my guess is that they are working hard to keep porn splogs in their place, but knowing who's who and what's what in porn and sexblogs is a pretty specialized skill, believe it or not, and I'm guessing that whoever might've tested their sex searches didn't know the difference between Comstock Films and your average skanky porn film site.
in the end, you really compare Google to Yahoo, Ask, Microsoft -- in any argument. why? because they don't even begin to compete with Google. practically speaking, there *is no other search engine*. and when someone (or something) has so much power over individuals and no transparency about their process... you live in constant fear that you're going to do something (you have no idea what) or that something will change (you never know when, or what), to make you disappear.
I've been unable to find any proper news or reportage on this, but I think the search engine is in the process of a major re-index or an algorithm revamp.Tony Comstock of Comstock Films (an adult film company that specializes in explicit fare starring real-life couples) says,
I've noticed my own search engine traffic go *insane* today. In the space of about an hour, I quadrupled my normal daily traffic through organic search engine hits only. And on a single post (on Francois Brunelle's Doppelgangers project, I think you guys covered it too) as well.
I assumed it was a Google re-index, but when I dug a bit deeper, I noticed that MSN and Yahoo spiked dramatically at the same time. Almost to the minute, actually.
I'm not quite sure what's happening here, but I think the two events could be somehow related.
Could the recent earthquake near Taiwan that downed a lot of major cables be affecting indexing (since the sites will be hitting a lot of 404's?)
I think it would be helpful to know through what process Google miscatagorized sites like mine or PrettyDumbThings, or TinyNibbles.com, sites that are virtually 100% original content (and content that has received many accolades for its quality and originality) as spam.
It's hard not to suspect that Google negatively weighted sexual language, especially colloquial sexual language, with little or no regard for the context in which it was being used.
It would also be helpful to know what process restored TinyNibbles.com and ComstockFilms.com, but so far has left PrettyDumbThings out in the cold. Is everyone who writes about sex in an adult way, using adult language going to have to petition Google for reestatement? Or will Google be willing to refine their algorithyms so that people who are looking for what PrettyDumbThings (and site like it) offer will be able to find them?
For me it's more than an academic question. In post production we have a couple whose sexual highjinx include some very lusty and raunchy fisting and ass***king [ anal sex ]. When it comes time for me to write about this film, would I be advised to tread lightly? To restrict the language I use to either medical or euphemistic?
I guess time will tell...