Believe it or not, Google wants you in their index.  In fact, they're willing to go considerably more than halfway to help you structure your site and your link pages in a manner that conforms to their ethical standards and self-imposed mandate to give web surfers links to pages untainted by deceitful SEO schemes and search-engine webspam. This is not merely altruism on their part.  One of the main reasons for Google's rocketing to the top of the search-engine totem pole was the simple fact that its index had more listings than anyone else's.  Google success has been based on inclusion, not exclusion.

Getting to the top is one thing, staying there is another.  The executives of Google are well aware that they are as vulnerable to competition as the rest of the SE industry.  What they did unto others could someday be done unto them.  To remain the number one choice of people searching for information on the internet, they have to index the largest possible number of sites consistent with their fair-usage standards.  In a perfect world, Google would like every single website on earth to be included in their pool of potential search returns.  But the cyber-world is as far from perfect as the dirt-and-turf one.  There are always people trying to bend the rules, cross the line, or - to put it bluntly - cheat in an attempt to improve their rankings.  So Google, like all the legitimate SEs, has a blacklist.  Bend the rules a bit too much, cross the line too far, try to spam their system with redirects, doorways, and high volume irrelevant link-farm compost and some of your pages - or your entire site may wind up on that blacklist, totally invisible on Google return pages.

There are always people trying to bend the rules, cross the line, or - to put it bluntly - cheat in an attempt to improve their rankings

But - and it's a big but - Google doesn't really want sites on their blacklist.  They'd much rather scamsters change their "evil" ways and they're more than willing to welcome back offenders who do so.  To this end, Google has recently changed its long-standing policy of banning websites without advising the operators why they were exiled.  They are also being more forthright about what black-listed webmasters must do to return to the fold.  These days, the Google Search Quality Team (GSQT) notifies site owners when they are placed on the blacklist, details the reasons for the blacklisting, and provides instructions on applying for re-listing.  A typical GSQT message:

While we were indexing your webpages, we detected that some of your pages were using techniques that were outside our quality guidelines, which can be found here.  In order to preserve the quality of our search engine, we have temporarily removed some webpages from our search results.  Currently pages from [removed url] are scheduled to be removed for at least 30 days.  Specifically, we detected the following practices on your webpages:  On [removed url], we noticed that pages such as [removed url] redirect to pages such as [removed url] using JavaScript redirects.  We would prefer to have your pages in Google's index.  If you wish to be reincluded, please correct or remove all pages that are outside our quality guidelines.  When you are ready, please submit a reinclusion request at [link] You can select "I'm a webmaster inquiring about my website" and then "Why my site disappeared from the search results or dropped in ranking," click Continue, and then make sure to type "Reinclusion Request" in the Subject: line of the resulting form.

Unfortunately, Google does not currently inform webmasters if their sites have been "sandboxed" rather than blacklisted.  In fact, Google does not publicly admit that a sandboxing filter exists and it has been suggested - though not by Google - that the "sandbox" is a fiction created by unscrupulous SEO experts as a scapegoat for poor - or even negative - results from their site tweaks.  However, the evidence is clear - or at least as clear as anything ever is in the arcane and high-security world of search-engine strategy -- that Google has maintained a "sandbox" (though it may well be called something else internally) since at least Q1 of 2004 and that the "sandbox" filter is largely based on Google's patented method of analyzing links.

Google does not publicly admit that a sandboxing filter exists and it has been suggested - though not by Google - that the "sandbox" is a fiction created by unscrupulous SEO experts as a scapegoat for poor - or even negative - results from their site tweaks

"I don't think there's a real question that it exists, the effect has been broadly noticed. The thing is that Google won't confirm it, as they don't talk about the algorithms they use and so no one knows precisely the parameters that trip the 'sandbox' filter," says Michelle Delio, an Internet consultant and frequent contributor to Wired News. "New sites, especially those that have been heavily optimized in an attempt to appeal to search engines, are much more likely to be sandboxed than old ones," Delio adds, "but posts on online message boards indicate that it's also happened to sites that have been around for a while."

"There's nothing you can do to get your site out of the 'sandbox' except correct the problems that got it there. And, once the problems have been fixed, there's no way to speed up the re-ranking process.  If you must appear high on Google while you're waiting to get un-sandboxed, consider buying advertising until your site is released.  And remember even if you are sandboxed, your site can still rank high in the other major search engines, such as Yahoo and MSN, which - as of yet - have not implemented sandbox filters."

Because sandboxing appears to be directly related to the quantity, quality, and method of acquisition of a site's links, it is important to stress the difference between being sandboxed and being blacklisted.  A blacklisted page or site is removed from the Google index and will not show up on any Google search return regardless of how many entries deep you look.  Sandboxing, on the other hand, does not remove your site from the Google index or search returns.  What it does is drop-kick a site a few dozen, hundred, or thousand spots further down the returns list than it would be otherwise.  If you suspect your site has been blacklisted but you haven't received a "courtesy" notice from Google, the easiest way to check is to do a Google search for your complete domain name - i.e. http://www.yoursite.com.  You can also run the Google toolbar, surf to your site, and check your page rank.  If the page rank box is gray, you may have been banned.  Blacklisting is very rare, generally unrelated to links or link pages and is, as noted in the GSQT e-mail above, almost always fixable.

Since Google does not officially admit the existence of the "sandbox," it is essentially impossible to be 100 percent certain that a specific site has been sandboxed. And don't bother to inquire how Google's algorithms determine the severity of the penalty, nobody outside Googleland knows.  It is, of course, entirely likely that nobody inside really knows either.  The "sandbox" filter is probably set up to weigh dozens of factors, apply a numerical value to each, and reach a decision about if and how much of a penalty should be assessed.  Chess playing computers, such as IBM's Deep Blue, work exactly the same way and there is no one - not the team that wrote the program nor the greatest chess player who ever lived - who can accurately predict exactly what the machine will do ten moves down the line.

Since Google does not officially admit the existence of the "sandbox," it is essentially impossible to be 100 percent certain that a specific site has been sandboxed

The best thing to do if your site takes a sudden tumble in the Google rankings is to take a close look in the mirror - in other words, ask yourself some serious questions and honestly answer them - before assuming you've been dumped in the gravel.

1. Do you consistently employ sound linking practices?
2. Do you studiously avoid links leading to "bad neighborhoods" and promiscuous-linking practices such as setting up 3-way link networks and multi-sites solely intended to artificially inflate your link count?
3. Do you avoid buying links in volume from sites that guarantee higher search engine rankings?
4. Do you avoid linking to "thin affiliates"?  (sites that do not publish any original content and publish significant affiliate links)?
5. Have you successfully resisted the temptation to "steroidize" your link pages via high volume full duplex link farming, links from guest-book pages, and other ways of, in Google's words, "gaining links from documents without editorial discretion on making links?"
6. Do your outbound links lead to sites containing content relevant to your own site?
7. Are your outbound links end-user oriented and useful for the end user?
8. Do you avoid inbound links come from low quality, low content, or sites totally irrelevant to your own site?

If the answer to one or more of those questions is "no" you could very well be in the "sandbox."  If so, take heart.  Getting out isn't terribly difficult and is automatic (based on our experience) once you correct your suspect linking practices. LinksManager, with its fail-safe anti-spamming technology, editor-based link-management structure, and dead-link checker, actually makes complying with Google's link policies a virtually painless procedure.  Noting that LinksManager can help you convert sandboxed link pages into organized and thoughtful, Google-compliant, end-user-friendly ones easy doesn't mean that being sandboxed isn't going to hurt.  Depending on the nature of your business, it can be quite costly -which is why you should always make linking decisions with the end user in mind.

Getting out (of the sandbox) isn't terribly difficult and is automatic once you correct your suspect linking practices

While everything we know about the practice is speculation, the more intelligent and well-connected speculators in the SEO arena estimate that the typical sandboxed site stays in the penalty box three to six months - a long time to be buried near the bottom of a Google search return.  What if your site takes a Page Ranking hit despite "best practices" coding, tagging and linking design?  Minor dips and advances are par for the course.  They happen to virtually every site and will generally average out over time.  The fact is that search-engine rankings, Google's and everyone else's, are neither carved in stone nor particularly logical.  For example, I just Googled "automobiles." Care to guess what the first return was?  Don't bother.  It was a New York Times road test of a Chevy minivan. (And, yes, this was a Google web search, not a news search.).  The first car manufacturer on the list?  Honda, number five in the returns -- probably due to their clever use of the automobiles.honda.com domain (as opposed to ford.com). OK, that explains Honda.  But how do you explain peugeot.com at number eight?  Peugeot makes fine automobiles, but it is far from a major manufacturer in America, the European Union, Asia, or anywhere else.  Next up among mainstream car makers was Citroen at the top of the second return page.  BMW and Mercedes checked in toward the middle of page three.  I gave up trying to find GM, Chrysler, and Ford after five pages, but I'm sure they're in the index somewhere.

The point is that a moderate decline in your ranking can be caused by something as simple as a competitor having a brainstorm that makes his page more enticing to Google than yours.  There may conceivably have been a time when Peugeot returned at five and Honda at eight.  Than someone at Honda said "Let's take advantage of the fact that we have our own name server by putting up automobile.honda.com instead of hondaautombile.com."  When it came time for a re-index, Honda went up and Peugeot went down.  Maybe.  Is the creative use of that particular URL the reason why Honda beats other car companies on that particular search?  Seems logical.  Except for the Times road test in the number one spot.  That tosses all logic out the window.  After all, road testing cars is hardly the New York Times primary interest and the test itself pertains to exactly one vehicle.  But the search term - automobiles -- was plural.  And while we're riddling things out, consider this.  Google gives Honda the fifth place in the returns and a page-ranking importance of 7/10 on the Google Toolbar.  Ford and GM both get 8/10 on the Toolbar, but don't make the first five pages in the "automobiles" search returns.

A moderate decline in your ranking can be caused by something as simple as a competitor having a brainstorm

The crucial things to understand are a.) your Google ranking for any given search term can, and almost certainly will, move up or down a bit every time your site is re-indexed, b.) whether your rank goes up or down may have nothing to do with changes in your site and everything to do with changes in your competitors' and c.) while your ranking on some keyword searches may move one way after a re-indexing, your rank on others may go in the opposite direction.  And then there's the beauty contest - the one innovation that Google bashers love to hate most.  One of Google's major "contributions" to search-engine methodology is factoring site popularity, as determined by traffic volume, into its rankings.  Googlephobes claim that using the number of hits a site receives as a ranking factor skews the returns in favor of sites supported by major advertising or promotional campaigns.  This, critics charge, produces a very uneven playing field for non-commercial sites competing with corporate sites and small companies competing with big ones.  They also claim that the "beauty contest" favors existing sites over new ones and helps existing web businesses get bigger at the expense of startups.

While these arguments are frequently overstated (popularity does not seem to figure heavily in final rankings), it only makes sense to periodically compare the number of hits your site has gotten over the past six months with your Google rankings.  If your hit count is substantially down compared to the six months prior, it might explain a drop in your rankings.  Obviously this is another area in which a sound linking strategy - one that focuses on creating quality link partnerships that generate relevant traffic to your site - can be a very positive factor in your Google return position.

The bottomline is that Google re-indexes sites frequently - every 30 days in some cases - and your page ranking will almost certainly change, at least in regard to some search terms, with each re-indexing

Other things that might move your ranking up or down involve "rules" changes.  Google is always refining its technology, which, in simple English means it is tinkering with how it analyzes web sites.  Google updates it's index every few months and is doing so as we finish this article.  Searching for "automobiles" after the latest Google update finds the New York Times dethroned from its number one position by automobiles.com and Honda advancing from number five to number three.  Bugatti and Peugeot, however, remain in place at seven and eight respectively.  The bottomline here is that Google re-indexes sites frequently - every 30 days in some cases - and your page ranking will almost certainly change, at least in regard to some search terms, with each re-indexing.

Before pushing the "I've been banned or sandboxed" panic button, check on whether or not you are still in the index using the one of the methods noted above.  If you are, Google each of your keywords/phrases.  Chances are your rank will have gone up on some, down on others, and remains about where it was on average.  If your site has been dropped from the index or if your return position has eroded substantially - 100 or more positions - regardless of search term, there's something wrong.  And that something is 99.9 percent certain to be on your end.  Go over your site's pages element by element, link by link, meta tag by meta tag, and you're almost sure to find it.