The Internet Archive discovers and captures web pages through many different web crawls.
At any given time several distinct crawls are running, some for months, and some every day or longer.
View the web archive through the Wayback Machine.
Web wide crawl with initial seedlist and crawler configuration from March 2011. This uses the new HQ software for distributed crawling by Kenji Nagahashi.
What’s in the data set:
Crawl start date: 09 March, 2011
Crawl end date: 23 December, 2011
Number of captures: 2,713,676,341
Number of unique URLs: 2,273,840,159
Number of hosts: 29,032,069
The seed list for this crawl was a list of Alexa’s top 1 million web sites, retrieved close to the crawl start date. We used Heritrix (3.1.1-SNAPSHOT) crawler software and respected robots.txt directives. The scope of the crawl was not limited except for a few manually excluded sites.
However this was a somewhat experimental crawl for us, as we were using newly minted software to feed URLs to the crawlers, and we know there were some operational issues with it. For example, in many cases we may not have crawled all of the embedded and linked objects in a page since the URLs for these resources were added into queues that quickly grew bigger than the intended size of the crawl (and therefore we never got to them). We also included repeated crawls of some Argentinian government sites, so looking at results by country will be somewhat skewed.
We have made many changes to how we do these wide crawls since this particular example, but we wanted to make the data available “warts and all” for people to experiment with. We have also done some further analysis of the content.
If you would like access to this set of crawl data, please contact us at info at archive dot org and let us know who you are and what you’re hoping to do with it. We may not be able to say “yes” to all requests, since we’re just figuring out whether this is a good idea, but everyone will be considered.
Last mission to repair the Hubble telescope Hubble space telescope discoveries have enriched our understanding of the cosmos. In this special report, you will see facts about the Hubble space telescope, discoveries it has made and what the last mission's goals are.
For their own good
Fifty years ago, they were screwed-up kids sent to the Florida School for Boys to be straightened out. But now they are screwed-up men, scarred by the whippings they endured. Read the story and see a video and portrait gallery.
Fill out this form to email this article to a friend
SheiKra shows her sexy side
And the Busch Gardens coaster isn't alone. More and more ad campaigns border on the risque to sell the most mundane of products.
By RODNEY THRASH
Published May 12, 2007
Call this number and you'll hear a come-on from an unlikely source: Busch Gardens. It's all in good clean fun, the park says.
[Times photo: Thomas M. Goethe]
[Times photo: Mike Pease]
Suggestive ads for a privately owned student residence blanketed the University of South Florida campus in Tampa in 2004. Marketers sometimes substitute sex for creativity, one expert says.
As commuters approach the city's northern end, the solicitations intensify.
"$0 down," says one billboard on Interstate 275.
"Free checking," says another.
Then just before the Busch Boulevard exit, this sign:
One last fling while I've still got it.
At the bottom, a toll-free number.
In the name of journalism, we dial.
"Hi, doll face," purrs the voice, a strange fusion of maternity and sensuality.
The woman on the line refers to callers as "honey" and "doll." She says "it's been a fun ride." She wants "one last fling-a-ding."
The horny pitchman?
The Tampa theme park, which caters to families and children, started placing the signs on I-275 and other locations and in newspapers late last month.
Written like personal ads, the advertisements are part of a campaign to promote SheiKra, the first dive coaster in North America. It closes May 28 and a new, floor-less version of the ride opens June 16.
"It's tongue-in-cheek," says Gerard Hoeppner, a spokesman for Busch Gardens.
"A play on words."
How about racy?
"Define for me what racy means," Hoeppner says.
He points out that the billboard has the word "SheiKra" by the phone number.
"It's not graphic in anyway. It's very much sterilized. We think that the ad will appeal to the smart and witty coaster fans that will get the humor."
Anything can be sexy
Sex has long been a sales gimmick, but marketers seem to have found ways to sexualize the least erotic things.
Last November, a woman stared seductively from a billboard on State Road 60. Scrawled across the left corner of the sign was MyRealSexLife.com. The link ultimately directed Internet users to the Web site of Relevant Church in Ybor City.
In a 2005 advertisement promoting its spicy BBQ burger, fast-food chain Carl Jr. featured hotel heiress Paris Hilton in a skin-tight swimsuit writhing, sudsy, atop a Bentley before taking a sloppy bite out of the burger. You must be 18 to view the commercial on YouTube.
Even college dormitories.
In ads that did nothing to calm nervous University of South Florida parents, the managers of Fontana Hall in 2004 plastered images of chiseled men and busty women across the privately owned building.
One ad showed a bare-chested man between two dripping wet women in lingerie.
shower with friends . . . by choice, the ad said.
Another sign showed a toned man embracing a woman in a nightgown, his hand gripping her waist.
it's better at fontana
Catching your eye
"We live in a very cluttered world, so it's very hard for marketers to break through and get attention," says Tim Calkins, a marketing professor at the Kellogg School of Management at Northwestern University in Evanston, Ill.
"That's why marketers keep pushing the edge. There's a desire to be different. Some people just substitute sex for creativity and that's unfortunate."
Or is it smart?
"People are driving," says Hoeppner, the Busch Gardens spokesman. "You can't overload the board with a lot of copy.
"At some point, you see the same outdoor boards all the time and they become landscape wallpaper."
Busch Gardens can't quantify the success of the new campaign. The billboards just went up a couple of weeks ago.
But, Hoeppner says, "by keeping the message fresh, we're able to keep the appeal of the park fresh."
That may be true to an extent, says Calkins, the marketing professor.
"If all you do is attract attention," he says, "you won't necessarily attract sales. The brand and the product gets lost."
Still, Calkins doesn't see sex disappearing from commercials, radio spots or print ads any time soon. As hypersexual as American culture has become, expect more of it, he says.
Head toward Fowler Avenue and 50th Street. There's another, tamer Busch Gardens billboard:
cute, had to look what a great way to advts, im sure you got plenty of lookers for this ad.got my attnchn. looking to book a stay just due to this diff. ad. adults are a good source of income for your resort. been there, and will be back again.
05/11/07 03:32 PM
Americans are so up tight when it comes to sex... compared to European countries this is "nothing!" It's fun to see Americans loosening up a little!
Subscribe to the Times
Click here for daily delivery
of the St. Petersburg Times.