FrederickStonehouse

COLLECTED BY

Organization: Internet Archive

The Internet Archive discovers and captures web pages through many different web crawls. At any given time several distinct crawls are running, some for months, and some every day or longer. View the web archive through the Wayback Machine.

Collection: Wide Crawl started March 2011

Web wide crawl with initial seedlist and crawler configuration from March 2011. This uses the new HQ software for distributed crawling by Kenji Nagahashi.

What’s in the data set:

Crawl start date: 09 March, 2011
Crawl end date: 23 December, 2011
Number of captures: 2,713,676,341
Number of unique URLs: 2,273,840,159
Number of hosts: 29,032,069

The seed list for this crawl was a list of Alexa’s top 1 million web sites, retrieved close to the crawl start date. We used Heritrix (3.1.1-SNAPSHOT) crawler software and respected robots.txt directives. The scope of the crawl was not limited except for a few manually excluded sites.

However this was a somewhat experimental crawl for us, as we were using newly minted software to feed URLs to the crawlers, and we know there were some operational issues with it. For example, in many cases we may not have crawled all of the embedded and linked objects in a page since the URLs for these resources were added into queues that quickly grew bigger than the intended size of the crawl (and therefore we never got to them). We also included repeated crawls of some Argentinian government sites, so looking at results by country will be somewhat skewed.

We have made many changes to how we do these wide crawls since this particular example, but we wanted to make the data available “warts and all” for people to experiment with. We have also done some further analysis of the content.

If you would like access to this set of crawl data, please contact us at info at archive dot org and let us know who you are and what you’re hoping to do with it. We may not be able to say “yes” to all requests, since we’re just figuring out whether this is a good idea, but everyone will be considered.

TIMESTAMPS

The Wayback Machine - https://web.archive.org/all/20110728143845/http://www.frederickstonehouse.com/

FrederickStonehouse.com

Maritime Historian, Writer, Lecturer, Teacher

Updated: 26May2011

Schedule

Programs Available

Books

Lighthouse Course

Maritime History Course

New Projects

Fees

Links

mailto:stonef@charter.net

Frederick Stonehouse has authored over thirty books on maritime history, many of them focusing on the Great Lakes and contributed to several others. The Wreck of the Edmund Fitzgerald and Great Lakes Lighthouse Tales are regional best sellers. Wreck Ashore, the U.S. Life-Saving Service on the Great Lakes, won a national publishing award and is the predominant work on the subject. Another book, Haunted Lakes, Great Lakes Maritime Ghost Stories, Superstitions and Sea Serpents, has opened an entirely new genre in Great Lakes study. His book, Final Voyage, is the first Great Lakes shipwreck book for children.

He has been a consultant for both the U.S. National Park Service and Parks Canada and has been an "on-air" expert for National Geographic, History Channel and Fox Family, as well as many regional media productions. Awards for contributions to Great Lakes maritime history have been received from Underwater Canada, Our World Underwater, Marquette Maritime Museum and Marquette County Historical Society. He is also the recipient of the 2006 Association For Great Lakes Maritime History Award for Historic Interpretation. The Award is presented annually in recognition of an individual making a major contribution over many years to the interpretation of Great Lakes maritime history in furtherance of the goals of the Association. In addition he was named the Marine Historical Society of Detroit’s “2007 Historian of the Year.” The award is the result of election by past MHSD Historians and recognizes persons who have actively contributed to the study of Great Lakes history. He holds a Master of Arts degree in History from Northern Michigan University, Marquette, Michigan.

Fred has taught Great Lakes maritime history at Northern Michigan University and is an active consultant for numerous Great Lakes oriented projects and programs. In April 2000, he began teaching two Internet based courses on Great Lakes lighthouses, the first time such instruction has been offered on the web. An Internet based course, Great Lakes Maritime History, HS 205 will be offered during the 2010 summer semester. See www.nmu.edu/summer for enrollment information.

Major presentations have been given at: the National Maritime Conferences, Great Lakes Aquarium, Erie Maritime Museum, Great Lakes Lighthouse Festival, Michigan Lighthouse Conference, Gales of November, (Duluth), Ghost Ships Festival (Milwaukee), Keeper of the Light Festival (Bayfield) Underwater Canada and Our World Underwater among many others.

His articles have been published in Lake Superior Magazine and Wreck and Rescue Journal as well as other publications. He is President of the Board of the Marquette Maritime Museum, President of the National Board of Directors of the U.S. Life-Saving Service Heritage Association and a former member of the Michigan Iron Industry Museum Advisory Board (Gubernatorial appointment). Fred makes his home in Marquette, Michigan with his wife Lois and son Brandon.

Fred is also active in the Marquette Community and was elected in November 2008 to the position of City Commissioner. In addition he is or has served on several boards and committees, including: Northern Michigan University Alumni Board, The Marquette County Economic Club, Marquette Harbor Advisory Committee (past chair) and the Northern Michigan University De Vos Art Museum Advisory Board. He also chaired the Mayor's Ad Hoc Task Force to develop the Marquette Harbor Master Plan. On December 16, 2006 Fred delivered the alumni welcome at Northern Michigan University's graduation .