3 September 2013 Last updated at 19:01 ET

Jonathon Fletcher: forgotten father of the search engine

By Joe Miller BBC News

As Google celebrates its 15th birthday, the web giant has become a byword for information retrieval.

But if you put Jonathon Fletcher's name into a Google search, none of the immediate results hint to the role he played in the development of the world wide web.

There is certainly nothing that credits him as the father of the modern search engine.

Yet 20 years ago, in a computer lab at the University of Stirling in Scotland, Mr Fletcher invented the world's first web-crawling search engine - the very technology that powers Google, Bing, Yahoo and all the major search tools on the web today.

Solving search

In 1993, the web was in its infancy.

Mosaic, the first popular browser with an interface that resembles the ones we use today, had just been released, and the total amount of web pages numbered in the thousands.

But the question of how to find things on the web had not been solved.

Mosaic had a page called What's New, which indexed new websites as they were created.

The problem was that in order for developers at Mosaic to be aware of a new website, its creators would have to write to the National Center for Supercomputing Applications (NCSA) at the University of Illinois Urbana-Champaign - where the browser's team were based.

At about this time, Jonathon Fletcher was a promising graduate of the University of Stirling, with an offer to study for a PhD at the University of Glasgow.

Before he could take his place, funding at Glasgow was cut, and Mr Fletcher found himself at a loose end.

"I was suddenly very motivated to find a source of income," he recalls, "so I went back to my university and got a job working for the technology department."

It was in this job that he first encountered the world wide web and Mosaic's What's New page.

'A better way'

While building a web server for the university, Fletcher realised the What's New page was fundamentally flawed.

Because websites were added to the list manually, there was nothing to track changes to their content. Consequently, many of the links were quickly out-of-date or wrongly labelled.

"If you wanted to see what had changed you had to go back and look," Mr Fletcher says of Mosaic's links.

"With a degree in computing science and an idea that there had to be a better way, I decided to write something that would go and look for me."

That something was the world's first web crawler.

Mr Fletcher called his invention JumpStation. He put together an index of pages which could then be searched by a web crawler, essentially an automated process that visits, and indexes, every link on every web page it comes across. The process continues until the crawler runs out of things to visit.

Ten days later, on 21 December 1993, JumpStation ran out of things to visit. It had indexed 25,000 pages.

To date, Google has indexed over a trillion pages.

Birth of search

Mr Fletcher quickly built an easy-to-navigate search tool for the index, stuck his website on Mosaic's What's New page, and the world's first modern search engine was in operation.

"I would say that he is the father of the web search engine," says Prof Mark Sanderson of the Royal Melbourne Institute of Technology, who has studied the history of information retrieval.

"There have obviously been computers doing searches for a very long time, and there were certainly search engines before the web. But Jonathon was the first person to create a search engine that had all the components of a modern search engine."

However, while Google's founders Sergey Brin and Larry Page are household names, Mr Fletcher, who now lives in Hong Kong, has received little recognition for his role in the evolution of the internet.

The fact that his project was ultimately abandoned may not have helped.

As JumpStation grew, it required more and more investment - something which the University of Stirling was not willing to provide.

"It ran on a shared server," explains Mr Fletcher. "There wasn't a lot of disk space and back then disks were small and expensive."

Space control

By June of 1994, JumpStation had indexed 275,000 pages. Space constraints forced Mr Fletcher to only index titles and headers of web pages, and not the entire content of the page, but even with this compromise, JumpStation started to struggle under the load.

And so did Mr Fletcher. "It wasn't my job," he says. "My job was to keep the student labs running and do system administration and technology odd jobs."

A job offer to go and work in Tokyo proved too strong to resist, and the university did little to try and keep him, or JumpStation, from leaving.

"I was obviously not very successful in convincing them of its potential," says Mr Fletcher.

"At the time I did what I thought was right, but there have been moments in the last 20 years where I've looked back."

Prof Leslie Smith, head of Computing Science and Mathematics at the University of Stirling, who remembers Mr Fletcher, acknowledged that JumpStation "proved to be ahead of its time", and told the BBC that "colleagues at Stirling are delighted he is gaining the recognition he deserves for his achievements".

Looking forward

But despite the disappointment Mr Fletcher suffered, his pioneering technology would be the foundation of all subsequent web search engines.

"The web community in 1993 was very small," says Prof Sanderson. "Anybody who was doing anything on the web would've known about JumpStation.

"By the middle of 1994 it was becoming clear that web search engines were going to be very important. Google didn't come out until 1998 and what Jonathon was doing was in 1993."

Mr Fletcher received some recognition for his achievements at a conference in Dublin a few weeks ago, where he was on a panel with representatives from Microsoft, Yahoo and Google. But in his speech, he talked about the future.

"In my opinion, the web isn't going to last forever," he told the audience. "But the problem of finding information is."

"The desire to search through content and find information is independent of the medium."

The current medium is making a lot of money for those who followed him, but the Scarborough-born pioneer has no regrets.

"My parents are proud of me, my wife is proud of me, my children are proud of me, and that's worth quite a lot to me, so I'm quite happy."