Out-Googling Google

Dubbed the “operating system of the internet”, Google has become the default term for web searching. It’s the top destination site for web searches, receiving more than 150 million queries daily. All of which makes the behemoth in Redmond, Washington, a trifle concerned.

Microsoft plans to dethrone Google. Remember the demise of internet darling Netscape? If all goes to plan for Microsoft, there will be a repeat of history – but this time it will be Google that’s toppled. Microsoft is already working on building a next-generation search engine for its MSN portal – the number three search engine by queries – to replace its current search technology (powered by outsource partners Overture, LookSmart, and Inktomi). MSN’s arch-rival Yahoo! recently purchased Inktomi and Overture, providing an impetus for Microsoft to build its own search engine in-house.

Microsoft’s search technology team is purportedly expanding this year from 70 engineers to over 300, according to US investment bank SoundView Technology Group. And Yusuf Mehdi, corporate vice president of MSN personal services and business, recently gave a clear hint of Microsoft’s ambitions, telling the Goldman Sachs internet conference, “We are investing a lot to go and build what we expect and hope will be the best-in-class search service in the near future.”

Clearly, something is afoot at Microsoft. During the past few months, webmasters around the world have noticed their websites being visited by a new search engine spider called MSNBOT. Search engine spiders surf the web all the time, following links and downloading millions – or even billions – of web pages that are all fed into one massive database that can be searched by keyword. But what makes MSNBOT’s activity significant is that it signals Microsoft’s seriousness in building its own search engine: it’s already started sending forth a spider to collect the contents of the web.

Microsoft itself declares it intends integrating the crawled contents of MSNBOT’s database into MSN results. But many believe this is just the beginning. Microsoft could very effectively leverage its desktop and web browser monopoly by hard-wiring access to its upcoming search engine right into Internet Explorer and into the Windows desktop. In all likelihood, Microsoft will incorporate a search box into the Internet Explorer browser toolbar that will use MSN Search by default, and embed a search function into the next version of Windows (code-named Longhorn) with an internet option, also pointing to MSN Search by default.

Taken a step further, internet search functionality could be embedded directly into Word documents, Excel spreadsheets and web pages.

A lot is riding on Longhorn, the successor to Windows XP that is slated for release in 2005. With Longhorn, Microsoft plans to roll out a totally new kind of file system based on its SQL Server database software. Documents will all be stored in a single database built into Windows to make it more easily searchable, more reliable, and more accessible across corporate networks and the internet. As part of Longhorn, Microsoft hopes to offer an all-encompassing search that mines the content of the user’s emails (both sent and received), spreadsheets, documents, databases, audio, video, images and more – whether stored locally on your computer, on your corporate network or on the internet.

This goal is a long-standing one, but thus far it has proved elusive. Microsoft believes it will nail it this time. Jim Allchin, group vice president in charge of Windows development, stated in a recent interview with the Seattle Times: “Google’s a very nice system, but compared to my vision, it’s pathetic.” Hubris aside, Allchin makes it clear in his statement that Microsoft’s vision for search encompasses a lot more than just a searchable repository of web documents, and as such, dethroning Google is probably just a stepping stone in the grand plan.

Does Microsoft’s upcoming search engine need to be better than Google’s? Probably not. It just needs to be good enough that the vast majority of users won’t bother fiddling with the default settings in Internet Explorer or Windows.

One ace Google could potentially play in all this: its technology is protected by several patents. It may prove challenging for Microsoft to “embrace and extend” Google’s functionality without violating some of these patents. In particular, the foundation of Google’s search ranking technology, the PageRank algorithm that is a big part of the magic that makes Google’s search results so relevant, was patented by Larry Page, co-founder of Google, in 2001.

Of course, entering a protracted legal battle with Microsoft wouldn’t come cheap.

By Stephan Spencer. This article first appeared on Unlimited in September 2003.