The Search Engine Unfriendliness of Web 2.0

Wouldn’t it be great if all those whiz-bang Web 2.0 interactive elements based on AJAX (Asynchronous JavaScript and XML) and Flash—such as widgets and gadgets and Google Maps mashups— were search engine optimal?

Unfortunately, that’s not the case. In fact, these technologies are inherently unfriendly to search engine spiders. So, if you intend to harness Web 2.0 technologies for wider syndication, increased conversion, improved usability and greater customer engagement, you’d better read on or you’ll end up missing the boat when it comes to better search engine rankings.

When it comes to AJAX and Flash, the onus is on you to render them search engine friendly. The major search engines just can’t cope with these Web 2.0 technologies very well at all.

Flash not friendly

Let’s start with Flash, a technology with which many of us already are familiar. Some search engines, including Google, have rudimentary means of extracting content and links from Flash. Nonetheless, any content or navigation embedded within Flash will, at best, rank poorly in comparison to a static, HTML-based counterpart, and at worst, not even make it into the search engine’s index.

Google’s view on Flash is that it doesn’t provide a user-friendly experience. Flash is wholly inaccessible to the vision-impaired, unrenderable on devices such as mobile phones and PDAs, and can’t be accessed without broadband connectivity.

In particular, Google frowns on navigational elements presented exclusively in Flash. Given this stance, Google isn’t likely to make big improvements on how it crawls, indexes and ranks Flash files anytime soon. So, it’s in your hands to either replace those Flash elements with a more accessible alternative like CSS/DHTML or to employ a Web design approach known as “progressive enhancement,” whereby designs are layered in a concatenated manner to provide an alternative experience for non-Flash users. This way, all users, including search engine spiders, will be able to access your content and functionality.

An example of progressive enhancement in action can be found at Amazon.com’s “Create Your Own Ring” on the Web. Simply turn off the JavaScript capabilities in your browser and build your ring—with or without the Flash interaction. All customers are equally served.

Problems with AJAX

AJAX poses similar problems to spiders as Flash does. That’s because AJAX also relies on JavaScript—that’s what the “J” in AJAX stands for, after all (the complete acronym stands for Asynchronous JavaScript and XML). Search engine spiders can’t execute JavaScript commands (or Java either, for that matter). AJAX can be used to pull data seamlessly in the background onto an already loaded Web page, sparing the user from the “click-and-wait” frustrations associated with more conventional Web sites. It’s a great timesaver for users, but the additional content that’s pulled in via AJAX is invisible to the spiders unless it’s preloaded into the page’s HTML and simply hidden from the user via CSS.

Here, progressive enhancement renders a non-JavaScript version of the AJAX application for spiders and JavaScript-incapable browsers. A low-tech alternative to progressive enhancement is to place an HTML version of your AJAX application within noscript tags (see TheCleanerMovie.com for an example).

Other options include rendering static HTML pages from product searches, as the vertical shopping engine Become.com does. Google’s guidelines warn that your search result pages must provide value to users to warrant inclusion in its index. So, extra care must be taken if employing this approach.

Widgets, the mini applications webmasters are encouraged to place on their sites to pull data from an external source, are in most cases inaccessible to search engine spiders. Most widgets are built in search engine unfriendly Flash or AJAX.

A well-loved widget in the blogosphere is Eurekster’s “Swicki,” with its “What’s Popular” buzzcloud, which you may have seen in the sidebars of popular blogs like TechCrunch. The “What’s Popular” buzzcloud. Under our tutelage, Eurekster made its widget more search engine friendly and reaped the benefits with a huge influx of search-referred traffic.

As you head down the road of Web 2.0, just remember that user-friendly doesn’t readily translate into search engine friendly without some assistance. But know that help is available, particularly in the form of progressive enhancement, when you embrace this brave new World Wide Web.