Google Now Crawls HTML Forms
Sunday, April 13th, 2008Anyone who has read Toni’s and my ramblings in other places will know that neither of us are big fans of Google. Sure, we play their game and abide by their rules but that doesn’t mean we like them or that we would do a whole lot of mourning if they just went away. And if the history of the Internet is anything to go by then perhaps one day they will … but till that day comes we keep up-to-date on what’s happening with Google and the way they crawl web pages.
So it was interesting to see that on Friday Google announced that they have begun to crawl HTML forms. Now there is no guarantee that they’re going to index what they find at the other end because, in their words:
If we ascertain that the web page resulting from our query is valid, interesting, and includes content not in our index, we may include it in our index much as we would include any other web page.
Google also undertakes not to crawl beyond any forms that require passwords or the entry of personal details and seeks to reassure webmasters by telling them that Googlebot is “ever-friendly” and is always a good Internet citizen … hmmm.
Google also says that Googlebot obeys “nofollow directives”. I must say that our experience of Googlebot in relation to nofollow has been a little different.
Google also claims that they can now crawl Javascript navigation and that may be so but we’re still seeing a lot of pages with Javascript navigation out there on the Web that have not been crawled or indexed despite having been online for a year or more.
I doubt that we’ll be using javascript navigation on any of our sites any time in the foreseeable future.
