Don't Tick Off the Googlebot

Whenever I visualise the Googlebot, I get this weird cross hatched version of a mutant R2D2 combined with Shelob the Spider from the Lord of the Rings. You want your pages to be crawled, but there are a few things you should remember to do – or not to do, as the case may be. Otherwise the spider may feel taunted and decide to squash you into jelly and suck you dry.

1. Do make sure you point to your new pages with good solid links. This gets them crawled and indexed much faster than submitting them for indexing – although that step won’t hurt either. If you are just starting out, consider creating a blog in tandem with your website, and making sure you keep the blog indexed and updated daily. Then you can instantly link to your new pages by way of posting on the blog.

2. Don’t use too much Flash. The rumour went around a while back that google now was indexing Flash – not QUITE the case. Google stopped completely ignoring Flash – as long as everything was properly tagged and there was some text to crawl.

3. Do tag like crazy, but BE CONSISTENT. This will help the spider get a balanced idea of what your site is about. You want keyword rich content, but not at the expense of readability. use your keywords wisely and well, and don’t forget to optimize anchor links, metatags and page descriptions.

4. Don’t ping Google unnecessarily. It irritates the spider to crawl all the way out on the web and not find a juicy fly due to a false alarm. Poking the web with a stem of grass just makes trouble. Only ping when you really do have new content.

5. Link, link, link. You can never have too many links, as long as they are all GOOD links. Don’t sdend the spider off on futile runs down hundreds of links that are merely reciprocal or threeway – or worse lead to sites that bear no relevance at all to your own. It wastes time and resources, and the spider is hungry for fresh meat.

6. Don’t forget a sitemap. A wel designed site map can make it very easy for the Googlebot to make short order of indexing your site. As long as you have good internal linking structure coupled with strng inbound links, you should be able to keep your site up to date on the Google index.

Make the spider happy. Feed it what it wants!