Getting a good organic rank for websites is the top priority today. Due to this Search Engine Optimization is increasingly been practiced by many business owners.

You can easily find a whole lot of information regarding various optimizing techniques, website architecture, crawling and indexing on the World Wide Web but when it actually boils down to the implementation process, you will be really amazed with all the nuances involved in it.

The primary question which generally bothers website owners is that how easy it is for search engines to crawl websites? Since Internet is vast and new contents are being regularly updated, it becomes virtually impossible for search engine robots to crawl the entire content present online. This means that only a percentage of this content is crawled by search engine spiders. Moreover amongst the crawled content, only a portion is indexed due to availability of limited resources.

Elaborating on this topic further, Susan Moskwa of Google, recently uploaded a presentation giving smart tips on how to optimize your crawling & indexing. She states that since URLs act as bridges between a website and a search engine crawler you should not make them complicated or redundant. In order to allow crawlers to be able to find and crawl your URLs, organize them in such a way that crawlers are directly led to distinct contents of your website. On doing this you will prevent crawlers from wasting time on going through empty web-pages and/or crawling same web content again and again via different URLs. Optimization of URLs allows crawlers to spend their time accessing the actual content of your website.

There are some important real life examples in this presentation that have been categorized under a “not to do” list. These include homegrown URL hacks and encoding, parameters camouflaged as part of the URL path and infinite crawl spaces. In addition to these, you will also find some other useful recommendations like removal of user-specific details from URLs, disallowing actions which Google-bot cannot normally perform and sticking to one URL for a particular set of content.

So if you too want to make search engine crawlers trace your website faster, do make sure to follow all the above Google SEO tips next time you optimize your website.