Search Engine Spiders Are Blocked?

Question

I have a feeling something on my site is blocking search engines from spidering the entire site. Anyone know of a quick and easy way to determine if spiders are picking up all the pages on my site?

Answer 1

First check your robot.txt for any /disallow tags. Secondly, check for nofollow tags in your internal links. Third, you need to badly sign up with Google Webmaster tools that analyzes your robot.txt, indexed pages, sitemap etc.

Answer 2

if you have misleading spiders than search engines will ignore your site, research blackhat seo, best of luck.

Answer 3

go 2 dogpile if that helps