What Are Spiders and Internal Links?
Spiders — automatic programs used by Internet search engines to regularly crawl the World Wide Web — look for websites and other content to add to the search engines’ databases. They follow the hyperlinks that connect websites on the Internet.
Search Engine Visibility provides several options for you to help search engine spiders and crawlers in navigating your site with ease. Spiders are crippled when they have no links to follow. If your website has broken or no internal links a spider will likely not go beyond the first link. If your site contains external links, the spider might follow those and leave the site behind.
If, however, all the Web pages on your site are linked to each other with functional hyperlinks, a visiting spider is able to visit every nook and cranny of the site.