Jump to content

Explain Spiders, Robots, and Crawlers

Recommended Posts

The Crawling is also known as Robot, Bot or Spider. They are the same and referred to by different names. These programs are used by search engines to browse the Internet, and to access web content found on websites automatically. Web crawling or spidering is the mechanism that performs a web crawler. Some sites, particularly search engines, use crawlers to keep the database updated.

Link to comment
Share on other sites


This topic is now archived and is closed to further replies.

  • Create New...