Jump to content

Which is better Robots.txt or Meta Robots Tag?


Recommended Posts

 - Robots Meta Directives. Robots meta directives (sometimes called “meta tags”) are pieces of code that provide crawlers instructions for how to crawl or index web page content. ... Below are the parameters that search engine crawlers understand and follow when they're used in robots meta directives.

 - Meta tags are used to dis allow a single page from crawling. ... The Meta tag option tells bots not to index individual files, whereas Robots.txt can be used to restrict access to entire directories. Both are supported by all crawlers.

Link to comment
Share on other sites


This topic is now archived and is closed to further replies.

  • Create New...