| The meta tag alternative to the robots.txt file |
|
It is possible to place a robots meta tag on a specific page to tell robots not to index the page, and not to follow the links on the page. The meta tag is placed within the head section of the source code on the page. <html> <head> <title>This is the page I don't want robots to crawl</title> <meta name="robots" content="noindex, nofollow"> </head> As with the robots.txt file, some robots may ignore the meta tag and index the page and follow the links even if the robots meta tag is present. There are several types of meta tags, so you must identify what type of meta tag it is by using name="robots". The content attribute tells the robots what to do. Possible values include "index", "noindex", "follow", and "nofollow" How to allow the page to be indexed, but restrict the links from being followed. <html> <head> <title>I want this page indexed, but don't want the links followed</title> <meta name="robots" content="index, nofollow"> </head> How to restrict the page from being indexed, but allow the links to be followed. <html> <head> <title>I don't want this page indexed, but do want the links followed</title> <meta name="robots" content="noindex, follow"> </head> How to restrict the page from being indexed, and the links from being followed. <html> <head> <title>I don't want this page indexed, or the links followed</title> <meta name="robots" content="noindex, nofollow"> </head> While other combinations are possible, they wouldn't make much sense. |