The ultimate guide
to the meta robots tag

The ultimate guide to the meta robots tag

Last update: 07 April, 2017

The meta robots tags you can have on your pages has been around for quite a while. Search engines have slowly been adding features to it. This page lists an overview of all the different values you can have in your tag. First, the official syntax of the meta robots tag:

Which search engine supports which robots meta tag values?

This table shows which search engines support which values:

Robots value Google Yahoo! MSN / Live Ask
index Yes Yes Yes Yes
noindex Yes Yes Yes Yes
none Yes Doubt Doubt Yes
follow Yes Doubt Doubt Yes
nofollow Yes Yes Yes Yes
noarchive Yes Yes Yes Yes
nosnippet Yes No No No
noodp Yes Yes Yes No
noydir No use Yes No use No use

Become a technical SEO expert with our Technical SEO 1 training! »

Technical SEO 1 training Info

The different robots meta tag values

An explanation of all the different values you can use in the robots meta tags:

index

Allow search engines robots to index the page, you don’t have to add this to your pages, as it’s the default.

noindex

Disallow search engines from showing this page in their results.

noimageindex

Disallow search engines from spidering images on that page. Of course, if images are linked to directly from elsewhere, Google can still index them, so using an X-Robots-Tag HTTP header is a better idea.

none

This is a shortcut for noindex,nofollow, or basically saying to search engines: don’t do anything with this page at all.

follow

Tells the search engines robots to follow the links on the page, whether it can index it or not.

nofollow

Tells the search engines robots to not follow any links on the page at all.

noarchive

Prevents the search engines from showing a cached copy of this page.

nocache

Same as noarchive, but only used by MSN/Live.

nosnippet

Prevents the search engines from showing a snippet of this page in the search results and prevents them from caching the page.

noodp

Used to block search engines from using the description for this page in DMOZ (aka ODP) as the snippet for your page in the search results. However, DMOZ doesn’t exist anymore.

noydir

Blocks Yahoo! from using the description for this page in the Yahoo! directory as the snippet for your page in the search results. No other search engines use the Yahoo! directory for this purpose, so they don’t support the tag. Since Yahoo! closed its directory this tag is deprecated, but you might come across it once in awhile.

The resources from the search engines

The search engines themselves have pages about this subject as well:

And of course there’s always the official robots.txt pages and Danny Sullivan’s big robots meta write up.

You can block all robots at once by the markup above, or just block one robot by specifying it specifically:

Google
GOOGLEBOT
Yahoo!
SLURP
MSN / Live
MSNBOT
Ask
TEOMA

 

 

 

Read more: ‘robots.txt: the ultimate guide’ »



Check out our must read articles about Technical SEO