The ultimate guide to the meta robots tag
If you use meta robots
tags on your pages, you can give search engines instructions on how you’d like them to crawl or index parts of your website. This page lists an overview of all the different values you can have in the meta robots tag, what they do, and which search engines support each value.
The different robots meta tag values
The following values (‘parameters’) can be placed on their own, or together in the content
attribute of tag (separated by a comma), to control how search engines interact with your page.
Scroll down for an overview of which search engines support which specific parameters.
- index
- Allow search engines to add the page to their index, so that it can be discovered by people searching.
- Note: When there are no directives relating to indexing, this is assumed to be the default.
- noindex
- Disallow search engines from adding this page to their index, and therefore disallow them from showing it in their results.
- Note: Informal messaging from Google suggests that, if a page is set to
noindex
for a long period of time, it may also be treated as if it were also set tonofollow
. The precise mechanics of this are unclear, and it’s unclear whether other search engines behave similarly. - follow
- Tells the search engines that it may follow links on the page, to discover other pages.
- Note: When there are no directives relating to following links, this is assumed to be the default.
- nofollow
- Tells the search engines not to ‘endorse’ (pass equity through) any links on the page. Note that this includes all links on the page, including, e.g., those in navigation elements, links to images or other resources, and so on.
- Note: It’s unclear (and inconsistent between search engines) whether this attribute prevents search engines from following links, or just prevents them from assigning any value to those links. In fact, Google will follow these for the purposes of discovery (but not passing equity).
- none
- A shortcut for
noindex, nofollow
. - all
- A shortcut for
index, follow
. - Note: This is assumed by default on all pages, and does nothing if specified.
- noimageindex
- Disallow search engines from indexing images on the page.
- Note: If images are linked to directly from elsewhere, search engines can still index them, so using an X-Robots-Tag HTTP header is generally a better idea.
- noarchive
- Prevents the search engines from showing a cached copy of this page in their search results listings.
- nocache
- Same as
noarchive
, but only used by MSN/Live. - nosnippet
- Prevents the search engines from showing a text or video snippet (i.e., a
meta description
) of this page in the search results, and prevents them from showing a cached copy of this page in their search results listings. - Note: Snippets may still show an image thumbnail, unless
noimageindex
is also used. - nositelinkssearchbox
- Prevents the search engine from showing an inline search box for your site.
- nopagereadaloud
- Prevents the search engine from reading your page’s content aloud via voice services/results.
- notranslate
- Prevents search engines from showing translations of the page in their search results.
- max-snippet:
[number]
- Sets a maximum number of characters for the meta description.
- Note: Omitting this tag may result in an implied value of
0
. A default value of-1
should be set to imply ‘no limit’. - max-video-preview:
[number]
- Sets a maximum number of seconds for a video in a preview.
- Note: Omitting this tag may result in an implied value of
0
. A default value of-1
should be set to imply ‘no limit’. - max-image-preview:
[setting]
- Sets a maximum image size for use in a preview (
none
,standard
orlarge
). - Note: Omitting this tag may result in an implied value of
none
. - rating
- Indicates that a page contains adult material.
- unavailable_after
- Tells search engines a date/time after which they should not show it in search results; a ‘timed’ version of
noindex
. - Note: Must be in
RFC850
format (e.g.,Monday, 15-Aug-05 15:52:01 UTC
). - noyaca
- Prevents the search results snippet from using the page description from the Yandex Directory.
- Note: Only supported by Yandex.
noydir- Blocks Yahoo from using the description for this page in the Yahoo directory as the snippet for your page in the search results.
- Note: Since Yahoo closed its directory this tag is deprecated, but you might come across it once in awhile.
Which search engine supports which robots meta tag values?
This table shows which search engines support which values. Note that the documentation provided by some search engines is sparse, so there are many unknowns.
Robots value | Yahoo | Bing | Ask | Baidu | Yandex | |
---|---|---|---|---|---|---|
Indexing controls | ||||||
index | Y* | Y* | Y* | ? | Y | Y |
noindex | Y | Y | Y | ? | Y | Y |
noimageindex | Y | N | N | ? | N | N |
Whether links should be followed | ||||||
follow | Y* | Y* | Y* | ? | Y | Y |
nofollow | Y | Y | Y | ? | Y | Y |
none | Y | ? | ? | ? | N | Y |
all | Y | ? | ? | ? | N | Y |
Snippet/preview controls | ||||||
noarchive | Y | Y | Y | ? | Y | Y |
nocache | N | N | Y | ? | N | N |
nosnippet | Y | N | Y | ? | N | N |
nositelinkssearchbox | Y | N | N | N | N | N |
nopagereadaloud | Y | N | N | N | N | N |
notranslate | Y | N | N | ? | N | N |
max-snippet:[number] | Y | Y | N | N | N | N |
max-video-preview:[number] | Y | Y | N | N | N | N |
max-image-preview:[setting] | Y | Y | N | N | N | N |
Miscellaneous | ||||||
rating | Y | N | N | N | N | N |
unavailable_after | Y | N | N | ? | N | N |
noodp | N | Y** | Y** | ? | N | N |
noydir | N | Y** | N | ? | N | N |
noyaca | N | N | N | N | N | Y |
* Most search engines have no specific documentation for this, but we’re assuming that support for excluding parameters (e.g., nofollow
) implies support for the positive equivalent (e.g., follow
).
** Whilst the noodp and noydir attributes may still be ‘supported’, these directories no longer exist, and it’s likely that these values do nothing.
Rules for specific search engines
Sometimes, you might want to provide specific instructions to a specific search engine, but not to others. Or you may want to provide completely different instructions to different search engines.
In these cases, you can change the value of the content
attribute to a specific search engine (e.g., googlebot
).
Note: Given that search engines will simply ignore instructions which they don’t support or understand, it’s very rare to need to use multiple meta robots tags to set instructions for specific crawlers.
Conflicting parameters, and robots.txt files
It’s important to remember that meta robots tags work differently to instructions in your robots.txt file, and that conflicting rules may cause unexpected behaviors. For example, search engines won’t be able to see your meta
tags if the page is blocked via robots.txt
.
You should also take care to avoid setting conflicting values in your meta robots tag (such as using both index
and noindex
parameters) – particularly if you’re setting different rules for different search engines. In cases of conflict, the most restrictive interpretation is usually chosen (i.e., “don’t show” usually beats “show”).
Adding a noindex
or nofollow
to a post or page is a breeze if you’re on WordPress. Read how to use Yoast SEO to keep a post out of the search results.
Resources from the search engines
- Google’s official documentation on the meta robots tag
- Bing has a webmaster help page for it
- Yandex has a help page for HTML meta tags
Coming up next!
-
Event
Wordcamp Asia 2025
February 20 - 22, 2025 Team Yoast is at Attending, Sponsoring, Yoast Booth Wordcamp Asia 2025! Click through to see who will be there, what we will do, and more! See where you can find us next » -
SEO webinar
Webinar: How to start with SEO (December 18, 2024)
18 December 2024 Learn how to start your SEO journey the right way with our free webinar. Get practical tips and answers to all your questions in the live Q&A! All Yoast SEO webinars »
Typo:
nofollow
Tells the search engines robots ***NOT*** to ‘endorse’ (pass equity through) any links on the page.
Note: It’s unclear (and inconsistent between search engines) whether this attribute prevents search engines from following links, or just prevents them from assigning any value to those links.
My comment is that this is also misleading. Yes, Google has said they will sometimes follow despite a nofollow. They haven’t said anything specifically about meta robots nofollow that I’ve seen.
One thing you don’t explain here is that metarobots nofollow directs a crawler to not follow ANY links on a page. That includes links in content, navigation links, links to images (i.e. google will not look at your images if you use this tag on a page), links to AMP (Google News bot won’t grab your AMP if you use this tag), links to other resources.
Hi Jack, thanks for spotting that rather critical typo. I’ve corrected it now!
I’ll tweak the description, too, to reinforce the implications of links not being crawled. Good feedback!
No need to change my current meta tags setting.
That’s great :)
Well written article, We thought Now s days Meta Tags Doest play major Role like it did before. Yes We use few Meta Tags But not like as you mentioned. Will give a try and let you know.
Thanks Again
No problem at all, good luck!