Sometimes Google does announcements about new features and we go “huh, why did they do that?” This week we had one of those. Google introduced a new set of robots meta controls, that allows sites to limit the display of their snippets in the search results. There is a reason for that, but they buried that reason far, far away.
With their newly introduced robots meta controls, you can “say” things like: I don’t want you to show more than 200 characters from my site! Or: I don’t want you to show images from my site. None of this made sense to us. Why would anyone who truly wants to optimize for Google’s search results, do any of that?
A French copyright law
Suddenly, we make a dramatic plot twist: France turns out to have introduced a new copyright law, which is the first implementation of Europe’s new copyright law. I am not a lawyer, but as Google explains it in this blog post, it requires for publishers to opt in to a display of their snippets, whereas in the past you had to opt out. Note that that blog post is in French, on their French blog, but there’s an English version in the bottom of that post.
It turns out that you can also use these new robots meta features to opt in to all those displays. Basically, use the meta robots to opt in to have your snippet text, videos and images being displayed at all.
robots meta?
Robots meta is hard enough as it is, read our ultimate guide to understand what you can do with it.
Because this French law implements the new European law, chances are that the other implementations of this copyright law across Europe are going to require similar action. This would mean more countries would need to opt in if they don’t want to lose their snippets. We don’t know that, of course, but of course: better safe than sorry.
A simple change to Yoast SEO
Because this potentially affects so many countries, we’ve decided to make a change for all Yoast SEO users. Every page will have the following robots meta bit added to its robots meta string if that page was not already set to noindex or nosnippet:
This makes sure that nobody runs into unexpected surprises and we’re ready for implementations of this law around Europe.
I want to change that robots meta value
If you want to change that robots meta value, we have a filter in our code that allows you to programmatically change that. That filter is called wpseo_robots, should you want to play with it.
You are opting in by using Yoast SEO
We realize that this means that we’re opting you in to all of these snippet features in Yoast SEO. I think it’s fair to say that if you use Yoast SEO to optimize your search results, we can assume that that’s what you want. At the same time, you might want somewhat more granular control over these values, if that’s true, please let us know in the comments!
The Yoast SEO release containing this change will be 12.2. This will be released next Tuesday, October 1.
Joost is an internet entrepreneur and the founder of Yoast. He has a long history in WordPress and digital marketing. On our blog, he has written a lot about SEO in general, technical SEO and important topics related to SEO.
November 29 - 30, 2024
Team Yoast is at Sponsoring WordCamp Netherlands 2024! Click through to see who will be there, what we will do, and more!
See where you can find us next »
19 November 2024
Learn how to start your SEO journey the right way with our free webinar. Get practical tips and answers to all your questions in the live Q&A!
All Yoast SEO webinars »
22 Responses to Robots meta changes for Google
Digital Shlok • 5 years ago
I really don’t think it is going to matter at all, because we still have crawling options to stop crawling.
Google’s official blog post listed five ways to control indexing:
Noindex in robots meta tags
404 and 410 HTTP status codes
Password protection
Disallow in robots.txt
Search Console Remove URL tool
So make sure you have apply above options on your website.
Thanks
Nhatty • 5 years ago
I cannot fully understand about the new google meta thing. But somehow if this would help every blogs then this is a must.
Andrea Bagnasco • 5 years ago
of course default opt-in for all Google snippet features is the way to go and the ‘same as it used to be’ approach that anyone not following up on SEO on a daily basis needs. Thank you for that!
Nethel • 5 years ago
Hi,
The story behind this meta is about money. Some french journalists wanted to get money from google for using/displaying in the search results their content without asking/pay for it…
To me it is a good thing you did opt-in for us all as we struggle for being visible in the search results. But indeed, legaly speaking, maybe you could let the option to turn it of globally (with the risk that newbies ask why they are not visible when turned off).
^ Post author
Joost de Valk • 5 years ago
Legally speaking, Google should make sure they have that opt in, if we have to do something to collect it on their behalf, it’s probably not even legal. I don’t know though, because IANAL :)
Lee • 5 years ago
Legally speaking, in the EU — and this is my lay opinion as a Brit which is offered free — people must opt-in not opt-out. This is the same with subscriptions. Though I believe you have taken the right technical decision, EU regulation and law derived thereof would be used to argue otherwise.
Can I suggest new installs of Yoast SEO offer authors/admins the option to opt-in to showing snippets in search results. The option cannot be pre-ticked — another EU regulation.
Thanks for all your work on Yoast.
^ Post author
Joost de Valk • 5 years ago
I don’t think we should be collecting opt ins on Google’s behalf. I’ll happily let them handle that themselves.
Joost • 5 years ago
I am so happy that you a Yoast take care of these kinds of thing, so this Joost does not need to bother with it.
QuestionUnanswered • 5 years ago
This is indeed away from the SEO mainstream media. It is good to know that Yoast team is preparing ahead of its massive implementation. Good move
Godswill Chijioke • 5 years ago
Not every one will use this. Besides, no one would like to remove any of those frm SERP.
Patrick Leonard • 5 years ago
I don’t think there’s much downside for the typical user, but when special circumstances arise it would still be nice to have some granular control.
Vered de Leeuw • 5 years ago
Thank you! Yes! I want to opt it to snippets. I’m sure you can solve any potential legal issue by having your lawyers add a simple disclaimer. Please don’t complicate things by making us manually opt-in as Alvern suggests.
Alvern @ Success Unscrambled • 5 years ago
Personally, I think it is a wise move because anyone using a plugin that improves their chances of gaining exposure in the SERPs should be happy with this feature.
However, European law enforcers may not be happy with your assumption. So, maybe it will make sense to place an opt-in box in the Yoast SEO release 12.2 allowing people to confirm their choice just so that you can avoid legal action.
Any new users should also be given the option to opt-in to allowing their content to show up in the snippets as well.
findtricks • 5 years ago
I use Yoast on my blog and if i do mistake or changes in robot.txt, then i have face any problem or not ?
Nathanaël Landais • 5 years ago
That is not how law works… You cannot “assume” that people opt-in.
An opt-in must be, well, an opt-in, an active decision from your users.
Jake Bohall • 5 years ago
This is why you are supposed to read the change-logs when you update a plugin.
I do think it would be a fair request to have a checkbox under general settings that lets you toggle this on/off … but just like any other feature, there are defaults when you install a plugin, and you should be aware of them and make choices that are the best fit for your business.
Dinesh • 5 years ago
It is always good to cope up with the change Google introduces. Looking forward to the plugin update and see how I can implement.
Claudia • 5 years ago
So is “max-snippet:-1” the right setting if I want the snippets in the search results?
^ Post author
Joost de Valk • 5 years ago
Yes!
Mentik • 5 years ago
So -1 meaning there are no max length of text in snippet?
NigerianWiki • 5 years ago
Hello… I don’t quite get it. I use Yoast on my blog and I don’t think this is a wise move. Making this decision on behalf of all Yoast users isn’t the way to go. This should only affect a section of bloggers and not all. Not fair in my opinion.
^ Post author
Joost de Valk • 5 years ago
Why don’t you like it? You don’t want to opt in to have your snippets show up everywhere? Or is a particular setting bothering you?
I really don’t think it is going to matter at all, because we still have crawling options to stop crawling.
Google’s official blog post listed five ways to control indexing:
Noindex in robots meta tags
404 and 410 HTTP status codes
Password protection
Disallow in robots.txt
Search Console Remove URL tool
So make sure you have apply above options on your website.
Thanks
I cannot fully understand about the new google meta thing. But somehow if this would help every blogs then this is a must.
of course default opt-in for all Google snippet features is the way to go and the ‘same as it used to be’ approach that anyone not following up on SEO on a daily basis needs. Thank you for that!
Hi,
The story behind this meta is about money. Some french journalists wanted to get money from google for using/displaying in the search results their content without asking/pay for it…
To me it is a good thing you did opt-in for us all as we struggle for being visible in the search results. But indeed, legaly speaking, maybe you could let the option to turn it of globally (with the risk that newbies ask why they are not visible when turned off).
Legally speaking, Google should make sure they have that opt in, if we have to do something to collect it on their behalf, it’s probably not even legal. I don’t know though, because IANAL :)
Legally speaking, in the EU — and this is my lay opinion as a Brit which is offered free — people must opt-in not opt-out. This is the same with subscriptions. Though I believe you have taken the right technical decision, EU regulation and law derived thereof would be used to argue otherwise.
Can I suggest new installs of Yoast SEO offer authors/admins the option to opt-in to showing snippets in search results. The option cannot be pre-ticked — another EU regulation.
Thanks for all your work on Yoast.
I don’t think we should be collecting opt ins on Google’s behalf. I’ll happily let them handle that themselves.
I am so happy that you a Yoast take care of these kinds of thing, so this Joost does not need to bother with it.
This is indeed away from the SEO mainstream media. It is good to know that Yoast team is preparing ahead of its massive implementation. Good move
Not every one will use this. Besides, no one would like to remove any of those frm SERP.
I don’t think there’s much downside for the typical user, but when special circumstances arise it would still be nice to have some granular control.
Thank you! Yes! I want to opt it to snippets. I’m sure you can solve any potential legal issue by having your lawyers add a simple disclaimer. Please don’t complicate things by making us manually opt-in as Alvern suggests.
Personally, I think it is a wise move because anyone using a plugin that improves their chances of gaining exposure in the SERPs should be happy with this feature.
However, European law enforcers may not be happy with your assumption. So, maybe it will make sense to place an opt-in box in the Yoast SEO release 12.2 allowing people to confirm their choice just so that you can avoid legal action.
Any new users should also be given the option to opt-in to allowing their content to show up in the snippets as well.
I use Yoast on my blog and if i do mistake or changes in robot.txt, then i have face any problem or not ?
That is not how law works… You cannot “assume” that people opt-in.
An opt-in must be, well, an opt-in, an active decision from your users.
This is why you are supposed to read the change-logs when you update a plugin.
I do think it would be a fair request to have a checkbox under general settings that lets you toggle this on/off … but just like any other feature, there are defaults when you install a plugin, and you should be aware of them and make choices that are the best fit for your business.
It is always good to cope up with the change Google introduces. Looking forward to the plugin update and see how I can implement.
So is “max-snippet:-1” the right setting if I want the snippets in the search results?
Yes!
So -1 meaning there are no max length of text in snippet?
Hello… I don’t quite get it. I use Yoast on my blog and I don’t think this is a wise move. Making this decision on behalf of all Yoast users isn’t the way to go. This should only affect a section of bloggers and not all. Not fair in my opinion.
Why don’t you like it? You don’t want to opt in to have your snippets show up everywhere? Or is a particular setting bothering you?