Skip to content

Robots meta changes for Google

Joost de Valk 26 September 2019 21 , ,

Sometimes Google does announcements about new features and we go “huh, why did they do that?” This week we had one of those. Google introduced a new set of robots meta controls, that allows sites to limit the display of their snippets in the search results. There is a reason for that, but they buried that reason far, far away.

With their newly introduced robots meta controls, you can “say” things like: I don’t want you to show more than 200 characters from my site! Or: I don’t want you to show images from my site. None of this made sense to us. Why would anyone who truly wants to optimize for Google’s search results, do any of that?

Suddenly, we make a dramatic plot twist: France turns out to have introduced a new copyright law, which is the first implementation of Europe’s new copyright law. I am not a lawyer, but as Google explains it in this blog post, it requires for publishers to opt in to a display of their snippets, whereas in the past you had to opt out. Note that that blog post is in French, on their French blog, but there’s an English version in the bottom of that post.

It turns out that you can also use these new robots meta features to opt in to all those displays. Basically, use the meta robots to opt in to have your snippet text, videos and images being displayed at all.

robots meta?

Robots meta is hard enough as it is, read our ultimate guide to understand what you can do with it.

Because this French law implements the new European law, chances are that the other implementations of this copyright law across Europe are going to require similar action. This would mean more countries would need to opt in if they don’t want to lose their snippets. We don’t know that, of course, but of course: better safe than sorry.

A simple change to Yoast SEO

Because this potentially affects so many countries, we’ve decided to make a change for all Yoast SEO users. Every page will have the following robots meta bit added to its robots meta string if that page was not already set to noindex or nosnippet:

max-snippet:-1, max-image-preview:large, max-video-preview:-1

This makes sure that nobody runs into unexpected surprises and we’re ready for implementations of this law around Europe.

I want to change that robots meta value

If you want to change that robots meta value, we have a filter in our code that allows you to programmatically change that. That filter is called wpseo_robots, should you want to play with it.

You are opting in by using Yoast SEO

We realize that this means that we’re opting you in to all of these snippet features in Yoast SEO. I think it’s fair to say that if you use Yoast SEO to optimize your search results, we can assume that that’s what you want. At the same time, you might want somewhat more granular control over these values, if that’s true, please let us know in the comments!

The Yoast SEO release containing this change will be 12.2. This will be released next Tuesday, October 1.

Avatar of Joost de Valk
Joost de Valk

Joost is an internet entrepreneur and the founder of Yoast. He has a long history in WordPress and digital marketing. On our blog, he has written a lot about SEO in general, technical SEO and important topics related to SEO.

Discussion (21)

  1. I really don’t think it is going to matter at all, because we still have crawling options to stop crawling.

    Google’s official blog post listed five ways to control indexing:

    Noindex in robots meta tags
    404 and 410 HTTP status codes
    Password protection
    Disallow in robots.txt
    Search Console Remove URL tool

    So make sure you have apply above options on your website.

    Thanks

  • I cannot fully understand about the new google meta thing. But somehow if this would help every blogs then this is a must.

  • of course default opt-in for all Google snippet features is the way to go and the ‘same as it used to be’ approach that anyone not following up on SEO on a daily basis needs. Thank you for that!

  • Hi,
    The story behind this meta is about money. Some french journalists wanted to get money from google for using/displaying in the search results their content without asking/pay for it…
    To me it is a good thing you did opt-in for us all as we struggle for being visible in the search results. But indeed, legaly speaking, maybe you could let the option to turn it of globally (with the risk that newbies ask why they are not visible when turned off).

    • Legally speaking, Google should make sure they have that opt in, if we have to do something to collect it on their behalf, it’s probably not even legal. I don’t know though, because IANAL :)

      • Legally speaking, in the EU — and this is my lay opinion as a Brit which is offered free — people must opt-in not opt-out. This is the same with subscriptions. Though I believe you have taken the right technical decision, EU regulation and law derived thereof would be used to argue otherwise.

        Can I suggest new installs of Yoast SEO offer authors/admins the option to opt-in to showing snippets in search results. The option cannot be pre-ticked — another EU regulation.

        Thanks for all your work on Yoast.

  • I am so happy that you a Yoast take care of these kinds of thing, so this Joost does not need to bother with it.

  • This is indeed away from the SEO mainstream media. It is good to know that Yoast team is preparing ahead of its massive implementation. Good move

  • Not every one will use this. Besides, no one would like to remove any of those frm SERP.

  • I don’t think there’s much downside for the typical user, but when special circumstances arise it would still be nice to have some granular control.

  • Thank you! Yes! I want to opt it to snippets. I’m sure you can solve any potential legal issue by having your lawyers add a simple disclaimer. Please don’t complicate things by making us manually opt-in as Alvern suggests.

  • Personally, I think it is a wise move because anyone using a plugin that improves their chances of gaining exposure in the SERPs should be happy with this feature.

    However, European law enforcers may not be happy with your assumption. So, maybe it will make sense to place an opt-in box in the Yoast SEO release 12.2 allowing people to confirm their choice just so that you can avoid legal action.

    Any new users should also be given the option to opt-in to allowing their content to show up in the snippets as well.

  • I use Yoast on my blog and if i do mistake or changes in robot.txt, then i have face any problem or not ?

  • It is always good to cope up with the change Google introduces. Looking forward to the plugin update and see how I can implement.

  • So is “max-snippet:-1” the right setting if I want the snippets in the search results?

  • Hello… I don’t quite get it. I use Yoast on my blog and I don’t think this is a wise move. Making this decision on behalf of all Yoast users isn’t the way to go. This should only affect a section of bloggers and not all. Not fair in my opinion.