Robots meta changes for Google

Sometimes Google does announcements about new features and we go “huh, why did they do that?” This week we had one of those. Google introduced a new set of robots meta controls, that allows sites to limit the display of their snippets in the search results. There is a reason for that, but they buried that reason far, far away.

With their newly introduced robots meta controls, you can “say” things like: I don’t want you to show more than 200 characters from my site! Or: I don’t want you to show images from my site. None of this made sense to us. Why would anyone who truly wants to optimize for Google’s search results, do any of that?

A French copyright law

Suddenly, we make a dramatic plot twist: France turns out to have introduced a new copyright law, which is the first implementation of Europe’s new copyright law. I am not a lawyer, but as Google explains it in this blog post, it requires for publishers to opt in to a display of their snippets, whereas in the past you had to opt out. Note that that blog post is in French, on their French blog, but there’s an English version in the bottom of that post.

It turns out that you can also use these new robots meta features to opt in to all those displays. Basically, use the meta robots to opt in to have your snippet text, videos and images being displayed at all.

robots meta?

Robots meta is hard enough as it is, read our ultimate guide to understand what you can do with it.

Because this French law implements the new European law, chances are that the other implementations of this copyright law across Europe are going to require similar action. This would mean more countries would need to opt in if they don’t want to lose their snippets. We don’t know that, of course, but of course: better safe than sorry.

A simple change to Yoast SEO

Because this potentially affects so many countries, we’ve decided to make a change for all Yoast SEO users. Every page will have the following robots meta bit added to its robots meta string if that page was not already set to noindex or nosnippet:

max-snippet:-1, max-image-preview:large, max-video-preview:-1

This makes sure that nobody runs into unexpected surprises and we’re ready for implementations of this law around Europe.

I want to change that robots meta value

If you want to change that robots meta value, we have a filter in our code that allows you to programmatically change that. That filter is called wpseo_robots, should you want to play with it.

You are opting in by using Yoast SEO

We realize that this means that we’re opting you in to all of these snippet features in Yoast SEO. I think it’s fair to say that if you use Yoast SEO to optimize your search results, we can assume that that’s what you want. At the same time, you might want somewhat more granular control over these values, if that’s true, please let us know in the comments!

The Yoast SEO release containing this change will be 12.2. This will be released next Tuesday, October 1.

Take your technical SEO to the next level

  • Get Technical SEO & Structured data training
  • Hands-on advice for your technical setup
  • Save $$ by buying both Yoast courses
  • Start online when you are ready for it
More info

22 Responses to Robots meta changes for Google

  1. Digital Shlok
    Digital Shlok  • 2 months ago

    I really don’t think it is going to matter at all, because we still have crawling options to stop crawling.

    Google’s official blog post listed five ways to control indexing:

    Noindex in robots meta tags
    404 and 410 HTTP status codes
    Password protection
    Disallow in robots.txt
    Search Console Remove URL tool

    So make sure you have apply above options on your website.

    Thanks

  2. Nhatty
    Nhatty  • 2 months ago

    I cannot fully understand about the new google meta thing. But somehow if this would help every blogs then this is a must.

  3. Andrea Bagnasco
    Andrea Bagnasco  • 2 months ago

    of course default opt-in for all Google snippet features is the way to go and the ‘same as it used to be’ approach that anyone not following up on SEO on a daily basis needs. Thank you for that!

  4. Nethel
    Nethel  • 2 months ago

    Hi,
    The story behind this meta is about money. Some french journalists wanted to get money from google for using/displaying in the search results their content without asking/pay for it…
    To me it is a good thing you did opt-in for us all as we struggle for being visible in the search results. But indeed, legaly speaking, maybe you could let the option to turn it of globally (with the risk that newbies ask why they are not visible when turned off).

    • Joost de Valk

      Legally speaking, Google should make sure they have that opt in, if we have to do something to collect it on their behalf, it’s probably not even legal. I don’t know though, because IANAL :)

      • Lee
        Lee  • 2 months ago

        Legally speaking, in the EU — and this is my lay opinion as a Brit which is offered free — people must opt-in not opt-out. This is the same with subscriptions. Though I believe you have taken the right technical decision, EU regulation and law derived thereof would be used to argue otherwise.

        Can I suggest new installs of Yoast SEO offer authors/admins the option to opt-in to showing snippets in search results. The option cannot be pre-ticked — another EU regulation.

        Thanks for all your work on Yoast.

        • Joost de Valk

          I don’t think we should be collecting opt ins on Google’s behalf. I’ll happily let them handle that themselves.

  5. Joost
    Joost  • 2 months ago

    I am so happy that you a Yoast take care of these kinds of thing, so this Joost does not need to bother with it.

  6. QuestionUnanswered
    QuestionUnanswered  • 2 months ago

    This is indeed away from the SEO mainstream media. It is good to know that Yoast team is preparing ahead of its massive implementation. Good move

  7. Godswill Chijioke
    Godswill Chijioke  • 2 months ago

    Not every one will use this. Besides, no one would like to remove any of those frm SERP.

  8. Patrick Leonard
    Patrick Leonard  • 2 months ago

    I don’t think there’s much downside for the typical user, but when special circumstances arise it would still be nice to have some granular control.

  9. Vered de Leeuw
    Vered de Leeuw  • 2 months ago

    Thank you! Yes! I want to opt it to snippets. I’m sure you can solve any potential legal issue by having your lawyers add a simple disclaimer. Please don’t complicate things by making us manually opt-in as Alvern suggests.

  10. Alvern @ Success Unscrambled
    Alvern @ Success Unscrambled  • 2 months ago

    Personally, I think it is a wise move because anyone using a plugin that improves their chances of gaining exposure in the SERPs should be happy with this feature.

    However, European law enforcers may not be happy with your assumption. So, maybe it will make sense to place an opt-in box in the Yoast SEO release 12.2 allowing people to confirm their choice just so that you can avoid legal action.

    Any new users should also be given the option to opt-in to allowing their content to show up in the snippets as well.

  11. findtricks
    findtricks  • 2 months ago

    I use Yoast on my blog and if i do mistake or changes in robot.txt, then i have face any problem or not ?

  12. Nathanaël Landais
    Nathanaël Landais  • 2 months ago

    That is not how law works… You cannot “assume” that people opt-in.
    An opt-in must be, well, an opt-in, an active decision from your users.

    • Jake Bohall
      Jake Bohall  • 2 months ago

      This is why you are supposed to read the change-logs when you update a plugin.

      I do think it would be a fair request to have a checkbox under general settings that lets you toggle this on/off … but just like any other feature, there are defaults when you install a plugin, and you should be aware of them and make choices that are the best fit for your business.

  13. Dinesh
    Dinesh  • 2 months ago

    It is always good to cope up with the change Google introduces. Looking forward to the plugin update and see how I can implement.

  14. Claudia
    Claudia  • 2 months ago

    So is “max-snippet:-1” the right setting if I want the snippets in the search results?

    • Joost de Valk

      Yes!

      • Mentik
        Mentik  • 2 months ago

        So -1 meaning there are no max length of text in snippet?

  15. NigerianWiki
    NigerianWiki  • 2 months ago

    Hello… I don’t quite get it. I use Yoast on my blog and I don’t think this is a wise move. Making this decision on behalf of all Yoast users isn’t the way to go. This should only affect a section of bloggers and not all. Not fair in my opinion.

    • Joost de Valk

      Why don’t you like it? You don’t want to opt in to have your snippets show up everywhere? Or is a particular setting bothering you?