Want to learn about HTTP headers?

Take our Technical SEO training!

  • Which technical SEO errors hurt your site?
  • Solve them and climb the rankings!
  • Improve your site speed on the go
  • On-demand SEO training by Yoast
More info

Recent HTTP headers articles



Preventing your site from being indexed, the right way

5 June 2017 by - 9 Comments

We’ve said it in 2009, and we’ll say it again: it keeps amazing us that there are still people using just a robots.txt files to prevent indexing of their site in Google or Bing. As a result, their site shows up in the search engines anyway. You know why it keeps amazing us? Because robots.txt …

Read: "Preventing your site from being indexed, the right way"
robots meta

HTTP status codes and what they mean for SEO

20 January 2017 by - 8 Comments

HTTP status codes, like 404, 301 and 500, might not mean much to a regular visitor, but for SEOs they are incredibly important. Not only that, search engine spiders, like Googlebot, use these to determine the health of a site. These status codes offer a way of seeing what happens between the browser and the server. …

Read: "HTTP status codes and what they mean for SEO"
HTTP status codes

Playing with the X-Robots-Tag HTTP header

3 January 2017 by - 2 Comments

Traditionally, you will use a robots.txt file on your server to manage what pages, folders, subdomains or other content search engines will be allowed to crawl. But did you know that there’s also such a thing as the X-Robots-Tag HTTP header? Here, we’ll discuss what the possibilities are and how this might be a better …

Read: "Playing with the X-Robots-Tag HTTP header"