HTTP headers

The entire web is based on HTTP, knowing the basics about the different types of HTTP headers you can send is a base requirement for anyone doing web development and website optimization.


HTTP status codes and what they mean for SEO

20 January 2017 by Edwin Toonen » - 2 Comments

HTTP status codes

HTTP status codes, like 404, 301 and 500, might not mean much to a regular visitor, but for SEOs they are incredibly important. Not only that, search engine spiders, like Googlebot, use these to determine the health of a site. These status codes offer a way of seeing what happens between the browser and the server. »

Category: Technical SEO
Tag:

Playing with the X-Robots-Tag HTTP header

3 January 2017 by Joost de Valk » - 2 Comments

x robots meta tag

Traditionally, you will use a robots.txt file on your server to manage what pages, folders, subdomains or other content search engines will be allowed to crawl. But did you know that there’s also such a thing as the X-Robots-Tag HTTP header? In this post we’ll discuss what the possibilities are and how this might be »

Category: Technical SEO
Tags: , , ,

HTTP 451: content unavailable for legal reasons

21 January 2016 by Joost de Valk »

At the end of last year, a new HTTP status code saw the light. This status code, HTTP 451, is intended to be shown specifically when content has been blocked for legal reasons. If you’ve received a take down request, or are ordered by a judge to delete content, this is the status code that »

Category: WordPress
Tag:

Should we move to an all HTTPS web?

13 January 2014 by Joost de Valk » - 45 Comments

There was a bit of tweeting in the SEO community today because Bing introduced an HTTPS version of their site and people thought that would mean they’d lose their keyword data. That’s not true, if you take the right precautions. I thought I’d write a bit of an intro in how all this works so »

Category: Technical SEO
Tags: , ,

HTTP 503: Handling site maintenance correctly for SEO

18 March 2013 by Joost de Valk »

Last week I got a few messages from Google Webmaster Tools, saying it couldn’t access the robots.txt file on a site of a client. Turns out the client didn’t handle scheduled downtime correctly, causing problems with Google. While this article covers some rather basic technical SEO the last bit might be interesting for more advanced »

Category: Technical SEO
Tag:

Preventing your site from being indexed, the right way

17 December 2009 by Joost de Valk » - 36 Comments

It keeps amazing me that I keep seeing people use robots.txt files to prevent sites from being indexed and thus showing up in the search engines. You know why it keeps amazing me? Because robots.txt doesn’t actually do the latter, even though it does prevent your site from being indexed. Let’s go through some terms »

Category: Technical SEO
Tags: , ,

Practical Guide to creating a 404 Error Page for WordPress

3 November 2009 by Joost de Valk » - 78 Comments

I make mistakes. You make mistakes. We all do. And some of these mistakes end up providing our readers with a 404 page. Chances are that page says “Error 404: file not found”. How does that help your visitor? Instead of just identifying the problem, your 404 error page needs to offer a solution. In »

Tags: ,