HTTP headers

The entire web is based on HTTP, knowing the basics about the different types of HTTP headers you can send is a base requirement for anyone doing web development and website optimization.

HTTP 451: content unavailable for legal reasons

21 January 2016 by Joost de Valk »


At the end of last year, a new HTTP status code saw the light. This status code, HTTP 451, is intended to be shown specifically when content has been blocked for legal reasons. If you’ve received a take down request, or are ordered by a judge to delete content, this is the status code that »

Category: WordPress

Should we move to an all HTTPS web?

13 January 2014 by Joost de Valk » - 45 Comments

There was a bit of tweeting in the SEO community today because Bing introduced an HTTPS version of their site and people thought that would mean they’d lose their keyword data. That’s not true, if you take the right precautions. I thought I’d write a bit of an intro in how all this works so »

Category: Technical SEO
Tags: , ,

HTTP 503: Handling site maintenance correctly for SEO

18 March 2013 by Joost de Valk »


Last week I got a few messages from Google Webmaster Tools, saying it couldn’t access the robots.txt file on a site of a client. Turns out the client didn’t handle scheduled downtime correctly, causing problems with Google. While this article covers some rather basic technical SEO the last bit might be interesting for more advanced »

Category: Technical SEO

Preventing your site from being indexed, the right way

17 December 2009 by Joost de Valk » - 36 Comments

It keeps amazing me that I keep seeing people use robots.txt files to prevent sites from being indexed and thus showing up in the search engines. You know why it keeps amazing me? Because robots.txt doesn’t actually do the latter, even though it does prevent your site from being indexed. Let’s go through some terms »

Category: Technical SEO
Tags: , ,

Practical Guide to creating a 404 Error Page for WordPress

3 November 2009 by Joost de Valk » - 78 Comments

I make mistakes. You make mistakes. We all do. And some of these mistakes end up providing our readers with a 404 page. Chances are that page says “Error 404: file not found”. How does that help your visitor? Instead of just identifying the problem, your 404 error page needs to offer a solution. In »

Tags: ,

Playing with the X-Robots-Tag HTTP header

20 January 2008 by Joost de Valk » - 31 Comments

Ever since the announcement on the Google Blog and more recently Yahoo’s announcement that they’ve enhanced their support for it, I’ve been meaning to play with the X-Robots-Tag header. This HTTP header allows you to do what you’d normally do in a robots meta tag, in an HTTP header, which has some pretty cool appliances. »

Category: Technical SEO
Tags: , , ,