HTTP 503: Handling site maintenance correctly for SEO

HTTP 503: Handling site maintenance correctly for SEO

Last update: 20 October, 2017

Sometimes, your site will need some down-time so you can fix things or update plugins. Most of the time, this tends to be a relatively short period in which Google will most likely not attempt to crawl your website. However, in the case that you need more time to get things fixed, chances are much higher that GoogleBot might come for a visit and be confronted with a website that’s down. So how do we prevent Google from deranking your website?

HTTP status codes and you

For those not familiar with HTTP status codes, here’s a brief summary of the ones that apply to you when dealing with site maintanence:

  • 200 OK. This status code dictates that the server was successful in returning a response.
  • 301 Moved permanently. This tells the browser that this page is no longer valid and will redirect to the correct page.
  • 302 / 307 Moved temporarily. There’s some history behind these two HTTP status codes, but what this dictates to the browser is that you’ll temporarily be redirecting the browser to a different page and that the current URL will eventually be back to its former state.
  • 404 Not Found. This status code means that the page you were trying to navigate to, couldn’t be found.
  • 503 Service unavailable. This is the one you want to be returning to Google when you’re dealing with site maintenance. It tells Google that you are in fact working on this page or that something else went wrong. Google knows that when this status code is returned, to check the page again later on. This is what we’ll be discussing a bit more.

Please note that Google will consider pages returning the 200 HTTP status code, despite there being an error (or very little content) on the page, as a “soft 404” in Google Webmaster Tools.

Telling Google you’re busy

If Google runs into a 404 whilst crawling your site, it’ll usually toss out that page from the search results until it comes back the next time to verify the page is back. However, if Google repeatedly runs into a 404 on that specific page, it’ll eventually postpone re-crawling which means that more time will pass before the page returns in the search results.

To overcome this potential longer loss of pagerank, you need to return a 503 status code whenever working on a particular page. The original definition of the 503 status code according to this RFC is:

The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. If known, the length of the delay MAY be indicated in a Retry-After header. If no Retry-After is given, the client SHOULD handle the response as it would for a 500 response.

What this means is that returning a 503 in combination with a Retry-After header which will tell Google how many minutes to wait before coming back. This does not mean Google will crawl again in exactly X minutes, but it’ll ensure Google doesn’t come back around to take a look anytime before then.

Adding the header

If you want to implement the header, there are a few options you can choose from.

Using the WordPress defaults

By default, WordPress already returns a 503 when updating plugins or WordPress core. WordPress allows you to override the default maintenance page by adding a maintenance.php to your wp-content/ directory. Please note that you’ll then be responsible for properly returning the 503 header. Plan on doing database maintenance? You’ll have to take care of that as well. Add a db-error.php file to your wp-content/ and make sure that you also properly return a 503 header here as well.

If you’re looking to add something fancier to your WordPress website, check out WP Maintenance Mode. This plugin also adds in a lot of extra features besides what we mentioned in the previous section.

If you’re just writing your own code and want a solution that’s easy to implement, you can add the following snippet to your codebase and call it in the code that determines if you’re in maintenance mode:


function set_503_header() {
    $protocol = 'HTTP/1.0';

    if ( $_SERVER['SERVER_PROTOCOL'] === 'HTTP/1.1' ) {
        $protocol = 'HTTP/1.1';
    }

    header( $protocol . '503 Service Unavailable', true, 503 );
    header( 'Retry-After: 3600' );
}

Note that the 3600 in the code snippet dictates the delay time in seconds. That means that the above sample will tell GoogleBot to return after an hour. It’s also possible to add a specific date and time in Retry-After, but you need to watch out with what you add here as adding faulty date might result in unexpected results.

Pro-tips

Caching

There are a few things you need to take into consideration when working with maintenance pages and returning 503 status codes. If you actively use caching, you might run into a situation where the cache isn’t properly passing the 503 status on, so please make sure you test this properly before actively using this on the live version of your website.

Robots.txt

Did you know it’s also possible to return a 503 status code for your robots.txt? Pierre Far from Google explains in this article that if you return a 503 status code on your robots.txt file, Google will halt crawling your website. The biggest advantage to this is less server load during maintenance periods.



Check out our must read articles about Analytics