Crawl directives archives

Indexing is an important part of what a search engine does. Without indexing, all the pages Googlebot crawls don’t have a place to live — and the ranking systems don’t have the input they need to do their work. If Google can’t index your site it can’t appear in the search results. The basics of …

Ranking in search engines requires a website with flawless technical SEO and great, relevant content. Luckily, the Yoast SEO plugin takes care of (almost) everything on your WordPress site. Still, if you really want to get the most out of your website and keep outranking the competition, some basic knowledge of technical SEO is a must. …

How does a new website start ranking? Does it just magically appear in Google after you’ve launched it? What things do you have to do to start ranking in Google and get traffic from the search engines? Here, I explain the first steps you’ll need to take right after the launch of your new website. …

We’ve said it way back when, but we’ll repeat it: it keeps amazing us that there are still people using just a robots.txt files to prevent indexing of their site in Google or Bing. As a result, their site shows up in the search engines anyway. Do you know why it keeps amazing us? Because …

The robots.txt file is a file you can use to tell search engines where they can and cannot go on your site. Learn how to use it to your advantage!

This guide discusses what hreflang is, what it is for and gives in-depth information on how to implement it for your multilingual websites.

With the meta robots tag you can control what search engine spiders do on your site. Learn what it does right here!

In this post, we’ll go into the basics of technical SEO. Now, discussing the basics of technical SEO might seem like a contradiction in terms. Nevertheless, some basic knowledge about the more technical side of SEO can mean the difference between a high ranking site and a site that doesn’t rank at all. Technical SEO …

Some of the pages of your site serve a purpose, but that purpose isn’t ranking in search engines or even getting traffic to your site. These pages need to be there, as glue for other pages or simply because regulations require them to be accessible on your website. If you regularly read our blog, you’ll know …

Links are an important part of SEO. Without links, Google (or other search engines) may not discover your pages, or might not think that they’re important. Sometimes, though, you might want Google not to follow a link. Or you might want to tell them a particular is sponsored, or added to your page by a …