Crawl directives archives

Recent Crawl directives articles



SEO for a new website: the very first things to do

How does a new website start ranking? Does it just magically appear in Google after you’ve launched it? What things do you have to do to start ranking in Google and get traffic from the search engines? Here, I explain the first steps you’ll need to take right after the launch of your new website. …

Read: "SEO for a new website: the very first things to do"
SEO for a new website: the very first things to do

Preventing your site from being indexed, the right way

19 March 2021 | 18 Comments | Joost de Valk

We’ve said it way back when, but we’ll repeat it: it keeps amazing us that there are still people using just a robots.txt files to prevent indexing of their site in Google or Bing. As a result, their site shows up in the search engines anyway. Do you know why it keeps amazing us? Because …

Read: "Preventing your site from being indexed, the right way"
Preventing your site from being indexed, the right way




What’s technical SEO? 8 technical aspects everyone should know

In this post, we’ll go into the basics of technical SEO. Now, discussing the basics of technical SEO might seem like a contradiction in terms. Nevertheless, some basic knowledge about the more technical side of SEO can mean the difference between a high ranking site and a site that doesn’t rank at all. Technical SEO …

Read: "What’s technical SEO? 8 technical aspects everyone should know"
What’s technical SEO? 8 technical aspects everyone should know

Which pages to noindex or nofollow on your site?

29 June 2020 | 11 Comments | Michiel Heijmans

Some of the pages of your site serve a purpose, but that purpose isn’t ranking in search engines or even getting traffic to your site. These pages need to be there, as glue for other pages or simply because regulations require them to be accessible on your website. If you regularly read our blog, you’ll know …

Read: "Which pages to noindex or nofollow on your site?"
Which pages to noindex or nofollow on your site?

What are sponsored, nofollow and ugc links, and why use them?

Links are an important part of SEO. Without links, Google (or other search engines) may not discover your pages, or might not think that they’re important. Sometimes, though, you might want Google not to follow a link. Or you might want to tell them a particular is sponsored, or added to your page by a …

Read: "What are sponsored, nofollow and ugc links, and why use them?"
What are sponsored, nofollow and ugc links, and why use them?