We’ve said it way back when, but we’ll repeat it: it keeps amazing us that there are still people using just a robots.txt files to prevent indexing of their site in Google or Bing. As a result, their site shows up in the search engines anyway. Do you know why it keeps amazing us? Because …Read: "Preventing your site from being indexed, the right way"
Crawl directives archives
Recent Crawl directives articles
The robots.txt file is a file you can use to tell search engines where they can and cannot go on your site. Learn how to use it to your advantage!Read: "The ultimate guide to robots.txt"
This guide discusses what hreflang is, what it is for and gives in-depth information on how to implement it for your multilingual websites.Read: "hreflang: the ultimate guide"
With the meta robots tag you can control what search engine spiders do on your site. Learn what it does right here!Read: "The ultimate guide to the meta robots tag"
In this post, we’ll go into the basics of technical SEO. Now, discussing the basics of technical SEO might seem like a contradiction in terms. Nevertheless, some basic knowledge about the more technical side of SEO can mean the difference between a high ranking site and a site that doesn’t rank at all. Technical SEO …Read: "What’s technical SEO? 8 technical aspects everyone should know"
Some of the pages of your site serve a purpose, but that purpose isn’t ranking in search engines or even getting traffic to your site. These pages need to be there, as glue for other pages or simply because regulations require them to be accessible on your website. If you regularly read our blog, you’ll know …Read: "Which pages to noindex or nofollow on your site?"
Links are an important part of SEO. Without links, Google (or other search engines) may not discover your pages, or might not think that they’re important. Sometimes, though, you might want Google not to follow a link. Or you might want to tell them a particular is sponsored, or added to your page by a …Read: "What are sponsored, nofollow and ugc links, and why use them?"
Some posts and pages should not show up in search results. To make sure they don’t show up, you should tell search engines to exclude them. You do this with a meta robots noindex tag. Setting a page to noindex makes sure search engines never show it in their results. Here, we’ll explain how easy it …Read: "Noindex a post or page in WordPress, the easy way!"
Your site needs to be up and running if you want to be found in search engines. If you aren’t blocking anything — deliberately or accidentally — search engine spiders can crawl and index it. You probably know that Yoast SEO has lots of options to determine what does and doesn’t need to be indexed, but …Read: "Yoast SEO & Ryte: Checking your site’s indexability"
Google doesn’t always spider every page on a site instantly. In fact, sometimes, it can take weeks. This might get in the way of your SEO efforts. Your newly optimized landing page might not get indexed. At that point, it’s time to optimize your crawl budget. We’ll discuss what a ‘crawl budget’ is and what you …Read: "How to optimize your crawl budget"