In our plugin, you can connect Google Search Console to Yoast SEO. This verifies your website for your Google Search Console account and allows you to view your crawl errors. Especially when you have a large site, the number of crawl errors might scare you. Here, we’ll explain a bit more about crawl errors and show you how to fix them, using Yoast SEO Premium.
- What are crawl errors?
- Why should you fix crawl errors
- How to connect Yoast SEO to Search Console to retrieve crawl errors
- View and resolve crawl issues in Yoast SEO
- An easy way to create redirects
Update May 2, 2019: Google has removed the API that allowed us to display Search Console errors in Yoast SEO. So currently, your Search Console report will be empty. Know that nothing is broken on your site and there’s no special action required from your side. You can read what our next steps are to fix this issue in the blog post Where did my Search Console errors go.
What are crawl errors?
Crawl errors occur when a search engine tries to reach a page on your website but fails at it. Google uses so-called Googlebots to crawl and index your page. Crawling, in layman’s language, is the process of Googlebot going over your pages, one link at a time. When crawling, its goal is to get to every important page on your site by following links on pages, in sitemaps, etc. Indexing, on the other hand, is what Googlebot does to take in all the content on your pages, to include it in its search result pages. Crawl errors might even prevent your page from appearing in search results.
There are two types of crawl errors:
- Site errors that affect your entire site. Think along the lines of connectivity issues with your web server, and problems fetching your robots.txt file.
- URL errors that affect a specific page on your website. Googlebot tried to crawl the URL but did not succeed somehow. It was able to resolve your DNS, connect to your server, fetch/read your robots.txt file, and then request the URL. But after that, something went wrong.
Google’s Crawl Errors Report documentation has more information on the types of crawl errors and what they mean.
Why should you fix crawl errors
Please monitor these errors, as too many errors could send a signal of low quality — or bad maintenance — to Google. Crawl errors might even prevent your page from appearing in search results, so you need to keep an eye on them. To fix, you can redirect 404 errors in Yoast SEO Premium, which we talk about a bit more at the end of this article. It is also possible to do that directly in your .htaccess file, but that’s a lot more work. After you fix your crawl errors in Yoast SEO, check the checkbox in front of the URL and click Mark As Fixed — this will just make sure the list is cleaned up, it won’t do much besides that.
We have found that a lot of people either ignore the errors here or forget to mark errors as fixed. This will only lead to a very long list of errors. Clean that list up now and then. If you want to check if any of these URLs are already fixed, click the link to find more information on the error and use Fetch as Google to see if the URL is now accessible for Google — or click Marked as fixed, of course.
How to connect Yoast SEO to Search Console to retrieve crawl errors
In our Yoast SEO plugin (free and paid), you can view the crawl errors that Google came across on your website. All you have to do is connect Google Search Console to Yoast SEO. In our plugin, we guide you through that process. We’ll explain the steps here as well.
The first thing you need to do to see your crawl errors in Yoast SEO is making a Google Search Console account and connect that to your website. If you haven’t yet, find out how to add your website to Google Search Console. After your site is verified in Search Console you can continue with the steps below to get the proper authorization:
1. Log in to your WordPress website.
When you’re logged in, you will be in your ‘Dashboard’.
2. Click on ‘SEO’.
On the left-hand side, you will see a menu. In that menu, click on ‘SEO’.
3. Click on ‘Search Console’.
The ‘SEO’ settings will expand providing you with additional options. Click on ‘Search Console’.
4. Under Settings, click on ‘ Click here to get a Google Authorization Code’.
5. Login to your Google account, if you are not logged in.
Please use an owner account. Restricted users may have limitations such as not being able to use the ‘Mark as fixed’ option.
6. Click ‘Allow’.
7. Copy the unique code that you see.
8. Paste the code in the plugin settings.
9. Click ‘Authenticate’ and save changes.
If you have more than one profile, select the profile for this website and click ‘Save changes’. Google Search Console allows you to claim each version of your site including HTTP, HTTPS, www and non-www. Each version is considered a separate property. Please select the property that matches your preferred domain format.
View and resolve crawl issues in Yoast SEO
Open the Search Console section in Yoast SEO and click the Desktop tab. Be sure to check the other tabs as well to find specific crawl errors. Here, you will find the information Yoast SEO collects from your Google Search Console account. In this table, you see the URL that gave an error, the date Google crawled it last, the date when Google detected the error first and the response code Google sent. In the screenshot, all response codes are 404 Not Found.
So, if you connect Google Search Console to Yoast SEO, you will have a great overview of how many crawl errors Google finds on your website. Now, you can go and create redirects for these 404s, or simply change them to 410s if that page is of absolutely no use to you anymore. More on status codes in this article. When you have ‘fixed’ the error, hover over the URL in Yoast SEO and click ‘mark as fixed’.
An easy way to create redirects
Yes! There is an easier way to complete this process, and it is called Yoast SEO Premium. Yoast SEO Premium is packed with cool features — one of our favorites is the redirect manager. This tool allows you to simply create your redirect in the plugin: Click ‘Create redirect,’ and Yoast SEO Premium will give you the option to create a redirect, or add another status code (301, 302, 307, 410, 451 are all possible). In case of a 301 redirect, like in the example, insert the URL you’d like that ‘old’ URL to redirect to. If you want to tell Google Search Console about this fix, leave the check ‘Mark as fixed’ as is and hit ‘Create Redirect.’ It’s as simple as that!
Read more: Which redirect shoud I use? »