xml sitemaps do you need them

What is an XML sitemap
and why should you have one?

What is an XML sitemap and why should you have one?

June 20th, 2017 – 18 Comments

A good XML sitemap is a roadmap to all important pages of a website. This roadmap guides Google to all main content on a website. Having an XML sitemap can be beneficial for SEO, as Google can retrieve essential pages of a website very fast, even if the internal linking of a site isn’t flawless. Here, we’ll explain what XML sitemaps are and how they help you with your rankings.

What are XML sitemaps?

You want Google to crawl every important page of your website. But it can happen pages don’t have any – internal – links to them, which will make them hard to find. You can use an XML sitemap to make sure Google can find and crawl all pages you deem essential on your website. An XML sitemap contains all important pages of a site to help Google determine the structure of it:

XML Sitemap Yoast

The XML sitemap of Yoast.com

 

The XML sitemap above shows the XML sitemap of the Yoast website, which the Yoast SEO plugin created. If you read further down the article, we’ll explain exactly how our plugin helps you create the best XML sitemaps. If you’re not using our plugin, it could be that your own XML sitemap looks a bit different but it will work the same.

As you can see the XML sitemap of Yoast shows several ‘index’ XML sitemaps: …/post-sitemap.xml, …/page-sitemap.xml, …/video-sitemap.xml etc. This categorization makes a site structure as clear as possible. If you click on one of the index XML sitemaps, you’ll see all URLs in that specific sitemap. For example, if you click on ‘…/post-sitemap.xml’ you’ll see all the post URLs of Yoast.com (click on image to enlarge):

XML Post Sitemap Yoast

The post XML sitemap of Yoast.com

The date at the end of each line tells Google when we’ve last updated the post. This is beneficial for SEO because you want Google to crawl your updated content fast. When a date in the XML sitemap changes, Google knows that there is new content to crawl and index.

Sometimes it’s necessary to split an index XML sitemap because of the number of URLs in it. The limit to the number of URLs in one separate XML sitemap is set to 50.000 URLs. This means, for example, that if your website has over 50.000 posts, you should add two separate XML sitemaps for the post URLs. So, you’re actually adding another index XML sitemap. We’ve set the limit to 1.000 URLs in the Yoast SEO plugin to keep your XML sitemap loading fast.

What websites need an XML sitemap?

If we look at Google’s documentation, they say that XML sitemaps are beneficial for “really large websites”, for “websites with large archives”, for “new websites with just a few external links to it” and for “websites which use rich media content”.

We agree that these types of websites will definitely benefit from having an XML sitemap. However, at Yoast, we think an XML sitemap is beneficial for every website. On each website, you want Google to easily find the most important pages and to know when you’ve last updated those pages. That’s why we’ve added this function to the Yoast SEO plugin. 

Optimize your site for search & social media and keep it optimized with Yoast SEO Premium »

Yoast SEO for WordPress pluginBuy now » Info

Which pages should be in your XML sitemap?

How do you decide which pages you need to include in your XML sitemap? Always start by thinking of the relevancy of a URL: when a visitor lands on a specific URL, is it a good result? Do you want visitors to land on that URL? If not, that URL probably shouldn’t be in your XML sitemap. However, if you really don’t want that URL to show up in the search results you’ll need to add a ‘noindex, follow’ tag. Leaving it out of your XML sitemap doesn’t mean Google won’t index the URL. If Google can find it by following links, Google can index the URL.

Example 1: a starting blog

Let’s take the example of a new blog. The owner wants Google to find new URLs of the blog fast to make sure his target group can find his blog in Google. So it’s a good idea to create an XML sitemap right away. The owner has created some describing categories for the first posts and he has written the first posts. He has also set up some tags to start with. However, he doesn’t have enough content yet to fill the tag overview pages with. Since these tag overview pages contain “thin content”, it’s not valuable to show them to the visitors yet. It’s, therefore, better to leave the tag’s URLs out of the XML sitemap for now. In this case, the tag pages could also be set to ‘noindex, follow’ because you don’t want people to land on those URLs from the search results.

Example 2: media & images

Another example of an unnecessary XML sitemap – in most cases – is the ‘media’ or ‘image’ XML sitemap. Since your images are probably used within your pages and posts, the images are already included in your ‘post’ sitemap or your ‘page’ sitemap. Adding a separate ‘media’ or ‘image’ XML sitemap would be redundant. We recommend always leaving this one out of your XML sitemap. Only when images are your main business you can make an exception. When you’re a photographer, for example, you probably do want to show a separate ‘media’ or ‘image’ XML sitemap to Google.

How to make Google find your XML sitemap

If you want Google to find your XML sitemap fast, you have to add it to your Google Search Console account. You can find the sitemaps in Search Console by navigating to ‘Crawl’ and then clicking on ‘Sitemaps’. You’ll immediately see if your XML sitemap is already added to Search Console. If not, click on the ‘Add/Test sitemap’ button which you see on the right of the arrow in the image below.

Google Search Console XML Sitemap Yoast

The XML sitemap of Yoast is added to Google Search Console

 

As you can see in the image, adding your XML sitemap can be helpful to check whether all pages in your sitemap are really indexed by Google. If there is a big difference in the ‘submitted’ and ‘indexed’ number of a certain sitemap, we recommend analyzing this further. Maybe an error prevents some pages from being indexed or perhaps you should just add more content or links to the content that’s not indexed yet.

Yoast SEO and XML sitemaps

Because of the importance of XML sitemaps, we’ve added this functionality to our Yoast SEO plugin. XML sitemaps are available for both the free and the premium version of the plugin.

Yoast SEO creates an XML sitemap for your website automatically. You can find it by clicking on ‘XML Sitemaps’ in the sidebar of your WordPress install:

Yoast SEO tabs in WordPress backend

The XML Sitemaps tab in Yoast SEO

 

 

 

 

 

 

In the screen that follows you can enable or disable the XML sitemaps of your website. In addition to that, you can click on the ‘XML sitemap’ button to check your XML sitemap in your browser:

XML Sitemap settings in Yoast SEO

XML sitemap settings in Yoast SEO

In the tabs below the ‘enabled’ or ‘disabled’ toggle, you can find the different sitemaps you can in- or exclude from your XML sitemap: Users/Authors, Post Types and Taxonomies. On top of that, you can also exclude specific posts from the XML sitemap if you think the content of that post isn’t valuable enough.

Check your own XML sitemap!

Now you’ve read this complete post, you know it’s important to have an XML sitemap, because having one can help your site’s SEO. Google can easily access your most important pages and posts if you add the right URLs to your XML sitemap. In addition to that, Google can also find updated content easily, so they know if a certain URL needs to be crawled again. Lastly, adding your XML sitemap to Google Search Console helps Google find your sitemap fast and, besides that, it allows you to check for sitemap errors.

Now go check your own XML sitemap and see if you’re doing all of this correctly!

Read more: ‘WordPress SEO tutorial: definite guide to higher ranking’ »


Leave a reply

Your email address will not be published. Required fields are marked *

18 Responses to What is an XML sitemap and why should you have one?

  1. Alexander Dolgan
    By Alexander Dolgan on 22 June, 2017

    What about css property background-image? How engines can recognize alt tag for them?

  2. Preeti Mittal
    By Preeti Mittal on 22 June, 2017

    XML site map gives an idea to google about structure of your website and webpages. It help google to index your website accordingly.

  3. Dominique Verniers
    By Dominique Verniers on 21 June, 2017

    Can a sitemap replace the href lang tags in your pages? I think I read that somewhere in the google search console support pages. Any tools to generate one?

  4. Mark
    By Mark on 21 June, 2017

    Hi Meike,
    Do you have to attent Google on a new XML-sitemap? Or do they know, once you have shown them the way?

  5. Saurabh tiwari
    By Saurabh tiwari on 21 June, 2017

    I should thank you for writing on this topic, very useful information about sitemaps you have given through this Post.

  6. Lrers
    By Lrers on 21 June, 2017

    Thanks for sharing great info with me

  7. Jase
    By Jase on 21 June, 2017

    Nice to finally see an article on this with clear instructions. Only 24hrs too late for me. Tried googling n thrashing thru Yoast help files for over 1.5 years on rare occasions til i gave up in frustration to suss this. I muddled onto it by trial n error just yesterday! yay – site fully seen by Google at last – just as you show.
    Put this in your help files please to help other lost souls – as Yoast attracts many beginners. ….
    Why clear instructions like this are not there already mystifies me.

    • Meike Hendriks
      By Meike Hendriks on 21 June, 2017

      Hi Jase,

      I’m sorry it’s too late for you! We will definitely consider your tip about adding a knowledgebase article about XML sitemaps. Thank you.

  8. joão l. b. almeida
    By joão l. b. almeida on 20 June, 2017

    Thanks for the post. I have created 4 properties for my domain (http with www), (http without www), (https with www) and (https without www). When I add the sitemap on Search Console, I have to choose one property. Which property should I choose? Or do I have to add the sitemap for each of the 4 properties?

    • Meike Hendriks
      By Meike Hendriks on 21 June, 2017

      Hi Joao,

      We recommend setting your actual domain as the preferred domain and also adding the XML sitemap to this preferred domain. Looking at your website, this should be the https version with www. Good luck!

  9. Jason Quinlan
    By Jason Quinlan on 20 June, 2017

    Nice post,I like how are you emphasized what doesn’t belong in your sitemap. It is such a newbie mistake thin pages and TOC pages to a site map. Great work!

    • Meike Hendriks
      By Meike Hendriks on 21 June, 2017

      Thanks, Jason!

  10. danilo alba
    By danilo alba on 20 June, 2017

    hi, i have the sitemap created by yoast and loaded in the GSC.
    However Semrush is pointing an error as it is not finding the /sitemap.xml, as the directory from the Yoast is slightly different.
    Do i have to install the /sitemap.xml in the root or it will give conflict with Yoast Sitemap? or i just disconsider semrush error?
    cheers

    • Meike Hendriks
      By Meike Hendriks on 21 June, 2017

      Hi Danilo,

      I recommend checking if Google Search Console is giving any errors regarding your XML sitemap. If everything is indexed properly according to Search Console, I would ignore the Semrush error. Google is the one you want to understand your XML sitemap:)

  11. Adrian Lyons
    By Adrian Lyons on 20 June, 2017

    This is great and all but are you guys going to get talking with elegantthemes and fix your XML generator to work properly?

    • Meike Hendriks
      By Meike Hendriks on 21 June, 2017

      Hi Adrian,

      Please create an issue on GitHub if something isn’t working properly:)

  12. Daniel Boswell
    By Daniel Boswell on 20 June, 2017

    Meike I notice that in your Google Search Console example, you only put the XML index file. I’ve been adding each individual sitemap listed in the index file when I set up my sitemaps in GSC. I was under the impression I had to list each one individually. I would much rather just list the index file and be done with it. I just want to confirm its cool to just list the index XML file from now on.

    • Meike Hendriks
      By Meike Hendriks on 21 June, 2017

      Hi Daniel,

      Adding your index XML sitemap is enough! Google will automatically find all the XML sitemaps within the index.


Check out our must read articles about Technical SEO