Before Google can rank your content, it needs to discover it, be permitted and able to evaluate it, and index it. If any of these processes go wrong, you might find that your pages don’t show up in the search results.
Most of the time, you can rely on Google to correctly index your content, all by itself. After all, this process is one of the foundational parts of what Google is, and does.
Simply putting your content online isn’t always enough, however.
If you have technical issues, low-quality content, or incorrect indexing controls, you might trip up during those processes of discovery, evaluation and indexing might go wrong.
In order to index a page, Google has to be able to find it. That means that somewhere has to link to it – whether that’s from other indexed pages in the same site, or from other sites.
Depending on the relevance and quality of the places it’s linked from, it might take a little time for Google to schedule following those links and finding your pages.
That also means that the page can’t be ‘hidden’ – which, for example, might mean content being password protected, blocked via robots.txt, or only available to users in certain countries.
When Google has discovered the page, it will digest the content (including the HTML code and related assets) to assess the quality and relevance.
During this process, there are a number of things which can result in Google choosing not to index a page. They include:
- When it determines that the content of the page is ‘low quality’. E.g., if there’s a very low word count, or if the content is a close/direct duplicate of another page. Particular ‘over-optimized’ or ‘spammy’ pages may also be ignored.
- When it discovers specific indexing instructions on the page (such as a meta robots tag, or a canonical URL tag pointing at a different page). Google will make a judgment call in cases like this whether it should honor the instructions, but chances are, it’ll choose not to include the page.
If you’ve passed all of those tests, then your content should be successfully indexed and should turn up when you search for it.
HINT: try doing a ‘site’ search on Google (e.g.,
site:https://www.example.com/example-page/) to see if a specific URL has been included in the index).
Bear in mind that, once a page is in the index, that doesn’t mean it’ll stay there forever! Google repeatedly crawls and re-evaluates content – so if your quality drops, or if you accidentally prevent Google from evaluating the content, then your page might get dropped out of the index.
Any questions? Let us know in the comments!
Read more: What is crawlability? »
Coming up next!
- Event August 07 - 15, 2021 Team Yoast is sponsoring Karnataka do_action, an online charity hackathon. Click through to see what this event is about! All events we will be attending »
- SEO webinar August 31, 2021 Register for our SEO news webinar in August 2021. Our SEO experts Jono and Joost will talk you through the latest Google and WordPress news. All Yoast SEO webinars »