Innovations by Yoast

You have read about our products, are probably using one of these, but a product doesn’t become a product overnight. Our products are under constant review. We align them with the latest changes in the search engines and try to make your life as easy as possible.

This process requires continuous innovation. We talk regularly with Google and others to do things that are not in other plugins yet, to give your website that well deserved edge to your competitors. We dive into new (web) techniques and see how we can use these to benefit our users.

To share these innovations and our thought process, we have set up this innovations page. On this page, we will guide you through some of the things we have done or are working on. To share why we did certain things, but even more to inspire you to stay on top of your game. If we all share these things, innovations become open source as well. And we’ll all make the web better together.

Structured data

We believe that structured data will play a huge role in the future of search. It’ll also change the wider web, and create new possibilities. Helping search engines and other systems to truly understand content unlocks incredible potential.

Unfortunately, many websites don’t take full advantage of structured data. That’s because implementing structured content is hard. It’s technically complex, it breaks easily, and it’s difficult to integrate correctly. That is, until now!

We’ve invested thousands of hours into defining and building a unique approach to integrating schema.org markup. Now, that approach is a core part of our software. Website owners and users can take advantage of rich, structured data, without having to worry about micromanaging their code. It’s all taken care of automatically, in the background, as they write.

But we wanted to think bigger. We wanted to truly solve the challenges of implementing structured markup, for everybody. So we’ve developed flexible APIs, which allow plugin and theme developers to extend or alter our approach. We’ve made it easy for other platforms, plugins and teams to connect their structured data. We’ve also made all of our technical documentation open-source so that anybody can build on the work we’ve done.

We’re paving the way for a more connected, more intelligent world wide web. We’re really excited to see what people build. You can read more about our schema.org integration here.

Morphology

In October 2018, we added word-form recognition for English, soon followed by a similar upgrade for German. This new functionality allowed us to bring our SEO analysis to the next level. Before, we were only able to look for exact matches of the keyphrase in the text. Now, Yoast SEO Premium can recognize all possible word-forms of the words that comprise the keyphrase. This change brought us closer to the way search engines see our users’ posts! Our team of linguists is currently making word-form recognition available for other languages —  Dutch (the native language of most Yoasters) and Spanish are at the top of the list.

Providing word-form recognition functionality was one of the first steps in a massive revision of SEO checks that Yoast SEO offers. We took a serious look at how the plugin analyzes texts and how the results of such analysis can be helpful to our users. Before releasing that big update, which we called the Recalibrated SEO analysis, we collaborated with thousands of our users to make sure the new analysis works great on all sorts of websites.

Indexables

An important part of what our plugin does is connecting your pages to each other. Think, for example, of our breadcrumbs, our structured data, and our internal linking features. When we talk about pages in this context, we’re not talking about the WordPress post type but rather every page on your website. That is what makes this complicated. Every page on your website can link to:

  • Things stored in the WordPress posts table such as posts, attachments, custom post types or WordPress pages.
  • Taxonomies such as tag or category pages stored in the WordPress taxonomies table.
  • Post type archives or date archives that aren’t represented in any table at all.
  • Custom pages from plugins that could originate from just about anywhere.

Connecting all these very different things, therefore, becomes quite a challenge. Not only are there many different scenarios to account for, but performance issues are constantly popping up. Many WordPress functions that allow discovering these things can be quite expensive. For example, if you look at the list of uncached functions WPVIP recommends plugins not to use, you’ll find many functions that you’d think would be suited to overcome exactly this challenge.

That leaves us with a problem: How do we efficiently gather information about all linked and related pages to best optimize your page while still keeping things blazingly fast? That’s where indexables come in. Indexables are a custom data type that represents anything that could be indexed on your site. Everything a page can link or relate to on your site will be an indexable, including that page itself.

This gives us a single source of information that we can easily optimize. That way, we ensure that all the data needed to best optimize your pages is always available quickly. In turn, this will allow us to provide search engines with the richest information possible while still keeping page speeds as fast as they can be.

Yoast SEO Schema FI

Privacy by design

At Yoast, we strongly believe in privacy by design and we go out of our way to achieve it. Our rule of thumb is to never collect data we don’t strictly need. A good example of that is that we designed our SEO and readability analyses to work fully in the browser. 

Most natural language processing tools require users to send data to servers. There, machine learning algorithms analyze the data and return the insights gained. You can imagine many companies wouldn’t be happy if their product announcement drafts were sent somewhere else before publication. Fortunately, Yoast SEO does all the language processing straight in a web browser. Your drafts are safe with us!

Privacy drives innovation

In order to make that possible, we have to utilize the technological capabilities of web browsers to the fullest. For example, to prevent the DOM from freezing during analysis, we make use of a web worker. This is a separate thread in the browser in which we can do all the heavy lifting of the analysis. Another improvement we are working on is to analyze an abstract tree representation of the content instead of raw HTML. This should lead to dramatic improvements in terms of performance and reliability. Those are problems we wouldn’t have to deal with on the server as much.

Privacy by design starts with being conscious about the data we collect. We are proud that we’re able to provide our users with smart technology without having to collect any data. We’ve also often opted to not develop certain functionality if it required us to store data on our servers. Instead, we’ve always looked for ways to provide similar benefits with tools users can run in their own environments.

Hackathons

Every six weeks (check) we have a hackathon-day. During this day, all developers work together on a specific project. We have a couple of different development teams at Yoast, but during the hackathon, we are all working together.

We have three different types of hackathons:

  1. Pressure cooker – achieving a common goal before the end of the day, usually building a prototype.
  2. Technology exploration – gaining experience with and knowledge about a technology that we don’t use regularly during daily programming tasks.
  3. Freeform playground – picking a technology to play with for a day to build something cool. 

During a hackathon, the developers are split up in pairs to work on a specific task or project together. At the end of the day, we present the end results and experiences gained during the work.

SEO quick wins

Agile processes

Our CTO Omar introduced Kanban not long after joining the company. This methodology is based on the pull-principle: whenever you have room to do work, you pick up the first thing that needs attention. This worked out great for all the development teams. Every team has made their own little tweaks to alter this methodology to fit their needs.

Those little tweaks are a very important part of how we approach the processes of the teams. They provide the freedom and flexibility to find optimizations in the implementation. Saving time, avoiding confusion and improving communication are the measurements we use to determine if something is an improvement. We’ve had a number of occasions where one of the teams introduced a new principle to their process, which was picked up by the other teams quickly and became a standard.

At the beginning of 2019, the first team started using Scrum to gain a better insight into delivery expectations and clarity of the tasks that need to be done. Soon, the other development teams followed suit. Scrum is proving to be a valuable tool to set up a “Product & QA → UX → Development” prioritization and planning structure in our company.