At Yoast, we like to release often. We’re a fan of continuous delivery and currently release every two weeks. As we sometimes get questions about our release cycle, we’d like to tell the entire story on why and how we release and test our plugins before they are ready to be installed on millions of websites worldwide.
Yoast SEO: two week release cycles
At Yoast, we’re a fan of the software engineering approach called continuous delivery. Continuous delivery means that the software teams create and release software in short cycles. At any given time, the software should be stable enough to be released. Continuous deployment is one step further. Continuous deployment means that the software gets delivered to the user at any time using an automated system. For the yoast.com website, we use continuous delivery and continuous deployment. For the plugins, this is unfortunately not possible in the current WordPress environment. Our current software process is as close to continuous delivery for WordPress plugins can get. This is important, as this will result in fewer bugs, more focus, and more collaboration.
Fewer bugs
The Yoast SEO plugins are installed on over 9 million websites worldwide. We know that when our release has a bug, this has an impact on a lot of websites. Even if a bug is a, so called, edge case and only appears on 0,1% of all websites that run Yoast SEO, we’re talking about over 80 thousand websites.
Our developers, but also our board, support team, and everyone involved releasing the plugins, are well aware of the impact a release can have. This is why we stand behind our continuous delivery approach. If something goes wrong, we can fix and release the fix within a couple of hours to a couple of days. This is because the developers know exactly what they worked on the past two weeks. If a bug is introduced in the newest release, they only have to debug two week’s worth of code, instead of months.
More focus
This automatically means that our developers have more focus. They plan in advance what they’re going to work on. Our releases have certain themes. Think about the Schema release, or the word forms releases! This way the developers can dive into one big feature and solve one big puzzle at a time. It allows them to release this finished puzzle before starting on another puzzle, especially as the company is growing. When I first started working at Yoast, back in September 2014, we were with just a handful of developers. Right now, there are four development teams. Two of those teams work on our plugins. By focusing on certain subjects, the team leads can discuss where the focus will be on.
More collaboration between the various teams
This collaboration is not only happening between our developers. Collaboration is extremely important at Yoast. This means that with short release cycles, the developers can discuss any important features or issues with the support engineers, but also with our content team, our testers, our research team, our marketing, and social team and the board. So, when there is an important release coming up, every team lead knows quickly who to inform about what.
Our code gets tested excessively
Fortunately, we have dedicated testers working on testing the plugin. Our approach to verify the quality of the plugins can be divided into four separate testing solutions.
1. Automated testing
Every night the plugins with the latest development changes are tested against automated tests. The results of these tests are processed each morning. This means that the development teams receive early feedback if new code affected any existing functionality. All new features receive automated tests before these features are released.
2. Pre-production testing
Every release candidate gets tested on several testing environments. Among this, is a testing environment that resembles yoast.com. We believe that our code cannot be put live if it breaks yoast.com, so we test this excessively.
3. Acceptance testing
Another part of testing is the so-called acceptance phase. All release candidates are tested by acceptance testers. The testers test based on user experience, they test our software on various browsers and operating systems.
4. Feedback processing phase
If a user comes across issues after a new release, the issues get analyzed. First, the focus lies on fixing any breaking issue as fast and as good as possible. It gets tested thoroughly again. After that, we decide whether any new testing needs to be included in the development cycle. It could be that there will be new automated tests, a new step will be added for acceptance testing or that there is another approach needed.
Multiple releases within a few days: the real story
Sometimes it, unfortunately, happens that a bug is found immediately after or the day after a regular release. If the bug is severe, meaning that websites can break, it’s not an edge case, multiple websites suffer from this and it is introduced in the previous release, there might be a patch release upcoming. Fortunately, we have an excellent support team with team members all over the world. Please realize, we do not take our decisions about patch releases lightly. The reasons for releasing are carefully studied and processed within the various teams. We do not bring out a patch release if it is not necessary.
As explained before, our plugins are installed on millions of websites worldwide. Although they all run WordPress, not one website is the same. One website might run thirty plugins and an outdated theme, while another might only run Yoast SEO and keeps everything up to date. And this is exactly what makes it hard. With so many WordPress versions going around on the web, various hosting platforms, and PHP versions and with over 54,000 plugins and over 7,000 themes on WordPress.org alone, it is impossible for us to tackle each and every setup combination.
This is why our testers and developers work with the most common setups and PHP versions and check the current process after every patch release. This is also the reason why we partnered up with various plugin developers and discuss possible bugs to solve this as soon as possible.
In the future, we want to focus on an automated release process where you as a user do not have to manually hit the update button every time a Yoast plugin gets released. Our continuous delivery process is part of a culture of continuous improvement within Yoast. We aim to always deliver improvements to our users as fast as possible.
Irene Strikkers
Irene is Yoast's Head of R&D. She is a web developer with a passion for linguistics.
November 29 - 30, 2024
Team Yoast is at Sponsoring WordCamp Netherlands 2024! Click through to see who will be there, what we will do, and more!
See where you can find us next »
26 November 2024
Get expert analysis on the latest SEO news developments with Carolyn Shelby and Alex Moss. Join our upcoming update! 📺️
All Yoast SEO webinars »
8 Responses to Continuous delivery at Yoast
James • 5 years ago
Do you ever consider rolling back a release to the last stable version instead of patching?
Omar Reiss • 5 years ago
Hey James, that’s a great question! Thanks! Rollbacks are definitely an option, but they’re also a last resort. Usually a release contains multiple changes, some bugfixes and enhancements. Rolling a release back also means taking those changes back from the end user. Even if it’s only for the time being, that’s a heavy measure. We’d consider doing that, but only when the negative impact of a release is really big and time is a big factor. Fortunately that hasn’t happened since we have the quality assurance process in place we have now. A rollback would however still mean we have to ship a new release with the code of the previous version, because we need to provide an upgrade path to the many users who have already updated.
Marilyn • 5 years ago
“In the future, we want to focus on an automated release process where you as a user do not have to manually hit the update button every time a Yoast plugin gets released. Our continuous delivery process is part of a culture of continuous improvement within Yoast. We aim to always deliver improvements to our users as fast as possible.“
I don’t like not having an option. I like to make sure everything in the release is working and identified so if I need to research an error I know what may have caused it. Especially structured data as per the 11.0 release.
Omar Reiss • 5 years ago
Hey Marilyn, thanks for sharing your input! We totally understand your usecase and will make sure automatic updates are optional when we ship them.
Gladys Strickland • 5 years ago
Thank you for explaining all of that! A lot of work does go into the plugin.
I do, however, have a concern about automated releases when there is a big update, such as 11.0. By being able to hear about the release and what it entailed, I then knew how to go in and turn off Schema in my theme so it wouldn’t conflict with Yoast. If this had been done automatically, my website would have been a mess while I tried to sort it out.
Again, thanks for sharing all the work that goes into Yoast before it is released.
Omar Reiss • 5 years ago
Hi Gladys, Thanks! I totally understand your concern. No worries, we’ll make automatic updates optional once we ship them.
MaAnna • 5 years ago
The big news here is in the last paragraph – an automated update feature. That is already happening for minor releases.
I’m not a fan of it happening for major releases – the 11.0 release being a prime example. Me and most of my clients needed to delay that update due our theme’s own schema markup conflicting. It took time for a resolution to be developed to turn off the theme’s output.
I certainly hope you will seriously consider a way for us to opt out of auto major updates and keep that u dear the co tool of site owners, which is where it belongs.
Omar Reiss • 5 years ago
Hey there! Thanks for raising your concern! We understand some sites need more control over updates. We are no different in that respect :-) We’ll definitely make automatic updates optional.
Do you ever consider rolling back a release to the last stable version instead of patching?
Hey James, that’s a great question! Thanks! Rollbacks are definitely an option, but they’re also a last resort. Usually a release contains multiple changes, some bugfixes and enhancements. Rolling a release back also means taking those changes back from the end user. Even if it’s only for the time being, that’s a heavy measure. We’d consider doing that, but only when the negative impact of a release is really big and time is a big factor. Fortunately that hasn’t happened since we have the quality assurance process in place we have now. A rollback would however still mean we have to ship a new release with the code of the previous version, because we need to provide an upgrade path to the many users who have already updated.
“In the future, we want to focus on an automated release process where you as a user do not have to manually hit the update button every time a Yoast plugin gets released. Our continuous delivery process is part of a culture of continuous improvement within Yoast. We aim to always deliver improvements to our users as fast as possible.“
I don’t like not having an option. I like to make sure everything in the release is working and identified so if I need to research an error I know what may have caused it. Especially structured data as per the 11.0 release.
Hey Marilyn, thanks for sharing your input! We totally understand your usecase and will make sure automatic updates are optional when we ship them.
Thank you for explaining all of that! A lot of work does go into the plugin.
I do, however, have a concern about automated releases when there is a big update, such as 11.0. By being able to hear about the release and what it entailed, I then knew how to go in and turn off Schema in my theme so it wouldn’t conflict with Yoast. If this had been done automatically, my website would have been a mess while I tried to sort it out.
Again, thanks for sharing all the work that goes into Yoast before it is released.
Hi Gladys, Thanks! I totally understand your concern. No worries, we’ll make automatic updates optional once we ship them.
The big news here is in the last paragraph – an automated update feature. That is already happening for minor releases.
I’m not a fan of it happening for major releases – the 11.0 release being a prime example. Me and most of my clients needed to delay that update due our theme’s own schema markup conflicting. It took time for a resolution to be developed to turn off the theme’s output.
I certainly hope you will seriously consider a way for us to opt out of auto major updates and keep that u dear the co tool of site owners, which is where it belongs.
Hey there! Thanks for raising your concern! We understand some sites need more control over updates. We are no different in that respect :-) We’ll definitely make automatic updates optional.