Conversion rate optimization: hypothesise first, then test

Since I began working at Yoast, I’ve been busy with a lot of things, but my personal favorite definitely is this one: conversion. Maybe this is because of my background in behavioral science. Or maybe I just prefer bigger numbers to smaller ones. In any case, the fact remains: I’m hooked. Now, for the people thinking I’m here to convert them to any kind of religion, be at ease. Conversion, in this case, means converting visitors of your website to actual customers (or returning visitors). And optimizing the conversion rate is an ongoing process on our website. During this process, I’ve come across a few things I want to share.

Before we dive in, if you want to learn more about conversion and other essential SEO skills, you should check out our All-around SEO training! It doesn’t just tell you about SEO: it makes sure you know how to put these skills into actual practice!!

Don’t believe everything you read

Maybe it’s because I’m looking for it, but I’m seeing more and more blogs, books and ‘experts’ in the field of conversion. This is fine, of course. In fact, I’ve learned a lot from all those people. But there’s something important you have to keep in mind at all times: what they’re saying or advising might’ve worked for them, but that doesn’t mean it will work for you.

So how will you know what works for you? Good question. The answer is actually simple: you should test that! Yes, this is a conscious choice of words, as it’s also the title of the book by Chris Goward. That book really changed my idea of conversion and made it understandable and clear. But most of all it convinced me of the need for testing. Use the ideas and advice from those blogs and books as hypotheses. And then test them.

A/B tests

The easiest way to test is the A/B test. A is your baseline, or in other words: what your website looks like before you change anything. B is your variation to A, so a changed button, changed text, or anything like that. You will then test these against each other to see which one works best. Now you can add as many variations as you want, making it an A/B/C/D/etc test. However, when using a lot of variations, you’re better off using multivariate testing. But that’s something I won’t go in to here.

At Yoast, we’re currently using Convert for our A/B tests, and we like it very much. It’s clean and simple enough for even me to understand. There are several others out there offering the same packages though. But using tools like these, you can actually set up variations to a page (or several of them) and see which variation converts best. And this immediately takes us to the next point.

Revenue trumps conversion

Now, no one can disagree on this one. If you’re making money with your website, the top thing you want is to make as much money as possible. This means revenue is more important than conversion. What I mean by this is the following: if having just 2 conversions a day is making you more money than having 4 (lower value) conversions a day, you’ll go for the 2 conversions.

The idea is simple: when testing hypotheses, the one making you the most money wins. So don’t just focus on the number of conversions, be sure to check what the tests are doing to your revenue as well. And also be sure your revenue is being calculated the right way. So never rely on just one tool, when setting up your tests. The best is to have an exact listing of all your transactions (conversions) with an amount of money attached to it. This way, you can always double check the revenue and be sure it’s being calculated the right way.

Patience converts

This may not exactly be true, but patience does play a very big role in conversion rate optimization. It’s never a good idea to rush your conversion rate optimization, as you might pick the wrong variation or option. You should always leave a test running for at least a week, and preferably two, no matter how much traffic you get. The reason for this is simple: people’s online behavior is different on the weekends than it is during the week.

Now guess which days are the weekends. Freaky, right? So your tests should run for at least a full week, to account for differences between the days. The longer you let a test run, the surer you can be of the results. And believe me, this requires patience. I’m from the “I want it and I want it now” age, and it can be hard to not just make the changes and hope for the best.

Make it a science

This last one I cannot stress enough. Don’t be afraid you’ll get in over your head; it’s actually really easy. What you have to do is formulate hypotheses for what you’re testing. And I understand it if you won’t dive into the scientific literature to find references and support for your claims (although this would be awesome). Just be sure you know what you’re 


 the outcome of the test will be and why. Redundant as this may seem, it’s actually really important.

Hypotheses are not things you just think of yourself. You have to have a reason why you believe something will work, other than your gut feeling. So if you really want to make it scientific, you could dig into the scientific literature. Believe me, there’s a lot of it out there that could be of use for your conversion optimization.

However, as I said, you don’t have to do this. In fact, there are quite a few people who probably already did the hard work for you. For instance, the amazing people at HelpScout have already done all the research in the strategies of pricing. So a good way to come up with hypotheses is to read blogs and books and see what they say worked for them (or their clients). Usually, there are good (psychological) explanations as to why this worked.

Formulating hypotheses

The next step is to formulate hypotheses from these sources. What will you be changing on your website, and what do you expect will change and why? Make sure you use your sources to explain the ‘why’ of these expected changes. This is what makes it a science: backing up your ideas and expectations with actual results from other people.

Formulating hypotheses makes sure you think about the tests you’re about to run. If you don’t put enough effort into thinking of logical hypotheses, you will lose precious time trying variations that make no sense.

So it’s really simple: you start with a good hypothesis, then you test whether you were right and then you implement the changes.

Now it’s up to you: have you already formulated some nice hypotheses, which you’ve already tested? If so, please do share!

Coming up next!

32 Responses to Conversion rate optimization: hypothesise first, then test

  1. Packers Movers pune
    Packers Movers pune  • 11 years ago

    Thanks for this post,it is a rain drop post…i like it really ..because it contain all the information about the exact topic…I am intrigued with much of your information and am persuaded to agree with you after reading your material. I’m hoping you’ll add more articles on this topic.

    • Thijs de Valk

      Good to hear you liked it!

  2. Steve eMailSmith
    Steve eMailSmith  • 11 years ago

    Hey Thijs…

    This is always a great topic – I was expecting to read only positive comments though… not sure how the negative ones have been sparked off, but heh… everyone is entitled to their opinions, I guess.

    Steve ? Master eMailSmith ? Lorenzo
    Chief Editor, eMail Tips Daily Newsletter

    • Thijs de Valk

      Hi Steve,

      Good to hear you liked it! I’m not sure which negative comments you mean though? One may be called critical, but I don’t see any negative comments? ;)

  3. roy
    roy  • 11 years ago

    i can see you dont use tags, why is it?

  4. klochcovigor
    klochcovigor  • 11 years ago

    ???????? ??????- ???????.

  5. Tracy
    Tracy  • 11 years ago

    Thanks for the advice Thijs!

    I notice a lot of these ‘conversion tracking’ articles that give advice are only helpful for websites that sell something, or have a specific end goal like downloading something r subscribing.

    What kinds of conversions should you be looking for on a blog, where the only thing visitors do is read?

    Thanks for your help!

    • Thijs de Valk

      Hi Tracy,

      Thanks for your reply! You mention that conversion optimization articles only talk about sales, end goals and subscriptions. This is actually right, because that’s what conversion is. Like I said in the beginning of this article: conversion means converting your visitors to paying or returning visitors.

      So to get to your question: I have to say a blog should also be looking at subscriptions. Sure, all the people do on your blog is read, but you have to make sure they keep coming back, don’t you? One way of doing that is using a newsletter or email subscription.

      Does that make it a bit more clear? ;)

      • Tracy
        Tracy  • 11 years ago

        That sounds good! Thank you.

  6. Zach Smith
    Zach Smith  • 11 years ago

    i will be back for more articles like this :)

  7. Mary Kay Lofurno
    Mary Kay Lofurno  • 11 years ago

    Yoast, certainly, I agree that you stay focused on the one that makes the most money. That said, the actual revenue calculation can be difficult depending upon your scenario. Much is often not factored in like discounts, cost of good sold etc.
    Next, ‘Patience’ and ‘making it a science’. This is all fine and true when your in a situation where you (1) own your company like yourself or (2) a environment that is supportive of testing.

    I agree that testing is important and I am a firm believer in it but these things you mention become very different in settings other than the two I mentioned in the previous paragraph.

    Best Regards, Mary Kay Lofurno

    • Thijs de Valk

      Hi Mary Kay,

      About the revenue: you’re right, of course. However, the examples you give here are things that should be easily dealt with. You always know the cost of your product, so you can always track revenue with the cost of your product deducted. Or you can even track them both. Discounts, admittedly, are a bit harder. However, all this doesn’t change anything about the fact that you should focus on the most profitable variation.

      I agree we’re in a somewhat luxurious position, because we can do whatever we want here ;) However, this post is written on how it should be, and how you *should* test. I understand this is not an option for some people, because they have their bosses breathing down their necks. But if that is the case, please do use this post to at least start convincing them. And have them read ‘You Should Test That’ as well. I hope it helps! :)

  8. Josh Brancek
    Josh Brancek  • 11 years ago

    Thijs, really great tips with the whole week testing!!! One thing I noticed is that on Saturdays and Sundays, my traffic is down like 60 percent, so I have to remember to not run any tests!!!

    Regards, Josh

    • Thijs de Valk

      Good to hear you like it Josh! But don’t stop testing your weekends; all data is important. Just make sure you leave your tests running for at least a week.

  9. Tony B
    Tony B  • 11 years ago

    Nice one Thijs, (and how do I pronounce that?)

    Loved the information you presented here, Split testing is something that scares the cr*p out of so many online marketers including me. Hey yep I’m going to hold my hands up here.. But YOU have put it into a way that I can really relate to ..

    I to have a love of figures and stats – but I feared the technical implementation of A/B tests. I would never consider paying for a split testing tool purely because of that fact alone.

    Following links I’m going to suck it up and try the wordpress plugin and maybe even the 30 day trial at Optimizely.
    Either way I’m up for trying a little imperfect action thanks to you TJ..
    Cheers Tony.B aka Mission0ps

    • Thijs de Valk

      Thanks for your kind words Tony! Do try it out, you’ll be happy you did. In the end, everything that makes you money, has cost you money at first: that’s what investment is ;)

  10. Philip
    Philip  • 11 years ago

    Awesome post and very informative. Looking further into formulating hypotheses and A/B testing for my sites.

    • Thijs de Valk

      Thanks a lot! Glad to see you’re working on conversion rate optimization; you’ll love it!

  11. Dror
    Dror  • 11 years ago

    As you said, testing is everything and that is the only way to really learn and figure out what would work best for your site and your service.

    By the way, what plugin (if it is a plugin) are you using for the optin box that shows up at the bottom right when scrolling?


    • Dave
      Dave  • 11 years ago
      • Tony B
        Tony B  • 11 years ago

        Gonna try that it does look good cheers Dave ;)

    • Thijs de Valk

      Good hear to you agree ;)
      Joost actually built the ‘scroll up’ subscription thingy himself, so it’s not a plugin. There are some out there doing this, but they don’t work as neatly.

  12. Andy
    Andy  • 11 years ago

    Really good post. I did some work on this some time ago and we produced a spreadsheet that displays the difference relating to investment to increase visitors as opposed to investment to increase conversion. Conversion wins everytime. See the free spreadsheet to illustrate this point. Thanks for the insight in this area.

  13. David Wells
    David Wells  • 11 years ago

    Great post Thijs! A/B testing is a critical component a lot of business owners overlook.

    Coming up with what to test can be a challenge for people new to a/b testing and I think a well thought out strategy is important.

    We built A/B and multivariate testing into our free wordpress plugin . Thought you might like to check that out as well =)

  14. giankar
    giankar  • 11 years ago

    A/B but also multivariate testing is very helpful to formulate hypothesis for what is working for your website and not.

    • Thijs de Valk

      Well that’s what this post is about basically: don’t use those tools to formulate hypotheses. Use those tools to test them.

  15. Todd
    Todd  • 11 years ago

    Great first post! We use optimizely and are thinking about testing out Crazy Egg as well. Still can’t believe how many people don’t optimize conversions.

  16. Richard Theuws
    Richard Theuws  • 11 years ago

    Haha guess what? Mine too! (as probably most website owners will say hahahaha)

    • Richard Theuws
      Richard Theuws  • 11 years ago

      blockquote fail. fml.

      • Thijs de Valk

        LOL I was wondering what that was ;-)

  17. Daniel Law
    Daniel Law  • 11 years ago

    Great first post Thijs. We’re currently debating whether we should use Optimizely as well, along with CrazyEgg.

    Great insights and looking forward to your next post!


    • Thijs de Valk

      Thanks a lot! You should definitely give Optimizely a try, it’s a very easy to use and handy tool. And no, we’re not affiliates ;-)

      Edit: we’re currently actually using Convert, as their customer support is just way superior, and they could actually track what we want.