Email Reliability: use an SPF record

A while back I outlined my system for preventing comment spam. One of the core fundamentals in there is that I send people an email to verify their email address before their comment is published. For this to work well, I need to trust on my email to be received. As it turns out, email reliability is far from easy, let alone email reliability for email coming from your own web server. One of the important things is setting up an SPF record.

There’s all sorts of factors that decide on whether your email is delivered or not, but one of the most important ones is a DNS record called SPF. SPF stands for Sender Policy Framework and let’s the receiving mail server reliably determine whether the server that is sending you email is actually allowed to send you email. Adding one will increase your email reliability incredibly.

As with most DNS type records, the syntax is quite hard to explain so I won’t even try. Let me link you to the Wikipedia page if you really want to know. What I found way more helpful though when I was searching and trying to figure out how to do this is Microsoft’s wizard for this stuff. It asks you a bunch of questions and will still require some time from your side but it got me to the desired end-result a lot faster.

SPF records, Google Apps for Domain and email reliability

I use Google Apps for domain to handle my email, which means that I do most of my sending through a Google SMTP server. Because of that I had set the SPF record in the manner Google suggests here. The thing missing from that is very, very subtle, but makes all the difference. It’s a few letters, let’s see if you can spot it. This is the SPF record Google gave me:

v=spf1 include:_spf.google.com ~all

This is what it actually needed to be:

v=spf1 a include:_spf.google.com ~all

You will have spotted the addition of “a ” after “spf1″. This simple addition means that all web servers that are identified in my domains A records, hence, all the web servers from which I run my domain and subdomains, are allowed to send email as well.

The include directive means that Google can setup SPF records for the domain _spf.google.com and thus add or remove mail servers without you having to change anything.

How to test email reliability

If you’re now thinking “I don’t know whether this has been set up correctly for my domain”, don’t fret. You can test it quite easily. Go to this SPF testing tool and use the 3rd form on the page. Using only the first and third input of that form, enter your web server’s IP address and your from address, which is usually wordpress@yourdomain.com if you’re running WordPress. With my initial test, it gave this result:

email reliability report

This means that this test won’t fail your email per se (as there was no SPF record), but for stricter email servers, it might, in other words, you’ve got quite low email reliability. Now I ran it with the SPF record I has just fashioned using Microsoft’s wizard and it gave me this:

email reliability high

That is the result you want to see, as that means my server’s email reliability just went up quite a bit.

WordPress specific email reliability: using another SMTP server

Quite often email from your own server won’t work as expected, especially when you’re on a shared host it can give issues. There are quite a few plugins out there to help you set up an external SMTP server, I’ve relied on Coffee2Code’s Configure SMTP plugin myself a few times. There’s some issues with that though, especially once you’re starting to get more visitors, as most SMTP servers for free email services have a limit to how many emails you can send per day.

If you can, just use your own server or outsource the email delivery to a third party that specializes in sending service emails. I’ve been testing SendGrid myself, but wasn’t completely happy with it. Their reliability was a lot higher but the costs are quite high too when you get several hundreds of comments. You might think “that’s only a couple of hundred emails”, well, it’s not.

If you have subscribe to comments enabled, which I highly suggest you do, it’ll be a lot more. I get anywhere from 20 to 200 comments on a post, average about 80 at the moment. If 50% subscribes to comment notifications, that adds up to an enormous amount of email being sent.

As for newsletters, there are plugins for sending those from your WordPress install too. I highly encourage you not to do that. Use Mailchimp, or any other newsletter service for that. Their reliability is way higher and you get awesome statistics and subscription services to boot. I use them for my WordPress Newsletter and never regret it.

Are you using other services that send email for you?

If you’re using other services that send email for you, for instance if you’re using Freshbooks (aff) for invoicing like I do, be sure to include them in your SPF too, just add another include like we did above for Google:

include:_spf.freshbooks.com

There are more services that support this, be sure to check if you’re using any services that send email on your behalf. Email reliability is important, but even more important when it’s sending your invoices!

Update from Antonio Romero in the comments, the same goes for MailChimp:

include:servers.mcsv.net

Just updated my own SPF with that as I use my own email address as a from address there.

Conclusion: make sure your email reliability is high!

Whether you run your own server or not, it can’t hurt to do the check above to see whether your email has a high chance of being delivered. If you’re sending email from your web server, be sure to do the above check and make the necessary changes to your SPF record if needed!

Bonus tip: DKIM

If you’re using Google Apps for domain, be sure to read this guide and implement everything in it. It’ll help you set up domain keys, which adds another layer of spam protection.

Tags: , ,


Yoast.com runs on the Genesis Framework

Genesis theme frameworkThe Genesis Framework empowers you to quickly and easily build incredible websites with WordPress. Whether you're a novice or advanced developer, Genesis provides you with the secure and search-engine-optimized foundation that takes WordPress to places you never thought it could go.

Read our Genesis review or get Genesis now!

22 Responses

  1. Arjan SnaterseBy Arjan Snaterse on 6 July, 2011

    For everyone who’s using the standard Google Apps lines for the SPF records (see http://www.google.com/support/a/bin/answer.py?answer=178723&hl=en), you need to add an ‘a’ to it, so it will be like Yoast’s example above..

    • Joost de ValkBy Joost de Valk on 5 December, 2011

      Did you “check” the email notifications box here? :)

  2. Rolands UmbrovskisBy Rolands Umbrovskis on 6 July, 2011

    Nice! Just updated almost all (with most used e-mails conversations) my Google Apps with new TXT record.

  3. Antonio RomeroBy Antonio Romero on 6 July, 2011

    Hello,
    Just in case, for Mailchimp you can also add this
    v=spf1 a include:servers.mcsv.net ~all
    BR

    • Joost de ValkBy Joost de Valk on 6 July, 2011

      Just the include: part is what you need to add to the SPF exampe above, but well spotted Antonio, added it to the article!

      • Antonio RomeroBy Antonio Romero on 6 July, 2011

        Hi again,

        I have google apps for email and mailchimp like you and trying the microsoft wizard I got this

        v=spf1 a mx mx:alt1.aspmx.l.google.com mx:alt2.aspmx.l.google.com mx:aspmx.l.google.com mx:aspmx2.googlemail.com mx:aspmx3.googlemail.com mx:aspmx4.googlemail.com mx:aspmx5.googlemail.com include:_spf.google.com include:servers.mcsv.net mx:servers.mcsv.net ~all

        Are mx register really needed??

        BR

        • Joost de ValkBy Joost de Valk on 6 July, 2011

          No you don’t need them, you already use the include: directive as specified above, so you won’t need to add all the different MX servers.

  4. GertBy Gert on 6 July, 2011

    An absolute must, if you don’t want you’re emails (email campaigns) being block by some (of the more mature) spam filters. Joost did again a perfect explication on how to implement. Personally this is one of the first things I do after registering a new domain.

  5. Haroun KolaBy Haroun Kola on 6 July, 2011

    Thanks for this post, I didn’t even bother to think about these issues before so implementing these practices are key for me.

    I’m still building to receiving at least 8 comments per my post :)

  6. Ankur JainBy Ankur Jain on 6 July, 2011

    Excellent article! I updated my SPF record to what you outlined above for GoogleApp’s. I have had instances where clients email servers would block me out, this should hopefully resolve that issue!

    P.S. I have been using MXToolBox to test my SPF records.

  7. Neil EdmondsonBy Neil Edmondson on 6 July, 2011

    my cpanel has DomainKeys SPF. In here it give a place to set this up, but it says
    “Additional Hosts that send mail for your domains (A):

    All the hosts you specify here will be approved for sending mail. You do not need to specify your primary mail exchanger or any server that an mx has been created for as they are already included automatically.”

    So I already have MX records for google apps … I guess this means I don’t have to do anything to activate an spf. Is this true?

    I use madmimi as a service … anyone know what to add for them?

    • Dean LevittBy Dean Levitt on 7 July, 2011

      Hey Neil, I believe you already chatted with us about this (and Sally’ll get back to you soon) but the simple answer is, you don’t need to take any action but you can if you’d like :)

      We’ll get some info over to you soon!
      Dean

      • Neil EdmondsonBy Neil Edmondson on 7 July, 2011

        Thanks Dean. Yes just got Sally’s reply. Love mimi … it’s great value for the mailings I do.

    • Neil EdmondsonBy Neil Edmondson on 7 July, 2011

      Here’s a note I got from Sally at Mad Mimi that answers the last part of my question … did I say I love mimi?

      You don’t have to change your SPF to get good delivery with Mad Mimi as we handle most validation for you including DKIM and most variants of SPF, but there are some edge and cosmetic cases where it can help. Just change what you have there to this:

      v=spf1 a mx ip4:174.36.102.72 include:auth.madmimi.com include:pass.madmimi.com ?all

      You could even change the ?all to ~all if you’re pretty sure there’s no more IPs you’ll be sending mail from. :)

  8. Chris VendilliBy Chris Vendilli on 7 July, 2011

    This is an excellent article thank you Joost!

    Does anyone have any guess what record needs to be added (if any) to increase deliverability with Aweber when using your domain’s google apps email address as the from/reply-to with the Aweber campaign?

    I read an article that said messages received from the auto-responder would use aweber’s SPF record but that article was 3 years old. Plus, seeing as how it helps with Mailchimp it’s a fair assumption it may help with Aweber as well…

  9. Daniel LyonsBy Daniel Lyons on 7 July, 2011

    I commented on a post before and I never got a confirmation email and my post never appeared. I did get a request for your newsletter which I accepted and receive.

  10. Patrick MulderBy Patrick Mulder on 8 July, 2011

    Nice article Joost! I added it for some customers. I was already struggling with this half a year ago. Hope it will work now!

  11. BrianBy Brian on 8 July, 2011

    Any idea how to do this if your using 1 and 1 servers with your DNS mail server pointing to google apps?

  12. Mullins FarmsBy Mullins Farms on 9 July, 2011

    That’s won’t get you very far unless you have sender id or domain keys also installed… and you really need spam feedback loop with all the major providers… but that’s just a starting point. MUCH more needs to be done.

  13. Frank OrlandoBy Frank Orlando on 12 July, 2011

    I was meaning to do this for quite some time. This article prompted me to go ahead and get it done.

    I am having trouble entering more then 1 spf. I have 4 different ones I need to enter. I ran test and each one works by itself, but I get permanent errors when I try to add all them.

    Any ideas?

  14. ErikBy Erik on 15 July, 2011

    @Frank You could try the microsoft SPF wizzard to get the correct spf records see : www microsoft com /senderid/wizard