Magento performance hosting

June 12th, 2009 – 39 Comments

The system requirements of Magento are quite extensive, it requires at least PHP 5.2.0 extended with mcrypt, PDO_MySql and simplexml. For the database Magento needs at least MySQL 4.1.20 with InnoDB storage engine. At MagentoCommerce.com you can find a complete list of requirements. But how do you know if your server meets these system requirements? Check this and follow these three steps.

How to get the most out your Magento install?

Magento is notorious for it speed and performance. But isn’t necessary to have a killer web server to get very reasonable performance, it is all about the configuration of your server and the code of your template. However, after the 1.3 release there are some performance improvements, like Frontend Flat Catalog, and people measure reduction of loading times up to 40%. Another recent development is the Magento Compiler this module compiles all Magento files. From tests this module gave between 25% to 50% better performance. This module is still in beta and should not be used in a production environment for now.

The question is, what else can you do to speed up your Magento install?

Optimize your template for speed

More information can be found at the Magento SEO article.

Enable compression of your files

Mod_deflate allows output from your server to be compressed. To enable it for Magento edit your .htaccess, around line 74, in Magento root directory.

<ifmodule mod_deflate.c>
############################################
## enable apache served files compression
## http://developer.yahoo.com/performance/rules.html#gzip
# Insert filter
SetOutputFilter DEFLATE
# Netscape 4.x has some problems...
BrowserMatch ^Mozilla/4 gzip-only-text/html
# Netscape 4.06-4.08 have some more problems
BrowserMatch ^Mozilla/4.0[678] no-gzip
# MSIE masquerades as Netscape, but it is fine
BrowserMatch bMSIE !no-gzip !gzip-only-text/html
# Don't compress images
SetEnvIfNoCase Request_URI .(?:gif|jpe?g|png)$ no-gzip dont-vary
# Make sure proxies don't deliver the wrong content
Header append Vary User-Agent env=!dont-vary
</ifmodule>

Apache Module mod_expires

mod_expires controls the setting of the Expires HTTP header and the max-age directive of the Cache-Control HTTP header in server responses. To enable Expires HTTP header for Magento you had to change the code underat your .htaccess to.

<ifmodule mod_expires.c>
ExpiresActive On
ExpiresDefault "access plus 1 month"
</ifmodule>

Use PHP Accelerator

Install a PHP opcode cacher such as APC or XCache. This seems to deliver rather large improvement in the responsive of the Magento install. Some people note a reduce of loading time up to 70%.

Tune your MySQL configuration

You can modify the configuration of the MySQL server to take advantage of the server’s RAM. The efficiency of this step are different, it seems to depends mainly on the number of products. When you have only 100 products, difference will be hardly to notice (below 100ms). But for shops with over 50.000 products there is a huge reduction of the loading time. To give an idea about the settings, for shops with only a limit number of products a query_cache_limit of 1MB will be enough. For larger stores the optimal query cache value can be 64MB. To get the optimal value you really need some testing. A good start point for your Query Cache configuration can be found here.

Speed up your Cache files

Magento makes extensive use of file-based storage for caching and session storage. The slowest component in a server is the hard drive, so if you use a memory-based file system such as tmpfs, you can save all those extra disk IO cycles by storing these temporary files in memory instead of storing them on your slow hard drive.

Let´s do this with tmpfs

Let’s say your Magento install is at /var/www/domain.com/ so your cache and session directories are /var/www/domain.com/var/cache/ and /var/www/domain.com/var/session/ For cache we will allocate a max of 256MB RAM and for sessions 64MB RAM. It is important to give access to everyone (e.g your webserver Apache)

mount -t tmpfs -o size=256M,mode=0744 tmpfs /var/www/domain.com/var/cache/
mount -t tmpfs -o size=64M,mode=0744 tmpfs /var/www/domain.com/var/session/

Now it might be a good idea to restore these volumes each time your server boots. Just add the following lines to your /etc/fstab:

tmpfs /var/www/domain.com/var/cache/ tmpfs size=256,mode=0744 0 0
tmpfs /var/www/domain.com/var/session/ tmpfs size=64,mode=0744 0 0

Please note, your tmpfs is temporary in the sense that nothing will be created on your hard drive. If you reboot, everything in tmpfs will be lost.

Save the sessions in your database

Another option for the sessions is to save them in your database. Magento supports this very well and it is one small step to make this work. Just edit app/etc/local.xmland set

<session_save><![CDATA[files]]></session_save>

to

<session_save><![CDATA[db]]></session_save>

it will then save all sessions in the database which is a much quicker access than the filesystem. When you use a cluster enviroment you had to use this option.

Enable Apache KeepAlives

Enable Apache KeepAlives, this allows persistent connections. These long-lived HTTP sessions allow multiple requests to be send over the same TCP connection, this can result in an almost 50% speedup in latency times for HTML documents with lots of images. An example setting can be:

KeepAlive On
KeepAliveTimeout 2

Need more Magento Performance?

Ok, you are a very successful online retailer and it is time to cluster, no problem. This means that one single-server doesn’t have enough power to keep your customers happy and so you need a clustered environment with two or more servers. A first step can be to start using a Content Delivery Network (CDN).

The most easy to start is using an separate media server, e.g. media.domain.com. Magento has native support for this. You can configure ‘Base Media URL’ secure and unsecure under web configurations. This allow you to serve media form one server without the needs to synchronize any directory. Another option is to start using a Content Delivery Network (CDN) for delivering the static files like images. MaxCDN has some options for Magento that are quite nice.

Another option is to use more than one database servers this tactic is called “database replication”. One database will be the “master” and all others will be called “slaves”. The master is the only database that accepts any sort of write-base queries. These write queries are then replicated to any of the slave servers in real-time. A post at the Magento blog will get you on the right track.

To summarize, 5 quick wins

  • move the the rules in the .htaccess files from the directories to virtualhost configuration directives
  • install Xcache or APC PHP cache accelerator
  • clean up your template and layout
  • make sure Apache KeepAlives is enabled
  • use a memory-based file system for Magento’s var/cache/ and var/session/ directory

Conclusion

This article give a introduction how to improve your Magento site performance, none of these tips are revolutionary. Because every website’s scenario is different, when you really want to get the most out of your Magento install you need to hire a professional. The only way to discover the optimal server configuration is testing, a really great tool to use is Browsermob, load testing with real web browsers so you will able to put even the Magento checkout process under load testing.

Now, all of the above you can’t do on your average host, you need a VPS that provides you full control of your server environment. Check out WestHost’s sister company, VPS.net, for a VPS environment that would give you full control.


39 Responses to Magento performance hosting

  1. Brian
    By Brian on 8 December, 2009

    Fantastic article, I’d never thought about the cache filesystem. xCache has really sped up our site as well. Thank you very much for a well written article.

  2. Ben Neale
    By Ben Neale on 2 December, 2009

    Really good article. I’ve applied most of these tweaks and it’s made an immediate noticeable difference.

    Quick question re: using tmpfs

    When using the ‘Refresh Cache’ options withing the Admin, will this flush the memory as well as delete the files from /var/cache?

    Many thanks

    Ben

  3. Kim Alders
    By Kim Alders on 19 November, 2009

    Thanks a bunch! Using info in this thread in my blog too (in Dutch, soz)

  4. qkin
    By qkin on 15 November, 2009

    Thank you. I linked this up on my blog, as usual, Yoast is on top of his game.

  5. koowii
    By koowii on 22 October, 2009

    Thanks for the tips, they are very useful. At Koowii we have just created a dedicated server for Magento sites, in order to be able to apply performance tweaks like this for all of our users.

  6. izzet
    By izzet on 14 September, 2009

    thanks for trips..

  7. Gravur
    By Gravur on 19 July, 2009

    This Article ist Great – makes further in such a way ;-)
    Best Regards from Germany
    Peter

  8. Patrick Lightbody
    By Patrick Lightbody on 15 July, 2009

    Fantastic writeup. Bonus points for mentioning BrowserMob :)

    BassKozz, I can try to answer your questions about RBUs vs VUs. There really is no “right” one to use. RBUs are much more realistic, as they use real browsers and follow all the standard traffic patterns browsers would do (ie: process AJAX faithfully, 6 connections per host, etc). VUs are lower cost, but they are single-threaded (1 connection total) and are a bit more difficult to script.

  9. GeSchenK GraVuR
    By GeSchenK GraVuR on 6 July, 2009

    Thanks for this artice.
    We will try few things … Best regards :-)

  10. BassKozz
    By BassKozz on 3 July, 2009

    I tested 2 versions of my Magento Store:
    1. With mod_deflate ON
    2. With mod_deflate OFF

    I tested using browsermob – 20 VU’s RAMP for 15minutes, and surprisingly the scripts ran FASTER with mod_deflate OFF.
    Could this be due to the fact that I used VU’s instead of RBU’s ?
    Any insight would be greatly appreciated.
    -BassKozz

    • Joachim Houtman
      By Joachim Houtman on 4 July, 2009

      That is no surprise, mod_deflate need some resources (CPU) of your server and slow it down a bite. What was the difference? What kind of hosting do you use?

      • BassKozz
        By BassKozz on 4 July, 2009

        I use a VPS on Dreamhost, I set my memory at 1500mb during the testing and the results were:

        mod_deflate – Enabled: MEM maxed out at 374MB, CPU maxed out at 4.81, number of transactions=11, average runtime (time_active)=42176.45 (7.03minutes)
        mod_deflate – DISabled: MEM maxed out at 378MB, CPU maxed out at 4.37, number of transactions=13, average runtime (time_active)=397310.08 (6.62minutes)

  11. Brett
    By Brett on 30 June, 2009

    Even with the mod_expires setup the http headers report back;

    Expires: Thu, 19 Nov 1981 08:52:00 GMT

    I can’t figure out why.

  12. BassKozz
    By BassKozz on 29 June, 2009

    I am checking out Browsermob per your recommendation, but I am curious to know how you created a script to run Browsermob?
    Did you use “Basic Browser Script (RBU)”, “Selenium Browser Script (RBU)”, or “Virtual User Script (VU)” ?

    Also would it be possible to just use “Selenium” to test on a local machine (FREE alternative)?
    Sorry I am new to performance testing.

    Thanks,
    -BassKozz

  13. Bill
    By Bill on 20 June, 2009

    These are useful tips – we’ve tried many of them and have seen performance improvements.

    But the one change that made by far the biggest difference (better than 2x page load times) was when Crucial (our host) switched us from Apache to Litespeed. The PHP code execution on Litespeed is remarkably better.

    • Chewynachox
      By Chewynachox on 7 December, 2009

      litespeed?? did you have any problems with the switch over or was it pretty seemless. We use Crucial as well.

      • Bill
        By Bill on 7 December, 2009

        The migration itself was painless and didn’t require any downtime, as Crucial knows how to do this. There were a few post-move problems with the Litespeed configuration (MIME types, mostly), but about 24 hrs afterward, we were back to normal. Btw, they were the ones who suggested the move and I figured that since they vouched for it, it was probably a good idea.

  14. Pierre
    By Pierre on 18 June, 2009

    Thanks for the tips :)

    1) the directory name for sessions is not “sessions” but “session”, there is this error everywhere in this article…

    2) Magento doesn’t delete automatically the session files so if you put only 64mo for the tmpfs, it will be a problem in few hours or few days…

    • Joachim Houtman
      By Joachim Houtman on 18 June, 2009

      Thanks Pierre. You are right, I have edited the post. About the size of tmpfs, these values are just a indication, with 64mb you can store about 40k sessions. Yes it is not a bad idea to clean the session files up with a cron job.

  15. Tom D
    By Tom D on 17 June, 2009

    Do you think with these performance enhancements that magento could run on the grid server from mediatemple? I know size matters, but what do you guys think?

  16. Dirk Henst
    By Dirk Henst on 16 June, 2009

    Most of the tips I already applied to our dedicated server, specially the cache setting in magento and the APC installed did the trick; extremly fast.

    Nice to see all these together again!

  17. Ashley
    By Ashley on 16 June, 2009

    We have an extremely high spec dedicated server and are using APC now which yields the best performance from the different setups we have tried. When we launched we fell over after a few hours due to database replication and a lack of APC but now all is fine. Also we had the same issue with Connection:KeepAlive.

  18. 123tweak
    By 123tweak on 16 June, 2009

    I think set KeepAliveTimeout 5

  19. 2020Media.com
    By 2020Media.com on 15 June, 2009

    We’ve hosted Magento for a couple of years and now we recommend hosting on a virtual server on practically every quote we do.
    We also offer to improve performance by adding tweaks.

    Your tips shown here are great suggestions.

    2020Media.com

  20. HSN
    By HSN on 13 June, 2009

    Indeed, I think that all website admins will spend some amount of time scouring the internet for performance tips and tricks. I know that in my case, I spent several weeks reading blogs and forums in order to find the best settings and learning about what those settings mean. In the meantime, I got a more powerful dedicated box, and I am still interested in squeezing the last drop of performance out of my server — that’s why I’m here on this particular post! :)

    However, I did write an article that I think your users would find interesting. In it, I cite those blogs and forums that have proven invaluable to my own development. In addition, I provide snippets of the httpd.conf and the my.cnf files.

    4 Tips for a faster website.

  21. Jae Jun
    By Jae Jun on 13 June, 2009

    Isnt keepalive on by default?
    Also I just read that APC and Zend Optimizer are not compatible making eaccelerator the stable solution. Is this true?

    • Joachim Houtman
      By Joachim Houtman on 13 June, 2009

      Yes you are right, keepalive is enabled by default but it is good to check it and think about the configuration. Different settings can have influence on the performance.
      About Zend Optimizer, as far as I know it is incompatible with APC, so you had to choose one of them.

  22. Alvaro Veliz
    By Alvaro Veliz on 12 June, 2009

    Amazing! Thanks for the tips. I really need this!

  23. Web Strategy 360
    By Web Strategy 360 on 12 June, 2009

    Great post. Always interested in ways of optimising Magento builds.

  24. FreewareMatter
    By FreewareMatter on 12 June, 2009

    Good article. I think these purposes here can be applied not only for Magento, but for most PHP applications. Thank you very much.

  25. Jelmer de Jong
    By Jelmer de Jong on 12 June, 2009

    Nice article! What is your opinion about selecting a Magento hosting provider? There are some nice cloud service as GoGrid, Amazon EC2 and MT. Are these solutions suitable for a scaling Magento shop? Or should we stick to a hoster as NEXCESS, specialist in Magento (according to their website).

    In short my question is: what to look for when building a Magento webshop that need to scale and deal with high peaks in traffic once a month, hosting wise?

    • Joachim Houtman
      By Joachim Houtman on 13 June, 2009

      It depends, e.g. where are geographical your users, how high is your peak, what is your budget. It might not be a good idea to go with a budget shared hosting plan ;) When you have really some peaks you have 2 options, I think. Go for a host with with Magento hosting in a cluster, for the Netherlands e.g. Byte. Or go with a cloud service like EC2, see Magento wiki or see this post

  26. Phil
    By Phil on 12 June, 2009

    It might be simpler/cheaper to pay for “a killer web server to get very reasonable performance” ;-)

    • Brian Mark
      By Brian Mark on 12 August, 2009

      A killer web server? We’ve had it across 3 multi-core dual cpu machines with 8 gigs of RAM each, a pair of similar configured machines for database servers, a machine with 32 gigs of RAM for memcached, plus using a CDN for images and even these fixes didn’t quite do it. There was more db tuning that needed done just to get the system usable, and now we can run it from 1 web server without much problem except during extreme peaks. A killer web server won’t ever quite do it… you’ve still got to have it tuned or you’re not going to get what’s possible out of the machine.

  27. Armetiz
    By Armetiz on 12 June, 2009

    Thanks for this tips & tricks ;)


Check out our must read articles about Analytics