Publishing content with WordPress hardly seems like rocket science. Download the free software, then install and add some plugins and – voilà – that’s it, right?
Indeed it is when you initially get started and just have a few visitors to your site and very little content. However, once you have a few thousand posts and lots of related plugins and functions, your server will become a slow mess. Now you can always upgrade to the biggest machine available but is that possible at a price point you can afford?
At Mighty Travels, we went a different route and have been investing in several caching layers since day one. Although we now see 100 page requests per second, we still operate with the same hardware we did when we started. The amazing thing is that we consume only 10% of those same server resources and from our test page, load time is much faster than ever.
Our goals have been to show a page (before any ads load) in less than 300ms on a fast connection and 2s on a slow connection…
How did we do that?
Cache layers are nothing new but adding too many makes it rather complex, as updates become tricky. It is easy to make sites that barely change VERY fast but it is a different beast with content that does change (and where content is being added on).
1) Reduce the size of your page
This sounds easy enough in the beginning and all it takes is to select a template in WordPress that has a smaller footprint. However, by the time you add pictures, content, comments and an advertising section, it often spirals out of control. 500KB sounds very little when you are on a 20Mbit connection but on a 1.5G connection it will take a looong time until it’s loaded. Many websites push 10MB towards you when you load them in a browser; now you can add a separate mobile interface but that would be one more design to create and maintain.
So minimalist and adaptive design it is, to make the same website design work on both mobile and desktop.
WordPress is composed of several layers working together – the MySQL database, the PHP from WordPress and server setting. Accessing a page takes CPU power and the computing takes time; the 200ms needed doesn’t sound so much but that’s added to the time a user needs to send and receive a request.
Many of the bigger pages or archive pages need a 1-2s MySQL request to finish, so every page request already takes several seconds until loaded. Most visitors are not that patient and want the page at least partially loaded within one second.
W3 Total Cache is really good at optimizing a request that hits your server and precompiled PHP, caches database queries in Memcache and enables browser caching. W3TC also minifies and compresses your content. A request is still served by your web server but in most circumstances it at least makes use of some cached data and pages can be constructed in less than 1s.
3) Varnish Cache
W3TC is a good tool but still relies on a lot of computing to put together all the cached portions of your HTML. A better tool has long been in use by many system administrators – Varnish Cache. The open source software does only one thing, which it does really well – it saves and serves static content from the computer’s hard drive.
Even with big pages, the Varnish system is very good at looking up content in a few milliseconds and then serving it and it runs in front of your web server, with minimal CPU usage. If Varnish has the content cached, it will serve it in just a few milliseconds to a user and the system will barely register a request.
Cloudflare works similar (the term used is reverse proxy) but it caches your content in locations all around the world. This works without the Internet routers knowing where your content is located – they just sooner or later hit one Cloudflare machine that serves the content right away instead of hitting your server (the latter is what happens with Varnish).
Cloudflare solves latency issues, as even the light takes 100-300ms to traverse around the globe (through all the fiber connections). A round-trip connection from Singapore to a US East Coast server is about 400-500ms and that’s without any delay serving the content on a web server side.
Most webpages are made of so many little pieces (that are dependent on each other) that the 500ms quickly add up to several seconds.
Since Cloudflare has 120 data centers now, you most likely hit Mighty Travels content under 30ms. That’s usually as good as Google data centers (they invested in their own local infrastructure many years ago). Google is usually the fastest site in each country because of its data centers with local content close to the customer (especially noticeable with YouTube files that are cached/not cached).
5) Cache Definition and Pre-loading
For each piece of content on Mighty Travels, we devised a length of caching by all these layers. The homepage is not a good idea to cache for a long time (since you all want to see updates) but static pictures generally won’t change much for a long time.
We also pre-load all important files constantly into the cache layer, even before the expiration, to improve the customer experience (since you never want to load the content first and see a request run through all the layers).6) Make sure Updates work
If all this sounds complicated, that’s because it is. Several caching layers can be perplexing even to experienced system admins, as it is hard to track down which layer did not update or shows stale content. You also want to make sure that updates are propagated quickly and that the homepage changes at the press of a button, although you are actually updating hundreds of web servers around the globe in real time.
7) Maintain it
Every update of WordPress (like the last one, Version 4.8) has the potential to influence how this system works in parallel.
Is it worth it?
We are all spending a lot of time online and the one thing we all feel is that slow pages are the death sentence of online content. Nobody has the patience to wait longer but a stable site that loads fast is one that I bookmark or add to my IFTTT RSS Feeds.