It’s been a while I haven’t published anything interesting. Okay, it’s 0440 hours in local time. I woke up too early today and writing this post on how I manage a server that running a blog that garners 6+ million page views every month at only a nominal cost of $6 ($5 server + 1$ backup).
Yes, you read it right, it is just a $5/month starter cloud server from the DigitalOcean(DO).
I’ve been managing the DO cloud server since 2015 and it’s been an incredible learning experience. It’s been an ultimate rollercoaster ride when working with an unmanaged cloud server.
If you don’t know about DigitalOcean and how happy I have been, you can follow this post.
There are tonnes of things that come into play when completely managing a server, unlike managed cloud hosting you have to deal with every single issue on your own.
I had many sleepless nights and unlimited make-and-break trials of command-line that finally helped to create a server that can bare extensive peak load.
Before getting started, here is the sneak peek of DO billing —
and Google Analytics performance graph of DigitBin.com for Nov 2019.
Without further ado, let get started —
Backend Server Configuration
When I say ‘Server’, it’s completely the backend invisible part that host’s the site. Here are the configuration and specs of the cloud server running behind DigitBin:
It’s configured on the very basic standard droplet in DigitalOcean which is available at $5 per month.
- CPU: virtual shared single-core (1 vCPU)
- Memory: 1 GB RAM
- Disk Space: 25 GB SSD
- Bandwidth: 1 TB transfer
- Location: NYC
I have also enabled the weekly droplet backup that additionally cost 20% of droplet cost, which is $1 for retaining 4 backups.
After multiple rounds of performance tweaks, I’ve now settled down to the below LEMP Stack configuration:
- Operating System: Ubuntu 18.04 Server
- Web server: Nginx
- PHP version: 7.3
- SQL: MySQL 8.0
- Mail server: Postfix
- Firewall: UFW
The server was initially created with the Apache2 web server (LAMP), however, a few years later we moved to Nginx (LEMP).
Now I am not going into detail about how I configured the software and hardware specs cause I already have a series of posts along with the video demos that you can refer to.
If you follow the guide carefully, I believe you will have the best and most secure server configuration to date.
Backend Performance Configuration
While basic server setup plays an important role in hosting and running the website, it’s performance tweaks that enhance the capability of the server.
I have tweaked every bit of the server configuration to make it powerful and sustainable at peak loads. For example;
I consider the Nginx to be the fastest when compared to the Apache webserver. It’s lightweight and has heavy processing capability.
To further improve its capabilities, I’ve added the snippets to execute browser caching headers, gzip compression, added security snippets, http2, and more. This will eliminate the reliance on third-party extensions in the front-end.
PHP7.3 (will be migrating to 7.4) is the fastest PHP version compared to predecessors. I have enabled the capability of FPM (FastCGI Process Manager) for processing requests during heavy-load.
This is also used for generating and serving the static html page from dynamic content using FastCGI nginx cache.
I use MySQL8.0 which is considered to be the fastest among predecessors (MySQL5.7) and likely has the capability of MariaDB.
Along with it also use the Redis Object Caching that helps in caching the complex and most requested queries within the RAM memory. Overall improving the processing speed and reducing the dependency on MySQL to process every request.
The cloud server setup guide covers hell lot of things that help in optimizing the overall server performance.
WordPress CMS Tweaks
DigitBin.com is running on the WordPress blogging CMS which has the rich capability and features to fully control, manage, and improve performance. Though I mostly rely on backend optimization, there are still a few minor tweaks that need to be made in the front end.
The WordPress itself is a pretty light-weight CMS that consumes a negligible memory and CPU for operating. But, things get clumsy when loaded with heavy memory intensive plugins.
Especially the caching and security plugins like W3 total cache, WP Supercache, Wordfence, etc. are pretty heavy on the server. Hence, I always look up for alternative which can be implemented in the backend. That’s where the FastCGI cache, nginx browser cache, redis object cache, UFW firewall, etc. act as a perfect substitute for the features these plugins are offering.
Always choose the plugins wisely. I add plugins if and only if there is no alternative or can’t be managed at the backend, like SEO plugins.
Offloading Static Assets
There are few static assets that often helps in loading and visualizing the site like, JS, CSS, Images, and HTML. Though HTML is an integral part of the front-end which cannot be offloaded completely, however, can be cached and distributed from the nearest edge caching server (more covered in CloudFlare CDN section below).
Free Images Offloading:
I am very thankful to WordPress for its very generous support that helped in offloading the images using photon CDN for free (clubbed into JetPack as Site Accelerator). If you do not know about photon and it’s API, you can read here.
With the Photon CDN, I’m able to save a few TBs of bandwidth against the images. Every image hosted on DigitBin is automatically synced and replaced with a photon CDN URL. Hence, saving the bandwidth cost and serving the image files from the photon’s edge caching server.
Apart from saving bandwidth, CDN also converts the image into webp format which is considered to be the superior lossless and lossy compression for images on the web.
Eventually, I am planning to migrate the images to CloudFlare CDN under sub-domains.
Offload CSS and JS:
CSS and JS are important assets that use for making front-end design attractive and interactive. I have (had) offloaded using plugins named commonWP and NGT jsDelivr. One can use either of the plugins to check which is compatible and working with your installation. However, I have used both interchangeably.
Both the plugins will rewrite the static CSS and JS asset URLs from the site to free open source CDN like JSDelivr. Example: https://restorebin.com/staticfiles.js will become https://cdn.jsdelivr.net/staticfiles.js. This further reduces the bandwidth usage as well as server load.
However, there are certain conditions for rewriting the URLs. Like:
- The active theme and plugins must be from the WordPress repository.
- The developer should commit the files within /tags/ sub-directory folder (not in our control).
You can refer to the individual plugins details page for more such limitations.
[Note: I recently stopped using the jsdelivr since I wanted to combine JS & CSS assets and serve as a single file. Additional details in Cloudflare section]
Asset optimization is another important part that helps in improving server performance as well as improve this site loading speed.
As mentioned above, the Photon CDN automatically converts the image and compresses into webp format hence further optimization is not necessarily required. However, if you think you can improve loading time further, then you can of course compress the image file using smush.it API.
There is a dedicated plugin for smush.it, however, I use WP-Optimize plugins that not only incorporates the smush.it but also helps in the database cleanup.
CSS, JS and HTML Optimization:
WP-Optimize also has the feature to minify, cache, compress, and preload HTML, JS, and CSS as well. However, note that if you alter or make any changes to JS and CSS files, then free asset offloading using jsdelivr won’t work.
As far as HTML caching is consider, we already have the Nginx FastCGI caching in place that pretty much works without any additional caching plugin.
I recently started using the Autoptimize plugins for minification and combining assets. Though this has increased the server load a bit since it requires memory for processing, its totally worth it.
Cloudflare CDN Settings
So finally, here is the section that I have already mentioned twice above.
I used Cloudflare CDN a couple of years ago (I do not remember exactly but maybe in 2015), but my experience wasn’t great in those days. The site performance dropped and it also the loading became slow. Hence, immediately switch back to the original server and stayed.
But, recently a year ago I heard quite good reviews in terms of improved site speed and it’s fast DNS resolving capability. Hence, I thought of giving a try again.
I switched from Google DNS to Cloudflare DNS and made mandatory setup and started serving the files from Cloudflare Edge CDN caching.
I currently use the page rules, service workers, Cloudflare DNS for speed, SSL certificate, and minimal WAF security.
Cloudflare Page Rules
The Page Rules in Cloudflare is a pretty helpful feature that helps in setting up the site performance. It also accepts the wildcard parameter * that makes it every more robust.
I currently have these two rules which cache everything within the /content/ or /wp-content/ and /wp-includes/ directory and store in edge cache for 7 days.
The files that are minified and combined using the Autoptimze plugin are automatically served using the below page rules. Hence, there won’t be any additional bandwidth consumption from the cloud server.
Edge caching helps in storing and sync the cached static files in the nearest edge server. When a user requests the webserver files, the Edge cache serves from the closest CDN pop datacenter. Making the site faster and smoother loading experience.
Cloudflare Service Worker
The Service worker was quite a new thing when I started using the Cloudflare again. The service worker helps in deploying the JavasSript worker on the Edge server that can control the website HTTP request.
I use a service worker to Cache Everything on the front-end part. This script helps in busting the cache when logged-in as a WordPress user.
The cache everything will help in edge caching the dynamic HTML in the Cloudflare server. So when the next web request is sent, Cloudflare automatically serves the HTML copy of the web page from the nearest CDN Edge server datacenter instead of hitting our original DigitalOcean cloud server.
This significantly reduces the server load and also makes the web serving extreme faster due to the Edge caching mechanism.
The service worker is chargeable. If you’re within the free tier limit, you won’t be charged a penny.
If you use the Let’s Encrypt free SSL certificate service, then you can skip this item. However, I moved to Cloudflare SSL and discounted the Let’s Encrypt. Cloudflare provides great options and controls to improve site security.
Cloudflare also has bot protection mode which is known as Web Application Firewall (WAF) that you can customize to any security level. I currently kept the Low mode since I have multiple crawler bots running on the site.
Note: Do not use the Cloudflare and Let’s Encrypt together. You may encounter the redirect loop error. Also, remove all the Let’s Encrypt rules that you might have added into the Nginx configuration file.
Bandwidth Served from Server
Since most of the static and dynamic assets are served using the Cloudflare and Photon CDN. The total bandwidth usage on the DO droplet is unsurprisingly low. Here are the stats of the DigitalOcean and Cloudflare server for last month:
Last month the Cloudflare CDN has helped me save 452 GB of bandwidth by serving all static assets (except Images) and cached HTML pages.
The Photon CDN helps to save unlimited image bandwidth. I do not have exact counter, however, I am guessing the bandwidth to be anywhere near to ~20TB. DigitBin is an image-heavy site with the average size of an image being 100KB.
Photon CDN not only helps in saving the bandwidth but also reduces the server requests. Making CPU and RAM available for other important task intensive processes.
Eventually, I am planning to move the images into Cloudflare CDN hence making the site even faster.
After cutting off the bandwidth over to Cloudflare and Photon CDN, the DigitalOcean has only received limited bandwidth transfer requests — 22 GB per month.
The data transfer size has increased on the DO droplet after installing the Autoptimize plugins and disabling the JSDelivr CDN plugins. Earlier it was less than 10GB (~8GB) per month.
Bottom Line: DigitBin Performance
So this was all about the server and website configuration of DigitBin that I currently manage at a nominal cost.
Though I have been charged additionally for Cloudflare service workers after crossing its graceful free limit. But, that almost negligible. These charges can be stopped if you do not use the service workers.
Overall I am pretty happy with the DigitalOcean server performance and free Cloudflare CDN as well as security setup.
I know you might have additional questions in your mind, feel free to drop in the comments below. Also. please drop any thoughts about my configuration or have suggestions.
Thank you! //Kushal