optimization

Ignore noisy logs with fluentd in EKS or other Kubernetes clusters

Recently, I decided to use the fluentd-kubernetes-daemonset project to easily ship all logs from an EKS Kubernetes cluster in Amazon to an Elasticsearch cluster operating elsewhere.

The initial configuration worked great out of the box—just fill in details like the FLUENT_ELASTICSEARCH_HOST and any authentication info, and then deploy the RBAC rules and DaemonSet into your cluster, and you're off to the races (assuming your Elasticsearch instance is configured to allow access from the cluster!).

But once I did that, I noticed the brand new EKS cluster was sending over 16,000 log messages per second to Elasticsearch. Doing a tiny bit of analysis (not much was required, honestly), I found that over 98% of the logs were coming from two EKS-specific noisy containers, efs-csi-node and ebs-snapshot-controller.

Make composer operations with Drupal way faster and easier on RAM

tl;dr: Run composer require zaporylie/composer-drupal-optimizations:^1.0 in your Drupal codebase to halve Composer's RAM usage and make operations like require and update 3-4x faster.

A few weeks ago, I noticed Drupal VM's PHP 5.6 automated test suite started failing on the step that runs composer require drupal/drush. (PSA: PHP 5.6 is officially dead. Don't use it anymore. If you're still using it, upgrade to a supported version ASAP!). This was the error message I was getting from Travis CI:

PHP Fatal error:  Allowed memory size of 2147483648 bytes exhausted (tried to allocate 32 bytes) in phar:///usr/bin/composer/src/Composer/DependencyResolver/RuleWatchNode.php on line 40

I ran the test suite locally, and didn't have the same issue (locally I have PHP's CLI memory limit set to -1 so it never runs out of RAM unless I do insane-crazy things.

My Mac is Running Really Slow - Help!

This article will help you to diagnose common problems such as lack of maintenance, permissions conflicts and Hard Drive glitches that cause your Mac to run slowly.

There are two main routes to take to make your Mac as lean and mean as it was when you bought it (or even more so!)—hardware upgrades and software maintenance. One of the easiest things to do is to purchase additional RAM. As a rule, I say you can never have too much RAM. If you don't have at least 2GB of RAM, you should immediately upgrade. In addition to RAM, you should consider getting a faster hard drive (an SSD drive is what I recommend–more on that later). If your Mac is more than five years old, however, you might want to consider purchasing a new one, especially if you're running newer applications or the latest version of Mac OS X.

The following are essential utilities for keeping your Mac running smoothly and for fixing little problems that may crop up (Click the links to go to their download sites):

My Windows PC is Running Really Slow - Help!

This article will help you to diagnose common problems, such as Spyware, Adware, Viruses, RAM and Hard Drive problems, that cause your Windows PC to run slowly.

Over the course of time, everything becomes dirty and needs some cleaning. Computers are no different; many different programs do many different things on your computer, and 'dirt' can result. It's a good idea to clean up your computer from time to time, and it's usually not very hard to do! Be sure to read through this whole article to find advice for keeping your PC running smoothly far into the future.

Running Apache Benchmarks: Drupal/Joomla core vs. Static Page Cache

I just discovered (after asking about it in the #drupal IRC channel) the wonderful little program ab, included in an Apache installation. This little nugget does one thing, and does it well: It beats the heck out of your server, then tells you how your server did in terms of page serving. I tested a few different configurations on a dedicated, 4-core, 4 GB RAM server from SoftLayer, and used the following two commands:

1. Download the specified URL 1,000 times, with KeepAlive turned off (each request gets a new http connection):

ab -n 1000 -c 5 http://ip.address.of.site/path-to-page.php

2. Downlaod the specified URL 1,000 times, with KeepAlive turned on (thus allowing the connection to be maintained for as many http downloads as you have set in your httpd.conf file):

ab -n 1000 -kc 5 http://ip.address.of.site/path-to-page.php

I ran these tests a few different ways, and here are the results of the tests I ran with KeepAlive on, with the number of pages per second ab reported listed after the method:

  • Drupal - normal page caching turned on, css/js aggregation, 55kb page – 12.5 pages/sec
  • Joomla - no page caching (disabled due to buggy 1.x caching), 65kb page – 8.2 pages/sec
  • Drupal - boost module enabled, serving up the boost-cached file – 3,250 pages/sec
  • Joomla - custom page caching system enabled, serving static html file – 2,600 pages/sec

Speed boost due to caching: ~250x faster!

Getting OSC's Drupal Install Optimized on Slicehost

Initially, when thinking about finally taking the plunge and purchasing a slice or two from Slicehost, I thought, "wow, this is going to be incredibly fast and awesome, compared to my Host Gator account!"

Slicehost + Drupal

But, after setting everything up and putting Open Source Catholic live on the fresh slice, running free -m, and looking at the results, reality set in: 256 MB of RAM is not much to work with if you're running a Drupal site on a LAMP stack! Drupal usually consumes 15-40 MB of RAM per page view for a logged-in user, and if you have a site with 10 or so logged in users at any moment... well, bad things can happen.

For anonymous users, using Boost will help your site fly no matter the amount of RAM you have. But even so, a bunch of requests to uncached pages will cause your site to load a heck of a lot slower, and will fill up your RAM faster than a fire hose fills up an 8 oz. glass!

Using default Apache, MySQL and PHP settings, free -m showed a full 250 MB of RAM used, along with 400-500 swap space used (swap should be kept to a minimum—if you have a lot of swap usage, that means the hard drive is being used instead of RAM, and the hard drive is inherently many times slower!). After performing a few quick modifications to Apache and MySQL, I was able to get this number down to 140 MB RAM / 40-60 MB swap, on average.

I modified the server configuration in two different places: Apache's httpd.conf, and MySQL's my.cnf:

Subscribe to optimization