volumes

For years, Mac users have dealt with slow filesystem performance for Docker volumes when using Docker for Mac. This is because the virtualized filesystem, which used osxfs for a while and will soon be upgraded to use VirtioFS.

But if you need to do large operations on huge codebases inside a shared directory, even using NFS to share from the Mac into Docker is a lot slower than running a native Docker volume or just using files inside the container's own filesystem.

Docker for Mac's shared volume performance saga continues!

After monitoring the issue File system performance improvements for years (discussion has moved to this issue now), it seems like the team behind Docker Desktop for Mac has finally settled on the next generation of filesystem sync.

For years, the built-in osxfs sync performance has been abysmal. For a Drupal developer like me, running a default shared volume could lead to excruciating slowdowns as PHP applications like Symfony and Drupal scan thousands of files when building app caches.

Or God forbid you ever have to install dependencies using Composer or NPM over a shared volume!

It got to the point where I started using NFS to speed up volume performance. Heck, the Docker team almost added Mutagen sync, which I tested successfully, but it caused problems for too many projects.

tl;dr: Docker's default bind mount performance for projects requiring lots of I/O on macOS is abysmal. It's acceptable (but still very slow) if you use the cached or delegated option. But it's actually fairly performant using the barely-documented NFS option!

July 2020 Update: Docker for Mac may soon offer built-in Mutagen sync via the :delegated sync option, and I did some benchmarking here. Hopefully that feature makes it to the standard Docker for Mac version soon.

September 2020 Update: Alas, Docker for Mac will not be getting built-in Mutagen support at this time. So, read on.

If that post title isn't a mouthful...

I'm excited to be moving a few EKS clusters into real-world production use after a few months of preparation. Besides my Raspberry Pi Dramble project (which is pretty low-key), these are the only production-grade Kubernetes clusters I've dealt with—and I've learned a lot. Enough that I'm working on a new book.

Anyways, back to the main topic: As of Kubernetes 1.11, you can auto-expand PVs from most cloud providers, AWS included. And since EKS now runs Kubernetes 1.11.x, you can have your EBS PVs automatically expand by just increasing the PVC claim size in spec.resources.requests.storage to a larger size (e.g. 10Gi to 20Gi).

To make sure this works, though, you need to make sure of a few things:

Make sure you have the proper setting on your StorageClass

You need to make sure the StorageClass you're using has the allowVolumeExpansion setting enabled, e.g.:

Using the official MySQL Docker image from Docker Hub, I recently ran into the error:

Host '172.18.0.1' is not allowed to connect to this MySQL server

The only change I had made to my docker-compose.yml file was:

mysql:
  image: mysql:5.6
  ports:
    - '3306'
  volumes:
    # Use this option to persist the MySQL DBs in a shared volume.
    - ./mysqldata:/var/lib/mysql:rw,delegated
    # Use this option to persist the MySQL DBs in a data volume.
    # - db_data:/var/lib/mysql

I switched from using a data volume (db_data) to mounting a volume from my host (mysqldata in the current directory), and after the next time I did a docker-compose down and docker-compose up, I started seeing the error about my host not being allowed to connect to the MySQL server.