Docker for Mac's shared volume performance saga continues!
After monitoring the issue File system performance improvements for years (discussion has moved to this issue now), it seems like the team behind Docker Desktop for Mac has finally settled on the next generation of filesystem sync.
For years, the built-in osxfs sync performance has been abysmal. For a Drupal developer like me, running a default shared volume could lead to excruciating slowdowns as PHP applications like Symfony and Drupal scan thousands of files when building app caches.
Or God forbid you ever have to install dependencies using Composer or NPM over a shared volume!
It got to the point where I started using NFS to speed up volume performance. Heck, the Docker team almost added Mutagen sync, which I tested successfully, but it caused problems for too many projects.
But (hopefully) no more! The latest version of Docker Desktop for Mac includes new toggles under the 'Experimental Features' section:
I tested with the defaults (both features disabled, and 'Use gRPC FUSE for file sharing' under the 'General' tab), and then with VirtioFS enabled, and the results speak for themselves:
- Defaults (gRPC FUSE): 93.750s
- Enable VirtioFS: 25.461s
That's a 114% speedup, and it makes a huge difference for my PHP development workflows using Docker on my Mac.
Update: Adding a note that on my new M1 Max Mac Studio, with Enable VirtioFS, the install completes in just 15.020s!
And this isn't a synthetic filesystem test benchmark, I tested installing my Drupal codebase three times in each condition, and those are the averages. I do this regularly, and a 4x speedup in this part of the workflow is huge.
Some users have reported issues with file permissions or running git operations from within the container, but since I manage my codebase on the Mac side, and don't do anything too exotic inside the running container, I haven't encountered an issue yet.
So far I'm not sure if similar improvements are on their way for Windows, but this longstanding issue's permanent resolution should bring more users across (even in the midst of some abandoning Docker Desktop because of their licensing changes).
Comments
A 100% speedup would mean halving the time, so I'm confused by how reducing the time to a little over a quarter equates to only a 114% speedup.
Wouldn't it be 93.75 / 25.461 = 3.68 (i.e. 368% of the original speed, or 268% faster than the original) ?
You're right, I'm honestly not sure where I pulled that number from—I'm guessing I just did a completely incorrect calculation somewhere along the line... Fire the proofreader!
I've updated it to just '4x faster' (it's about 3.7x faster).
Hey as I am myself interested in a m1 air, did you ever try out to have a amd64 VM? Is that viable or just not worth it. Sadly some stuff I need isn't available on arm64, so would need to have a VM for that stuff, as compiling it myself would be just too big of a hassle
No need for VM. On your docker compose file just add
platform: linux/x86_64
It will run a bit slower than native arm64 but is the best workaround I found
I have a rather complex setup here, in your test you used a docker container with `docker -v` only or a docker-compose stack? I still failed to reproduce that magic gain in performance here. docker-compose.yml with `:cached` volumes in my case.
Went and had a look at the repository the author mentioned in the article and it looks like it's via docker compose using a bind mount volume with the delegated flag, at least for the Drupal side of things.
Hi Jeff, I am trying to test this myself. I was running into some really unexplained slow page loads via browser using lando. Bizarrely didnot notice this with drush or cli ops. I think what game me the most speed boost is enableing opcode cache as described here:
https://github.com/Deutsche-Digitale-Bibliothek/ddbpro/blob/8f1ce397a34…
Before that I also tried viritiofs and didn't notice a boost. In the Docker preferences it says: "his improves I/O performance for operations on volumes shared with '-v'." Do you know what that mean and how we can verify the volumes are shared with "-v" using docker volume inspect?
Finally, the biggest boost I noticed with db operations, is not mounting the db data volume. 2GB database import was done in 9 minutes vs 30-40 mins.
VirtioFS has been broken on Docker for macOS since it was released. It doesn't release file handles until the Docker application is completely restarted (not just restarting the Docker service, a full shutdown of Docker is required). I opened a GitHub issue for this over a year ago, but Docker has not even acknowledged the issue. https://github.com/docker/for-mac/issues/6783