macOS Finder is still bad at network file copies

In what is becoming a kind of hobby for me, I've just finished testing another tiny NAS—more on that tomorrow.

But as I was testing, I started getting frustrated with the fact I've never been able to get a Raspberry Pi—regardless of internal storage speeds, even with 800+ MB/sec PCIe-based storage—to consistently write more than around 100 MB/sec write speeds over the network, with either Samba or NFS.

NFS would be more consistent... but it ran around 82 MB/sec:

NFS file copy to Raspberry Pi 5 stalled at 80 MB per second

Samba would peak around 115 MB/sec, but it was wildly inconsistent, averaging around 70 MB/sec:

Samba file copy to Raspberry Pi 5 wild undulations

I have a problem: I use macOS1.

This blog post isn't about whether macOS is good or bad, but it is about network shares on macOS. And quantitatively, through the years, network shares on Mac OS, Mac OS X, and macOS have always been bad.

No matter what incantations I tried, with NFS, Samba, client, or server—and yes, I've even spoken to one of the Samba devs about it—there was no way to get beyond 100 MB/sec write speeds on the Pi from my Mac2.

Read speeds were always fine, when copying from the Pi to my Mac. I could peg it at 122 MB/sec over the Pi's 1 Gbps connection, and 230 MB/sec over a 2.5 Gbps connection (courtesy of the Pineberry Pi HatNET! 2.5G).

I started wondering if the problem was truly on the Raspberry Pi after tkaiser recommended monitoring more deeply with atop. I didn't see anything amiss, and the Pi had a ton of headroom (CPU, network, interrupts, and disk IO were all well in hand).

So that led me down a rabbit hole of testing:

  • I verified using iperf3 that my network connections were at line speed. iperf3 showed 940 Mbps to and from on the 1 Gbps LAN, and about 2 Gbps on the 2.5 Gbps connection—lower than the normal 2.35 Gbps because it's through a PCIe Gen 2 packet switch.
  • I verified using iozone that my storage was writing through at over 800 MB/sec to a 4-drive RAIDZ1 array of SATA SSDs (tested with a 50 GB test file in 1M chunks).
  • macOS Sonoma doesn't seem to have an /etc/nsmb.conf file by default, but according to many, in past macOS releases adding signing_required=no in this file would speed things up by disabling Samba packet signing (a security feature not really required on LANs).
  • It didn't seem like signing was active on this connection, from all my research, but I did spend an hour or so force-disabling it everywhere... which made no difference.
  • I tried a bunch of other server-side tweaks, none of them seemed to make any difference.
  • I tried using cp and rsync to see if it was just the Finder where this issue cropped up—both were more consistent in their write speed to the share, but slower overall.
  • I fired up Transmit, my SFTP client, and copied the same directory over using SSH / scp, and it copied at 112 MB/sec, very consistently. Yay for SSH file copies being faster than Samba, I guess!

I'll spare you many hours of debugging—I eventually booted up my Windows 11 PC and ran all the same tests, with the same 50 GB video project folder3.

On there?

Samba file copy to Pi 5 Windows 11

Consistent 108 MB/sec write speeds for the entire copy over the 1 Gbps connection.

Samba file copy to Pi 5 from Windows 2.5 Gbps

And then it ran at 150 MB/sec over the 2.5 Gbps connection. (The bottleneck, in this case, is the PCIe Gen 2 switch, used to install a 2.5G PCIe NIC in addition to four SATA SSDs on the Pi 5's single PCIe lane. More on that tomorrow!)

Read speeds are the same from macOS to Windows, though. Since ZFS doesn't need to do the parity calculation and writes, it can read out from the four drive array a bit faster.

Samba file copy from Pi 5 to Windows 2.5 Gbps

What I still don't understand is where the bottleneck lies on macOS's side. I don't see anything that screams bottleneck when monitoring with Activity Monitor or htop on my Mac. And I don't know of any equivalent to atop that will let me monitor interrupts and other resources like I can on Linux.

Any other ideas why macOS is so bad at writes to network shares?

I guess I should be happy network shares work at all though... in the past I would fight against macOS's built-in NFS support to even get them mounted!

  1. At least for my primary workstation where I edit video and build and test infrastructure automation. ↩︎

  2. On my HL15 NAS—which also runs on Arm, though a beefier server Ampere chip—I can get 500+ MB/sec, though even there, the write speed goes up and down a lot. I just thought that was normal... but may not. ↩︎

  3. Whenever you're benchmarking network storage, you should try to use file or directory sizes that are many times larger than the RAM on the server, that way Linux filesystem caches or ZFS caching won't give you false results. You should have files much larger than any potential cache size so you can test access all the way through to the disks. ↩︎

Further reading

macos
mac
windows
finder
samba
nfs
networking
raspberry pi
nas

Comments

I used to do MacOS drivers for a small company shipping one of the first full speed 10GbE NICs in the mid 2000s. At the time, I found that AFP was way better than NFS or SMB on MacOS.

Network storage on MacOS was so bad at the time that we partnered with companies that sold overpriced NAS solutions into tv / movie studios that let video editors access shared storage at close to 10GbE. I don't recall if they had their own FS, or just tuned APFS or SMB correctly.

Try AFP :)
I know it's old, but performance should be way better!

I wonder if the MacOS SMB client lacks support for SMB3's multi steam protocol.

Filing a bug may be a good idea.
developer-dot-apple-dot-com-slash-bug-reporting

I'm using a Mac mini M2 Pro and a Synology DS1621+ NAS. Write speed around 750MB/s and read speed around 1020MB/s. No problems.

I suggest pointing out to Apple that their OS isn't meeting your expectations in this regard.
https://www.apple.com/feedback/
They might not care about one person's opinion, but if enough people tell them, they'll listen.
Best to include specific details about what can be improved.

I can relate to your frustration. My network connection from a Mac to the office LAN is slow and requires frequent disconnecting / reconnecting. It has always been a mystery.

Coincidentally, this issue was the last straw before I switched from Mac to Linux for my daily driver. I tried most of the things you did, but got wildly inconsistent transfer speeds across the board. We do still have one Mac in the house, and I've since found that sshfs works better for the occasions when I need to transfer things to/from our server.

Have you tried the following?

https://gist.github.com/jbfriedrich/49b186473486ac72c4fe194af01288be

It have been a very long time, since I last read about transfer speed performance with Samba, but as far as I recall, the Samba developers noticed an increase in transfer speed between Samba and Windows when disabling SMBv1.

I do not own a Mac and I am never getting one, but I am guessing you have to explicit, if you want to use SMBv3?

Have you tested copying multiple files from multiple/different Finder windows/tabs? Are you copying from SATA or NVMe? Are you copying many small files or one large or a mish mash? How exactly are you measuring performance?

I use ATTO iSCSI software to connect to TrueNAS Core and see a dramatic increase in performance over Samba. I use the NAS for backup, so iSCSI works fine for my purposes.

Have you tried CIFS/AFP using Samba with vfs_fruit? For that matter, have you configured Samba in depth?

My understanding (from long time past but may still be true from limited googling) is Samba is limited because it is single-threaded (by Microsoft design). netatalk was limited because it used a database to maintain file info, although switching from the default database helped. Also, if copying from one drive, you are limited by the single drives read performance... although other factors can further limit performance. For example, if writing to a single RAIDZ vdev, writes are limited to the write performance of the single worst drive. RAID5/6 works differently (but if you aren't using ZFS, you don't care about your data anyway). Other possible factors: your disk controller chipset, the PCIe version of the NIC and disk controller (client and server), bad network cables, etc. Many also think LACP will improve single workstation throughput, it does not.

I thought the issue was limited to finder, I store photo backup on my pi NVME - about three years worth per folder and the access times are abysmal even just spotlight loading a photo which is a few megs in size. I really appreciate your work & documenting of the issue Jeff

Apple has a serious problem looking outside their own ecosystem. They seem to assume everyone just stays within their walls and never ventures outside. I found this out when I was beta testing years ago for Apple's internal Appleseed beta. They seem only concerned with Apple ecosystem functions and differed any third party issues as having to be solved by them not us. That is the frustrating thing about Apple.

The thing is, they don't even have an ecosystem solution for network file storage, so it's no like "get an Apple server" is even an option.

At any rate, in my case, I did find that NFS was significantly faster than SMB, so I use that whenever I need to transfer lots of stuff. It's not mission critical for me, though: I feel sorry for anyone in that situation.