network

Last year, after I started a search for a good out-of-the-box all-flash-storage setup for a video editing NAS, I floated the idea of an all-M.2 NVMe NAS to ASUSTOR. I am not the first person with the idea, nor is ASUSTOR the first prebuilt NAS company to build one (that honor goes QNAP, with their TBS-453DX).

But I do think the concept can be executed to suit different needs—like in my case, video editing over a 10 Gbps network with minimal latency for at least one concurrent user with multiple 4K streams and sometimes complex edits, without lower-bitrate transcoded media (e.g. ProRes RAW).

For the past year, I've slowly upgraded parts of my network to 10 Gigabit. But 10 Gigabit switches, NICs, and even cabling is a bit more expensive and sometimes annoying to deal with than the very-cheap 1 Gbps equipment most homelabbers are used to.

I dipped my toes into the 2.5 Gbps waters once I got a NAS with 2.5G ports—you can use standard USB NICs that cost less than $50, or PCIe cards for even less. And cabling is easier, since 2.5G works fine over Cat5e (which I already have run to most of my house).

So in order to install a new WiFi 6 Access Point upstairs—and get it's full bandwidth—I upgraded my main 1 Gbps PoE+ switch to a 2.5 Gbps PoE+ switch.

Looking around at options, most switches with more than 4 2.5 Gbps ports with PoE+ seem to cost upwards of $300. And knowing that I'd like to expand my network a bit in the future, I finally splurged a bit and bought this 20-port monstrosity:

Video: This blog post is a companion piece to my video: Raspberry Pi does what Microsoft can't!

With a new Network Install feature, a Raspberry Pi can now set itself up—without any flash drive or other computer—directly over the Internet.

Since the Raspberry Pi Compute Module 4 was introduced last year, I've been testing a variety of PCI Express NICs with it. One of the main types of NIC I'm interested in is cheap 2.5 Gigabit Ethernet adapters.

2.5 Gigabits is about the highest reasonable bandwidth you can get through the PCI Express Gen 2.0 x1 lane on the Raspberry Pi, and it's also a lot more accessible than 10 Gigabit networking, especially for home users who might already have Cat5e runs that they are loathe to swap out for Cat6 or better cabling.

In my testing, besides discovering that not all 10 Gbps SFP+ transceivers are created equal, I found out that when it comes to performance, the Linux driver you're using matters—a lot.

There could be a thousand reasons something like this happens, but here's my situation:

A few months ago, when I was testing 10 Gigabit networking on the Raspberry Pi, I noticed something funny when I was doing iperf3 speed tests between devices connected through my MikroTik CRS305 10G switch.

If you read the title of this blog post and are thinking, "10 Gbps on a Pi? You're nuts!," well, check out my video on using the ASUS XG-C100C 10G NIC on the Raspberry Pi CM4. Back? Good.

To be clear: it's impossible to route 10 gigabits of total network throughput through any Raspberry Pi on the market today.

But it is possible to connect to a 10 gigabit network at 10GBase-T speeds using a Raspberry Pi Compute Module 4 and an appropriate PCI Express 10G NIC. And on my Pi PCI Express site, I documented exactly how I got an ASUS XG-C100C working on the Raspberry Pi. All it takes is a quick recompile of the kernel, and away it goes!

Now that I have a half-height rack and a 3D Printer, I figured I should finally move all my Raspberry Pis from sitting in odd places in my office to the rack. And what better way than to print my own 1U Raspberry Pi Rack mount unit?

The rack unit you see above was assembled from 6 'frames', 6 hot-swappable Pi carrier trays, 2 rack mount ears, and a couple lengths of threaded rod for rigidity.

It was printed from these plans from russross on Thingiverse; Russ Ross also made an assembly video, and shows how you can build a 2U 12-Pi enclosure using the same basic design, with interchangeable Pi trays!

Video

There is more detail and a full walkthrough of my home rack in this video:

Since the day I received a pre-production Raspberry Pi Compute Module 4 and IO Board, I've been testing a variety of PCI Express cards with the Pi, and documenting everything I've learned.

The first card I tested after completing my initial review was the IO Crest 4-port SATA card pictured with my homegrown Pi NAS setup below:

But it's been a long time testing, as I wanted to get a feel for how the Raspberry Pi handled a variety of storage situations, including single hard drives and SSD and RAID arrays built with mdadm.

I also wanted to measure thermal performance and energy efficiency, since the end goal is to build a compact Raspberry-Pi based NAS that is competitive with any other budget NAS on the market.

tl;dr: EFS is NFS. Networked file systems have inherent tradeoffs over local filesystem access—EFS doesn't change that. Don't expect the moon, benchmark and monitor it, and you'll do fine.

On a recent project, I needed to have a shared network file system that was available to all servers, and able to scale horizontally to anywhere between 1 and 100 servers. It needed low-latency file access, and also needed to be able to handle small file writes and file locks synchronously with as little latency as possible.

Amazon EFS, which uses NFS v4.1, checks all of those checkboxes (at least, to a certain extent), and if you're already building infrastructure inside AWS, EFS is a very cost-effective way to manage a scalable NFS filesystem. I'm not going to go too much into the technical details of EFS or NFS v4.1, but I would like to highlight some of the painful lessons my team has learned implementing EFS for a fairly hefty CMS-based project.

Pi Hole is a nifty open source project that allows you to offload the task of blocking advertisements and annoying (and often malicious) trackers to a Raspberry Pi. The installation is deceptively simple (a curl | bash affair), but I wanted to document how I set up mine headless (just plugging the Pi into power and the network).

Set up Raspbian Lite

I bought a Raspberry Pi model 2 B along with the official Raspberry Pi foundation Case. Then I bought a Samsung Evo+ 32GB microSD card (which comes with a full-size SD card adapter), and did the following steps on my MacBook Pro to set up the Pi's OS: