raid

Earlier this year, I pitted the $549 ASUSTOR Lockerstor 4 NAS against a homebrew $350 Raspberry Pi CM4 NAS, and came to the (rather obvious) conclusion that the Lockerstor was better in almost every regard.

Well, ASUSTOR introduced a new lower-cost NAS, the $329 Drivestor 4 Pro (model AS3304T—pictured above), and sent me one to review against the Raspberry Pi, since it make for a better matchup—both have 4-core ARM CPUs and a more limited PCI Express Gen 2 bus at their heart.

Around the same time, Radxa also sent me their new Taco—a less-than-$100 Raspberry Pi Compute Module 4 carrier board with 5x SATA ports, 1 Gbps and 2.5 Gbps Ethernet, an M.2 NVMe slot, and an M.2 A+E key slot. (The Taco will soon be available as part of a kit with a CM4 and case for around $200.)

The specs evenly matched, at least on paper:

This is a simple guide, part of a series I'll call 'How-To Guide Without Ads'. In it, I'm going to document how I set up bcache on a Raspberry Pi, so I could use an SSD as a cache in front of a RAID array.

Getting bcache

bcache is sometimes used on Linux devices to allow a more efficient SSD cache to run in front of a single or multiple slower hard drives—typically in a storage array.

In my case, I have three SATA hard drives: /dev/sda, /dev/sdb, and /dev/sdc. And I have one NVMe SSD: /dev/nvme0n1.

I created a RAID5 array with mdadm for the three hard drives, and had the raid device /dev/md0.

I then installed bcache-tools:

$ sudo apt-get install bcache-tools

And used make-bcache to create the backing and cache devices:

When I heard about Radxa's Taco—a Raspberry Pi Compute Module 4-powered NAS/router-in-a-box—I knew what must be done.

Load it up with as much SSD storage as I can afford, and see what it can do.

And after installing five Samsung 870 QVO 8TB SSDs and one Sabrent Rocket Q NVMe SSD—loading up every drive slot on the Taco to the tune of 48TB raw storage—I found out it can actually do a lot! Just... not very fast. At least not compared to a modern desktop.

Special thanks to Lambda for sponsoring this project—I was originally going to put a bunch of the cheapest SSDs I had on hand on the Taco and call it a day, but with Lambda's help I was able to buy the 8TB SSDs to make this the most overpowered Pi storage project ever!

This is a simple guide, part of a series I'll call 'How-To Guide Without Ads'. In it, I'm going to document how I set up a ZFS zpool in RAIDZ1 in Linux on a Raspberry Pi.

Prequisites

ZFS does not enjoy USB drives, though it can work on them. I wouldn't really recommend ZFS for the Pi 4 model B or other Pi models that can't use native SATA, NVMe, or SAS drives.

For my own testing, I am using a Raspberry Pi Compute Module 4, and there are a variety of PCI Express storage controller cards and carrier boards with integrated storage controllers that make ZFS much happier.

I have also only tested ZFS on 64-bit Raspberry Pi OS, on Compute Modules with 4 or 8 GB of RAM. No guarantees under other configurations.

Installing ZFS

Since ZFS is not bundled with other Debian 'free' software (because of licensing issues), you need to install the kernel headers, then install two ZFS packages:

This is a simple guide, part of a series I'll call 'How-To Guide Without Ads'. In it, I'm going to document how I create and mount a RAID array in Linux with mdadm.

In the guide, I'll create a RAID 0 array, but other types can be created by specifying the proper --level in the mdadm create command.

Prepare the disks

You should have at least two drives set up and ready to go. And make sure you don't care about anything on them. They're gonna get erased. And make sure you don't care about the integrity of the data you're going to store on the RAID 0 volume. RAID 0 is good for speed... and that's about it. Any drive fails, all your data's gone.

Note: Other guides, like this excellent one on the Unix StackExchange site, have a lot more detail. This is just a quick and dirty guide.

List all the devices on your system:

Last week, I posted about the Pi NAS hardware build, and compared setting up an off-the-shelf ASUSTOR Lockerstor 4 to the same thing, but with a Raspberry Pi Compute Module 4 an IO Board.

Some people wondered why not just use a Raspberry Pi 4 model B with USB hard drives, and there are a few reasons:

Just posting to the blog for reference; I posted this video on YouTube recently, in which I built (what I believe to be) the world's tiniest NVMe SSD RAID array, using the Raspberry Pi Compute Module 4 and three diminutive WD SN520 NVMe drives (which are M.2 2230 size, which makes them each about the size of a quarter):

I ran some benchmarks in RAID 5 and RAID 0, as well as one drive by itself, and found one surprising thing: the Pi's overall IO bandwidth is already saturated by just one drive, so putting NVMe disks in RAID doesn't really help with performance, like it does with slower spinning hard drives.

For Pi Day, I'm going to livestream my second attempt at getting 16 hard drives (well, 12 hard drives and 4 SSDs) recognized by a Raspberry Pi.

The first attempt went decently well... but I wound up running into power supply issues.

This time around, I will hopefully have those issues solved, and also we may have a little fun building a software-RAID-on-hardware-RAID (depending on how crazy we want to get). It probably won't work like I expect, but that's what makes it fun!.

A few months ago, I posted a video titled Enterprise SAS RAID on the Raspberry Pi... but I never actually showed a SAS drive in it. And soon after, I posted another video, The Fastest SATA RAID on a Raspberry Pi.

Well now I have actual enterprise SAS drives running on a hardware RAID controller on a Raspberry Pi, and it's faster than the 'fastest' SATA RAID array I set up in that other video.

A Broadcom engineer named Josh watched my earlier videos and realized the ancient LSI card I was testing would not likely work with the ARM processor in the Pi, so he was able to send two pieces of kit my way: