With the Raspberry Pi 5's exposed PCI Express connector comes many new possibilities—which I test and document in my Pi PCIe Database. Today's board is the Geekwork X1011, which puts four NVMe SSDs under a Raspberry Pi.
Unlike the Penta SATA HAT I tested last month, this carrier uses thinner and faster NVMe storage, making it a highly-compact storage expansion option, which has the added benefit of freeing up the top of the Pi 5 for other HAT expansion options.
The board isn't without its flaws and oddities, though—for example, power is a little strange. I opted for a 27W GaN USB-C charger from Argon forty for my testing:
I noticed there was also a "DC5V" connector on the X1011, so I initially assumed I'd need to find a 5V DC barrel plug power supply, but when I went to Geekworm's Wiki, I found this line:
Don't power the X1011 via DC powe [sic] jack and the Raspberry Pi5 via USB-C at the same time.
It seems like they have a bit of a dumb power circuit that might not have protection from backfeeding, so I would heed their caution and not plug two power supplies in at the same time.
Want a video instead? I reviewed this board with a little more detail over on my Level 2 Jeff YouTube channel. Watch here: Tiny Pi NAS: It's impossible to recommend.
But how does the board get power from the Pi 5? Raspberry Pi maintains you shouldn't draw more than 5W through the PCIe FFC, so where does the board get up to 5A 5V power? It's through these pogo pins:
Pogo pins aren't unheard of in Pi undermount boards. In fact, there's a grid of test points on the bottom of the Pi—5V power rail included—and tapping into them with a spring-loaded connector is a clever hack that avoids mechanical connectors or use of the GPIO header.
Personally I have run a couple Pi Zeros with pogo pin connections for USB and power before, and only had issues one time when one of the pins bent a little so wasn't making great contact. And I'm guessing Raspberry Pi doesn't finish off test point solder to a degree of precision where every pogo pin contact will be perfect... but it worked in my testing on this board with this Pi 5, so I won't complain too much.
If you're considering using this in a high-vibration environment, maybe look elsewhere; that's the achilles heel of even the best pogo pins!
Moving on to something that should be a lot less controversial, the FFC (Flat Flexible Cable) that connects the board to the Pi 5's PCIe header uses a double-90° design like I first saw in Pimoroni's NVMe BASE, which allows for easy access to the microSD card slot on the edge of the Pi 5. I like that design, and the impedance-matched cable seems well designed and durable enough. Geekworm even includes two, in case you damage one or it gets flaky.
Back to something that is a little disappointing:
Because the Pi 5 only exposes one lane (x1) of PCI Express Gen 2, you need a PCI Express switch chip (much like a network switch) to split that single lane into four connections.
The ASM1184e is a solid, reliable PCIe Gen 2 switch, but it is limited to Gen 2 speeds (5 GT/sec), whereas the Pi's internal bus can be pushed to PCIe Gen 3 speeds (8 GB/sec).
Considering the speed of even cheaper NVMe SSDs, you're sacrificing that much more performance splitting up the lower Gen 2 bandwidth among four devices. In reality, the Pi's other IO is an even bigger bottleneck here (it only has 1 Gbps built-in Ethernet, though you can get 2.5 Gbps using a USB 3 adapter), so it's not as big a problem as it may seem. But it is a problem—across four NVMe SSDs in RAID 0, the maximum raw throughput will top out around 430 MB/sec:
| Benchmark | Result |
|---|---|
| iozone 4K random read | 46.43 MB/s |
| iozone 4K random write | 138.80 MB/s |
| iozone 1M random read | 418.14 MB/s |
| iozone 1M random write | 393.81 MB/s |
| iozone 1M sequential read | 430.45 MB/s |
| iozone 1M sequential write | 398.13 MB/s |
This is where a device like Friendlyelec's CM3588 shines—the RK3588 SoC used in it includes four PCIe Gen 3 lanes, for a total of 32 GT/sec of bandwidth. That's... a lot more :)
The switch chip is also the hottest part of this entire setup, running at 55°C or so after an hour of repeated storage benchmarks in open air. If you put this in a case, I would highly recommend either a fan, or a heatsink over the chip to prevent any performance issues.
The drives themselves don't seem to heat up that much—but that's dependent on what NVMe SSDs you install.
I put in some very inexpensive TLC drives suitable more for read-heavy use, some Inland TN320 SSDs I picked up at Micro Center. They seemed to stay under 50°C even after many read/write benchmarks, writing through a few TB of data in a ZFS RAIDZ1 array.
Interestingly, in addition to the four NVMe activity LEDs on top of the Geekworm board, the Inland SSDs actually have an integrated green LED, so no matter what way you flip the board, you'll get some blinkenlights:
Always welcome in my homelab.
I mentioned the Friendlyelec's CM3588 above—it's probably the most appropriate board to compare to, and LTT did a video on it, testing OMV. So comparing on price alone, how do the boards stack up?
| Item | Pi 5 + X1011 | Friendlyelec CM3588 |
|---|---|---|
| 4x NVMe carrier board | $51 | $95 |
| SBC/SoM | $59 (4GB Pi 5) | $35 (4GB RK3588) |
| Power Supply | $10 | $10 |
| microSD (for boot) | $12 | $12 |
| NVMe Drives (4x 256GB) | $111.96 | $111.96 |
| TOTAL | $243.96 | $263.96 |
Pricing doesn't take into account durability, software support, or anything else. For example, you can already buy a PoE HAT for the Pi 5, meaning you could run this entire system over PoE. Running the CM3588 over PoE would require an external adapter to break out PoE power. And Pi OS already runs great with ZFS, OMV, and other storage solutions, though neither of these builds runs more popular NAS OSes like Unraid (see latest Unraid on Arm discussion) or TrueNAS (see latest TrueNAS on Arm discussion).
That said, I've been running vanilla ZFS on two Arm NASes in production for months, with zero hiccups and no extra overhead... so do you really need all the fancy chrome and shiny buttons one of those NAS UIs give you?
I also haven't found anyone designing a 3D case for the X1011 yet, but I'll update this post with a link to one if I do. There's a CM3588 case design from sgofferj with space for two small fans.
One caveat: until this firmware issue is resolved, you can't boot direct off an NVMe drive behind a PCIe switch (like on the X1011) without having the first stage bootloader on another storage device (like USB or microSD card). Hopefully that's resolved soon!
For the latest testing and info about this board, and other PCI Express devices for Raspberry Pi, check out the issue on GitHub: Geekworm X1011 PCIe to Four M.2 NVMe SSD HAT.
Watch the that goes along with this blog post:
Comments
Few people still see the alure of the Raspberry Pi after the shortage. As a former Pi 4 8gb owner, anything Raspberry is last on my list.
I think for some segments of the homelab market, that may ring true; however, sales volumes (and traffic patterns for blog posts and videos on it) indicate the wider market hasn't written off Raspberry Pi.
I agree, I used to have an army of Pi 4 boards doing various things around the house. Now I just have a bunch of VMs doing the same thing. Raspberry Pi still serves a purpose in my opinion, but I think some of us got carried away :) Also before anyone straps four NVMe drives up to their Pi, remember that for not much more you can do the same with SATA flash drives that will have a better $/TB ratio, but you can actually make use of that bandwidth with 2.5/5/10Gbps network hardware either built into the board or as an add-in card.
And you'll get even more value (cost per TB) from spinning rust still :)
A lot of the projects I work on are more of a 'can it do this' thing, rather than a 'you should do this' thing, and I try to make that very clear (especially since I've only recommended people buy a couple of the Pi 5 projects I've featured on this blog this year, and only if it's suitable for their niche use case).
The Pi 4 was (technically still is) cheap enough to throw at almost any non-intense task, so people got carried away (as you said!). I think the shortage was a reality check... VMs and Docker/containers are more efficient for simple use cases, especially if you don't enjoy putting together a bunch of hardware nodes on your network!
I don't know...
I have been daily driving the Pi 4 8GB for about 4 years now, and it's only been 5 days since I've recompiled the kernel :D
I'm literally just building out a CM3588 after it arrived on a slow-boat from China.
Like a lot of boards, the installation process doesn't seem to be great, but made the multi-installer SD image, installed Debian to the eMMC, and then converted to a DietPi install.
However, performance seems reasonable so far, transferring files from my old SATA based 'NAS' onto 2x1TB Crucial P3 plus drives in an LVM JBOD with XFS has been running at a constant 110 MiB according to rclone.
Rclone is copying 17 threads simultaneously and is using ~40% of the performance cores.
Its running at 57c without a fan in a 23c room so not bad, but I will get a fan on it.
Its nice to be able to just jam some NVMe drives into the system without having to faff around with daughterboards, ribbon cables etc, so all in all, pleased so far.
👀 seems complicated!
Is there an Armbian image for the board? It seems like it should be easier to get up and running with one of these.
I'm designing a case in blender but totally over drawings and it's my first ever try at 3d design.
I don't access to a 3d printer of myself.
Just to say, don't expect too much.
If anyone is willing and daring to print the case once I think it's finished enough, I'll be making the board available (open source) .
Just drop me a mail on Stefan.vantheemsche Gmail com
Jeff, your content is gold as usual. What would be your opinion on grabbing the cm3588 vs the geekworm board? I've been looking at the cm3588 for a bit and going for something higher end (16GB or higher) does drive up the price and I'm wondering if there is value. The friendly elec seems to allow for a lot more production from the NAS itself.
The CM3588 would probably be the better bet, especially if you're just going to use it as a NAS.
Thanks Jeff. I'm most comfortable with the PI ecosystem and was leaning that way, but felt this was the better way to go based on your videos and a cpl others floating around. I'll let you know how it shakes out.
Thank you for the article. I want to retire my Synology NAS and get more out of my RPI5 8Gb. I am interested in mounting 4 NVME with PCIe Gen3. I have searched, but can only find the Geekworm one from Gen2. Do you know if they plan to upgrade the HAT X1011 to Gen3?
Hey Rual, if you don't mind me asking: how is your research going? I am also trying to move away from a Synology DS 4 bay SHR-5 into a SBC ssd array. I'll buy a more expensive board if I have to but this is quite a lot for a relative beginner. I just recently got into proxmox PVE and that turned me onto ZFS
Hey Jeff, was wondering in theory if something like this would work with Rpi5
A Geekwork X1011 with 4 (ebay uk item 466376135224) m2 to mini SAS slots =4 x 4 sata 16 drives
Just wondering?
Hey Jeff, appreciate the content! I recently purchased a GeekPi N16 Quad M.2. This is a link to the product. I noticed this allows me to boot off of a NVMe and just like your experience, no additional tweaking required. How does this board differ from yours? Can this be tweaked to allow 4 lane PCIe? Is four lane mandatory for a NAS? Sorry for all the Qs.