Every so often, when you're debugging weird hardware issues on SBCs like the Raspberry Pi, it's useful to get way down into the guts of how the Pi represents its hardware to Linux.
And the Linux kernel uses a method called Device Tree overlays to do it. On the Pi 5 (and other Pis), these overlays are stored as .dtb files inside the /boot/firmware directory, and there's an overlay for every major Raspberry Pi hardware model.
I've had to modify the dtb files in the past to increase the PCIe BAR space for early GPU testing on the Compute Module 4. And recently I've had to mess with how the PCIe address space is set up for testing certain devices on the Raspberry Pi 5.
The problem is, you can't just hand-edit a .dtb file—they're in a format readable only by the Linux kernel. You have to decompile the .dtb file to a .dts (source) file, edit it, then recompile it to a .dtb.
As an example, I needed to change the msi-parent for the Pi 5's external PCIe connector to allow for full MSI-X support for a Google Coral TPU (work on getting it to work is ongoing):
# Back up the current dtb
sudo cp /boot/firmware/bcm2712-rpi-5-b.dtb /boot/firmware/bcm2712-rpi-5-b.dtb.bak
# Decompile the current dtb (ignore warnings)
dtc -I dtb -O dts /boot/firmware/bcm2712-rpi-5-b.dtb -o ~/test.dts
# Edit the file
nano ~/test.dts
# Change the line: msi-parent = <0x2f>; (under `pcie@110000`)
# To: msi-parent = <0x66>;
# Then save the file.
# Recompile the dtb and move it back to the firmware directory
dtc -I dts -O dtb ~/test.dts -o ~/test.dtb
sudo mv ~/test.dtb /boot/firmware/bcm2712-rpi-5-b.dtb
Check out elinux's Device Tree Reference for more useful background info.
I consider myself an absolute noob at Device Trees in Linux... but I've now done this enough times that I'd like a simple reference and most of the ones out there assume you are an expert-level wizard in all things Linux/hardware!
Comments
Hi Jeff,
can you please adjust the title and remove the '(device tree overlay)' part since you're explaining how to convert between dts and dtb while with overlays (dtbo) there exists the concern that this doesn't reliably work bidirectional and you always should start with the original dts to create a new dtbo: https://forums.raspberrypi.com/viewtopic.php?p=2072951&sid=96cdcb15ed6b…
(it's really just about the 'overlay' part that's confusing – at least I arrived here via Google while searching for overlays)
Changed!
So it seems that every time you run apt-get upgrade, you'll have to do this. I had it working and then the upgrade undid the changes.
Woow, you saved for using an asm1064 4 sata ports m2 card in rpi5 with this topic.
Oh nice! If you don't see your card over on this site (https://pipci.jeffgeerling.com), might you consider adding an issue for it? Would love to track your experience with it.
Hi Jeff, I have installed it in pineberry top rpi5 pcie adapter. How could I add an issue for it?
Tarjeta de expansión M.2 NVME a Mini SAS SFF-8087, compatible con 4 puertos SATA3.0, 6Gbps, HDD, SSD, controlador SATA, SFF8087 a M2, adaptador NVME
I have four disks connected and added to raidz zpool.
root@rpi5:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
volume1 7.06T 755G 7.06T /volume1
root@rpi5:~# zpool status
pool: volume1
state: ONLINE
config:
NAME STATE READ WRITE CKSUM
volume1 ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N0CYSY9L ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N0XJ91HP ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N2ZLUHU0 ONLINE 0 0 0
ata-WDC_WD30EFRX-68EUZN0_WD-WCC4N3FX7ENX ONLINE 0 0 0
errors: No known data errors
root@rpi5:~# lspci | grep -i ASME
0000:01:00.0 SATA controller: ASMedia Technology Inc. ASM1064 Serial ATA Controller (rev 02)
Hey, I've been following your guide in order to get a B+M keyed M.2 coral TPU accelerator. Using a Pimoroni NVME hat.
I ran into an issue on this step, when I booted up i would get some PCI errors and the card would not be detected.
However, after reading the forum post. I noticed that the trick was to get the pcie@110000 device to use itself as an msi-parent. And that address had changed from 0x66 in your docs. To 0x6d in my case.
You can inspect the ref a little further down in the pcie@110000 entry to find the ref (if it changes again)
Your comment saved me. I had it worked completely but I decided to reinstall pi5 os. But this time I couldnt get it to work. turns out mine also changed to 0x6d.
If I may ask, How were you able to figure that out in case it ever changes again
Of course!
I was reading the parent post to this (https://www.jeffgeerling.com/blog/2023/pcie-coral-tpu-finally-works-on-…)
And this article was linked, I also followed the guide but then noticed my PCIe card wasn't showing up at all.
In the parent article, the forum post was linked where the author figured this out. And I read through that.
The actual change you're making to the dtb is changing the msi-parent from &mip1 to &pcie1.
I looked through the dts file and found that &pcie1 was now at address 0x6d instead of 0x66, so I used that address in the pcie@110000 entry and that fixed it.
So, if anyone reading this in future encounters a similar issue. The information you need is in the .dts file. You will need to find the &pcie1 address that's being used, and update pcie@110000's mis-parent to use that address
To clarify for any other travelers that find this post. Look at the bottom of the pcie@110000 section for the "phandle" value. Mine was 0x6e. I replaced the msi-parent value to this and my SATA controller: ASMedia Technology Inc. ASM1064 Serial ATA Controller was up and running.
Cheers Draxx for the lead!
Robert