gpu

LLMs accelerated with eGPU on a Raspberry Pi 5

After a long journey getting AMD graphics cards working on the Raspberry Pi 5, we finally have a stable patch for the amdgpu Linux kernel driver, and it works on AMD RX 400, 500, 6000, and (current-generation) 7000-series GPUs.

With that, we also have stable Vulkan graphics and compute API support.

When I wrote about getting a Radeon Pro W7700 running on the Pi, I also mentioned AMD is not planning on supporting Arm with their ROCm GPU acceleration framework. At least not anytime soon.

Luckily, the Vulkan SDK can be used in its place, and in some cases even outperforms ROCm—especially on consumer cards where ROCm isn't even supported on x86!

AMD Radeon PRO W7700 running on Raspberry Pi

Raspberry Pi 5 with AMD Radeon PRO W7700 graphics card

After years of work among a bunch of people in the Pi community (special callout to Coreforge!), we finally have multiple generations of AMD graphics cards working on the Raspberry Pi 5.

We recently got Polaris-era GPUs working (like the RX460), but in the past month we've gotten 6000 and 7000-series GPUs up and running. And many parts of the driver work at full performance—well, as much as can be had on the Raspberry Pi's single PCIe Gen 3 lane (8 GT/sec)!

I've been testing tons of modern AAA games, like Doom Eternal and Crysis Remastered, and can get 10-15 fps at 4K with Ray Tracing on, or 15-20 fps at 4K. Dropping down to 1080p is not enough to overcome the Pi's CPU bottleneck—only at resolutions under 720p does the Pi's CPU and the single PCIe lane not seem to get in the way quite as much.

Use an External GPU on Raspberry Pi 5 for 4K Gaming

After I saw Pineboards 4K Pi 5 external GPU gaming demo at Maker Faire Hanover, I decided it was time to set up my GPU test rig and see how the Pi OS amdgpu Linux kernel patch is going.

GLmark2 running on Pi 5 with AMD RX 460 external GPU

I tested it out on a livestream over the weekend, but I thought I'd document the current state of the patch, how to apply it, and what else is left to do to get full external GPU support on the Raspberry Pi.

I also have a full video up with more demonstrations of the GPU in use, you can watch it below:

How many AMD RX 7900 XTX's are defective?

I'm working on a project—a very dumb project, mind you—and I was trying to acquire the two current-gen flagship GPUs: an Nvidia RTX 4090, and an AMD Radeon RX 7900 XTX.

In some weird stroke of luck (it has been difficult to find either in stock), I was able to get one of each this week.

AMD RX 7900 XTX Reference Sapphire edition versus Nvidia RTX 4090 Gigabyte boxes

(lol at the size difference...)

Besides exorbitant price gouging, Nvidia's ownership of the crown in terms of GPU performance remains in this generation, as the 4090 blows past any competing card so far. But AMD's 7900 XTX was poised to be the best value in terms of price, performance, and efficiency (at least compared to any Nvidia offering).

External graphics cards work on the Raspberry Pi

AMD Radeon HD 7450 Graphics card with Raspberry Pi Compute Module 4

In October 2020, after Raspberry Pi introduced the Compute Module 4, I started out on a journey to get an external graphics card working on the Pi.

At the time, it'd been over a decade since the last time I'd built a PC, and I had a lot to learn about PCI Express, the state of graphics card drivers in Linux, and PCI Express support on various ARM SoCs.

Tried Nvidia's GTX 1080 - still no external GPU on a Pi

Earlier today I did a livestream on my YouTube channel to attempt using an Nvidia GeForce GTX 1080 on a Raspberry Pi Compute Module 4.

MSI Nvidia GeForce GTX 1080 Graphics Card GPU

As with all my testing, I'm documenting everything I learn in this GitHub issue, which is part of the Raspberry Pi PCI Express Card Database website.

It's only been a few hours, but I've already gotten good suggestions for better debugging than I was able to do on the stream. And someone pointed out it might be the case, due to 32-bit memory limitations on the BCM2711's PCIe bus, that no GPU with more than 4 GB of onboard RAM could work. Though it's hard to confirm there'd be no software workaround—even 1 and 2 GB graphics cards (AMD and Nvidia) are crashing the kernel in similar ways.

The full livestream is available on replay and is embedded below:

Three more graphics cards on the Raspberry Pi CM4

Last year I tested two older graphics cards—a Radeon 5450 and a GeForce GT710—on a Raspberry Pi Compute Module 4.

Jeff Geerling holds NVidia and ASRock Rack GPU and Raspberry Pi Compute Module 4 with quizzical look

This year, I've been testing three more graphics cards—a GeForce GTX 750 Ti, a Radeon RX 550, and the diminutive ASRock Rack M2_VGA.

The Compute Module 4, if you didn't know already, exposes the BCM2711's single PCI express lane, and the official IO Board has a nice, standard, 1x PCIe slot into which you can plug any PCI express device.

External GPUs and the Raspberry Pi Compute Module 4

The Raspberry Pi Compute Module 4 eschews a built-in USB 3.0 controller and exposes a 1x PCI Express lane.

The slightly older Raspberry Pi 4 model B could be hacked to get access to the PCIe lane (sacrificing the VL805 USB 3.0 controller chip in the process), but it was a bit of a delicate operation and only a few daring souls tried it.

Raspberry Pi Compute Module 4 with Zotac Nvidia GeForce GT 710 GPU

Watch this video for more detail about my experience using these GPUs on the CM4:
GPUs on a Raspberry Pi Compute Module 4!