Blog

LLMs accelerated with eGPU on a Raspberry Pi 5

After a long journey getting AMD graphics cards working on the Raspberry Pi 5, we finally have a stable patch for the amdgpu Linux kernel driver, and it works on AMD RX 400, 500, 6000, and (current-generation) 7000-series GPUs.

With that, we also have stable Vulkan graphics and compute API support.

When I wrote about getting a Radeon Pro W7700 running on the Pi, I also mentioned AMD is not planning on supporting Arm with their ROCm GPU acceleration framework. At least not anytime soon.

Luckily, the Vulkan SDK can be used in its place, and in some cases even outperforms ROCm—especially on consumer cards where ROCm isn't even supported on x86!

AMD Radeon PRO W7700 running on Raspberry Pi

Raspberry Pi 5 with AMD Radeon PRO W7700 graphics card

After years of work among a bunch of people in the Pi community (special callout to Coreforge!), we finally have multiple generations of AMD graphics cards working on the Raspberry Pi 5.

We recently got Polaris-era GPUs working (like the RX460), but in the past month we've gotten 6000 and 7000-series GPUs up and running. And many parts of the driver work at full performance—well, as much as can be had on the Raspberry Pi's single PCIe Gen 3 lane (8 GT/sec)!

I've been testing tons of modern AAA games, like Doom Eternal and Crysis Remastered, and can get 10-15 fps at 4K with Ray Tracing on, or 15-20 fps at 4K. Dropping down to 1080p is not enough to overcome the Pi's CPU bottleneck—only at resolutions under 720p does the Pi's CPU and the single PCIe lane not seem to get in the way quite as much.

M4 Mac mini's efficiency is incredible

I had to pause some of my work getting a current-gen AMD graphics card running on the Pi 5 and testing a 192-core AmpereOne server to quickly post on the M4's efficiency.

M4 Mac mini on desk

I expected M4 to be better than M1/M2 (I haven't personally tested M3), and I hoped it would at least match the previous total-system-power efficiency king, a tiny arm SBC with an RK3588 SoC... but I didn't expect it to jump forward 32%. Efficiency gains on the Arm systems I test typically look like 2-5% year over year.

The M4 mini I just bought reaches 6.74 Gflops/W on the HPL benchmark.

UPDATE 2024-11-20: I re-ran all my tests without my 10 Gbps Ethernet plugged in, and the efficiency is even better. I also re-tested LLM performance, and made this quick video:

Popular Rockchip SBC distro in limbo after maintainer burns out

Recently Joshua Riek posted he's dropping off from GitHub. If you haven't heard of him, he's one of the few reasons working with Linux on Rockchip SBCs is so much easier today than it was just a few years ago.

His Ubuntu Rockchip distribution is built for Ubuntu 22 and 24, and they've been maybe the most popular and stable way to run Ubuntu on Rockchip devices.

So popular, in fact, that manufacturers who use Rockchip, like Turing Pi, build their own official images on top of Joshua's.

XKCD Dependencies

Now, if you're reminded of XKCD #2347, yeah, I am too.

Home Assistant and CarPlay with the Pi Touch Display 2

After a decade, Raspberry Pi finally upgraded their official Touch Display from 480p to 720p, while keeping the price and overall aesthetic the same.

Raspberry Pi Touch Display 2 - Home Assistant Dashboard

I've had early access to the Touch Display 2, and have been testing it in a variety of scenarios. Generally, Linux touchscreen support isn't wonderful. And Pi OS, being a fairly customized UI focused on simple use cases, is not quite to a usable state if you go touchscreen-only, considering I had trouble getting the onscreen keyboard to work in Chromium half the time, and it would overlay things I was typing even in fully-supported apps like Terminal.