llm

RISC-V is the new entrant into the SBC/low-end desktop space, and as I'm in possession of a HiFive Premier P550 motherboard, I am running it through my usual gauntlet of benchmarks—partly to see how fast it is, and partly to gauge how far along RISC-V support is in general across a wide swath of Linux software.

From my first tests on the VisionFive 2 back in 2023 to today, RISC-V has seen quite a bit of growth, fueled by economics, geopolitical wrangling, and developer interest.

The P550 uses the ESWIN EIC7700X SoC, and while it doesn't have a fast CPU, by modern standards, it is fast enough—and the system has enough RAM and IO—to run most modern Linux-y things. Including llama.cpp and Ollama!

Compiling Ollama for RISC-V Linux

I'm running Ubuntu 24.04.1 on my P550 board, and when I try running Ollama's simple install script, I get:

OpenAI, which is only really open about consuming all the world's energy and half a trillion of our taxpayer dollars, just got rattled to its core.

Deepseek, a new AI startup run by a Chinese hedge fund, allegedly created a new open weights model called R1 that beats OpenAI's best model in every metric.

After a long journey getting AMD graphics cards working on the Raspberry Pi 5, we finally have a stable patch for the amdgpu Linux kernel driver, and it works on AMD RX 400, 500, 6000, and (current-generation) 7000-series GPUs.

With that, we also have stable Vulkan graphics and compute API support.

When I wrote about getting a Radeon Pro W7700 running on the Pi, I also mentioned AMD is not planning on supporting Arm with their ROCm GPU acceleration framework. At least not anytime soon.

Luckily, the Vulkan SDK can be used in its place, and in some cases even outperforms ROCm—especially on consumer cards where ROCm isn't even supported on x86!