Hardware

As I hinted in my last post, I’ve had a chance to dig into hardware a bit. I did a lot of research online, and mostly found information that confirmed my prior assumptions. That always makes me suspicious, but honestly I think hardware matters less than it ever did at this point. For the majority of situations and workloads, pretty much any hardware with enough ram and storage is “good enough” today.

The classic move for someone in my situation is to pick up a single x86 machine and use proxmox to spin up 7 VMs, 3 for a Control Plane and 4 for your workers running Ceph Rook and some workloads. You can expand to multiple x86 machines from there if you need to spin up more than your x86 machine can handle.

Another popular option is to use some sort of ARM SBC, like a Raspberry Pi. In fact, going with a Raspberry Pi is attractive because they have a robust ecosystem and all of the underlying software is well tested on the hardware. However, there are other interesting options if you go beyond the Raspberry. Turing Pi has a really neat ITX board that accepts 4 of their blades, but unfortunately the peripherals available to each blade vary too much to build the kind of cluster I want. However, that turned me onto the Rockchip based boards, of which there are quite a few interesting boards to choose from.

To make a long story short, the most interesting board I found was the Orange Pi 5 Plus. It comes with 8 cores, half of them Cortex A76 and the other half A55, 16gb of RAM, two 2.5gb NICs, an M.2 slot for an SSD AND an M.2 slot I can put a SATA interface on. My only concern is the software ecosystem, fewer OSes support the Orange Pi boards. However, it’s hard to beat the value in these compared to the Raspberry Pi 5.

As you can see, by the time you’ve built out each configuration you’re looking at pretty much the same cost per node. The added cost (and complexity) of the Raspberry Pi is due to the M.2 hat only accepting 2230 and 2242 M.2 cards. These compact cards are more expensive for larger storage sizes.

But what kind of performance difference are we looking at? I didn’t run a robust set of test suites or anything, but I did install DietPi on each system and do some benchmarking. Unfortunately I accidentally deleted the file with the results, but they aligned with the DietPi avg benchmarks in general. Let’s pull out the Orange Pi 5 Plus and the Raspberry Pi 5 rows for comparison. (Pulled on 2024-12-31, may be different by the time you look.)

As you can see the Orange Pi 5 Plus beats out the Raspberry Pi 5 in every way. While there’s clearly a major difference in sampling and quality control here, my own testing put both machines close to their average. I would love to dig into the raw numbers here, as it seems obvious to me that there are some outlier reports for the Raspberry Pi 5, based on min and max. However, this is enough data for me to draw some conclusions.

Both boards perform well enough for my purposes, but there is an obvious difference in capacity per dollar. The Raspberry Pi 5 has the same number of high performance Cortex A76 cores, but lacks the 4 A55 cores that will let us have reduced power consumption when loads are light. The Raspberry Pi comes with only 8gb of RAM in a single bank, vs the 2 banks totalling 16gb that the Orange Pi has. Temperatures, even with a heatsink and constant fan, are generally 10° cooler on the Orange Pi. The Orange Pi comes with M.2 slots built-in, meaning I can add a SATA card and directly attach all my drives. You have to add a HAT to the Raspberry Pi for NVME and then use USB-SATA adaptors for storage beyond that.

With the decision to go with Orange Pi 5 Plus made, we can wrap up hardware discussions and move on the the next phase of this project. In my next post I will be talking about how I am going to package all of this hardware up in a functional and aesthetically pleasing form.

Discussions

If you have any thoughts about a topic I've written about, check out the GitHib Discussions. I’d be thrilled to hear from you.