(This is not entirely about maker projects, but it is about Raspberry Pi and other single board compute devices, so hopefully it is sufficiently on-topic and of interest to the forum).
I recently peeked at someone’s Kubernetes dashboard on a Mac, and was quite fascinated by it - it had around 300 containers under management! I fancy brushing up my devops skills, so have been pondering how to spin up a K8S cluster in a way that will keep me learning. The boring way is to use a 32GB laptop and six or so virtual machines at 2GB each, but I don’t know whether that will keep my easily-distracted brain engaged.
The ⚡️ interesting approach 🎉 is to build a hardware cluster with a network switch. I can get 10 NUC boxen from eBay for £400, which is probably rather too many. I probably want about 4-5 running, but it is cheaper to buy them in bulk when a job lot comes up. However, I’d much rather get a lean 4x Raspberry Pi CM4 cluster, say in 4GB RAM modules. But I saw one of these on eBay UK, and the seller wants £999 for the kit. Based on a box count, that’s about five times as expensive!
There are other boards, such as the Nvidia Jetson Nano (2GB), which on Pimonori goes for £60 (it is presently out of stock). I wonder if this board is also scarce - scalpers on eBay UK are asking for £200 and upwards, with only a handful being available in China. That is starting to look like the £1000 price of the horrendously-expensive RPi cluster, which is hard to justify for a learning project.
Are there any other SBC devices (say with 2-4GB RAM) that are worth considering? I should think anything that can run generic Linux (e.g. Ubuntu) would do the trick.
I’d seen the first link, but yes, I should think Zeros are out - I think Kube needs 2GB per node to run. So normal RPis would be required (only 4s sadly, again due to RAM limitations).
The second one looks great, I had not found that one. The 4GB version goes for around £60, and five of those makes £300. It would be the same price as using Mini PCs (Intel NUCs) but with a much reduced power requirement. So this would be viable if RPi4s were widely available! 😬
(Interestingly the ClusterCTRL Stack doesn’t offer any network switch facility - seems like a missed opportunity, given that clusters are always wired up on the same LAN).
Hmm, the shop has a few Raspberry Pi 400 devices knocking about, costing £80 each. One could just fish out the Pi itself (see teardown) and then build a case that can contain three or four of them. 🤔
(One source on the web says that you can get the Pi 400 board separately, but I expect they are OOS).
Yeah, seen that video - very cool. It looks great, but it’s not cost-effective per node, and the CM units aren’t available anyway. [Edit: ah, I am confusing the scalper prices on eBay with the RRP!]
I found another SBC called a Zima Board, at 120 USD (~£100). It’s still nearly twice the cost of using NUCs, but the power consumption would be much more efficient.
Ooh, this clusterboard looks pretty good - the board for seven modules is $100 USD, and then compute modules are $33 each. The downside is that the modules are only 2GB of RAM, and shipping outside of the US looks a bit ropey. I think the company is based in China/HK.
Plus, the warranty is poor, and some of the product descriptions suggest the testing and UX polish is a bit lacking compared to, say, an RPi.
He has several videos on Pi Clusters. Should be one or two economy versions in there, lol. It would be the way I would have to go if i was going to try it.
And probably a few “who can afford that” versions.
Ooh, I might have a scout about for those today - ta very much 👍
I have discovered a new approach to this problem. On eBay I found various “thin client” machines from Dell (under the brand name of “Wyse”), which sit in the £35-55 bracket. They generally have 2-4GB of RAM and all have gigabit Ethernet. They have very limited storage (8GB is about the median) and the CPU clock speeds tend to be low in order to conserve power. I think that they’ll be perfect, assuming they can run the K8S tools.
I bought four “mini” Lenovo business PCs - they are about the cross-section of a shoebox, but only an inch or so wide. I found a seller on eBay UK who really knows their CPU/MB specs, and I could customise quite a bit. Each unit cost about £110, if memory serves: 16GB RAM, 120GB SSD, no WiFi/BT, Gigabit Ethernet for each.
In the end I only used three, but I might still add the fourth. I put Ubuntu Server onto each of them, then Canonical’s MicroK8S, and then joined them in a cluster. I added a cron script on all of them that puts each to sleep at 1am. I then added an always-on Raspberry Pi to act as an external bastion and as a wake-up server. The wake script just uses Wake-On-Lan to unsnooze the nodes, and they wake up within five seconds or so.
The total current consumption when alive (but with no effective workloads) is around 70W, the consumption once sleeping is 8W, most of which is the Pi.
