Ever since I rebuilt my system, I decided it was time to go all-in on Linux. I don’t game as much as I used to while programming more, so what is Windows still doing for me? So I went ahead and changed my OS drive into a dual boot drive with ArchLinux as the default OS.

As a downside, I expected to regularly switch back to Windows whenever I wanted to play a game. However, I’m pleasantly surprised with Steam’s Proton compatibility layer making most games I like, run near-seamless on Linux.

As I mentioned in the previous post, I’m now running my services on a PI 4 cluster on the 2GB and 4GB ram models. Boy did I underestimate how much memory server applications like nowadays. Some applications like GitLab (4GB), ElasticSearch (4GB) and SonarQube (2GB) are some of the applications that are severely constrained in performance because I had to limit them in memory usage. Initially, I even ran GitLab and ElasticSearch on the same 4GB PI. GitLab’s nightly backup would push the memory demand too far and cause the OOM-killer to shoot the ElasticSearch cluster.

To alleviate the memory pressure on Yui (The node running GitLab and ElasticSearch), I configured a second 4GB PI as secondary ElasticSearch node (On 64-bit ARM this time!). But as ElasticSearch on Yui was heavily constrained in memory to share its place with GitLab, it actually ran out of memory while trying to replicate the data to the new ElasticSearch node on Agil. To get it done I had to shut down some of GitLab’s components in order to free up extra memory for ElasticSearch to replicate the data.

But let’s point out another thing. Using a package manager is super convenient on Linux. That is if the packages you need are in there. As I used ArchLinux on ARM, I could only use whatever they packaged for me. Many of the things I run, were not part of these. So my dabbling into Arch’s build system began. For some packages, I could create packages using the published ARM binaries. Yet for other packages, I had to compile from source as they did not support ARM out of the box or only provided Raspbian packages. To make it even more fun, the ArchLinux image for ARM was still 32-bit (ARMv7) even though the PI 4 is 64-bit (ARMv8) So once I started with the second ElasticSearch node, I had to learn about cross-compiling to different architectures. This impacted packages built from the source the most. As depending on the language’s compiler cross-compiling is set up slightly different.