> ## Content Index > Fetch the complete content index at: https://articles.akadata.co.uk/llms.txt > Use this file to discover other available public pages before exploring further. # ⚙️ Turbocharging Banana Pi R4 Kernel Builds with distcc and ccache - URL: https://articles.akadata.co.uk/turbocharging-banana-pi-r4-kernel-builds-with-distcc-and-ccache/ - Published: 2025-07-04T04:05:54.000Z - Updated: 2025-07-04T04:23:16.000Z - Author: AKADATA Building Linux kernels on embedded devices like the Banana Pi R4 can be painfully slow. Out-of-the-box, a full kernel build on the R4 can take over two hours. Fortunately, we can massively accelerate this by offloading compilation to powerful x86\_64 machines using `distcc` and caching results with `ccache`. Here’s how to set up a distributed build system and cut build times dramatically. --- ## Hardware Setup We’re working with: ### Banana Pi R4 \*3 - CPU: 4-core ARM Cortex-A73 - RAM: 8GB - OS: Debian Bookworm (arm64) - Storage: eMMC, 64GB SD card, 256Gb NVMe ### Homer (Intel i9-13900K) - Architecture: x86\_64 - Cores: 24 cores (32 threads) - RAM: 128 GB DDR5 - Host OS: Arch Linux: Chroot: Aarch64 Debian Bookworm ### Worker (Xeon E5-2683 v4) - Architecture: x86\_64 - Cores: 16 cores (32 threads) - RAM: 256 GB - Host OS: Arch Linux: Chroot: Aarch64 Debian Bookworm Combined, Homer and Worker boast 384 GB RAM and dozens of cores to share the build load. --- ## Why distcc and ccache? **distcc** distributes compile jobs across multiple machines. Instead of compiling locally, the source code is preprocessed on the R4 and sent to the x86 hosts for compilation, then the resulting object files are returned. **ccache** caches compiler outputs. If a file hasn’t changed, ccache returns the object file instantly, avoiding repeated compilation. Used together: - **distcc** speeds up builds by distributing work. - **ccache** avoids unnecessary work. - Both reduce build times and heat on the Banana Pi R4. In practice, distcc and ccache can reduce a kernel build on the R4 from \~2 hours to under 20 minutes. --- ## Setting up the chroot environment The goal is to let Homer and Worker compile aarch64 code even though they’re x86\_64\. We use: - **Debian Bookworm rootfs** (arm64) - **qemu-aarch64-static** - **binfmt\_misc** for automatic binary translation ### Create the chroot On Homer and Worker: ```bash # Bootstrap the Debian arm64 rootfs debootstrap --arch=arm64 bookworm /mnt/bpi-chroot http://deb.debian.org/debian ``` Copy QEMU into the chroot: ```bash cp /usr/bin/qemu-aarch64-static /mnt/bpi-chroot/usr/bin/ ``` Bind mount essential filesystems: ```bash mount -t proc /proc /mnt/bpi-chroot/proc mount --bind /sys /mnt/bpi-chroot/sys mount --bind /dev /mnt/bpi-chroot/dev ``` Enter the chroot: ```bash chroot /mnt/bpi-chroot /bin/bash ``` Inside the chroot: ```bash apt update apt install distcc ccache build-essential ``` --- ## Installing distcc and ccache ### On Debian/Ubuntu ```bash sudo apt install distcc ccache qemu-user-static ``` ### On Arch Linux ```bash sudo pacman -S distcc ccache qemu-user-static ``` --- ## The distcc wrapper script The kernel build calls compilers directly. We must force it to use distcc. On the R4: ```bash mkdir -p ~/distcc_bin cat > ~/distcc_bin/aarch64-linux-gnu-gcc <<'EOF' #!/bin/bash exec distcc aarch64-linux-gnu-gcc "$@" EOF chmod +x ~/distcc_bin/aarch64-linux-gnu-gcc export PATH=~/distcc_bin:$PATH ``` This ensures any call to `aarch64-linux-gnu-gcc` uses distcc automatically, simplifying setup and avoiding the need to modify kernel Makefiles directly. --- ## Configuring distcc hosts On the R4, edit: ``` /etc/distcc/hosts ``` Example contents: ``` 172.16.0.1 172.16.0.2 ``` Replace these with the IPs of Homer and Worker. --- ## Starting distccd Inside each chroot on Homer and Worker: ```bash /usr/bin/distccd \ --pid-file=/var/run/distccd.pid \ --log-file=/var/log/distccd.log \ --daemon \ --allow 127.0.0.1 \ --allow 172.16.0.0/24 \ --allow 172.17.0.0/24 \ --listen 172.16.0.1 \ --nice 10 \ --jobs 28 ``` Adjust `--listen` and `--jobs` based on hardware capacity. --- ## Testing distcc This simple test verifies distcc setup: On the R4: ```bash echo 'int main() { return 0; }' > test.c DISTCC_VERBOSE=1 distcc aarch64-linux-gnu-gcc -c test.c -o test.o ``` ### Confirming distributed builds: - Use `distccmon-text 1` on Homer and Worker to view live job distribution. - Inspect `/var/log/distccd.log` for compile job logs. - Use `ifstat` to monitor network traffic indicating distributed builds. If your Banana Pi R4 shows high CPU load but no network activity, distcc is misconfigured. --- ## Building the kernel Compile the kernel from the Frank-W repo: ```bash export ARCH=arm64 export CROSS_COMPILE=aarch64-linux-gnu- make -j60 ``` - The `-j60` flag utilizes up to 60 threads. - Check load to avoid memory exhaustion. - Check cache efficiency with `ccache -s`. ```bash ccache -s ``` --- ## Why binfmt\_misc? Without binfmt\_misc, an x86 host can’t run ARM binaries. This Linux kernel feature automatically invokes QEMU for aarch64 binaries, allowing transparent cross-compilation in the chroot environment. --- ## Real-World Results - **Banana Pi R4 local build:** \~2 hours - **Using distcc:** \~20 minutes or less - Network traffic peaks around \~10 Gbps during builds. This setup transforms the Banana Pi R4 from a slow single-board computer into a serious development platform. Happy hacking! ![](https://articles.akadata.co.uk/content/images/2025/07/image-13.png) ## Conclusion With the right tooling, you don’t have to settle for 2-hour kernel builds. Combining **distcc, ccache, and smart chrooting** turns the Banana Pi R4 into a serious development platform, slashing build times and keeping your workflow productive. --- **Want to replicate this?** Feel free to reach out or check the ongoing work at: > [https://articles.akadata.ltd/how-to-speed-up-bpi-r4-development-with-nfs-root-and-cross-chrooting/](https://articles.akadata.co.uk/how-to-speed-up-bpi-r4-development-with-nfs-root-and-cross-chrooting/)