Compiling Krita for ARM: an AppImage tale

The AppImage running on a Raspberry Pi 3B+. It's slow, but it works 😄

Someone on #krita, can’t remember their exact nick, asked if it was possible to run Krita on ARM-based computers, specifically the Raspberry Pi 3B+. AFAIK, no one has tried to do so, so I will tell you: yes, it is possible! (Although it will run as slow as a turtle!) This work took me the whole weekend, but it was an excellent experience as well as a wonderful way to test our infrastructure.

A key warning before moving on: DO NOT TRY THIS ON YOUR PI. It will be unbearably slow 😄 I built mine with a Ryzen 7 with 12 threads and it still took me two 12-hour shifts!

This post covers three steps: setting up the build environment, compiling the dependencies and Krita itself, and finally packaging the AppImages themselves. As per the official instructions, we’ll target Ubuntu 16.04 ARM. I chose the armhf port to match the Raspberry Pi’s default distro, Raspbian. I also tested aarch64 – see the last section for the necessary changes.

Setting up the build environment

KDE’s official way to build AppImages is through a Docker image. This image is based off 16.04 x64. We’ll port this image to ARM and run it under the QEMU emulator. This is done by changing the base image to multiarch/ubuntu-debootstrap:armhf-xenial.

We also need to build four tools that do not have official, up-to-date binaries available:

CMake
AppImageTool
patchelf
linuxdeployqt

To make this easy, I’ll just show you the finished Dockerfile. I build a base layer which matches almost perfectly the original image, then build the four tools off it, and finally copy it back to the final layer. The key difference is that I changed the USER to an unprivileged one, and the ENTRYPOINT to bash (through QEMU).

FROM multiarch/ubuntu-debootstrap:armhf-xenial as base

# Start off as root
USER root

# Setup the various repositories we are going to need for our dependencies
# Some software demands a newer GCC because they're using C++14 stuff, which is just insane
# AMY: CMake must be compiled manually, Kitware does not provide ARM packages
RUN apt-get update && apt-get install -y apt-transport-https ca-certificates gnupg software-properties-common wget
RUN add-apt-repository -y ppa:openjdk-r/ppa

# Update the system and bring in our core operating requirements
RUN apt-get update && apt-get upgrade -y && apt-get install -y openssh-server openjdk-8-jre-headless

# Some software demands a newer GCC because they're using C++14 stuff, which is just insane
# We do this after the general system update to ensure it doesn't bring in any unnecessary updates
# AMY: Add universe repo at this time, debootstrap does not include it; we'll need it for gstreamer
RUN add-apt-repository -y ppa:ubuntu-toolchain-r/test && add-apt-repository universe && apt-get update

# Now install the general dependencies we need for builds
RUN apt-get install -y \
    # General requirements for building KDE software
    build-essential git-core locales \
    # General requirements for building other software
    automake gcc-6 g++-6 libxml-parser-perl libpq-dev libaio-dev \
    # Needed for some frameworks
    bison gettext \
    # Qt and KDE Build Dependencies
    gperf libasound2-dev libatkmm-1.6-dev libbz2-dev libcairo-perl libcap-dev libcups2-dev libdbus-1-dev \
    libdrm-dev libegl1-mesa-dev libfontconfig1-dev libfreetype6-dev libgcrypt11-dev libgl1-mesa-dev \
    libglib-perl libgsl0-dev libgsl0-dev gstreamer1.0-alsa libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev \
    libgtk2-perl libjpeg-dev libnss3-dev libpci-dev libpng12-dev libpulse-dev libssl-dev \
    libgstreamer-plugins-good1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base \
    gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly libtiff5-dev libudev-dev libwebp-dev flex libmysqlclient-dev \
    # Mesa libraries for everything to use
    libx11-dev libxkbcommon-x11-dev libxcb-glx0-dev libxcb-keysyms1-dev libxcb-util0-dev libxcb-res0-dev libxcb1-dev libxcomposite-dev libxcursor-dev \
    libxdamage-dev libxext-dev libxfixes-dev libxi-dev libxrandr-dev libxrender-dev libxss-dev libxtst-dev mesa-common-dev \
    # Kdenlive AppImage extra dependencies
    liblist-moreutils-perl libtool libpixman-1-dev subversion

# Setup a user account for everything else to be done under
RUN useradd -d /home/appimage/ -u 1000 --user-group --create-home -G video appimage

# Get locales in order
RUN locale-gen en_US en_US.UTF-8 en_NZ.UTF-8

###
### Prepare cmake
###

FROM base as cmake

# TODO test if pip install cmake works
RUN cd /tmp && \
    wget -c https://cmake.org/files/v3.17/cmake-3.17.0.tar.gz && \
    tar xf cmake-3.17.0.tar.gz && \
    cd cmake-3.17.0 && \
    ./bootstrap --prefix=/tmp/cmake --parallel=$(nproc) && \
    make -j $(nproc) && \
    make -j $(nproc) install

###
### Prepare appimagetool
###

FROM base as appimagetool

# Install appimagetool
RUN cd /tmp && \
    wget -c "https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-armhf.AppImage" && \
    chmod +x appimagetool*AppImage && \
    mkdir appimagetool && \
    cd appimagetool && \
    qemu-arm-static /tmp/appimagetool*AppImage --appimage-extract && \
    chmod +rx squashfs-root/usr/lib/appimagekit

###
### Prepare patchelf
###

FROM base AS patchelf

RUN cd /tmp && \
    wget -c https://nixos.org/releases/patchelf/patchelf-0.9/patchelf-0.9.tar.bz2 && \ 
    tar xf patchelf-0.9.tar.bz2 && \
    cd patchelf-0.9/ && \
    ./configure --prefix=/tmp/patchelf && \
    make -j$(nproc) && \
    make -j$(nproc) install

###
### Prepare linuxdeployqt
###

FROM base as linuxdeployqt

COPY --from=patchelf /tmp/patchelf /

COPY --from=appimagetool  /tmp/appimagetool/squashfs-root/usr/ /usr/local

RUN apt-get install -y \
    qt5-default qtbase5-dev qttools5-dev-tools

RUN cd /tmp && \
    git clone https://github.com/probonopd/linuxdeployqt.git src && \
    cd src && \
    qmake CONFIG+=release CONFIG+=force_debug_info linuxdeployqt.pro && \
    make -j$(nproc) && \
    mkdir -p linuxdeployqt.AppDir/usr/bin && \
    mkdir -p linuxdeployqt.AppDir/usr/lib && \
    cp ./bin/linuxdeployqt linuxdeployqt.AppDir/usr/bin/ && \
    chmod +x linuxdeployqt.AppDir/AppRun && \
    ./bin/linuxdeployqt linuxdeployqt.AppDir/linuxdeployqt.desktop -verbose=3 -appimage -executable=linuxdeployqt.AppDir/usr/bin/linuxdeployqt && \
    cd /tmp && \
    mkdir linuxdeployqt && \
    cd linuxdeployqt && \
    qemu-arm-static /tmp/src/linuxdeployqt*AppImage --appimage-extract

###
# Merge all tools (into /usr/local, that folder is ignored by Krita scripts)
###

FROM base

LABEL Description="KDE Appimage Base (armhf)"
LABEL vendor="amyspark.me"
LABEL me.amyspark.release-date="2020-07-13"

# Install cmake

COPY --from=cmake /tmp/cmake /usr/local

# Install patchelf
COPY --from=patchelf /tmp/patchelf /usr/local

# Install linuxdeployqt
COPY --from=linuxdeployqt /tmp/linuxdeployqt/squashfs-root/usr/ /usr/local

# Install appimagetool
COPY --from=appimagetool  /tmp/appimagetool/squashfs-root/usr/ /usr/local

USER appimage

ENTRYPOINT [ "/usr/bin/qemu-arm-static", "/bin/bash" ]

I build this image with:

docker build . --tag krita:latest

After a good half day, I got a freshly built base for my AppImage needs.

Building dependencies and Krita

To build our dependencies, KDE uses a script available at Krita’s repo under packaging/linux/appimage/build-deps.sh. In ARM, this script will fail out of the box. It needs the following changes (replace X with the number of threads):

Add -DSUBMAKE_JOBS=X to the cmake line. This enables multithreading the Qt build.
Append -- -jX to the end of each dependency line below it, except the following (these use make directly):
- expat
- python
- sip

The following dependencies will not build, comment them out:

OpenColorIO (ext_ocio)
- it needs this patch applied
libx265 (inside ext_heif)
- append -DENABLE_ASSEMBLY=FALSE to the end of the CMAKE_ARGS line in the ext_x265 block of 3rdparty/ext_heif/CMakeLists.txt
- this was reported to MultiCoreWare as #486
python (ext_python)
- it will randomly hang when building the tests
- I just kill -HUP the offending processes, or edit the BUILD_COMMAND and change the make target to build_all
GSL (ext_gsl)
- Linux needs the shared version, replace if (ANDROID) with if (UNIX) in 3rdparty/ext_gsl/CMakeLists.txt
Vc (ext_vc)
- there is no fix available for this

Assuming your working directory is where you cloned Krita’s repository (it should have a src folder), run:

docker run --rm -v $(pwd):/krita -ti krita:latest -- /krita/src/packaging/linux/appimage/build-deps.sh /krita/appimage_build_armhf /krita/src

After some 6 or 7 hours, you should have a fully working set of libraries.

Building Krita

This does not need any changes. Run:

docker run --rm -v $(pwd):/krita -ti krita:latest -- /krita/src/packaging/linux/appimage/build-krita.sh /krita/appimage_build_armhf /krita/src

Packing the AppImages

The AppImage packaging process is run through two scripts, build-gmic-qt.sh and build-image.sh.

These scripts hardcode two assumptions:

that the platform is x86_64
that the ABI (i.e. the platform triplet) is x86_64-linux-gnu We’ll fix both.

In build-gmic-qt.sh add before the final mv command these lines:

ARCH=`dpkg --print-architecture`
TRIPLET=`gcc -dumpmachine`

and replace the mv with:

mv gmic_krita_qt*$ARCH.AppImage gmic_krita_qt-$ARCH.appimage

In build-image.sh, the process is a bit more convoluted, so apply this diff:

diff --git a/packaging/linux/appimage/build-image.sh b/packaging/linux/appimage/build-image.sh
index 1e3660b1dd..41579edc1d 100755
--- a/packaging/linux/appimage/build-image.sh
+++ b/packaging/linux/appimage/build-image.sh
@@ -22,7 +22,9 @@ export DEPS_INSTALL_PREFIX=$BUILD_PREFIX/deps/usr/
 export DOWNLOADS_DIR=$BUILD_PREFIX/downloads/
 
 # Setup variables needed to help everything find what we built
-export LD_LIBRARY_PATH=$DEPS_INSTALL_PREFIX/lib/:$DEPS_INSTALL_PREFIX/lib/x86_64-linux-gnu/:$APPDIR/usr/lib/:$LD_LIBRARY_PATH
+ARCH=`dpkg --print-architecture`
+TRIPLET=`gcc -dumpmachine`
+export LD_LIBRARY_PATH=$DEPS_INSTALL_PREFIX/lib/:$DEPS_INSTALL_PREFIX/lib/$TRIPLET/:$APPDIR/usr/lib/:$LD_LIBRARY_PATH
 export PATH=$DEPS_INSTALL_PREFIX/bin/:$PATH
 export PKG_CONFIG_PATH=$DEPS_INSTALL_PREFIX/share/pkgconfig/:$DEPS_INSTALL_PREFIX/lib/pkgconfig/:/usr/lib/pkgconfig/:$PKG_CONFIG_PATH
 export CMAKE_PREFIX_PATH=$DEPS_INSTALL_PREFIX:$CMAKE_PREFIX_PATH
@@ -50,8 +52,10 @@ cp -r $DEPS_INSTALL_PREFIX/share/sip $APPDIR/usr/share
 cp -r $DEPS_INSTALL_PREFIX/translations $APPDIR/usr/
 
 # Step 2: Relocate x64 binaries from the architecture specific directory as required for Appimages
-mv $APPDIR/usr/lib/x86_64-linux-gnu/*  $APPDIR/usr/lib
-rm -rf $APPDIR/usr/lib/x86_64-linux-gnu/
+if [ -d $APPDIR/usr/lib/$TRIPLET ] ; then
+  mv $APPDIR/usr/lib/$TRIPLET/*  $APPDIR/usr/lib
+  rm -rf $APPDIR/usr/lib/$TRIPLET/
+fi
 
 # Step 3: Update the rpath in the various plugins we have to make sure they'll be loadable in an Appimage context
 for lib in $PLUGINS/*.so*; do
@@ -126,7 +130,7 @@ linuxdeployqt $APPDIR/usr/share/applications/org.kde.krita.desktop \
   -appimage 
   
 # Generate a new name for the Appimage file and rename it accordingly
-APPIMAGE=krita-"$VERSION"-x86_64.appimage
+APPIMAGE=krita-"$VERSION"-$ARCH.appimage
 
-mv Krita*x86_64.AppImage $APPIMAGE
+mv Krita*$ARCH.AppImage $APPIMAGE

Now you should be able to run:

docker run --rm -v $(pwd):/krita -ti krita:latest -- /krita/src/packaging/linux/appimage/build-image.sh /krita/appimage_build_armhf /krita/src
docker run --rm -v $(pwd):/krita -ti krita:latest -- /krita/src/packaging/linux/appimage/build-gmic-qt.sh /krita/appimage_build_armhf /krita/src

and get your own set of AppImages, fresh from the oven!

Release!

The AppImage is at my S3 dumping ground.

Integrity check:

d1b982ddbe8cc0e3fbd6032a9db7067f7de30ad034aa5c8f1d5f291e350296ac  krita-4.3.1-alpha-79267b7-armhf.appimage

And what about aarch64?

This one took me another whole day (well, 12 hour shift – plus 4 more hours, thanks to a bug in KDE’s Dockerfile).

There are two big bugs in aarch64. First, to build Qt’s X11Extras module, the key dependency is xkbcommon-dev. However, in all platforms it is brought in by libegl1-mesa-dev– except on aarch64, courtesy of libmirclient-dev. You need to add it manually to the Dockerfile.

Then, AppImageKit has a weird naming convention for this platform. While Debian names 32-bit ARM (hard floating point) armhf, and 64-bit arm64, AppImageKit calls the latter aarch64. This will need extra tinkering in the build scripts (which I did not care to do at this point, seeing how much it takes to compile).

Also, libx265 will not work on aarch64 because we run 3.2.1, which does not support this platform at all.

This is the Dockerfile for aarch64. You can use it with the commands above (just change the armhf suffix for arm64 where necessary):

FROM multiarch/ubuntu-debootstrap:arm64-xenial as base

# Start off as root
USER root

# Setup the various repositories we are going to need for our dependencies
# Some software demands a newer GCC because they're using C++14 stuff, which is just insane
# AMY: CMake must be compiled manually, Kitware does not provide ARM packages
RUN apt-get update && apt-get install -y apt-transport-https ca-certificates gnupg software-properties-common wget
RUN add-apt-repository -y ppa:openjdk-r/ppa

# Update the system and bring in our core operating requirements
RUN apt-get update && apt-get upgrade -y && apt-get install -y openssh-server openjdk-8-jre-headless

# Some software demands a newer GCC because they're using C++14 stuff, which is just insane
# We do this after the general system update to ensure it doesn't bring in any unnecessary updates
# AMY: Add universe repo at this time, debootstrap does not include it
RUN add-apt-repository -y ppa:ubuntu-toolchain-r/test && add-apt-repository universe && apt-get update

# Now install the general dependencies we need for builds
RUN apt-get install -y \
    # General requirements for building KDE software
    build-essential git-core locales \
    # General requirements for building other software
    automake gcc-6 g++-6 libxml-parser-perl libpq-dev libaio-dev \
    # Needed for some frameworks
    bison gettext \
    # Qt and KDE Build Dependencies
    gperf libasound2-dev libatkmm-1.6-dev libbz2-dev libcairo-perl libcap-dev libcups2-dev libdbus-1-dev \
    libdrm-dev libegl1-mesa-dev libfontconfig1-dev libfreetype6-dev libgcrypt11-dev libgl1-mesa-dev \
    # AMY: on arm64, libegl1-mesa-dev does not bring in libxkbcommon-dev
    libxkbcommon-dev \
    libglib-perl libgsl0-dev libgsl0-dev gstreamer1.0-alsa libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev \
    libgtk2-perl libjpeg-dev libnss3-dev libpci-dev libpng12-dev libpulse-dev libssl-dev \
    libgstreamer-plugins-good1.0-dev libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-base \
    gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly libtiff5-dev libudev-dev libwebp-dev flex libmysqlclient-dev \
    # Mesa libraries for everything to use
    libx11-dev libxkbcommon-x11-dev libxcb-glx0-dev libxcb-keysyms1-dev libxcb-util0-dev libxcb-res0-dev libxcb1-dev libxcomposite-dev libxcursor-dev \
    libxdamage-dev libxext-dev libxfixes-dev libxi-dev libxrandr-dev libxrender-dev libxss-dev libxtst-dev mesa-common-dev \
    # Kdenlive AppImage extra dependencies
    liblist-moreutils-perl libtool libpixman-1-dev subversion

# Setup a user account for everything else to be done under
RUN useradd -d /home/appimage/ -u 1000 --user-group --create-home -G video appimage

# Get locales in order
RUN locale-gen en_US en_US.UTF-8 en_NZ.UTF-8

###
### Prepare cmake
###

FROM base as cmake

# TODO test if pip install cmake works
RUN cd /tmp && \
    wget -c https://cmake.org/files/v3.17/cmake-3.17.0.tar.gz && \
    tar xf cmake-3.17.0.tar.gz && \
    cd cmake-3.17.0 && \
    ./bootstrap --prefix=/tmp/cmake --parallel=$(nproc) && \
    make -j $(nproc) && \
    make -j $(nproc) install

###
### Prepare appimagetool
###

FROM base as appimagetool

# Install appimagetool
RUN cd /tmp && \
    wget -c "https://github.com/AppImage/AppImageKit/releases/download/continuous/appimagetool-aarch64.AppImage" && \
    chmod +x appimagetool*AppImage && \
    mkdir appimagetool && \
    cd appimagetool && \
    qemu-aarch64-static /tmp/appimagetool*AppImage --appimage-extract && \
    chmod +rx squashfs-root/usr/lib/appimagekit

###
### Prepare patchelf
###

FROM base AS patchelf

RUN cd /tmp && \
    wget -c https://nixos.org/releases/patchelf/patchelf-0.9/patchelf-0.9.tar.bz2 && \ 
    tar xf patchelf-0.9.tar.bz2 && \
    cd patchelf-0.9/ && \
    ./configure --prefix=/tmp/patchelf && \
    make -j$(nproc) && \
    make -j$(nproc) install

###
### Prepare linuxdeployqt
###

FROM base as linuxdeployqt

COPY --from=patchelf /tmp/patchelf /

COPY --from=appimagetool  /tmp/appimagetool/squashfs-root/usr/ /usr/local

RUN apt-get install -y \
    qt5-default qtbase5-dev qttools5-dev-tools

RUN cd /tmp && \
    git clone https://github.com/probonopd/linuxdeployqt.git src && \
    cd src && \
    qmake CONFIG+=release CONFIG+=force_debug_info linuxdeployqt.pro && \
    make -j$(nproc) && \
    mkdir -p linuxdeployqt.AppDir/usr/bin && \
    mkdir -p linuxdeployqt.AppDir/usr/lib && \
    cp ./bin/linuxdeployqt linuxdeployqt.AppDir/usr/bin/ && \
    chmod +x linuxdeployqt.AppDir/AppRun && \
    ./bin/linuxdeployqt linuxdeployqt.AppDir/linuxdeployqt.desktop -verbose=3 -appimage -executable=linuxdeployqt.AppDir/usr/bin/linuxdeployqt && \
    cd /tmp && \
    mkdir linuxdeployqt && \
    cd linuxdeployqt && \
    qemu-aarch64-static /tmp/src/linuxdeployqt*AppImage --appimage-extract

###
# Merge all tools (into /usr/local, that folder is ignored by Krita scripts)
###

FROM base

LABEL Description="KDE Appimage Base (arm64)"
LABEL vendor="amyspark.me"
LABEL me.amyspark.release-date="2020-07-14"

# Install cmake

COPY --from=cmake /tmp/cmake /usr/local

# Install patchelf
COPY --from=patchelf /tmp/patchelf /usr/local

# Install linuxdeployqt
COPY --from=linuxdeployqt /tmp/linuxdeployqt/squashfs-root/usr/ /usr/local

# Install appimagetool
COPY --from=appimagetool  /tmp/appimagetool/squashfs-root/usr/ /usr/local

USER appimage

ENTRYPOINT [ "/usr/bin/qemu-aarch64-static", "/bin/bash" ]

The AppImage running on Libre Computer's 2GB Le Potato. It is way faster UI-wise, but the canvas slows down when hovering...

The AppImage is at my S3 dumping ground.

Integrity check:

1c388f527c23783d58ab23d5e07d186c1a914ded7e9d1d3b7df7a7153dd6dd5f  krita-4.3.1-alpha-aba6da1-aarch64.AppImage

And this is all from me today! Please chime in on the Krita Artists thread if you have any feedback.

Cheers, ~amyspark